caleblloyd.com

hardware, software, and life

Automatically Take EBS Snapshots and Delete Old Ones with PHP Script

Posted on

I am a big fan of Amazon’s RDS product, which takes automated nightly snapshots of your RDS Storage and deletes old snapshots after a specified amount of time. After reading that Amazon’s EBS drives sustain a 0.1% – 0.5% Annual Failure Rate, I also wanted to take automated nightly snapshots of my EBS drives. Deleting snapshots after a certain number of days was also of interest, because this way I am not over-paying for snapshot storage at Amazon or removing snapshots manually. I also wanted to email the results to myself after backups were taken.

I did some searching on Google and found a nice script by Chris at Applied Trust Engineering, Inc. that runs PHP from the command line to create an automatic snapshot. I used this script as an example to setup my script, so thanks, Chris. My script has support for:

  • Backup Multiple EBS Volumes
  • Protect against script running again and creating another snapshot too soon
  • Delete Snapshots after a Specified Period of Time
  • Script outputs Detailed Snapshot Information for these 5 categories:
    • Snapshots that succeeded
    • Snapshots that failed and had errors
    • Old Snapshots that were removed
    • Old Snapshots that had errors while trying to remove
    • Snapshots that were preserved
  • Includes PHPMailer code to email results of script to yourself

Script Setup

You will need: (all links open in new window)

  • My ebs_backup.php Code (download is a zip) (or look below)
  • Stores snapshot information in “./snapshot_information.json” – Make sure PHP can write this file
  • Configure the lines of code within the configuration comment blocks
  • Run script periodically to your needs with CRON or whatever
  • Requires AWS PHP SDK be configured for your AWS Account: http://aws.amazon.com/sdkforphp/
  • Optional PHPMailer Support to email results to yourself: http://phpmailer.worxware.com/ (configure PHPMailer at the very bottom of the script)

Screenshot of Output

Automatic EBS Snapshots

Source Code

snapshot_information.json: (just initialize it to an empty list and make sure PHP can write to it)

[ ]

ebs_backup.php:

<?

/**************************************************************************
|
|	Script to Automate EBS Backups
|	Run this script with CRON or whatever every X period of time to take
|	automatic snapshots of your EBS Volumes.  Script will delete old
|	snapshot after Y period of time
|	
|	Version 1.01 updated 2012-08-02
|
|	Copyright 2012 Caleb Lloyd
|
|
|	I offer no warrant or guarentee on this code - use at your own risk
|	You are free to modify and redistribute this code as you please
|
|	Requires AWS PHP SDK be configured for your AWS Account:
|		http://aws.amazon.com/sdkforphp/
|
|	Optional PHPMailer Support to email results to yourself
|		http://phpmailer.worxware.com/
|
|	Stores snapshot information in "./snapshot_information.json"
|		Make sure PHP can write this file
|
**************************************************************************/


/**************************************************************************
|	Begin Configuration
**************************************************************************/

//Declare the volumes that you want to backup
//The Volume ID's are the keys of the array, you can store any custom information you
//want in value array, or just keep it blank.  Make sure you keep it as a blank array
//because the script will fillthis up with values...
$volumes=array(	'vol-11111111'=>array(),
				'vol-22222222'=>array()
);

//Do not take a snapshot more than every X hours/minutes/days, etc. (uses strtotime)
//This prevents the script from running out of control and producing tons of snapshots
$snapshot_limit = '23 hours';

//Keep snapshots for this amount of time (also uses strtotime)
$keep_snapshots = '7 days 12 hours';

//Your path to the Amazon AWS PHP SDK
require_once 'path_to_aws_php_sdk/sdk.class.php';
//EC2 Region, view path_to_aws_php_sdk/services/ec2.class.php for definitions
$region='ec2.us-east-1.amazonaws.com';

//Your path to PHP Mailer (if you don't want to eamil yourself the results, you can get rid of this)
require_once('path_to_PHPMailer/class.phpmailer.php');
//Go to bottom of script to configure PHP Mailer settings


/**************************************************************************
|	End Configuration
**************************************************************************/

function snapshot_info($s)
{
	$info='<p>';
	$info.='Volume: '.$s['volume'].'<br />';
	$info.=(!empty($s['volume_name'])?'Volume Name: '.$s['volume_name'].'<br />':'');
	$info.=(!empty($s['snapshot'])?'Snapshot: '.$s['snapshot'].'<br />':'');
	$info.=(!empty($s['instance'])?'EC2 Instance: '.$s['instance'].'<br />':'');
	$info.=(!empty($s['device'])?'Device: '.$s['device'].'<br />':'');
	$info.=(!empty($s['error'])?'Error: '.$s['error'].'<br />':'');
	$info.=(!empty($s['datetime'])?'Date/Time: '.$s['datetime'].'<br />':'');
	$info.='</p>';
	return $info;
}

$success=array();
$failure=array();
$preserve=array();
$success_delte=array();
$failure_delete=array();

$ec2 = new AmazonEC2();
$ec2 = $ec2->set_region($region);

$latest_snapshot=array();

if (file_exists('snapshot_information.json'))
	$json=file_get_contents('snapshot_information.json');
else
	$json='[]';
$snapshots=json_decode($json,TRUE);
foreach ($snapshots as $s)
{
	if (!empty($lastest_snapshot[$s['volume']]))
	{
		if ($s['timestamp']>$lastest_snapshot[$s['volume']]['timestamp'])
		{
			$lastest_snapshot[$s['volume']]=$s;
		}
	}
	else
	{
		$lastest_snapshot[$s['volume']]=$s;
	}
}

foreach ($volumes as $volume => $v)
{
	$v['volume']=$volume;
	$v['instance']='Not Attached to an Instance';

	$volume_information = $ec2->describe_volumes(array('VolumeId' => $volume));
	$v['volume_name'] = '(volume has no tags)';
	if (!empty($volume_information->body->volumeSet->item->tagSet->item->value))
	{
		$v['volume_name'] = (string)$volume_information->body->volumeSet->item->tagSet->item->value;
	}
	$description = 'Volume '.$volume.(empty($v['volume_name'])?'':' ('.$v['volume_name'].')');
	
	if (!empty($volume_information->body->volumeSet->item->attachmentSet->item->status))
	{
		if ($volume_information->body->volumeSet->item->attachmentSet->item->status == "attached")
		{
			$v['device'] = (string)$volume_information->body->volumeSet->item->attachmentSet->item->device;
			$v['instance'] = (string)$volume_information->body->volumeSet->item->attachmentSet->item->instanceId;
			$description.=' attached to '.$v['instance'].' as '.$v['device'];
		}
	}
	else
	{
		$description.= ' ('.$v['instance'].')';
	}
	
	if ((!empty($lastest_snapshot[$volume]))&&($lastest_snapshot[$volume]['timestamp']>strtotime('-'.$snapshot_limit)))
	{
		$error=TRUE;
		$v['datetime']=date('Y-m-d H:i:s');
		$v['timestamp']=time();
		$v['error']='An Automatic Snapshot Already Exists for that volume in the past '.$snapshot_limit;
		$failure[]=$v;
	}
	else
	{
		$response = $ec2->create_snapshot($volume, array('Description'=>$description));
		if ($response->isOK())
		{
			$v['datetime']=date('Y-m-d H:i:s');
			$v['timestamp']=time();
			$v['snapshot']=(string)$response->body->snapshotId;
			$success[$v['snapshot']]=$v;
		}
		else
		{
			$error=TRUE;
			$v['datetime']=date('Y-m-d H:i:s');
			$v['timestamp']=time();
			$v['error']=(string)$response->body->Errors->Error->Message;
			$failure[]=$v;
		}
	}
}

if (!empty($snapshots))
{
	foreach ($snapshots as $snapshot => $s)
	{
		$s['snapshot']=$snapshot;
		if ($s['timestamp']<strtotime('-'.$keep_snapshots))
		{
			$response = $ec2->delete_snapshot($snapshot);
			if ($response->isOK())
			{
				$success_delete[$snapshot]=$s;
			}
			else
			{
				$error=TRUE;
				$s['error']=(string)$response->body->Errors->Error->Message;
				$failure_delete[$snapshot]=$s;
			}
		}
		else
		{
			$preserve[$snapshot]=$s;
		}
	}
	$snapshots_json=json_encode(array_merge($success,$preserve));
}
else
{
	$snapshots_json=json_encode($success);
}
file_put_contents('snapshot_information.json',$snapshots_json);

$message='';

if (!empty($success))
{
	$message.='<p><strong>The following Snapshots Succeeded:</strong></p>';
	foreach ($success as $v)
	{
		$message.=snapshot_info($v);
	}
}

if (!empty($failure))
{
	$message.='<p><strong>The following Snapshots Failed and had Errors:</strong></p>';
	foreach ($failure as $v)
	{
		$message.=snapshot_info($v);
	}
}

if (!empty($success_delete))
{
	$message.='<p><strong>The following old Snapshots were removed:</strong></p>';
	foreach ($success_delete as $v)
	{
		$message.=snapshot_info($v);
	}
}

if (!empty($failure_delete))
{
	$message.='<p><strong>The following old Snapshots had errors while trying to remove:</strong></p>';
	foreach ($failure_delete as $v)
	{
		$message.=snapshot_info($v);
	}
}

if (!empty($preserve))
{
	$message.='<p><strong>The following Snapshots were preserved:</strong></p>';
	foreach ($preserve as $v)
	{
		$message.=snapshot_info($v);
	}
}

echo $message;


/**************************************************************************
|	Begin PHPMailer Script
|	Remove Below This Line if you don't want to EMail results to yourself
|	This is the SMTP Example
|	For other examples in PHPMailer, go to path_to_PHPMailer/examples
**************************************************************************/

$mail = new PHPMailer(true); // the true param means it will throw exceptions on errors, which we need to catch

$mail->IsSMTP(); // telling the class to use SMTP

try {
  $mail->Host       = "mail.yourdomain.com"; // SMTP server
  $mail->SMTPDebug  = 2;                     // enables SMTP debug information (for testing)
  $mail->SMTPAuth   = true;                  // enable SMTP authentication
  $mail->Host       = "mail.yourdomain.com"; // sets the SMTP server
  $mail->Port       = 26;                    // set the SMTP port for the GMAIL server
  $mail->Username   = "yourname@yourdomain"; // SMTP account username
  $mail->Password   = "yourpassword";        // SMTP account password
  $mail->AddReplyTo('[email protected]', 'First Last');
  $mail->AddAddress('[email protected]', 'John Doe');
  $mail->SetFrom('[email protected]', 'First Last');
  
  $mail->Subject = 'EBS Snapshot Backup Information for '.date('Y-m-d').' - '.($error?'ERRORS':'Success');
  $mail->MsgHTML($message);
  
  $mail->Send();
  echo "Message Sent OK</p>\n";
} catch (phpmailerException $e) {
  echo $e->errorMessage(); //Pretty error messages from PHPMailer
} catch (Exception $e) {
  echo $e->getMessage(); //Boring error messages from anything else!
}

/**************************************************************************
|	End PHPMailer Script
|	Remove Above This Line if you don't want to EMail results to yourself
**************************************************************************/
?>