shell script option http://chetanjm.wordpress.com/2009/07/16/simple-s3-backup-database-and-code-script-using-s3cmd/

mysqldump –databases db_name1 db_name2 db_name3

Thanks for your useful script first,but I have a question with it
See:
@mysql_database = “your-mysql-database”

Is it means that you can only backup a single database with this script?

If I want to backup all the databases,what should I change with your script?

Thanks

This resulted in the path and filename being different.

Thanks for the code. This is was a great starting point. Is the updated code available outside of installing ec2onrails? I recently moved off AWS and I now use Mosso for everything.

Thanks for pointing out that the full backup should delete the binary logs. That’s a bug, I’ll try to find time to fix it.

You’re right, when any of the S3 calls fail we should retry. I recently switched the backup feature in EC2 on Rails (which this example code is based on) to use the right_aws S3 library instead of aws/s3 because it automatically retries a few times with an increasing pause between failures.

Regarding the other issues: I look at this as just mirroring the MySQL data on S3, not archiving it. If you screw up your database in any way then as you pointed out you’ll be backing up your screwed up version. I think it’s a good idea to archive it as well, and I archive mine regularly (but to a different S3 bucket entirely).

I’m not sure what you mean about having “log_bin only”.

In my.cnf I had log_bin only which results in the binary logs being named mysqld.bin thus I needed to edit incremental_backup.rb and restore.rb

I was concerned that there was only one full backup in s3. Isn’t it possible for my DB to be damaged and then overwrite the good copy in s3? Assuming the dump is done daily I moved the last dumpfile before pushing the latest one.

last_dump = File.basename(dump_file) + (Date.today-1).wday.to_s
# allows for 7 days of dumps to be in S3
begin
AWS::S3::S3Object.rename(File.basename(dump_file), last_dump, @s3_bucket)
rescue AWS::S3::NoSuchKey
puts last_dump + ” did not exist. continuing”
end

If someone accidentally deleted the database before the full backup would we push an empty dump file? Just in case I also check the size of the dump and only push to s3 if the size is greater than an arbitrary 1000 bytes

I also added code to remove the binary logs from S3 on full backup.. I have not added locking yet to prevent the scripts from overlapping.. Any reason why you didn’t share the whole solution? :-)

Another question: what happens if AWS::S3::S3Object.store fails. Seems like that would be grounds for a retry. (another reason why I like having 7 days of backups in s3. hopefully one is good. )

Major differences are using ruby’s Tempfile rather than managing your own temp dir, externalizing config so you can store it safely in source control, and some specs to make sure it all works.

Great article. I’m wondering how easy/hard it would be to roll encryption into this (PGP, maybe openssl etc) to encrypt the tar dumps before putting them on S3.