MySQL Database Migration: S3 to EBS

Objective

This tutorial will explain how to migrate a running MySQL setup that saves backups to S3 to a MySQL setup that uses EBS snapshots for backups--all without any downtime!  If you do not have a running MySQL database setup, you can follow the Create a MySQL-EBS Database Setup tutorial.

Instead of using the "MySQL Bootstrap" and "MySQL Additional" ServerTemplates for your redundant master/slave database servers, you will use the "MySQL EBS" ServerTemplate.  The key concept to keep in mind is that the "MySQL EBS" is based on EBS snapshots.  When a new server is launched with the EBS template, it will build a server and create/attach an EBS volume from the most recent EBS snapshot instead of trying to download a large .tar backup file from S3 (Master-DB), which could take a long time for large databases.

NOTE:  Be sure to lower the TTL setting for your DNS entries for your master/slave databases.  The Master/Slave promotion cannot be completed until the Master database's TTL has expired.

Step 1: Launch a Server using the "MySQL EBS" ServerTemplate

Go to your active deployment that contains your running master/slave databases that are using the "MySQL Bootstrap/Additional" ServerTemplates.

The first step is to launch a server using the new "MySQL EBS" template and connect it to the current Master-DB (ex: s3-db1 master) so that you can create your first EBS snapshot.

Click the Add (EC2) Server button. 

Provide the nickname "ebs-db1" and select your SSH Key and Security Group.  Keep the default availability zone (us-east-1a).  (NOTE:  When you create an EBS volume later in this tutorial, you will need to create the volume in the same availability zone as the server that you will attach it to.) Click Add.

At the deployment level, click the Inputs tab and configure the following input parameters:

Go back to the Servers tab and launch the "ebs-db1" database server.

Step 2: Copy data to the new MySQL EBS Server

Once the "ebs-db1" server is fully operational, select the "ebs-db1" server and go to its Scripts tab. 

Run the DB EBS slave init from non-EBS master operational script. 

This script will perform the following key tasks:

• Create an EBS volume using the "EBS_DB_NAME" value as its prefix.  (ex: mydb-i-0488216d) 
• Attach the EBS volume to the "ebs-db2" server.
• Copy all the data from the current Master-DB (ex: s3-db1 master) to the attached EBS volume

(NOTE:  If you have a large database, it may take a long time to complete this action.) 

Wait for the script to be 100% complete and for the volume to be fully attached to the "ebs-db1" server.

You can monitor its progress under the Server's Audit Entries tab.  

Step 3: Promote Server to be Master-DB and create an EBS Snapshot

We now have a running server (ebs-db1) that has an attached EBS volume that matches the data that is currently in the Master-DB server, but we still don't have a snapshot.  The next step is to promote the "ebs-db1" server to become the new Master-DB.

Under the "ebs-db1" server's Scripts tab, run the DB EBS promote to master operational script. 

This script will perform the following key tasks:

• "ebs-db1" will become the new Master-DB.
• A snapshot of the volume attached to "ebs-db1" will be taken (ex: mydb-master-200810221957). 

Once again, you must wait for the script to be 100% complete.

You can monitor its progress under the Server's Audit Entries tab.  

Now go to Manage -> Storage -> EBS Snapshots.  You must wait for the snapshot to be 100% complete before continuing to the next step.  Notice that since we're taking a snapshot of a Master-DB, "master" will be included in the snapshot's filename.  You now have an EBS snapshot that you can use to build a Slave-DB server.

Step 4: Launch a Slave-DB Server

Add a new server using "MySQL EBS" template and call it "ebs-db2."

Launch the "ebs-db2" server. 

This time you can keep the default input parameters because we now have an EBS snapshot.  You can also initialize the new Slave-DB server with the running Master-DB server. Keep INIT_SLAVE_AT_BOOT set to "True."

Once it's operational, you now have a redundant database using EBS.  You can now safely terminate your old MySQL database servers that were saving backups to S3 (s3-db1 master and s3-db2 slave).

Remember, if you haven't already done so, you should make sure that the DNS entries for your master/slave databases have a low TTL setting.  Otherwise, users might still try to access the old database servers instead of using the new ones.