{"id":2936,"date":"2018-12-06T11:44:06","date_gmt":"2018-12-06T11:44:06","guid":{"rendered":"http:\/\/blog.designed79.co.uk\/?p=2936"},"modified":"2018-12-06T16:47:08","modified_gmt":"2018-12-06T16:47:08","slug":"setting-up-mysql-replication-without-the-downtime","status":"publish","type":"post","link":"https:\/\/blog.designed79.co.uk\/?p=2936","title":{"rendered":"Setting up MySQL replication without the downtime"},"content":{"rendered":"<p>I clearly don\u2019t need to expound on the benefits of master-slave replication for your MySQL database. It\u2019s simply a good idea; one nicety I looked forward to was the ability to run backups from the slave without impacting the performance of our production database. But the benefits abound.<br \/>\nMost tutorials on master-slave replication use a read lock to accomplish a consistent copy during initial setup. Barbaric! With our users sending thousands of cards and gifts at all hours of the night, I wanted to find a way to accomplish the migration without any downtime.<\/p>\n<p>enabling bin-logging and taking a non- locking dump with the binlog position included. In effect, you\u2019re creating a copy of the db marked with a timestamp, which allows the slave to catch up once you\u2019ve migrated the data over. This seems like the best way to set up a MySQL slave with no downtime, so I figured I\u2019d document the step-by-step here, in case it proves helpful for others.<br \/>\nFirst, you\u2019ll need to configure the master\u2019s \/etc\/mysql\/my.cnf by adding these lines in the [mysqld] section:<\/p>\n<div class=\"codecolorer-container text default\" style=\"overflow:auto;white-space:nowrap;\"><div class=\"text codecolorer\">server-id=1<br \/>\nbinlog-format = mixed<br \/>\nlog-bin=mysql-bin<br \/>\ndatadir=\/var\/lib\/mysql<br \/>\ninnodb_flush_log_at_trx_commit=1<br \/>\nsync_binlog=1<\/div><\/div>\n<p>Restart the master mysql server and create a replication user that your slave server will use to connect to the master:<\/p>\n<div class=\"codecolorer-container text default\" style=\"overflow:auto;white-space:nowrap;\"><div class=\"text codecolorer\">CREATE USER replicant@&lt;&lt;slave-server-ip&gt;&gt;;<br \/>\nGRANT REPLICATION SLAVE ON *.* TO replicant@&lt;&lt;slave-server-ip&gt;&gt;<br \/>\nIDENTIFIED BY '&lt;&lt;choose-a-good-password&gt;&gt;';<\/div><\/div>\n<p>Note: Mysql allows for passwords up to 32 characters for replication users.<\/p>\n<p>Next, create the backup file with the binlog position. It will affect the performance of your database server, but won\u2019t lock your tables:<\/p>\n<div class=\"codecolorer-container text default\" style=\"overflow:auto;white-space:nowrap;\"><div class=\"text codecolorer\">mysqldump --skip-lock-tables --single-transaction --flush-logs --hex-<br \/>\nblob --master-data=2 -A &nbsp;&gt; ~\/dump.sql<\/div><\/div>\n<p>Now, examine the head of the file and jot down the values for MASTER_LOG_FILE and MASTER_LOG_POS. You will need them later:<\/p>\n<div class=\"codecolorer-container text default\" style=\"overflow:auto;white-space:nowrap;\"><div class=\"text codecolorer\">head dump.sql -n80 | grep &quot;MASTER_LOG_POS&quot;<\/div><\/div>\n<p>Because this file for me was huge, I gzip&#8217;ed it before transferring it to the slave, but that\u2019s optional:<\/p>\n<div class=\"codecolorer-container text default\" style=\"overflow:auto;white-space:nowrap;\"><div class=\"text codecolorer\">gzip ~\/dump.sql<\/div><\/div>\n<p>Now we need to transfer the dump file to our slave server (if you didn\u2019t gzip first, remove the .gz bit):<\/p>\n<p>While that\u2019s running, you should log into your slave server, and edit your \/etc\/mysql\/my.cnf file to add the following lines:<\/p>\n<div class=\"codecolorer-container text default\" style=\"overflow:auto;white-space:nowrap;\"><div class=\"text codecolorer\">server-id = 101<br \/>\nbinlog-format = mixed<br \/>\nlog_bin = mysql-bin<br \/>\nrelay-log = mysql-relay-bin<br \/>\nlog-slave-updates = 1<br \/>\nread-only = 1<\/div><\/div>\n<p>Restart the mysql slave, and then import your dump file:<\/p>\n<div class=\"codecolorer-container text default\" style=\"overflow:auto;white-space:nowrap;\"><div class=\"text codecolorer\">gunzip ~\/dump.sql.gz<br \/>\nmysql -u root -p &lt; ~\/dump.sql<\/div><\/div>\n<p>Log into your mysql console on your slave server and run the following commands to set up and start replication:<\/p>\n<div class=\"codecolorer-container text default\" style=\"overflow:auto;white-space:nowrap;\"><div class=\"text codecolorer\">CHANGE MASTER TO MASTER_HOST='&lt;&lt;master-server-<br \/>\nip&gt;&gt;',MASTER_USER='replicant',MASTER_PASSWORD='&lt;&lt;slave-server-<br \/>\npassword&gt;&gt;', MASTER_LOG_FILE='&lt;&lt;value from above&gt;&gt;', MASTER_LOG_POS=<br \/>\n&lt;&lt;value from above&gt;&gt;;<br \/>\nSTART SLAVE;<\/div><\/div>\n<p>To check the progress of your slave:<\/p>\n<div class=\"codecolorer-container text default\" style=\"overflow:auto;white-space:nowrap;\"><div class=\"text codecolorer\">SHOW SLAVE STATUS \\G<\/div><\/div>\n<p>If all is well, Last_Error will be blank, and Slave_IO_State will report \u201cWaiting for master to send event\u201d. Look for Seconds_Behind_Master which indicates how far behind it is. It took me a few hours to accomplish all of the above, but the slave caught up in a matter of minutes. YMMV.<br \/>\nAnd now you have a newly minted mysql slave server without experiencing any downtime!<\/p>\n<p>A parting tip: Sometimes errors occur in replication. For example, if you accidentally change a row<\/p>\n<div class=\"codecolorer-container text default\" style=\"overflow:auto;white-space:nowrap;\"><div class=\"text codecolorer\">STOP SLAVE;SET GLOBAL SQL_SLAVE_SKIP_COUNTER = 1;START SLAVE;<\/div><\/div>\n<p>\u201cMySQL server has gone away\u201d error while running the initial dump. The solution he found was to add the following during the import on slave:<\/p>\n<div class=\"codecolorer-container text default\" style=\"overflow:auto;white-space:nowrap;\"><div class=\"text codecolorer\">[mysqld]<br \/>\nmax_allowed_packet=16M<\/div><\/div>\n<p><a href=\"https:\/\/blog.designed79.co.uk\/wp-content\/uploads\/2018\/12\/Setting-up-MySQL-replication-without-the-downtime.pdf\">Setting up MySQL replication without the downtime<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>I clearly don\u2019t need to expound on the benefits of master-slave replication for your MySQL database. It\u2019s simply a good idea; one [&hellip;]<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-2936","post","type-post","status-publish","format-standard","hentry","category-info-on-tech"],"_links":{"self":[{"href":"https:\/\/blog.designed79.co.uk\/index.php?rest_route=\/wp\/v2\/posts\/2936","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blog.designed79.co.uk\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.designed79.co.uk\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.designed79.co.uk\/index.php?rest_route=\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.designed79.co.uk\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=2936"}],"version-history":[{"count":0,"href":"https:\/\/blog.designed79.co.uk\/index.php?rest_route=\/wp\/v2\/posts\/2936\/revisions"}],"wp:attachment":[{"href":"https:\/\/blog.designed79.co.uk\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=2936"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.designed79.co.uk\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=2936"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.designed79.co.uk\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=2936"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}