Archive for the 'database' Category

Simple and efficient MongoDB Backup using script

MongoDB Backup types and strategies are neatly explained in its documentation, which you can check here. In case you are not familiar with MongoDB backup types and strategies, please have a look at its documentation.

What I am describing here is a simple script which we are using since months to take MongoDB backup and transfer it over to our Backup server. Here are few things its doing:

  • As we have multiple MongoDB Replica Sets, the script identify current replica set and check whether current server is Master or Slave, exit if its Master. We take backup only from Slave host.

Fix mysql memory table error: The table xtable is full

Replication just stopped in one Slave server with error: The table xtable is full which means no more records are permitted to insert in this table by MySQL and hence this has broken the replication.

I checked that xtable is having storage engine as Memory. In such tables, the max. no. of records you can insert is controlled by variable max_heap_table_size. When checking the size of this variable, I found that this is having default value:

mysql> show variables like 'max_heap_table_size';
+---------------------+----------+
| Variable_name       | Value    |
+---------------------+----------+
| max_heap_table_size | 16777216 |
+---------------------+----------+
1 row in set (0.00 sec)

So we need to increase the value of this variable and then issue Alter Table command to make it effective. Also do not forget to add variable with new value in your my.cnf.

Speed up large MySQL backup, dump, restore processes

In general, MySQL is quite fast at restoring data, but I observed that while restoring 20gb of backup its taking more than the usual time. This can happen when you don’t have enough memory or if key_buffer_size is not set high enough, then it can take very long time to re-index the data. In CentOS 5.2 server with 6gb of RAM, I noticed key_buffer_size is set to just 800M which is very low. You should set it at least 20-25% of total RAM. After increasing the value to 2gb, MySQL is able to reload data quite fast.

quick web based php script to check replication status of MySQL

This is a basic PHP script to check replication status of single or multiple MySQL servers. You have to update variables in the script to run in browser like user, password, ip of server etc.

Here is the script code:

<?
##########
#
# dbrepstatus.php by Jagbir Singh (jags@jagbir.com)
#
#
#########

?>
 
<html><head>
<title> Online DB Server status </title>
<body>
<form name="ShowStatus" action="<? echo $_SERVER['PHP_SELF']; ?>" method="POST">
<br/>
<table border=1 align=center>
<tr align=center bgcolor=gray><td colspan=2><b> Online DB Server Status </b></td> </tr>
 
<!--- TODO: replace your server name in the select list below !>
 
<tr ><td align=centre> Select Server: <select name="server">
<option value="s1" <? if ($_POST[server] == "s1") echo "selected"; ?> > Server 1 
<option value="s2" <? if ($_POST[server] == "s2") echo "selected"; ?> > Server 2 
<option value="s3" <? if ($_POST[server] == "s3") echo "selected"; ?> > Server 3
<option value="s4" <? if ($_POST[server] == "s4") echo "selected"; ?> > Server 4 
</select> </td> 
 <td align=centre> <input type=submit name='show' value='Show Status'></td></tr>
</form>
</table>
 
<? 
$serverUp=1; ## if any of your server not configured, set this variable to 0 below
$serverString="First Server"; ## server name
$serverIP="w.x.y.z"; ## default IP, anyhow will be changed later
$dbuser="demouser"; 
$dbpassword="demopass"; 
 
$imgShow="<img src='led-green.gif' />";
 
if ($_SERVER['REQUEST_METHOD'] == 'POST'){
	switch($_POST[server]){
		case "s1":
		 $serverString="Server 1"; 
		 $serverIP="IP1";
		 break;
		case "s2":
		 $serverString="Server 2"; 
		 $serverIP="IP2";
		 break;
		case "s3":
		 $serverString="Server 3"; 
		 $serverIP="IP3";
		 break;
		case "s4":
		 $serverString="Server 4"; 
		 $serverIP="IP4";
		 $serverUp=0;
		 break; 
	}
 
?>
	<br/>
	<table border=1 align=center width=50%>
	<tr align=center bgcolor=gray><td colspan=4><b> Status of <? echo $serverString." : ". $serverIP; ?> </b></td> </tr>
	<tr > <th bgcolor=lightgray> Status </th> <th bgcolor=lightgray > Replication </th> <th bgcolor=lightgray> Delay </th> <th bgcolor=lightgray> Error </th> </tr>
	<?
	if (!$serverUp) {
	?>
			<tr> <td align=center colspan=4> This server is not configured yet! </td> </tr>
	<?
	}
	else {
		$contactAdmin="<a href='mailto:dbadmin@yourmail.com?subject=Replication issue in $serverString'> Contact Admin </a>";	
		$RepStatus = `mysql -h $serverIP -u $dbuser -p$dbpassword -e "show slave status\G;" | egrep "Slave_IO_Running: | Slave_SQL_Running: | Seconds_Behind_Master:" | cut -d':' -f2`; $RepStatus= trim($RepStatus);  
		list($RepRun, $SqlRun, $RepDelay) = split(" ",$RepStatus);
		 $RepRun = trim($RepRun);
		 $SqlRun = trim($SqlRun);
		 $RepDelay = trim($RepDelay);
 
//		echo "<br/> Reprun=$RepRun, IO Run =$SqlRun, Replication delay =$RepDelay";
 
		if($RepRun == "Yes") { $RepRun = "OK"; $contactAdmin=""; } else { $RepRun = "Replication (IO) Not Running!"; $imgShow="<img src='led-red.gif' />"; }
		if($SqlRun == "Yes") { $SqlRun = "None"; $contactAdmin=""; } else {$SqlRun = "SQL thread stopped! "; $imgShow="<img src='led-red.gif' />";}
		if($RepDelay == 0 ) { $RepDelay = "None"; } else { $RepDelay .= " seconds."; $contactAdmin=""; }   
 
 
	?>
		<tr> <td align=center> <? echo $imgShow; ?> </td> <td align=center> <? echo $RepRun; ?> </td> <td align=center> <? echo $RepDelay; ?>  </td> <td align=center> <? echo $SqlRun; ?>  </td> </tr>
 
	<?
	if ($contactAdmin != "") { echo "<tr> <td align=center colspan=4> $contactAdmin </td></tr>"; }
 
	} 
 
	?>
	</table> 
<?
}
?>
 
</html>
</body>

In my case, there are 4 servers. First is master and all other slaves so what I assumed here is that all servers have same user/pass to access information. Don’t forget to change user/pass/ip etc. before trying to run this script in browser. Let me know through comments if you any suggestion/enhancement/issue regarding this.

Install and configure PhpMyAdmin to manage multiple MySQL Servers

If you have read and implemented my previous article: how to run multiple MySQL services on a single box, you may like to manage such multiple installation of MySQL or multiple separate MySQL hosts by using a single PhpMyAdmin installation. This is easy enough to do quickly.

I assume you have Apache (httpd) and PHP installed where you want to setup PhpMyAdmin. Let’s proceed to install PhpMyAdmin:

$ cd /var/www/html
$ wget http://downloads.sourceforge.net/project/phpmyadmin/phpMyAdmin/2.11.11.3/phpMyAdmin-2.11.11.3-english.tar.gz?use_mirror=nchc
$ tar xzf phpMyAdmin-2.11.11.3-english.tar.gz 
$ mv phpMyAdmin-2.11.11.3-english phpmyadmin
$ cd phpmyadmin

You can of course, download and install PhpMyAdmin automatically using Yum, but here I’ve installed it manually. Now make a config file for it and update that:

$ cp config.sample.inc.php config.inc.php
$ vim !$