Linux Admin Zone » mysql

Fix mysql memory table error: The table xtable is full

jagbir — Mon, 20 Feb 2012 08:34:55 +0000

Replication just stopped in one Slave server with error: The table xtable is full which means no more records are permitted to insert in this table by MySQL and hence this has broken the replication.

I checked that xtable is having storage engine as Memory. In such tables, the max. no. of records you can insert is controlled by variable max_heap_table_size. When checking the size of this variable, I found that this is having default value:

mysql> show variables like 'max_heap_table_size';
+---------------------+----------+
| Variable_name       | Value    |
+---------------------+----------+
| max_heap_table_size | 16777216 |
+---------------------+----------+
1 row in set (0.00 sec)

So we need to increase the value of this variable and then issue Alter Table command to make it effective. Also do not forget to add variable with new value in your my.cnf.

Change the variable value:

mysql> set  max_heap_table_size=268435456;
Query OK, 0 rows affected (0.00 sec)
 
mysql> show variables like 'max_heap_table_size';
+---------------------+-----------+
| Variable_name       | Value     |
+---------------------+-----------+
| max_heap_table_size | 268435456 |
+---------------------+-----------+
1 row in set (0.00 sec)

Now let’s truncate the table and issue Alter table on it to make the value effective for this table:

mysql> truncate table xtable;
Query OK, 0 rows affected (0.00 sec)
 
mysql> alter table xtable ENGINE=MEMORY;
Query OK, 88 rows affected (0.06 sec)
Records: 88  Duplicates: 0  Warnings: 0

After this fix, the issue has been resolved. Make sure that you do not run the truncate command on table which have important data. I have issued truncate because it contains temporary/transactional data so we are fine with removing all records here.

Beware about admin commands in MySQL replication

jagbir — Fri, 19 Nov 2010 11:08:09 +0000

Here is the situation: We are doing replication of only 1 table from a database (live) to other host (backup). so we expect that updates which are on others DBs/tables except on this table should not replicate. Wrong! they got replicated.

We shifted one DB say DB1 from live env to Backup DB host and then removed that DB1 from Live. Even replication was only for a single table of a particular DB say DB5, DB1 got removed from Backup host as well, living us in vaccum

Luckily we had backup so restored immediately. Its like a confirmation now that all administrative commands like creating/dropping databases/tables etc get replicated to your client even if you are doing ‘selective’ replication.

Compile mysql 5.1 with innodb and optimize for heavy usage

jagbir — Wed, 21 Apr 2010 13:11:41 +0000

MySQL stopped default inclusion of InnoDB in latest 5.1.x, so if you need InnoDB, you have to compile it from source. I have done following steps in CentOS 5.4 server to compile MySQL and optimize it later for a heavy site:

1. Remove earlier installation of MySQL, if any and download source rpm from MySQL site. I’ve also removed earlier installation of PHP here to upgrade it:

$ rpm -e php-mysql mysql-server php php-devel php-pear

Download the latest source rpm from MySQL download site, you will get source rpm like below and install it. Installation of source rpm will put it as tar file in /usr/src/redhat/SOURCES, make sure to create /usr/src/redhat/SOURCES directory before installation:

$ ls
MySQL-community-5.1.45-1.rhel5.src.rpm

Add mysql user/group if its already not there:

$ groupadd mysql
$ useradd -g mysql mysql
 
$ rpm -i MySQL-community-5.1.45-1.rhel5.src.rpm
warning: user mysqldev does not exist - using root
warning: group mysqldev does not exist - using root
warning: user mysqldev does not exist - using root
warning: group mysqldev does not exist - using root
 
ignore above warnings. 
 
$ cd /usr/src/redhat/SOURCES
$ ls
mysql-5.1.45.tar.gz

2. Install required dependencies to compile MySQL:

$ yum install gcc gcc-c++ ncurses-devel

Uncompress and compile it:

$ tar xzf mysql-5.1.45.tar.gz
$ cd mysql-5.1.45
$ ./configure --prefix=/var/lib/mysql  --localstatedir=/var/lib/mysql/data --sysconfdir=/etc/mysql --with-mysqld-user=mysql --with-plugins=innobase,myisam --enable-thread-safe-client
$ make
$ make install

I need only InnoDB and MyISAM so enabled these storage engines with threads in configure command above. If you need, you can convert your existing MyISAM DB into InnoDB quickly. for more configure options you can check documentation here.

3. After compilation, you can check that everything related to MySQL is there in /var/lib/directory, change owner of this directory from root to mysql:

$ cd /var/lib/ 
$ ls mysql/
bin  docs  include  lib  libexec  mysql-test  share  sql-bench
$ chown -R mysql:mysql mysql/

Create data directory and initialize default mysql database:

$ cd mysql
$ bin/mysql_install_db --user=mysql

Copy startup script to /etc/init.d

$ cp /usr/src/redhat/SOURCES/mysql-5.1.45/support-files/mysql.server /etc/init.d/mysql
$ chmod +x /etc/init.d/mysql
$ chkconfig --level 345 mysql on

4. Create my.cnf file, start MySQL and update root password:

Create initial my.cnf file by coping sample file, tuned for heavy usage using InnoDB:

$ cp /usr/src/redhat/SOURCES/mysql-5.1.45/support-files/my-innodb-heavy-4G.cnf /etc/my.cnf

Open my.cnf and set values of some variables according to specs of your server:

$ vi /etc/my.cnf
innodb_data_file_path=ibdata1:2000M;ibdata2:10M:autoextend
innodb_log_file_size=100M
innodb_buffer_pool_size=5G
innodb_additional_mem_pool_size=20M
innodb_thread_concurrency=8

Here I’ve set buffer_pool_size as 5G, keep it increasing upto 80% of RAM in the server. Set thread concurrency according to available cpus*2. Here you can see more optimization tips. Start MySQL service now:

$ /etc/init.d/mysql start
Starting MySQL..                                           [  OK  ]

Jump into mysql prompt and update password:

$ mysql 
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 2
Server version: 5.1.45-log Source distribution
 
Type 'help;' or '\h' for help. Type '\c' to clear the buffer.
 
mysql> use mysql; 
mysql> UPDATE user SET Password=PASSWORD('yourpass') WHERE User='root' ;
Query OK, 3 rows affected (0.00 sec)
Rows matched: 3  Changed: 3  Warnings: 0
 
mysql> flush privileges; 
Query OK, 0 rows affected (0.00 sec)
 
mysql> quit;

5. We are done with MySQL. You can also run Multiple MySQL servers in a single machine, I’ve already blogged steps to implement that. You are good at this point about MySQL, continuing reading if you need to install/config latest PHP also. I’m going to install PHP 5.2.x using CentOS’s Testing repository:

Create testing repository:

$ cat > /etc/yum.repos.d/CentOS-Testing.repo
[c5-testing]
name=CentOS-5 Testing 
baseurl=http://dev.centos.org/centos/$releasever/testing/$basearch/
enabled=1
gpgcheck=1
gpgkey=http://dev.centos.org/centos/RPM-GPG-KEY-CentOS-testing

Press Ctrl+d (once) to save file.

Install PHP stuff:

$ yum install --enablerepo=c5-testing php-mysql php php-devel php-pear php-mcrypt php-mbstring

6. Install PhpMyAdmin to provide a GUI for MySQL:

$ cd /opt/
$ wget http://downloads.sourceforge.net/project/phpmyadmin/phpMyAdmin/3.3.2/phpMyAdmin-3.3.2-english.zip?use_mirror=space
$ unzip phpMyAdmin-3.3.2-english.zip
$ mv phpMyAdmin-3.3.2 phpmyadmin
$ cd phpmyadmin
$ cp config.sample.inc.php config.inc.php

Update phpmyadmin config.inc.php according to your environment. Add an alias to access phpmyadin in your httpd.conf file:

$ vi /etc/httpd/conf/httpd.conf
### add following lines in this file
Alias /phpmyadmin/ "/opt/phpmyadmin/"
 
<Directory "/opt/phpmyadmin">
    Options Indexes MultiViews
    AllowOverride None
    Order allow,deny
    Allow from all
Directory>

Restart httpd service to apply new upgrade of PHP and to access PhpMyAdmin and check. I also noticed one issue while trying to connect MySQL from remote, like from PhpMyAdmin. The programs searches for MySQL sock but unable to find. to resolve this create a symlink in to /var/lib/mysql:

$ ln -s /tmp/mysql.sock /var/lib/mysql

Check again and everything should run as expected.

How to quickly convert mysql databases from MyISAM to InnoDB

jagbir — Sat, 10 Apr 2010 10:06:30 +0000

Recently I was in need to convert all databases of a MySQL server from MyISAM engine to InnoDB. They are facing severe issues due to table level locking in MyISAM and wanted move to InnoDB. This task can be a mammoth work but if you work little smarter, all you need to execute few statements and look at the progress while having cup of coffee. Also note that you can convert in both side, means from MyISAM to InnoDB and from InnoDB to MyISAM using these commands.

There can be many approaches to do same work, like dumping all databases and replacing myisam with innodb in all create table commands in dump file and then restoring all databases. But this can take very long time and you also need to delete data from mysql before restore etc. so I preferred following approach:

1. Extract name of all databases into a text file and remove system databases from it:

$ mysql --skip-column-names -e "show databases;" > alldb.txt
$ vi alldb.txt ## Remove system databases like 'information_schema, mysql' or any other db you want to exclude from this file.

I’ve used –skip-column-names option here to tell MySQL do not add any column/head name in output.

2. Download MaatKit mk-find tool. This is like system find command but for database:

$ cd /opt
$ wget http://www.maatkit.org/get/mk-find

3. Instead of specifying password in mysql command line, I prefer storing it in ~/.my.cnf while doing such works to avoid typing passwords every time. you can create this file or supply password as you wish.

$ cat ~/.my.cnf
[client]
user=root
password="password"

4. Before executing the command to convert storage engine make sure that ‘skip-innodb’ is not in your my.cnf file. This is because in envrionment of all MyISAM, its normally placed to disable storage engines which are not in use. You need to delete/comment this line and restart mysql service first. Execute the following command to start conversion:

$ for dbname in `cat alldb.txt`; do ./mk-find $dbname --engine MyISAM --exec "ALTER TABLE %D.%N ENGINE=INNODB" --print; done

This will extract all database names one by one and supply that name to mk-find tool which execute alter table command on all of its tables to convert their storage engine from MyISAM to InnoDB. This can take much time depending no. of tables and their size.

Find out the clients of your MySQL server

jagbir — Tue, 16 Mar 2010 08:49:58 +0000

Sometimes in large deployments, there are cases when MySQL server, setup by you long time back which has been in use by multiple teams in your organization, needs some change or update or intrupption in its service and you are in need to know how many clients are there which connects to this server.

One way to know is to check user table in ‘mysql’ database, you might have created individual users/databases for your clients/users. But over the time, unless taking care seriously, we tend to forget the exactly how many users/scripts/machines are there making connections. May be you have used same read-only user in various scripts scattered over many machines which you’ve created just for a single client.

The other way, I’ve used here is to monitor your connections over a period of time and noting down IP addresses of machines making connections. After some time, you will have a good idea about your clients machines/users. In MySQL server, the ‘processlist’ command shows information about clients connected to server.

I have 1 MySQL master server and few of its slave servers for which I want to know who are using these servers. Store IP addresses of these servers in “serversip.txt” file. In an unfortunate case, if you have forgotten your mysql root password, check here to quickly reset it. Execute “show processlist” command into each of servers and extract and save IP addresses from output in a file. We have to execute this frequently, so you can run through cron (every minute) or just add infinite loop and sleep to do the same, like below:

$ vi searchclient.sh

#!/bin/bash
 
while [ 1 ]; do
 
for serverip in `cat serversip.txt`; do 
        mysql -u root -p password -h $serverip -e "show processlist;" | grep -v "Id\|system user\|Commands" | awk '{ print $3 }' | cut -d':' -f 1 >> clientsip.txt  
done
 
sleep 60
done

execute this script and keep it running for some time, preferrably for several hours to fetch all IPs who tried connecting your servers except system (replication etc) connections. Then kill this script as it will not stop automatically. Now we have got client’s IP addresses in “clientsip.txt” but there will be lots of duplicate IPs. Let’s clear out duplicates:

# cat clientsip.txt | sort | uniq > ipclients.txt

Your clients IP list is in “ipclients.txt” file. further, if you want to know hostnames, you can issue another command:

# for ips in `cat ipclients.txt`; do host $ips >> clienthosts.txt; done

Your client host names are in “clienthosts.txt”. Let me know your suggestions/tips for any better way to do this.