Debug : AWS High CPU Load on MySQL – AWS, Amazon RDS

Using the same netstat command and filtering on the PID we can find which connections does this process has.

Here you can see it has a couple of memcached connections.

Few MySQL connections (to the same host, which is usually a bad idea) and connections to some external web server.

You can use strace -p to see what the host is doing, it often gives a clue. 

In this case, I, for example, found the process is stuck in pool() system call reading from a network. Using netstat can give you an idea of what it can be but if you do not like guessing you can use gdb -p. 

It will not print your exact line of code in PHP which is running but can give you some good ideas – for example in this case I could find stack trace originated from PHP stream functions not from libmysql or memcache.so, which means it is not MySQL or memcache connections leaving the last candidate as the only choice. 

I also could see some of the variables in GDB “bt” command output which also hinted what could be the problem.

Yet another great tool that you can use is server-status if you’re running apache. 

This way you will see the URL which that process is processing and so get a few more hints on what may be happening or even get a repeatable example in some cases.

The tools I have mentioned regarding figuring out what is happening with the process are not only helpful to debug sleeping connections with MySQL but many other cases when you see web applications locking up or starting to run in the tight loop consuming too much CPU time.