Linux — How to Troubleshoot High Disk I/O

In this article, let’s do some war gaming and have fun with a high disk I/O troubleshooting use case.

Case Preparation

Let’s run the following docker container on a Linux host to simulate some high disk I/O scenario:

$ docker run --name=app -p 80:80 -itd tonylixu/word-pop
9bc4614419bc606064b3d35c5ba86be3fb55617dcd0e61791b4326f6df7c8c37

$ docker ps
CONTAINER ID   IMAGE             COMMAND            CREATED         STATUS         PORTS                               NAMES
9bc4614419bc   feisky/word-pop   "python /app.py"   6 minutes ago   Up 6 minutes   0.0.0.0:80->80/tcp, :::80->80/tcp   app

Once the container is up and running, let’s visit the url “http://127.0.0.1” using curl command:

$ curl http://127.0.0.1
hello world

We can see that the case starts normally. Now let’s query a specific endpoint:

$ time curl http://127.0.0.1/popularity/word
{
  "popularity": 0.0,
  "word": "word"
}real 0m44.134s
user 0m0.000s
sys 0m0.008s

From the above result, you can see that it took 44 seconds to get the result back! What’s going on? Why the query takes so long?

Troubleshooting Steps