top button
Flag Notify
    Connect to us
      Site Registration

Site Registration

MongoDB High iowait, no disk usage

+2 votes
1,002 views

We are seeing some issues with iowait. On a machine with 16 cores and 60 GB memory and SSD (max IO around 250 mb/s) (virtual machine on google) we see around constantly 20% iowait on top and iotop shows its the mongoDB process. Only around 10MB/s are actually read from the disk and as I said, we know from other tests that the machine can do much much more.

The query producing this pattern sorts a result from a query by an attribute (all indexed). After running queries like this a few times it works better. The total index size is around 4GB, so I assume it gets loaded in memory directly.

Is there any suggestions on how to debug this? I cant see how there can be such a high iowait when all computing resources are not even barely utilized.

Note: Running MongoDB 3.0.10 on Ubuntu 14.04

posted May 26, 2016 by anonymous

Share this question
Facebook Share Button Twitter Share Button LinkedIn Share Button

1 Answer

0 votes

High iowait on the MongoDB process implies that MongoDB is waiting for the disk to read/write data. Are the resources consistently not being utilized, or does this only happen for a period of time after a cold start?

There are a number of potential reasons for your initial performance issues, but collecting more detailed metrics may be able to provide a hint:

Does this happen after a cold start of the machine or mongod? E.g., does restarting mongod exhibit this issue as well?
Do you have multiple mongod running in the machine? E.g. on different ports, using Docker, or any virtualization?
Do you see anything suspicious in the server logs or the mongod logs? You could try using mtools which is a collection of tools to analyze MongoDB deployment by analyzing the log files, for example: mloginfo --queries, mplotqueries, etc.
Can you include the output of explain(true) for the query?
What is the storage engine used: MMAPv1 or WiredTiger?
What size and type of SSD is provisioned on the server?
How many times do you need to run the query before it gets to an acceptable level of performance, and does the performance increase significantly? Is there any difference in the log files between the two cases?
What is the size of the collection?
You may find the following links useful:
Optimizing Persistent Disk and Local SSD Performance
Optimization Strategies for MongoDB

Running MongoDB 3.0.10 on Ubuntu 14.04
I would recommend to upgrade to the latest 3.0 version, which is currently at 3.0.12 for bugfixes and improvements.

answer Nov 17, 2016 by Manikandan J
Similar Questions
0 votes

I have problem with mongods high cpu usage. We get 3000 data per second, and I insert it per 100,000.(insert with c++ driver, using vector)

I expect even progress of cpu, but I intermittently found high cpu of mongod. Although there is no result of currentOp() command. Why cpu status shows like below?

Insert amount of 459~742 second is not bigger than before, but mongods cpu usage is much bigger.

What can be cause of this status?

0 votes

Can Google cloud disk snapshots give consistent MongoDB backups? I am using WiredTiger engine with journalling enabled, both data and journal files are on the same disk.

+1 vote

I am planning to add more storage to my exiting mongodb by adding external disk.
Can anyone suggest how to connect external disk to mongodb?

+2 votes

What is CouchDB ? and which one is better mongodb or couchdb please explain with example.

0 votes

In our solution we have asynchronous processes with state persistence (aka sagas) where each saga has single state document persisted in mongodb. Depending on the saga type, this document may lead to high level of concurrent updates. So, we simply introduced optimistic concurrency there to retry handling of incoming messages. But as a result of that we face very high rate of retries what results in significant performance degradation. In the past we used pessimistic locking of saga document/record when used to store state in rdbms (mssql, oracle etc). I know that at the moment there is no such feature in mongodb, but maybe it would be useful/possible to have it for single document updates? I believe this feature could simplify many solutions.

Are there any other concurrency handling options? Suggestions?

...