Installing MongoDB on AWS

ByRafael Novello in

MongoDB install is made on EC2, the Amazon virtual server. In this environment it is possible to provision the speed of disks (called EBS) by configuring IOPS; for MongoDB, the faster the disk, the better! However, an important detail is that you can not configure the disk with many IOPS; instead, you need to pre-warm it!

The pre-warming operation is necessary only during the boot, wether is a new disk or a disk created from an image. Without this preparation, the disc may be 5% to 50% slower (yes, up to 50% slower!) and this was one of the factors that made me suffer with query performance for sure. You can read more about pre-warming checking Amazon documentation, but it consists of umount the disk and run the following command:

sudo dd if=/dev/zero of=/dev/<strong>xvdf</strong> bs=1M

Replace the unit in <strong> by the unit you are using and wait — the command can take few hours to complete, depending on the disk size. A good idea is to use Linux screen to not lose work if the session falls.

Another very important point for MongoDB performance which is independent of the used host is the system ulimit setting. Most Linux systems come configured by default to prevent a user or process consumes too much server resources, but sometimes these limits are too low and interfere with MongoDB performance. You can read more about at 10gen documentation page, but the recommendation is as follows:

file size: unlimited
cpu time: unlimited
virtual memory: unlimited
open files: 64000
memory size: unlimited
processes/threads: 64000

With these two settings I was able to dramatically improve the performance of my MongoDB installation, so I think these tips can help everyone.

Hardware allocation

Another point that I realized was very important for MongoDB performance is choosing the right server. When I started to deal with this database, I believed that the disk would be the most important resource and, although not totally wrong, I actually see that the RAM memory is the most important thing here. During database operation, if there is no RAM memory, you will need to take many changes in memory content, increasing disk usage and impairing performance. The recommendation is for at least the indexes fit in memory; you can see that with the stats command on each collection in MongoDB console:

> db.sua_colecao.stats()

The command will show the size of each index in bytes, then just add up and see if it fits in RAM memory. For those who use AWS, it was released a new type of EC2 instance with memory optimization, the R3 instances. They are a great option for MongoDB, as shown in this mongodirector article.

MongoDB version

There are several reasons why is always recommended to use the newest systems versions, but in the case of MongoDB there are important issues for our discussion. For instance, the version 2.4 had global locks in the database, which means that any write operation would block the database completely and no other writing could be made. In version 2.6 the locks came out to be in the database level, so it became possible to make writing and reading operations simultaneously in different databases. In version 2.8, according to this MongoDB blog post, locks will be at the document level, which will cause a huge performance impact on the system.

Leave a comment! 0

Your e-mail will not be visible. Mandatory fields are shown with *

Commenting as Anonymous

read more
This project is sponsored by these companies:
Hosted by: