Amazon Web Services - Linux - RAID

2011-12-17T18:09:00.001-08:00

First off, I’m no Linux expert. I’ve always liked it but it’s a struggle every time. When something goes wrong it takes me forever to figure it out.

I’m running three micro instances on AWS using Amazon’s Linux Distro. They provide the MongoDB backend for Clean Chow. Amazon’s recent affinity for rebooting instances combined with my lack of Linux knowledge (and stupidity) almost caused my site to go down.

I use RAID to improve database performance. What I didn’t know is that the RAID configuration would not survive a reboot. So when Amazon rebooted two of my instances MongoDB didn't come back up. Since I still had one instance running the database just went into read-only mode. The site stayed up because with the exception of some caching it doesn’t need write access. I was lucky. I know I should have rebooted the servers after adding RAID to the equation. At the time I was just happy to get it going.

So in the end these are the steps I had to go through to get RAID working on AWS. Note: I’m using RAID 0 which does NOT provide for any redundancy, it’s purely for performance. I believe these steps should work for other RAID levels, but I haven’t tested it.

1) Start a new instance (I used the Micro instance with the ‘Basic 64-bit Amazon Linux AMI 2011.09')

2) SSH into the instance and execute “sudo yum update”. I prefer to update it though I don’t know that it's necessary.

3) Create two new EBS volumes. Mine were 10GB each. Be sure to select the same availability zone the instance is in.

4) Name the newly created EBS volumes. I named mine ‘Amazon RAID A’ and ‘Amazon RAID B’. My real volumes have names like 'Mongo 2 DATA B'.

5) Attach the ‘A’ volume to the instance using the device id “/dev/sda2”.

6) Attach the ‘B’ volume to the instance using the device id “/dev/sda3”.

7) Watch the EBS Volumes page on the AWS Management Console to ensure both volumes are successfully attached.

8) Execute ‘ls /dev/sda*’ and make sure there are three volumes listed: sda1, sda2, sda3

9) Execute ‘sudo mdadm --create /dev/md0 --chunk=64 --level=raid0 --raid-devices=2 /dev/sda2 /dev/sda3’

10) Execute ‘sudo yum install xfsprogs’ to get the XFS formatting utilities (I’m using XFS because that sounded like a best practice for MongoDB).

11) Execute ‘sudo mkfs.xfs /dev/md0’

12) Execute ‘sudo mkdir /mongoData’. I’m using mongoData as a mount point, but you could use anything.

13) Execute ‘sudo mount /dev/md0 /mongoData -o noatime,nouser’ to mount the new RAID volume. I’m specifying the noatime and nouser options as a MongoDB optimization.

14) Now this is the part I initially messed up. If you reboot now the RAID array will not be reattached. So we need to setup the RAID array so that it will come back and define the mount point in the /etc/fstab file.

15) Switch to a root CSH shell by executing ‘sudo csh’. After you do this you don’t need to run sudo because everything executes as root. I did this because in the next step we have to pipe outut to a file that only root can write to and I don’t know enough about sudo to make this work, so I cheated ;-)

15) Execute ‘mdadm --detail --scan > /etc/mdadm.conf’. This will make the RAID array survive a reboot

16) Execute ‘cat /proc/mounts’ to list the current runtime configuration

17) Copy the line that includes a reference to your mount point. In my case its this line “/dev/md0 /mongoData xfs rw,noatime,attr2,nobarrier,logbsize=64k,sunit=128,swidth=256,noquota 0 0”.

18) Modify the /etc/fstab to include that line. I fixed up some of the spacing

20) Create a file in the /mongoData directory (this should be saved to the RAID volume).

21) Reboot instance and execute ‘df’. This should show the RAID volume and the file created in the /mongoData directory should still exist.

References:

http://aws.amazon.com/articles/1074

http://www.devil-linux.org/documentation/1.0.x/ch01s05.html

http://www.ducea.com/2009/03/08/mdadm-cheat-sheet/

http://joslynesser.com/blog/archives/2010/09/28/mongodb-sharding-guide-server-setup-on-ubuntu-ec2/

ReSharper

2009-09-09T21:23:00.000-07:00

Resharper is amazing! It has changed the way I work, increased my productivity and most importantly improved the quality of my code. I bought it myself and it was worth every penny.

When it comes to software I'm a minimalist. The more software loaded on a machine the more opportunity for trouble. Trouble can come in the form of software conflicts, performance degradation and security vulnerabilities just to name a few. Plus I can't stand the clutter. Resharper earns its keep every day.

A few of my favorite features:

Refactoring
Refactoring tools make it easier to refactor, which means you're more likely to do it. This means cleaner code which is always good.

Test Runner
The built in test runner is really handy. It makes running tests quick and convenient. They make it really easy to run tests in the debugger which is great.

Code Navigation
There are tons of keyboard shortcuts to help move around code more efficiently.

Code Formatting
ReSharper has a great engine for formatting code. This makes it really easy to keep the formatting consistent and professional looking. I don't want to waste time making sure my property declarations are bundled together and in order or worry about how many lines are after curly braces. I just bang out the code without concern for any formatting then hit CTRL-e, Ctrl-f and let ReSharper apply my formatting rules.

There's a free 30 day trial I highly recommend. I'd encourage anybody to give it a try, its worth the time.

http://www.jetbrains.com/resharper/

Steve Wiggins

Amazon Web Services - Linux - RAID

ReSharper