Benchmarking Rackspace dedicated vs cloud

People keep telling me that Magento performs better on dedicated hardware. I haven’t been able to find any numbers to support this, but I’ve heard it so often it’s either a very popular myth, or it’s true.

Now that our Rackspace dedicated box is online, I’m trying some benchmarks toput some numbers against the comparison. I wanted to test different operating systems and server sizes, so I booted 10 servers, one of each Ubuntu 10.04 LTS and RHEL 5.5 at 0.5, 1, 2, 4, 8 GB.

In order to automate the setup, I ran the following command on the Ubuntu boxes:

mkdir .ssh && chmod 700 .ssh && echo ssh-rsa <<snip>> > .ssh/authorized_keys && chmod 600 .ssh/authorized_keys && locale-gen en_GB.UTF-8 && update-locale LANG=en_GB.UTF-8 && apt-get update && apt-get --yes dist-upgrade && apt-get --yes install sysbench screen && reboot now && exit

Then on the RHEL boxes, something vaguely similar:

mkdir .ssh && chmod 700 .ssh && echo ssh-rsa <<snip>> > .ssh/authorized_keys && chmod 600 .ssh/authorized_keys && yum -y update && rpm -Uvh http://download.fedora.redhat.com/pub/epel/5/i386/epel-release-5-4.noarch.rpm && yum -y install screen sysbench && reboot && exit

To automate the actual testing, I created a script bench.sh and uploaded it to each of the servers. It’s a simple nested for loop to run each test 3 times.

#!/bin/bash

for threads in 1 4 8 16 32 64
do
	for r in 1 2 3
	do
		sysbench --num-threads=$threads --test=cpu run > sysbench_cpu_${threads}_threads_$(date +%Y-%m-%d_%H-%M-%S).log
		sleep 30
	done

	for r in 1 2 3
	do
		sysbench --num-threads=$threads --test=memory run > sysbench_memory_${threads}_threads_$(date +%Y-%m-%d_%H-%M-%S).log
		sleep 30
	done

	sysbench --num-threads=$threads --test=fileio --file-test-mode=rndrw prepare

	for r in 1 2 3
	do
		sysbench --num-threads=$threads --test=fileio --file-test-mode=rndrw run > sysbench_fileiorndrw_${threads}_threads_$(date +%Y-%m-%d_%H-%M-%S).log
		sleep 30
	done
done

Then I connected to each server, uploaded the script (actually copied / pasted into vim, seemed quicker) chmod +x and ran it. The scripts are running now on 10 machines…

Results

I started writing this post about 2 months ago and haven’t yet published it. The bottom line was that memory comparisons were roughly even between virtual and physical environments. However, disk IO was hugely variable on the virtual hardware, at the top end it was comparable to the dedicated hardware, at the bottom end about 10% of that.

I didn’t notice any difference between operating systems, but I didn’t look for it very hard either. The results for the cloud servers were all over the place, while the dedicated box was very consistent.

My takeaway result was that disk is unpredictable in the cloud. If you’d like to see the actual results to make a more detailed analysis, let me know in the comments and I’ll dig out the numbers. For now I’m going to finally publish this! šŸ™‚

Advertisements

2 thoughts on “Benchmarking Rackspace dedicated vs cloud”

  1. Hi,
    I am working on cloud storage..just now i started learning and i have read that sysbench and sysbench-graph are good tools to benchmark the mysql database.I tried on my system with this tool.
    Now, I want to know how i can use this tool for benchmarking the system which is either DAS,NAS,SAN.

    Thanks

    1. I don’t think I understand your question, and I’m not sufficiently experienced with DAS, NAS or SAN to provide any useful advice. Best of luck, please feel free to share any successes you have.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s