Amazon EC2 HighMemory Instances

Posted by Jonathan

Very very nice: 34.2 GB RAM and 68.4 GB RAM instances on EC2

# free -m
             total       used       free     shared    buffers     cached
Mem:         70007       2205      67801          0         28        595
-/+ buffers/cache:       1581      68425
Swap:            0          0          0

And

# cat /proc/cpuinfo
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.760
cache size	: 8192 KB
physical id	: 0
siblings	: 1
core id		: 0
cpu cores	: 1
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu tsc msr pae mce cx8 apic mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx lm constant_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr dca popcnt lahf_lm
bogomips	: 5336.34
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

[..]

We just included support for those machines in Scalarium, our EC2 cluster management plattform.

Upcoming events and talks

Posted by Jonathan

The conference season is starting again for me and I wanted to note where I will be/speak during the next couple of weeks.

First, there is Ruby Fools Copenhagen (April 1st and 2nd) where I will speak in the Ruby Performance track about Rails on AWS and how to leverage EC2, S3, and SQS in your application. The lineup at Ruby Fools looks really good with speakers like Glenn Vanderburg, Michael Koziarski, Evan Phoenix, Dr. Nic Williams, Dave Thomas, and Matz himself. Unfortunately I will not have too much time in Copenhagen as I have to leave early for Scotland on Rails in Edinburgh.

I'm really looking forward to be in Edinburgh again. After living, studying, and working there it feels like a second home. At Scotland on Rails (April 4th and 5th) I will talk about Rails Patterns: typical problems and scenarios in Rails applications like asynchronous operations (image processing, calculations, ..), authentication or deployment and common solutions and best practices.

In Mai I will be at Linuxtag 2008 in Berlin and hopefully talk about Ruby on Rails Security, but this talk has not been confirmed yet. Further, there is a chance that I will be speaking a the iX Cebit Forum 2008 about our internal Software Development Process and Agile Development.

Web 2.0 Expo Berlin

Posted by Jonathan

I'm just back from today's Web 2.0 Expo sessions and I'm not sure I will attend tomorrow. Many have written about this before, but the creative, social atmosphere is missing due to the conference labyrinth halls. Boy, I'm happy I haven't spend > 1.000 Euros on this. No real food, a lot of product presentations, not enough room for socializing and to many suits for my taste.

Still, I had some nice conversations and met some interesting people.

I did again a session on scaling with Amazon EC2 and S3, the slides can be found here.

This time a also talked a bit about how we use S3 and EC2 to drive our Webmail Portal product, PeritorMail at Peritor.

SlideShare | View

Also nice the AWS announcement of S3 being available in EU data centers. Now I'm only waiting for EC2 in the EU...

EC2 gets new instance types

Posted by Jonathan

Wow

Amazon EC2 gets two new types of instances, large and extra large EC2 instances. Basically a large instance that has 4 times the capacity (CPU, RAM, HDD) of the old, now default small instance type while the extra large instance type has 8 times the capacity.

Small Instance (default)

1.7 GB memory
1 EC2 Compute Unit (1 virtual core with 1 EC2 Compute Unit)
160 GB instance storage (150 GB plus 10 GB root partition)
32-bit platform
I/O Performance: Moderate 
Price: $0.10 per instance hour

Large Instance

7.5 GB memory
4 EC2 Compute Units (2 virtual cores with 2 EC2 Compute Units each)
850 GB instance storage (2 x 420 GB plus 10 GB root partition)
64-bit platform
I/O Performance: High 
Price: $0.40 per instance hour

Extra Large Instance

15 GB memory
8 EC2 Compute Units (4 virtual cores with 2 EC2 Compute Units each)
1,690 GB instance storage (4 x 420 GB plus 10 GB root partition)
64-bit platform
I/O Performance: High 
Price: $0.80 per instance hour

The idea is that you specify the instance type in the RunInstances API call. All old tools that do not specify this parameter start a default instance type.

Very nice to see this so fast after the recent S3 SLAs.

If they would now allow to run EC2 instances in Europe there are no excuses left not to run nearly all applications on EC2.

Amazon S3 Service Level Agreement

Posted by Jonathan

Finally:

Basically, we commit to 99.9% uptime, measured on a monthly basis. If an S3 call fails (by returning a ServiceUnavailable or InternalError result) this counts against the uptime. If the resulting uptime is less than 99%, you can apply for a service credit of 25% of your total S3 charges for the month. If the uptime is 99% but less than 99.9%, you can apply for a service credit of 10% of your S3 charges.

Very nice to see this shortly after the AWS presentation by Jeff Barr at the Berlin Ruby User Group. But I guess that for many people in big corporate settings this is not enough. But still, it's a start. And it makes selling AWS S3 as part of our solutions easier.

I'n now waiting for a EC2 SLA.

Get the details here.

My Rails Konferenz 2007 slides

Posted by Jonathan

Rails Konferenz 2007 is over and like last year it was a lot of fun, meeting other developers and learning some new stuff.

My talk on scaling Rails applications with Amazon S3 and EC2 went well and I had some interesting discussions afterwards. The talk was based on my Linuxtag talk but had a lot more info on Switiply and load-balancing.


(Photo by phil76)

The slides are available here as a PDF, a video will soon be available as I’m told.

Skalieren von Rails Anwendungen mit Amazon S3 und EC2 (PDF)

MeinProf featured on the German Amazon Affiliate Blog

Posted by Jonathan

It is not a secret that I am a fan of the Amazon Web Services. Besides EC2 and S3, I also use the Amazon Ecommerce Services (ECS) API at MeinProf.

Our implementation is now featured on the German Amazon Affiliate Blog.

We use the ECS API to let professors and students search for books and study guides on Amazon and then recommend the books to other students. So besides rating a course a student can also recommend supporting literature to others. This is the first step in our plans to increase MeinProfs value to students. We introduced this feature only a short while ago but already have many professors who register and recommend books to their students through our system.

The implementation uses the amazon-ecs ruby library that internally uses Hpricot, the great HTML/XML parser by _why.

The basic usage looks like this:

require 'amazon/ecs'

Amazon::Ecs.options = {
  :aWS_access_key_id => SECRET_KEY,
  :associate_tag => ASSOCIATE_TAG,
  :response_group => 'Medium',
  :country => :de
}

result_set = Amazon::Ecs.item_search('Programming Ruby', {
  :search_index => 'Books' # use Blended if you want also foreign books (usefull in DE and FR)
})

if result_set.has_errors?
  raise "Problem with ECS!" 
end

result_set.items.each do |item|
  puts item.get('title')
  puts item.get('asin')
  puts item.get('detailpageurl')
end

The difficult part is filtering and pre-processing the search term entered by the user in order to reduce the result set.

Linuxtag 2007 slides on Amazon S3 and EC2

Posted by Jonathan

I gave a presentation (in German) on how to scale web applications using Amazon S3 and EC2 at the Linuxtag 2007 in Berlin, Germany.

The talk was a broad introduction to S3 and EC2 and had some examples and scenarios using a Ruby on Rails application.

The slides are available as PDF here and there is also a mp3 recording thanks to http://www.digitalwarenmanufaktur.de/blog/.

I will present a more Rails centric variant of this talk at the upcoming Rails Konferenz in Frankfurt, Germany. Further I will be talking about S3/EC2 and Rails at the RailsConfEurope 2007 here in Berlin, Germany.

Skalieren einer Web Anwendung mit Amazon S3 und EC2 (PDF)

Skalieren einer Web Anwendung mit Amazon S3 und EC2 (MP3)