Rails Conf

Utilizing Amazon S3 and EC2 in Rails

Jonathan Weiss, Peritor Wissensmanagement GmbH RailsConf Europe Berlin, 2007

Starting Point

One machine: • Apache • Ruby / Rails • MySQL

2

Worst Case Populariy

3

A Difficult Path

4

Problem: Backup

• High availability • Redundancy • Very big data sets

5

Problem: File System • Important files have to be accessed by many servers • NFS / Samba not practical

6

Problem: Spontaneous Traffic

7

Problem: Load Fluctuation

8

Don‘t reinvent the wheel!

9

Amazon Web Services

Simple Storage Service - S3

Elastic Compute Cloud - EC2

Redundant Data Store

Virtual server per hour

$0,15 per GB data per month

$0,10 per CPU hour

$0,10 - $0,20 per GB traffic

$0,10 - $0,20 per GB traffic

E-Commerce S.. …….

10

S3 - Simple Storage Service • Redundant storage - as much as you like • max. 5 GB per object • Organized in „Buckets“ • Web Service API for uploads • Downloads via • Web Service • HTTP / HTTPS • BitTorrent

11

S3 - Buckets S3 • Unique over all S3 • Contains many key-value-metadata tupel • Cannot contain other buckets! • Key can contain „/“

MyBucket_name

www.peritor.com

foto_5.jpg

site/screen.css

backups/januar.zip

site/script.js

fotos/2007/001.png

FreeBSD-6.2.iso

12

S3 with AWS::S3 Upload

Download

13

EC2 - Elastic Compute Cloud • Based on XEN virtualization • On demand virtual servers - controlled with Web Service API • 1,7 GHz • 1,75 GB RAM • 160 GB local file system (not persistent!) • Use your favorite Linux distro (Linux 2.6.16), Amazon Machine Images (AMI) are stored on S3 • ACLs for hosts/ports access control

14

EC2 Tools List availabe images

Start a new instance

Login with SSH

Shutdown instance

15

amazon-ec2 gem Setup

Usage

16

And now? How does this solve my problems?

17

S3 - Backup

• s3sync.rb

• S3Browser

• Brackup

• Firefox S3 Organizer extension

• Jungle Disk

•…

• S3 FUSE • s3DAV • Duplicity

18

s3sync.rb Backup

Restore

19

S3 Asset Host

20

S3 Asset Host Setup DNS

Rails configuration

21

S3 Asset Host welcome.rhtml template

Output

22

S3 - Authenticated User Data

23

attachment_fu Rails plugin Setup

24

attachment_fu Rails plugin Upload

Download

25

On-Demand Computing with EC2

Time based, e.g. with cron

26

On-Demand Computing with EC2

Load based, e.g. with Monit

27

On-Demand Computing with EC2

28

EC2 for extra capacity

29

Load Balancer / Proxy Example mod_proxy_blancer: • Talks to multiple backend servers (Mongrel) • Central Proxy/Load-Balancer configuration that has knowledge about nodes • Typically proxy restart on config change

30

Swiftiply Swiftiply Proxy: • Multiple backend clients have a persistent connection to the backend port • Proxy listens on cluster port for requests and forwards them

No re-configuration

31

Swiftiply Proxy Installation

Configuration

Start

32

Swiftiplied Mongrel • Gem plugin that patches Mongrel • Transforms Mongrel in Swiftiply client • Experimental

Start

or

33

EC2 on Demand before Swiftiply

34

EC2 on Demand with Swiftiply

35

EC2 on Demand with Swiftiply

36

Ressources • Amazon Web Services http://aws.amazon.com • Swiftiply

http://swiftiply.swiftcore.org • Attachment_fu http://svn.techno-weenie.net/projects/plugins/attachment_fu/ • Elastic Rails http://www.elasticrails.com • Capazon http://capazon.rubyforge.org • RightScale http://www.rightscale.com

37

Peritor Wissensmanagement GmbH Lenbachstraße 2 12157 Berlin Internet: www.peritor.com Email: [email protected]

Telefon: +49 (0)30 69 40 11 94 Telefax: +49 (0)30 69 40 11 95

© Peritor Wissensmanagement GmbH - All Rights Reserved 38