Why?
Data storage applications (caches, queues, databases, etc) implemented in Ruby suffer two major problems
You cannot use multiple CPU cores
Pausing GC causes serious delays for large data sets
What does Arabesque Promise?
Scalability to multiple CPU cores Constant performance profile regardless of queue size Can be used stand alone or embedded
Secret Sauce?
Berkeley DB
Single writer multiple reader concurrency in a multi-process or multi-threaded environment Data is stored in a shared memory area away from Ruby's GC Master-slave data replication Create data structure exclusively in memory
Benchmarks
Multi CPU Scaling 20000
18000
16000
14000
Operations / Sec
12000
10000
8000
6000
4000
2000
0 Single Process
Dual Processes
Number of Processes
GC Time % of GC Time to application time
12.00%
10.00%
8.00%
6.00%
4.00%
2.00%
0.00%
Zero
100K
Queue Size
1M
10M
Deployment Options
In-Process Library
Pros
Very fast Much easier to setup and use Less moving parts
Cons
Does not scale to multiple physical servers If you do that you lose ordering (might not be an issue)
Standalone Server
Pros
Allows multiple front end servers to use the same queue Allows the queue to scale up or scale out independently from the front end
Cons
More moving parts Added latencies of TCP/HTTP communication overhead Loses strict queue ordering in scale out scenarios
Thank You!