All posts by Preeti Bhalla

SpreeCommerce Product Customization

As an eCommerce business you can do better than your competition by allowing your customers to customize product that they want to order. Of course not all products can be customized but if you deal in a product which renders itself … Continue reading →

Read more at the source

SpreeCommerce Marketing Extension – All you need to target the right audience.

When it comes to marketing it’s important to target the right people. Say, you have the perfect marketing strategy for increasing retention or just the right content for promoting a product. What else do you need for a successful marketing … Continue reading →

Read more at the source

Vinsol’s Spree Commerce Open Source Android App is out now!

Android dominates the smartphone market with a share of 82.8%**. Talking about Ecommerce, it is a little older than 20 years now and needless to say, is growing exponentially. Quick question! What would be the fastest and cheapest way out for … Continue reading →

Read more at the source

Vinsol is managing Spree Commerce Open Source Efforts. Time to contribute back to the community!

We have been delivering Ecommerce solutions for almost two decades now. For us it has been about delivering experiences and not just documented products. As we grew, our quality centric mindset has motivated us to create something better everyday. After First … Continue reading →

Read more at the source

Storehouse: Distributed Rails Page Cache

January 4, 2013

What?

Storehouse provides a cache layer that wraps Rails’ page caching strategy. It provides a middleware that returns content from a centralized cache store and writes files to your local machine on-demand, allowing distribution to multiple servers. Cache stores can be easily defined by using or creating an adapter.

Features:

Distributed Rails Page Cache
Cache Expiration
Cache Clearing on demand
Many potential storage engines
Optional file storage in addition to shared cache (fallback to be served by nginx/apache)
Single generator / In-progress locks

Logic

With any application, it’s a good idea to cache whatever pages you can. When dealing with an application which spans many servers, it’s a good idea to cache pages in a common place so those objects can be shared. We couldn’t find an existing gem which met the features described above, so we built Storehouse.

The main goal of Storehouse was the ability to have a common cache which can be expired. This means that I can cache a user’s profile, but tell all servers to update their cache when a change is made to his attributes. I don’t want any web server to communicate directly with any other web server to do this. Using a shared disk (NFS) was too slow, as was S3. In production we started off by using Riak as the store. We quickly figure out that we could fit the body of our cache in Redis which is much faster. We are currently using Storehouse with a redis backend.

You can see from the diagram the hierarchy of cache hits. Serving a static html file through nginx will always be the fastest, so you want to prefer that whenever possible. Next fastest is loading a fully rendered page from Storehouse, and then finally a rails request. One thing to note is that files you opt to keep on disk will not be expired without a deployment or periodic `rm *` on disk, and this will only happen under special circumstances.

How?

Storehouse supports the following storage engines (add more if you want):

Memcached
Dalli
Redis
Riak
S3
In-memory (great for tests)

Storehouse appends the rails page cache using the method you define. You can configure expiration times for your content and rules for which types of pages to cache (more detail in the readme). Because we make use of Railties, Storehouse only works for Rails 3.0 and up.

Storehouse is resilient to errors. If your storage backend is taking too long or cannot be reached, Storehouse will then pass the request down to the Rails stack to render the page in the normal way, and try to save the result after the request has been fulfilled to the user.

The Storehouse middleware listens for custom headers from your application in order to know what to do:

To make Storehouse push the rendered content into the backend, add a X-Storehouse header with a string value of ‘1’: response.headers['X-Storehouse'] = '1' if cache_page?
Optionally, you can pass an expiration time for the content: response.headers['X-Storehouse-Expires-At'] = 10.minutes.from_now.to_i.to_s
You can also tell Storehouse to distribute the content: response.headers['X-Storehouse-Distribute'] = '1'

These headers will never reach your end user as they are always stripped out.

Distribution is a great way to share cached resources on a multi-box setup. If you would like Storehouse to distribute the rendered page across all boxes, simply add the X-Storehouse-Distribute header. This will do three things:

Add the content to the backend
Mark the content as distributable
Lay the file on the server handling the current request. This means only one server does the work, but your entire system reaps the benefits.

So how much better is it?

On average, we have been seeing about a 40% reduction in page load time for Storehouse cached pages.

A few caveats:

We expire cached pages every 10 minutes. We do this to ensure that content is always fresh for list-view. However, this means that until you have at least 4.5K hits/month on average per cached paged, the cache is actually wasting your time. We do have a few cached pages with traffic that low, so we saw an increase in page load time for those pages (bringing the average down).
For pages like the above, the increase in page load time is accounted for by the additional overhead of storing the cached pages.

Panic Mode

Storehouse provides an especially useful tool which allows you to switch your site into “panic” mode. Panic mode is for when you’re experiencing massive load due to a traffic spike. Serving files from disk is always going to be more efficient so Storehouse will attempt to make use of that. This is a destructive operation in that any content coming from your backend or from your app with the X-Storehouse header will be written to disk. It will also render expired content read from the cache instead of passing control to your app.

Storehouse is available now. Enjoy!