30 · 07
Google AppEngine provides an high-level cloud service which means that your application will be distributed automatically on top of the Google platform. All of your code will depends on the AppEngine SDK, so it could be risky to develop complex application on it.
I develop a webservice application for content delivering and content publishing at
Zoomorama. We currently use Akamai CDN as a simple cache layer to improve data delivering accross the world. It is interesting for me to use AppEngine in the same way: without changing anything on my existing code base. I have found some posts on blogs dealing with this AppEngine usage, but they are not focused on dynamic HTTP caching like a real CDN.
Principle is very simple, all HTTP requests on my AppEngine application will be copied to the AppEngine Datastore. Moreover data which are delivered through AppEngine are cached by AppEngine servers. The code below is a tiny proof of concept:
# HTTP caching on Google App Engine
# - by shad <shad@zaphod.eu>
#
import web # webpy 0.3x
from google.appengine.ext import db
from google.appengine.api import urlfetch
origin = 'http://my.website.com'
urls = (
'(/.*)', 'Root'
)
class Cache(db.Model):
data = db.BlobProperty(default=None)
headers = db.ListProperty(str)
class Root(object):
def GET(self, request):
cache = self.readCache(request)
if cache is None:
cache = self.writeCache(request)
for h in cache.headers:
print h
return cache.data
def readCache(self, key):
cache = cache = Cache.get_by_key_name(key)
if cache:
return cache
def writeCache(self, request):
url = origin + request
response = urlfetch.Fetch(url=url)
if response.status_code != 200:
raise web.NotFound()
cache = Cache(key_name=request)
cache.data = db.Blob(response.content)
cache.headers = []
for k, v in response.headers.iteritems():
cache.headers.append('%s: %s' % (k, v))
cache.put()
return cache
if __name__ == '__main__'
app = web.application(urls, globals())
app.cgirun()
I use webpy to depends on the AppEngine SDK as less as possible.
I have almost finished the production version of this application. I am doing some performance tests. This application is closed source for now. But I am going to release the code source in few weeks.
This version will include:
- Fetch from Memcache (about 10 times faster).
- Headers forwarding.
- Read "Cache-Control" and "Expires" to define a TTL (rfc 2616).
- Multi origins (according to url mount points).
- Other small features (force TTL, ignore query string, etc...).
It is important to note that AppEngine does not keep running instances of your application (your CGI is distributed and it is executed on demand). So this application have to start very quickly (no configuration file, no dynamic generation, etc...).
