Cache me if you can

From the producers who brought you 'Tengo and Cache', we present 'Cache me if you can'.

The concept is all about HTML delivery into iOS devices alongside a mechanism for updating the application without going through the Apple app store. Loading HTML from a local file is not a new idea, as a few companies and projects have sprung up around this type of mobile app development have emerged, most notably Phonegap and more recently Apparatio.

The original iOS app I had built, Tengo and Cache, created a way that took this idea in a slightly different direction. The projects above deliver HTML within an application and would rely on Javascript to update any content within the application, using local storage to persist data. What Tengo and Cache offered was a way to download new HTML documents in their entirety, storing the files in writable areas of the file system on a device. This worked by providing a manifest file, based on the HTML5 manifest, on the domain which was the iOS app was trying to download files from.

During a hack day for the Guardian, I extended this work to add a further downstream cache. Although every effort is made to cache resources before loading, some files may be requested which are not included in the manifest. Overriding the NSURLCache class, the application can intercept any calls which attempt to go out to the web. This cache then retrieves the file, stores it, and then serves this instead of continuing down the pipeline. The next time this file is requested, the cache serves the file on disk instead of hitting the web at all.

Now that I had an iOS application that should pre-cache and intercept, I needed some content to install onto a device. I had wanted to use a Wordpress blog, or RSS feed, but felt that this content could prove difficult. Without knowing what content would be delivered, I couldn't build a manifest file that would capture the entire content. Also, I thought it might prove to be bad user experience if something that would work on a web page when online might not work offline. Even something as ubiquitous as search might look to have failed miserably.

I decided to use something I knew I had control over, the Guardian Content API. Using a small NodeJS application, I could retrieve a query and present that as HTML, alongside an appropriate manifest file. Then all that was required was a small property change in the iOS app and a native application was ready to be launched.

Cache me if you can

View more presentations from Robbie Clutton.

Json and Grails

Recently I was trying to return JSON data from a Grails app and I was having some issues. Grails has some built in conversion utilities to enable code like

render myObject as JSON

This is all well and good, but I have a few fundamental issues with this. First off, attributes like fully-qualified class name and the database primary key are returned. I don't really want to expose internal information to client side applications. It also doesn't allow any renaming of the attributes. Say you have a String called firstName using camel case in your domain object, it returns that to the client. What if you want the API to return first_name?

The answer was a lot simpler then my searching could muster, and that is to use the template system, Sitemesh in this case, to render exactly what you want in JSON as you would if you were rendering HTML. OK, it's a bit of a headache, but you have complete control over the content and style. Combine that with a one liner in the controller to set the content type, and you're flying.

response.contentType = "text/javascript"

Spider and Validate

One of the things we've been trying to achieve is automation is every sense. As my old mate Otu once said "automate until it hurts!" Well, hopefully this will make it a little less painful. We wanted to point to a single page, have the application spider the entire site finding pages which return anything apart from 200 OK, and validate all those who do return 200 OK.

By combining two PHP projects, PHPCrawl (http://sourceforge.net/projects/phpcrawl/) and the frontend-test-suite (http://github.com/NeilCrosby/frontend-test-suite) it's been possible to do just that. Throw a little Ant build script in there too and deploy into a continuous integration container like Hudson, and you start to get a feel for the state of websites in development, or even in production as part of a monitoring tool.

To make use of this, edit the build.xml file to change the SITE_URL to be the endpoint which you want to test and run 'ant validate'. This will do an Ant copy with filtering to create a file called test.php. Ant then runs this and then captures the output from this. It collects which pages may have returned a 500, a 404 and 200 OK pages. It then passes an array of 200 OK pages into the frontend-test-suite which uses PHPUnit to report on HTML validation.

There's still plenty of work to do, mainly introducing the ability to supply your own W3C validator endpoint and introducing CSS validation. It'll come though, so keep an eye out!

The result of this is over at Github - http://github.com/robb1e/Validator

As a side note, to get frontend-test-suite as a submodule on Github I followed instructions from FND, thanks!

Apple, standards and plugins

I hate to sound dramatic but with Apple sounding off about web standards over plugins with their latest products I was wondering just how much Apple was dragging he industry forward, or if it was doing the opposite.

Some years ago, developing web applications meant having to know the details about the browsers your audience were using. We're still suffering from this today with discussion over different box models and questions about just when are Microsoft going to release a standards compliant browser. One example of this is anyone who uses Exchange for email. They might know the pain of using the web client given how rich it is in Internet Explorer and how average it is in any other browser.

The advent of cross browser JavaScript libraries helped reduce many of these pains and it's great to see a lot of work going into the new HTML5 specs that are taking lessons direct from these libraries. However, whilst singing the praises of a few websites that are using the new HTML5 audio and video elements, Apple seem to be creating their own events for touch that I can't see anywhere in the specs. It seems Mozilla are also playing the same game also and I'm worried that we're returning to a world where we have to tailor lots of code to target specific browsers.

You only have to go to Nike.com or log into Gmail to see iPad websites that have orientation and swipe support. It's all very nice but how much extra work is going to have to take place to get a website to work in different browsers and different devices? What happens when a new device comes out with different events, or even as with the new iPhone and it's different screen resolutions, how will that change the work already in the wild?

This raises further questions for me, especially after reports of JavaScript execution speed on the iPad , even though some frameworks claim this isn't so much of a problem, such as Spoutcore Touch
and others go further saying that no framework should be used at all to ensure performance

Although Apple have created fairly complete documentation about building web content for their devices, it does beg the question why there should be differences if we are supposed to live in a standards world? How long will it be before you hit a webpage and have it say "The website is best viewed with iPad", or any other device or browser for that matter?

Should we ignore the devices ad go for standards like the BBC iPlayer for big screens like PS3 and iPad , use libraries to augment sites like how apple.com/iphone allows the user to swipe the image carousel, tailor something specific for each device such as Nike.com or should we just be building native applications?

How should we progress? Should we let our analytics decide? Should we take the standards route or device specific route? This will certainly be one to keep an eye on.