Cappuccino comes with a comprehensive set of tools for developing, debugging, optimizing, and deploying your Cappuccino applications. Those who have been following Cappuccino since the 0.7 release will notice some changes in the tools. The most important being that Cappuccino’s tools are now written entirely in JavaScript and Objective-J.

The 0.7 release used Rake, a Ruby build tool similar to “make”, to manage the build process of Cappuccino applications, as well as Cappuccino and Objective-J itself. In an effort to speed up the tools, reduce the number of dependencies, and allow us to focus on a single language family, we have replaced the Rake dependency with a simple port of Rake for JavaScript, aptly named Jake. Since Jake is written in JavaScript, other JavaScript and Objective-J build tools like the Objective-J compiler can run within the same process, speeding up the build time significantly.

Additionally, we have expanded the command-line/server-side JavaScript environment used for Cappuccino’s tools, now available as a separate project called Narwhal. Narwhal aims to support the emerging CommonJS module and standard library specifications on multiple JavaScript engines.

We started working with the Rhino engine, since the existing build tools were built on Rhino, and have recently added support for the JavaScriptCore / SquirrelFish engine. The performance of Narwhal on JavaScriptCore is an order of magnitude faster than Rhino, vastly improving build times. Currently “narwhal-jsc” supports Mac OS X, but other platform support is in progress.

Another major change was the refactoring of the “press” tool, which attempts to strip unnecessary files from your application bundle. As part of that refactoring we moved the “–flatten” feature, which inlines all code and files into one or more JavaScript files, into a separate tool, unsurprisingly called “flatten”. Flatten now supports splitting your application into multiple files which will be downloaded by the browser in parallel, via the “–split N” option.

For more information on all of these tools, check out the Tools wiki page.

The entire 280 crew just got back from an awesome time at JSConf EU in Berlin, where we got to show off some of the cool new developments coming with Cappuccino 0.8. I wanted to take the time to share one in particular in more depth here: image spriting.

The Problem with Spriting

Image spriting is the act of taking all the images in your app (or framework, or library, or whatever) and combining them down to one single image. This has the advantages of being smaller (since each individual image has overhead associated with the format), as well as allowing you to grab it from your server with one request:

This is faster in an absolute sense, but even more so psychologically since it allows you to show the contents of your app faster without having all the images “flash in” later. Cappuccino currently uses a similar technique with the source code in your application: concatenating it all together and minifying it, but up until now we’ve had no automatic, or built-in, support for spriting, you’ve instead had to do it entirely yourself. And truthfully, no framework has really good support for this.

This is because the fundamental problem with spriting today is that the process isn’t really automated and the results are thus subpar. Sure, there are scripts which will put images together for you, but they all require you to configure them first and to update them as your use of the images in question changes. This is because traditional spriting is dependent on how you use your images. Whether you choose to repeat, stretch, scale, or even draw to a canvas affects which images can be sprited together, and even which can’t be sprited at all. This has a number of unfortunate side effects:

• Rarely do you actually end up with just one image: Instead, you can end up with 2, 3, or even more. This is because images have to be “grouped” by their use. For example, vertically repeating images can be sprited together, but not with horizontally repeating images.
• You may have to actually change your code: Since images are being mutated, the code you write needs to take into account these new images. If you are lucky you have a system that is relatively good at doing this for you. However, if you decide to use an image in a new way (such as drawing it to a canvas), you either have to update your configuration files or choose to code it differently. This is easy to forget.
• Your images are still shipped separately from your code: Even in the best case where you are lucky enough to successfully sprite all your images together, you still have to wait for them separately from when your code is ready, potentially leading to noticeable delays from latency, or a “flash in” effect.
• Inflexible due to loss of data: There exist cases where your code is meant to be used by others, such as with libraries and frameworks. In this case, images can’t be used in any way other than how you intended them to if they are sprited, because the original images are gone or would require a redundant second download.

So unfortunately there is currently no good one-size-fits-all solution for image spriting the way there is with “code spriting”. All of them require the user to actually become involved in the optimization process, and even still can produce less than stellar results. This is clearly not a solution that can scale, and most everyone agrees to this.

But we’re hoping to change this with the release of Cappuccino 0.8, as we’re introducing a whole new, completely cross-platform, way to sprite: base64 images. By encoding images as base64, we create a lossless text representation of images, allowing us not only to use them in whatever way we please, but to actually ship them with the code:

There are many advantages to this:

• One file, guaranteed: All images can always be sprited together regardless of how you plan to use them, and can be included with the actual source code. This has the added benefit that gzip can work its magic on the entirety of your web app as one, producing better results.
• No need to ever modify code or configuration files: Since we’ve eliminated the ambiguous part of spriting images, the Cappuccino build tools are able to perform this optimization on your code automatically without tedious configuration files or having to “learn” how to sprite.

Yes, This Works in IE 6 and 7.

I’m sure most people are wondering how we are pulling this off in versions of IE before 8, since they do not support data URLs. Notice that earlier I didn’t specifically mention data URLs though, I instead only referred to the more broad technology of base64 images. As it turns out, IE has had support for base64 images since version 6 (!) with a little-known technology called MHTML. MHTML allows you place all your resources in one “resources” file, which incidentally can be any file in your website… including the same file that contains all your code.

Cappuccino is already smart enough to be able to automatically download and use different code depending on what browser is being used (and with no server configuration), so we now simply ship data URL versions of this technique to modern browsers, and MHTML versions to older copies of IE:

This is a very exciting feature for us. This has been a weak point in Cappuccino and its nice to finally have a solution that not only works, but is drop dead simple to use. Our tests have been proven incredibly promising, giving us the fastest load times we’ve ever seen with Cappuccino, and absolutely fantastic perceived speed as well. Our tools have all been honed to use this at every level: Apps, frameworks, and themes will automatically sprite your images for you.

This is just one of the many enhancements coming with Cappuccino 0.8, and the best part is as usual you won’t have to change a single line of code to get all the benefits.

Elias Klughammer has taken the time to put together a demo of using the new Tornado web server to bring push to Cappuccino applications. Tornado is a brand new non-blocking server recently open sourced by Facebook which was built to deal with the high intensity demands of FriendFeed. When combined with Cappuccino, the results are pretty impressive:

Check out the project on GitHub to start hacking on your own real time Cappuccino services!

Over at 280 North, we announced our next product, called Atlas, at the Future of Web Apps conference in Miami this week.

Atlas is a visual development tool for creating web applications using the Cappuccino framework. The best way to explain Atlas is to show it:

Think Vitamin has an article discussing Atlas in more detail.

We’re really excited about the ways Atlas could change the process of developing web applications. Atlas will allow developers to worry less about boilerplate user interface and glue code, just as the Cappuccino framework gives developers many common features expected by users, letting them focus on the ones specific to their applications.

Furthermore, Atlas allows non-programmers, such as many graphic designers, to join in on the process of actually building an application, rather than just providing mockups that must then be replicated in code by a developer.

Atlas will be available this summer. Sign up for updates on 280atlas.com and we’ll let you know of the progress on Atlas.

Note: please check the tools page on the wiki for the latest information on Cappuccino’s tools.

In the final installment of the Cappuccino Tool article series (for now), we cover “bake”, an automatic deployment tool. Writing a Cappuccino application does not require using “bake”, but can help with more advanced deployments.

Introduction

“bake” is somewhat analogous to Capistrano, a deployment tool often used for deploying Ruby on Rails and other applications, but “bake” has several features specifically for deploying Cappuccino applications.

The basic idea behind “bake” is to assemble a complete deployable copy of your application by pulling the pieces from various places (Git, Subversion, and local or remote directories via rsync), building it, packing it up, and deploying it to your server(s).

One key feature of bake is the management of multiple deployments over time, which allows three things: atomic deployments, keeping your client side code (the Cappuccino application) synchronized with your server side code, and enabling aggressive caching on all your static resources (code, images, etc). We’ll see later this is done by putting all the new resources in place, then “swinging” a single file, the index.html with a <base> tag.

“Atomic deployments” means that the deployment is updated to the new version in a single operation, which is either successful or not, nothing in between. Every client loading your application receives either the previous deployment, or the new deployment, but never a mixture of the two.

Keeping client code synchronized with server code is (sometimes) important for making sure users who have already loaded the application can continue using it even after you deploy a new version of the server side code that is incompatible with older versions of the client side code.

Aggressive caching of static resources reduces the load time of Cappuccino applications for repeat useres. This is possible because every resources URL is unique in each deployment, so we can set the “Expires” header far in the future (e.x. ‘Expires “Thu, 15 Apr 2010 20:00:00 GMT”‘). While the actual file name is not unique (for example Objective-J/Objective-J.js), in the deployed application the path will be unique (www.yourserver.com/YourApp/1225273279/Objective-J/Objective-J.js where the version number changes for each deployment)

Summary

The configuration for a bake deployment is specified in a “bakefile”, which is in the JSON format with a structure like the following:

{
	"sources" : [
	    {
	        "type"    : "git",
	        "path"    : "git://github.com/280north/cappuccino.git",
	        "parts"   : [
    	        {
    	            "src"       :   "Objective-J",
    	            "dst"       :   "Frameworks/Objective-J",
    	            "build"     :   "ant -DBuild=BUILD_PATH",
    	            "copyFrom"  :   "Release/Objective-J"
    	        },
	            {
    	            "src"       :   "Foundation",
    	            "dst"       :   "Frameworks/Foundation",
    	            "build"     :   "steam build -c Release -b BUILD_PATH",
    	            "copyFrom"  :   "Release/Foundation"
	            },
        	    {
    	            "src"       :   "AppKit",
    	            "dst"       :   "Frameworks/AppKit",
    	            "build"     :   "steam build -c Release -b BUILD_PATH",
    	            "copyFrom"  :   "Release/AppKit"
        	    }
	        ]
	    },
	    {
	        "type"    : "rsync",
	        "path"    : "/Users/username/projects/NewApplication",
	        "parts"   : [
	            {
    	            "src"       :   "",
    	            "dst"       :   "Blah"
	            }
	        ]
	    },
	    {
	        "type"    : "svn",
	        "path"    : "https://svn.youserver.com/Project/trunk",
	        "parts"   : [
	            {
    	            "src"       :   "subdirectory",
    	            "dst"       :   "Something"
	            }
	        ]
	    }
	],
	"deployments" : [
	    { "host" : "deploy@myserver.com", "path" : "/var/www/mysite/public" }
	],
	"templateVars" : {
        "APPLICATION_NAME" : "My Application",
        "BACKGROUND_COLOR" : "black",
        "TEXT_COLOR" : "black"
	}
}

This bakefile contains three “sources”: a git repository, a local directory, and a svn repository. The git repository is the main Cappuccino git repository, hosted on GitHub. You could replace this with a different branch if you needed a specific version, or even your own if you have one. This source contains three “parts”, Objective-J, AppKit, and Foundation. The “src” specifies where in the checkout the part is located, while “dst” specified where in the final built application it should be placed. You can optionally specify a “build” command, which is run from the specified “src” directory. This command should include the placeholder “BUILD_PATH”, which will be filled in by “bake” with the temporary directory where the build results are placed (equivalent to $STEAM_BUILD for any “steam” commands). Additionally, if you specify a “build” command you also need to specify a “copyFrom” parameter, which tells build the subdirectory of the build directory it should copy the built part from.

Once each piece of the application has been built it is copied to the appropriate subdirectory of a uniquely named (using the current Unix timestamp) “versioned” directory. Rather than deploying this versioned directory directly, it is placed within another directory which contains a special index.html file. This file includes a <base> tag that points to the versioned directory, which causes the browser to treat it as the base for all relative URLs, thus loading the correct version of all resources without requiring the application be located at a unique URL. “bake” will fill in “$VERSION” template variable in the base tag of the template with the correct

Additionally, “bake” will calculate the size of all code which needs to be loaded, in order to provide an accurate loading progress bar. The calculated size is inserted into the template variable “$FILES_TOTAL”. You can also specify other template variables such as “$APPLICATION_NAME”, “$BACKGROUND_COLOR”, and “$TEXT_COLOR” in the bakefile.

If you use your own template, the only required variable is the “$VERSION” one, but you can specify as many custom variables as you wish.

Finally, once the application has been built and assembled, it is optionally run through “press”, then gzipped. If the “–deploy” command line parameter was given then the gzipped application is transmitted to the deployment servers, un-gzipped, and copied to the deployment path using a specific sequence of command to ensure the deployment is atomic:

tar xzf 1225273279.tar.gz
mkdir -p /path/to/deployment
mv 1225273279/1225273279/ /path/to/deployment/1225273279
mv 1225273279/index.html path/to/deployment/index.html
rm "1225273279.tar.gz
rmdir 1225273279
cd /path/to/deployment
ln -nsf 1225273279 Current

(version “1225273279″, deplyoment directory “/path/to/deployment”)

One important thing to note about “bake” is that since the old deployment remains live (but hidden) if you deploy a fix for a security vulnerability you should manually purge all the old deployments from your server. If the change doesn’t introduce any incompatibilities between the server and client, you could set up a rewrite rule to direct all requests to the purged deployments to the latest, which is always symlinked to “Current”.

Conclusion

Once again I should stress that this tool is by no means required for writing a Cappuccino application. It was written to help deploy our production application, 280 Slides, but we have made it available for anyone who has similar deployment requirements. If it doesn’t quite fit your requirements, please feel free to suggest improvements, or even better, make improvements yourself and contribute them back!

Note: please check the tools page on the wiki for the latest information on Cappuccino’s tools.

In the third installment of the build tools series we tackle one of the most advanced build tools: “press”. Recall from the overview that “press” is a tool for stripping code and other optimizations. Additionally, it can convert an entire Objective-J application into a single pure JavaScript file, which we’ll see can be useful.

press

The primary goal of “press” is to strip unused code from your application and frameworks and perform other optimizations. It needs to know about the entire application, including your application code and the Cappuccino frameworks, in order to determine the dependencies and remove unused files. It is called with two parameters, the path to your application, and the output directory:

press root_directory output_directory

This will run “press” on your application located at root_directory, and output a similar but smaller version (with unused files stripped) at output_directory.

Additionally, there are several useful options:

• ‐‐main path: The relative path (from root_directory) to the main file (default: ‘main.j’)
• ‐‐frameworks path: The relative path (from root_directory) to the frameworks directory (default: ‘Frameworks’)
• ‐‐png: Run pngcrush on all PNGs (pngcrush must be installed!)
• ‐‐flatten: Flatten all code into a single Application.js file and attempt to add script tag to index.html (useful for Adobe AIR and CDN deployment)
• ‐‐nostrip: Don’t strip any files. Mostly useful for debugging.

‐‐flatten

The most interesting of these options is “‐‐flatten”, which converts your application into a single pure JavaScript file that can be imported with a standard “script” tag. While the Objective-J load system is great in most scenarios, there are currently a few limitations that a pure JavaScript file helps overcome. Namely that the load system doesn’t work in Adobe AIR due to strict security restrictions, and the load system doesn’t work across domains, such as with a CDN, also due to browser security restrictions. By flattening all Objective-J code into a JavaScript file, we can use a standard script tag to load our application, which works cross-domain and in Adobe AIR.

Internals

If you’re interested in how “press” works, read on. It starts by loading your application while noting which global variables are defined in each file. It then “walks” along the dependency “graph”, including every imported file, with one important exception: framework imports like “import <AppKit/AppKit.j>” and “import <Foundation/Foundation.j>” are ignored since these types of imports would result in the entire framework (including unused files) being imported. Instead, press looks at the global dependencies, and only keeps the files that are referenced by any other file that has been imported.

Conclusion

“press” is a already a great tool, but there is much room for improvement. In the future we would like to add finer-granularity code stripping, such as method-level rather than file-level. Additionally, we’re working on an automatic image spriting and optimization feature which determines exactly which images are required for your application and includes them in a single image, vastly reducing the number of HTTP requests to load your application.