WAIF - Webcomic Archive Interchange Format

While shopping for a place to put up my webcomic, I visited the technical support sites of quite a few webcomic hosting services. I'll probably be writing more about that next time because I observed several interesting things and an article about some of the rest of them would be interesting too. But in support forum after support forum, I noticed one plea, with many variations, over and over on every service, and there was absolutely no service where it had a good answer.

The plea is simple enough: it usually goes like this example from Paul Gadzikowski. I want "... to mirror my existing webcomic. There are 952 strips in its archive. Is there a way to install multiple strips at once, or must it be done individually?"

The reason this plea has no good answer is simple: it requires interoperability, and interoperability requires a standard. And as a webcartoonist who may one day need a new hosting service myself, I now have a personal interest in the existence of such a standard.

So I'm proposing one. I'm first asking for input from as many people as I can, and then I'm going to be writing software that functions as a reference implementation, and then I'm going to be distributing that software in the public domain to make it as usable as possible to everybody.

The objective of Webcomic Archive Interchange Format is to allow people to migrate entire archives of webcomics from any host that supports WAIF to any other host that supports WAIF.

I'm volunteering to coordinate the discussion about it, try to find either a consensus resolution for technical problems or find design means to eliminate them, and actually write a reference implementation that does automatic updates and builds a set of webpages given a valid WAIF archive.

WAIF 1.0 will not be applicable to all webcomics, but it is my intent to make it applicable to the large majority of webcomics. WAIF 1.0 won't support CGI-driven, Flash or Tarquin-engine pages, but it will support javascript and tables.

Design Objectives:

1. It should be practical in most cases for database-enabled hosting services with a little scripting to construct WAIF archives for existing comics.
2. It should be easy for hosts to support importing WAIF archives and generate the pages and/or database records for them.
3. It should be easy both for hosting services and individuals to maintain archives for new comics in a WAIF-compatible form.
4. The standard reference implementation should handle updating and page generation without requiring database access, so that ordinary people on any host can use it (assuming they can at least use 'cron' or 'cgi' to update webpages) and keep their archive in WAIF format so that they can migrate between WAIF-enabled hosts or self-hosting at will.

Straw Proposal:

This proposal is an overview, and doesn't go down to the last level of bytes and syntax. It is intended to allow some room for design and redesign as the requirements become clearer.

The form of a WAIF archive is a Zipfile containing a WAIF file in the root directory, and graphics files in a directory structure. HTML templates in the WAIF file may refer to static graphic resources using template tags with the names given those resources by the filenames and directory structure recorded in the Zipfile. This does not imply that that directory structure and filename set will be reproduced on the host machine; the graphic files may be renamed, moved, or virtualized when the archive is installed, but when the HTML templates stored in the WAIF file are instantiated, the image URL's in the resulting pages must refer to the correct graphics.

Here is the proposed content for a WAIF file, plus some discussion about what it means.

1. Administrivia (Base URL, Time offset, update periodicity, etc)
2. Static page templates (cast page, background, etc)
3. Dynamic (ie, update-driven) page templates
   1. First installment template
   2. Last/current installment template
   3. Middle installments template
   4. Archive page templates
   5. Multiple-update templates(partial archives, etc)
4. For each update:
   1. Graphic Link or "graphicset" with multiple graphic links
   2. Text fields called for by templates
   3. Time of each update
   4. Thread ID's for each update

The 'Administrivia' section gives named values so that you can use those names in template tags in all the templates. This is to facilitate changing the values in one place and having it take effect in all pages.

The 'Static Page Templates' name and enumerate the static pages and gives the HTML/Javascript for each of them. These pages might include character bios, cascading style sheets, and whatever other bits of HTML etc your pages need. These static pages may refer to images in template tags using the names of the image files in the WAIF archive, and may refer to other static pages with a template tag and the given name of the static page, and may refer to update-driven pages using a template tag giving a dynamic template name and instantiation number.

Note that the Static Page Templates, like all other HTML content in the WAIF file, may not refer to any graphic or other page from the WAIF archive by its relative or absolute location in the filesystem. The template tags must be used, because hosting services need the freedom to unpack WAIF archives moving the other pages and the graphics files to appropriate locations on their site and the template tags can be used to refer to these elements at their new locations.

Note also that with Static Page Templates, a bunch of names will be reserved for common usages, such as 'CAST', 'TIMELINE', 'FAQ', 'ARCS', 'MERCHANDISE', 'AUTHOR', and so on, because hosts will want to be able to identify those pages and unpack them in specific locations in order to make a comic imported from a WAIF archve as compatible as possible with their existing hosting system and layout.

The WAIF file next enumerates the dynamic templates: that is, the templates that require information from the updates to form pages. The obvious ones are for reading the archives one comic at a time, that is: first page, middle page, last/current page. Other archives may require information from many different updates, or even all updates, to complete, such as an archive page with a link to every comic, or full-week or full-month page containing all updates posted during that week or month. They have the same means of referring to graphics, static pages, and other template-generated pages as the static pages do. In addition, they may refer to named information from updates, including the ability to select sets of updates for their instantiation.

Finally, the WAIF file gives the updates, where each update gives the 'missing data' for the template tags from the corresponding types of dynamic templates. This includes named graphics tables giving the relative positions of graphics to use, named text fields, thread ID's for each update, and the time of each update.

Click on the thumbnail for full-sized repetition and redundancy!

The time of each update is special; it is up to a WAIF-compatible system to assure that pages implied by an update that has not happened yet are not visible on the site, and that the 'last page' template is used for the last update actually rendered. Also, no graphic should be viewable unless it is required by a static page or by an update that has already happened.

Thread ID's are an optional field. If the author includes thread ID's, then multiple sets of first/previous/next/last links may appear at the bottom of a comic, one for each thread ID. (For example, you may have a thread ID for each character, so someone can page forward and backward to comics containing that particular character even though those comics are scattered through the archive).

So now the discussion: What are the common operations or needs that you can't find a way to do or represent in this proposed form of WAIF format? What do you think needs added?

Bo Lindbergh pointed out in the WCN forum that some webcomics, like the fun parody of giant-robot anime on the left from Sluggy Freelance, reuse images; A single image may occupy multiple simultaneous positions in the comic. This is no problem for the proposal in its current form, since the template tags may simply point at the same graphic and the graphic sets allow for a set or sequence of image tags to replace any template tag calling for a graphic.

Click on the thumbnail for full-size multi-page updates!

Joey Manley, the main operator here at WCN, pointed out a more difficult issue that I need to account for in specific design, which he framed as multiple-page updates. Cheshire Crossing, a recent cover of which shown at the right, is an example of a comic that uses multi-page updates. A single update, which goes live all at once, spans multiple pages. There is no problem with multiple updates in a single day; you can just schedule them a thousandth of a second apart in the WAIF file and that will happen. But the multi-page update should form a "branch" off the main archive navigation; when you reach the multi-page update, you should be looking at the first page of it, even if you reach it by hitting the back button from the subsequent installment or going to the main site before another installment happens. And that's not consistent with what happens when you have multiple updates on a single day.

More generally, WAIF needs to support branching archives. The multi-page update is a special case of a larger problem. Stories flow like rivers. They can branch and they can join. I envisioned thread ID's as allowing branching and joining story sections, but the specific application of multi-page updates requires me to make them more general than I had considered. The general form of a multi-page update under WAIF is that "normal" updates all have the "default" thread ID. The first page of a multi-page update has the "default" thread ID and the "branch" thread ID. Subsequent pages in a multi-page update have the "branch" thread ID, so they link back and forth to each other. So far, so good: the first page of the multi-page update is reachable by next/last buttons from previous and subsequent main-branch updates, and has a next button for updates in its own branch as well as a next button for the "updates" in its own thread. The pages within the multi-page update have next/last buttons that navigate within the branch, and everything works....

But there's desired behavior that I didn't anticipate with the thread ID's, and that is that the pages within a branch may want forward/back buttons referring to updates outside the branch. IOW, when you hit the end of the branch, it would be nice to have a 'next' button that goes to the next update outside the branch, even though the 'back' button in that update refers to a different page, the page at the beginning of the branch. And in my design until I carefully considered multi-page updates, my thread ID's were bidirectional only, never one-way. So to handle this consideration there'll have to be a template tag for a non-reciprocal link to next/last updates in another branch.

If you have input for this project, Write me at the address given in my profile, including the tag '[WAIF]' somewhere in the mail title, or join us on the forums!