Lifestreams' summarize operation, 30 years later

In 1996, Eric Freeman and David Gelernter published Lifestreams: A Storage Model for Personal Data. The paper proposed replacing named files, directories, and the desktop with a single time-ordered stream of documents and five operations: new, clone, transfer, find, and summarize.

I found the paper while reading about Jef Raskin's Canon Cat. The two ideas are very alike, and neither could be fully built at the time.

Most of the paper is ordinary now. Timelines are everywhere, saved searches work like its substreams, and nobody is surprised that one document can appear in several collections.

summarize is different. It was the most ambitious of the five, and it is still the furthest from what the authors wanted.

The paper introduces it in one sentence: "The last operation, summarize, takes a substream and compresses it into an overview document." Then it adds the hard part: the content of that overview "depends on the type of documents in the substream." Stock quotes compress into a chart of your securities and net worth. A list of tasks becomes a prioritized to-do list. Phone call records yield a report; timecards yield a billing statement.

No matter how many documents fall into a given category, the system should be capable of summarizing the whole lot on a single screen.

That goal is about more than layout. Lifestreams wanted to subsume "many separate desktop applications", and the paper compares itself with Quicken. But an application also gives you a data schema, an interactive loop, and a way of looking at the data. Move your records out of Quicken into a chronological stream and you still need something that shows you a budget. summarize had to do that job. It was where the application was supposed to go.

In the 1996 prototype, summarize is a menu item. You pick a substream, open a menu of summary types, and choose one, such as "summarize by portfolio." As the paper describes them, these options depended on documents of known types: a business card, a phone call record, a stock quote.

That brings back the problem Lifestreams set out to solve. Someone still had to define the document type in advance, and someone had to write the summary in advance, for a need they had guessed. Each menu entry was a small application. Views nobody predicted did not exist.

Raskin ran into the same problem from another direction. His 1987 Canon Cat had no applications or files, only one continuous stream of text navigated by modeless search with the LEAP key. Archy, his later project, made commands findable the same way as content. But his commands were still programs someone wrote before the user needed them.

Both designs removed the application, and both then needed something that could write, on demand, the view the application used to provide. In those years, only a human programmer could do that.

Supertag started from a smaller complaint. I kept adding features to my note system, and each one added paperwork to writing: where a note belongs, its title, its tags. One day I noticed I was spending far more time developing the system than writing notes. So I removed fields and the dedicated capture, and what stayed was plain notes plus a database for reading them.

That is also what this problem needs: notes you can query without declaring a schema, and something that can write a view over them.

Supertag runs inside Emacs and works on standard Org files. The Org text is the only truth. Supertag projects it into a local database, an in-memory store with indexes, relations, and change events, persisted to disk. Every Org heading that carries an ID becomes a node with its properties, states, timestamps, and freeform text. A stream holds nothing of its own; it is a query over those nodes. No schema is enforced at ingestion.

There are two parts to the view side. A Summary Panel looks at one tag as a whole. A Stream Agent works through its notes one by one. I read them as two answers to the same problem: getting a view nobody wrote in advance.

A Summary Panel covers the notes of one tag. Instead of picking from a menu, you describe it in a sentence: "For each active project, show open tasks and highlight whichever has waited longest."

A model gets a sample: the property names, and the titles, states, and tags of six real records from your notes, with no note body and no non-date property value. It writes a layout as code. If the result isn't what you wanted, another sentence rewrites it.

The 1996 paper relied on known document types so that summaries could exist. Here nobody declares a type. A language model can look at your actual notes, however messy, and write a layout for them.

A panel is kept as a file. It queries the database every time it opens, so it shows current state. You can change a TODO state or jump to an entry from the panel, and the panel redraws. The paper's overview was a document placed on the stream; a panel does some of what an application does.

For now, a panel compresses facts: records, titles, states, tags, dates, and counts. A layer in which a model reads note bodies and reports what happened in them is not yet in the package.

The paper's phone call example shows the division of labor. A personal agent fills in a phone call record; summarize later reports over those records. A Stream Agent is closer to the paper's personal agents than to its summarize, but its view still comes from a sentence and not from a programmer.

From a prompt like "People I haven't contacted for thirty days rise to the top; when contacted, a keypress stamps the date and sinks them," the model writes the grouping, the ordering, the line under each title, the fields it writes, and the key bindings. A reply that fails the checks goes back to the model. For one that passes, the model is shown what it draws and either approves or corrects it, before anything is kept.

Once the agent is kept, the model is no longer involved:

The paper split this work between summarize and its personal agents. In Supertag, panels and agents cover both, without a programmer writing the application in advance.

Freeman and Gelernter set out to replace named files, directories, and explicit storage with a stream. Raskin's Canon Cat was a sealed, purpose-built machine with no visible operating system, and Archy carried its single stream forward as an environment of its own.

But the Lifestreams prototype was a client/server system on ordinary Unix workstations, and it let users keep their familiar applications: the paper names "emacs, xv, and ghostview" as tools for creating and viewing documents. Running inside Emacs is not a departure from that. The Cat, for its part, was discontinued after about six months, in what the Raskin Center's account calls a business decision, amid weak marketing and little internal sponsorship at Canon.

The difference I'd point to is who holds the data. In the Lifestreams prototype, the server stores all stream documents. In Supertag, the stream is a layer over plain text.

Because the Org files remain the source of truth, your notes stay ordinary text, searchable by grep, syncable by Git, and readable without Supertag. What Supertag adds lives beside them: tag hierarchy in .supertag-metadata.eld, panels and agents as small files in the vault.

In 1996, summarize could only be a menu, because every view had to be built by hand before the data arrived. A view can now be written from a sentence, over the notes you already have, so the operation can start to do what the paper asked of it.