I have been working on developing the prerequisites for Forester to emit RSS and Atom feeds for blogs, and I realised that the problem I was trying to solve earlier this month is a more multifaceted than I originally thought. It comes down to analysing what is needed for Forester to be a good citizen of the World Wide Web: in particular, if we emit an RSS feed that has hyperlinks to some trees in it, those links must refer to an actual page on the actual web rather than something specific to Forester’s ontology.

This may seem downright obvious in hindsight, but you must understand that for the longest time I was not thinking of Forester as a tool for progressively enhancing the Web, but rather as a tool for building fully-local life-wikis or Zettelkästen; I no longer believe that my former viewpoint is reasonable, and I have concluded that we must integrate Forester into the Web or else we will be buried under friction. This post is the start of a design for how to do this.

Forget what you know about how either Forester 5.0 or previous versions currently work; in order to solve these problems in a reasonable way, we cannot be bound by the past versions of an experimental tool. What we are bound by is the architecture of the World Wide Web, and that will be reflected in the design.

Here is the essence of the proposal:

1. We get rid of the forest://host/addr scheme. Instead, trees are globally addressed by a canonical URL.
2. The canonical URL of a tree can in principle be arbitrary, but in practice you will want it to be a place where that tree can be viewed — e.g. the place to which it will be uploaded and served via HTTP(S). Indeed, a default scheme will be provided so as to enable files to be rendered with names and relative locations consistent with the intended global addressing scheme; it is also possible to imagine customisation of this without disturbing the overall design.
3. The canonical URLs are now the vertices of the graph.
4. In Forester source code, a hyperlink like would be resolved right away to or something, using information supplied in the user’s forest; the same goes for transclusion.
5. Links to trees in foreign forests must, for now, be totally explicit (but we can imagine relaxing this in the future). Importantly, this approach does not require knowing what is in the forest at evaluation-time.

It may seem annoying to have canonical URLs. For example, a forest that contains vital information might need to be published in multiple places. That much is true, but the fact that the physical publication of a forest is replicated should not allowed to impact the graph or fill it with redundant vertices and edges (e.g. should two mirrors become federated). So the only problem with replication is that hyperlinks might take you to the original forest instead of keeping you in the mirror, but I think this should be resolved by some kind of middleware that rewrites links, just as the Wayback machine rewrites links in its snapshots. That can be handled outside of Forester.

Most of the time, an author is working with their forest on their own machine rather than on the web. It is important that links and transclusions point to the local content rather than whatever (if anything) is stored in the “global” canonical URL. I believe this is not actually a problem: although things like RSS feeds and perhaps even published websites would have all the hyperlinks point to the canonical URLs, there is no reason that this should be required for all renderers. It is easy to imagine making this a configurable flag for the default renderer, and for the upcoming “dynamic”/interactive HTML server we would emit links back to the local server rather than to the canonical URLs.

Similarly, there may be projects where there is no intention at all of online publication. In such cases, the scheme for assigning canonical URLs can be arbitrary.

Forester does not currently support any kind of access control, but this is indeed an important area that we are considering carefully in order to enable institutional use of Forester, and ease the burden of collaboration in the usual case of a forest that contains a mixture of data with varying levels of confidentiality. I believe that the current design is compatible with essentially any approach to access control that we might adopt, but I am interested in feedback to the contrary.

There are several extant schemes for identifying individuals, organisations, and artefacts. Some are centralised, and others are decentralised. Centralisation of identity is not necessarily a bad thing, but it is most viable when nearly everyone agrees on the central authority; on the other hand, decentralisation can help in situations where a single central authority has not accumulated enough trust or prestige to be viable.

Although Forester aims to become a better citizen of the Web and integrate with emerging protocols, it is a non-negotiable design constraint that Forester still remain usable by people who don’t control a domain name, cannot run software on their web host, cannot set DNS records, and could not care less what a DID is. I also have a feeling that there will not be a single protocol that fits all use cases; what I am noticing, however, is that there are commonalities to all the protocols, and that we ought to be informed by these commonalities. For example, in every case an identity is resolved from a URI of some kind—for example, DIDs and DOIs and ORCIDs all have canonical URIs.

Therefore, it strikes me that Forester’s approach to global identity must rest on the axiom that an identity is nothing more or less than a URI; we can place no constraints whatsoever on what form this URI takes, and we should also remain flexible as to compatibility with future replacements of URIs (whether in the form of IRIs, or the URL non-“standard”, etc.).

If we start from that point of view, there some problems to address:

1. Even if an identity is not canonically addressed by a tree in a forest, an identity still often needs to have a tree in the forest. One wants to store biographical and bibliographic information, and maybe even personal notes, etc., and at the very least it is very important to be able to browse backlinks on a biographical page even if the page itself has no content of its own.
2. Not only must we be able to attach a tree to an identity: we must be able to attach many trees to an identity. This is a requirement of federation.

Building on my analysis, I propose that Forester allow any tree to declare that it “describes” a given global identity in the form of a URI. At a first cut this can be done via datalog (but we would probably hide this behind something):

Now it remains to explain how we shall surface the fact that a given entity is described by some tree.

1. For any identity in this relation, we should automatically create a “disambiguation page” that transcludes all the attached trees.
2. When a hyperlink points to a URI that lies in this relation, it should be directed to the disambiguation page.

There are further implications for such a feature—for instance, in the future we might automatically populate bibliographic information, etc. (But we have to be careful due to the near-universal unusably low quality of bibliographic databases keyed by DOIs, etc.)

We will need to provide guidance as to how identities should be assigned to (e.g.) people who don’t control an online identity, etc. The rule of thumb should be that we always defer to the preferences of the described person, and to the version of record in the case of an artefact. When there is no canonical choice, users of Forester should do what they like, but they should be willing to update their references in the future should a canonical global entity emerge.

I am hoping to hear other people’s thoughts on this proposal, including any constructive criticisms or suggestions for how we might go about implementing it. You can write to me or the mailing list with your feedback.

After spending two years exploring the design of tools for scientific thought that meet the unique needs of real, scalable scientific writing in hypertext, I have created a tool called Forester which has the following benefits:

1. Forester is tag-based like Gerby, and can therefore power large-scale generational projects like Stacks and Kerodon.
2. Forester produces static content that can be uploaded to any web hosting service without needing to run or install any serverside software.
3. Forester is easy to install on your own machine.
4. To prevent bitrot, Forester is a single tool rather than a composition of several tools.
5. Forester satisfies all the requirements of serious scientific writing, including sophisticated notational macros, typesetting of diagrams, etc.

Forester combines associative and hierarchical networks of evergreen notes (called “trees”) into hypertext sites called “forests”.

Trees correspond roughly to what are referred to as “tags” in the Stacks Project.

In this section, we will walk through the installation of the Forester software.

Now that you have installed the Forester software, it is time to set up your forest. Forester provides a simple command to initialise a fresh forest within a folder. We’ll call our folder forest, but you can call it anything you want.

Now that we are inside our new directory, we can instruct Forester to initialise a forest.

forester init

This command initialises a git repository with the skeleton of a forest, which contains a configuration file named forest.toml; this file specifies the locations of your trees, assets, etc. There is also a git submodule bound to the theme/ directory (pointing to the base theme repository) that contains the stylesheets that web browsers will need in order to render your forest as HTML.

forester build forest.toml

python3 -m http.server 1313 -d output

The first tree that you should create is a biographical tree to represent your own identity; ultimately you will link to this tree when you set the authors of other trees that you create later on. Although most trees will be addressed by identifiers of the form xxx-NNNN, it is convenient to simply use a person’s full name to address a biographical tree. My own biographical tree is located at trees/people/jonmsterling.tree and contains the following source code:

Let’s break this code down to understand what it does.

1. The declaration sets the title of the tree to my name.
2. The declaration informs Forester that the tree is biographical. Not ever tree needs to have a taxon; common taxa include person, theorem, definition, lemma, blog, etc. You are free to use whatever you want, but some taxa are treated specially by Forester.
3. The subsequent declarations attach additional information to the tree that can be used during rendering. These declarations are optional, and you are free to put whatever metadata you want.
4. Like in HTML, paragraphs must be wrapped in .

Do not hard-wrap your text, as this can have visible impact on how trees are rendered; it is recommended that you use a text editor with good support for soft-wrapping, like Visual Studio Code.

You can see that the concrete syntax of Forester's trees looks superficially like a combination of \LaTeX and Markdown; Markdown-style links are used both for links to other trees and for links to external URLs. Forester's concrete syntax is not fully documented, but it is less ambiguous than both \LaTeX and Markdown.

Creating a new tree in your forest is as simple as adding a .tree file to the trees folder. Because it is hard to manually choose the next incremental tree address, Forester provides a command to do this automatically. If your chosen namespace prefix is xxx, then you should use the following command in your shell to create a new tree:

forester new forest.toml --dest=trees --prefix=xxx

In return, Forester should output the location of the new tree, e.g. trees/xxx-0002.tree. If we look at the contents of this new file, we will see that it is empty except for metadata assigning a date to the tree:

Most trees should have a annotation; this date is meant to be the date of the tree's creation. You should proceed by adding further metadata: the title and the author; for the latter, you will use the address of your personal biographical tree.

Tree titles should be given in lower case (except for proper names, etc.); these titles will be rendered by Forester in sentence case. A tree can have as many declarations as it has authors; these will be rendered in their order of appearance.

Now you can begin to populate the tree with its content, written in the Forester markup language. Think carefully about keeping each tree relatively independent and atomic.

You may be used to writing \LaTeX documents, where you work from the top down: you create some section headings, put some text under those headings, make some deeper section headings, put more text, etc. Forests work in the opposite way, from the bottom up: you start by writing independent, atomic notes/trees and then only later start to (sparingly) assemble these into a hierarchy in order to reify the emerging structure.

Forester’s bottom-up approach to section hierarchy works via something called transclusion. The idea is that at any time, you can include (“transclude”) the full contents of another tree into the current tree as a subsection by adding the following code:

This is kind of like \LaTeX ’s command, but much better behaved: for instance, section levels are computed on the fly depending on the position in the hierarchy. This entire tutorial is cobbled together by transcluding many smaller trees, each with their own independent existence. For example, the following two sections are transcluded from an entirely different part of my forest:

One of the immediate impacts and strengths of Forester’s transclusion model is that a given tree has no canonical “geographic” location in the forest. One tree can appear as a child of many other trees, which allows the same content to be incorporated into different textual and intellectual narratives.

A tree in Forester is a single file written in a markup language designed specifically for scientific writing with bottom-up hierarchy via transclusion. A tree has two components: the frontmatter and the mainmatter.

Now that you have created your forest and added a few trees of your own, it is time to upload it to your web host. Many users of Forester will have university-supplied static web hosting, and others may prefer to use GitHub pages; deploying a forest works the same way in either case.

1. First, make sure your forest is built using the earlier instructions.
2. Then take the entire contents of your output directory and upload them to your preferred web host.

I am eager to see the new forests that people create using Forester. I am happy to offer personal assistance via the mailing list.

Many aspects of Forester are in flux and not fully documented; it will often be instructive to consult the source of existings forests, such as this one.

Have fun, and be sure to send me links to your forests when you have made them!