+2

scripts/atom.xml

··· 0 0

··· 1 + <?xml version="1.0" encoding="UTF-8"?> 2 + <feed xmlns="http://www.w3.org/2005/Atom"><id>https://jon.recoil.org/atom.xml</id><title type="text">Jon's blog</title><updated>2025-04-24T15:41:17-00:00</updated></feed>

+9

scripts/dune

··· 0 0 0 0 0 0 0 0 0

··· 1 + (executable 2 + (name gen_atom) 3 + (preprocess 4 + (pps ppx_deriving_yojson)) 5 + (libraries syndic uri unix ptime bos odoc.odoc yojson ISO8601)) 6 + 7 + (executable 8 + (name gen_blog_index) 9 + (libraries str))

+622

scripts/feed

··· 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

··· 1 + <?xml version="1.0" encoding="UTF-8"?> 2 + <feed xmlns="http://www.w3.org/2005/Atom"> 3 + <contributor> 4 + <name>Jon Sterling</name> 5 + <uri>https://www.forester-notes.org/jonmsterling/</uri> 6 + </contributor> 7 + <updated>2025-03-25T13:44:56Z</updated> 8 + <title>Forester Blog</title> 9 + <id>https://www.forester-notes.org/30FM/</id> 10 + <link rel="alternate" href="https://www.forester-notes.org/30FM/" /> 11 + <link rel="self" href="https://www.forester-notes.org/30FM/atom.xml" /> 12 + <entry> 13 + <title>Towards Forester 5.0 II: a design for canonical URLs</title> 14 + <published>2025-03-25T13:44:56Z</published> 15 + <updated>2025-03-25T13:44:56Z</updated> 16 + <author> 17 + <name>Jon Sterling</name> 18 + <uri>https://www.forester-notes.org/jonmsterling/</uri> 19 + </author> 20 + <link rel="alternate" type="text/html" href="https://www.forester-notes.org/JVIT/" /> 21 + <id>https://www.forester-notes.org/JVIT/</id> 22 + <content type="xhtml"> 23 + <div xmlns="http://www.w3.org/1999/xhtml"> 24 + <p>One of the goals of <a href="https://www.forester-notes.org/jms-011P/">Forester 5.0</a> is <em>lightweight federation</em>—the ability to have two forests participate in the same graph and therefore provide backlinks, etc. In a previous post (<a href="https://www.forester-notes.org/OYOJ/">Towards <a href="https://www.forester-notes.org/jms-011P/">Forester 5.0</a>: a design for global identity</a>), I talked about some of the difficulties that arise when dealing with identities of people and references that have global scope but could nonetheless be described by trees in many forests. I proposed that such things should be addressed by canonical URIs (e.g. DIDs, DOIs, etc.) and that Forester should grow the ability to bind a canonical URI to multiple trees, which are then gathered into a disambiguation page.</p> 25 + <p>Today I want to broaden the discussion to cover the difficulties of addressing trees themselves (as opposed to the global entities they may describe). <mark>This is a proposal and I welcome feedback.</mark></p> 26 + <section> 27 + <header> 28 + <h2>Forester must become part of the Web</h2> 29 + </header> 30 + <p>I have been working on developing the prerequisites for Forester to emit RSS and Atom feeds for blogs, and I realised that the problem I was trying to solve <a href="https://www.forester-notes.org/OYOJ/">earlier this month</a> is a more multifaceted than I originally thought. It comes down to analysing what is needed for Forester to be a good citizen of the World Wide Web: in particular, <mark>if we emit an RSS feed that has hyperlinks to some trees in it, those links <em>must</em> refer to an actual page on the actual web rather than something specific to Forester’s ontology.</mark></p> 31 + <p>This may seem downright obvious in hindsight, but you must understand that for the longest time I was not thinking of Forester as a tool for progressively enhancing the Web, but rather as a tool for building fully-local life-wikis or Zettelkästen; I no longer believe that my former viewpoint is reasonable, and I have concluded that we must integrate Forester into the Web or else we will be buried under friction. This post is the start of a design for how to do this.</p> 32 + <p>Forget what you know about how either <a href="https://www.forester-notes.org/jms-011P/">Forester 5.0</a> or previous versions currently work; in order to solve these problems in a reasonable way, we cannot be bound by the past versions of an experimental tool. What we <em>are</em> bound by is the architecture of the World Wide Web, and that will be reflected in the design.</p> 33 + </section> 34 + <section> 35 + <header> 36 + <h2>What is the proposal?</h2> 37 + </header> 38 + <p>Here is the essence of the proposal:</p> 39 + <ol><li>We get rid of the <code>forest://host/addr</code> scheme. Instead, trees are globally addressed by a canonical URL.</li> 40 + <li>The canonical URL of a tree can in principle be arbitrary, but in practice you will want it to be a place where that tree can be viewed — e.g. the place to which it will be uploaded and served via HTTP(S). Indeed, a default scheme will be provided so as to enable files to be rendered with names and relative locations consistent with the intended global addressing scheme; it is also possible to imagine customisation of this without disturbing the overall design.</li> 41 + <li>The canonical URLs are now the vertices of the graph.</li> 42 + <li>In Forester source code, a hyperlink like <code><]]></code> would be resolved right away to <code><]]></code> or something, using information supplied in the user’s forest; the same goes for transclusion.</li> 43 + 44 + <li>Links to trees in foreign forests must, for now, be totally explicit (but we can imagine relaxing this in the future). Importantly, this approach does not require knowing what is in the forest at evaluation-time.</li></ol> 45 + </section> 46 + <section> 47 + <header> 48 + <h2>What about replication and mirroring?</h2> 49 + </header> 50 + <p>It may seem annoying to have canonical URLs. For example, a forest that contains vital information might need to be published in multiple places. That much is true, but the fact that the physical publication of a forest is replicated should not allowed to impact the graph or fill it with redundant vertices and edges (e.g. should two mirrors become federated). So the only problem with replication is that hyperlinks might take you to the original forest instead of keeping you in the mirror, but I think this should be resolved by some kind of middleware that rewrites links, just as the <a href="https://web.archive.org/">Wayback machine</a> rewrites links in its snapshots. That can be handled outside of Forester.</p> 51 + </section> 52 + <section> 53 + <header> 54 + <h2>What about viewing my forest locally?</h2> 55 + </header> 56 + <p>Most of the time, an author is working with their forest on their own machine rather than on the web. It is important that links and transclusions point to the local content rather than whatever (if anything) is stored in the “global” canonical URL. I believe this is not actually a problem: although things like RSS feeds and perhaps even published websites would have all the hyperlinks point to the canonical URLs, there is no reason that this should be required for all renderers. It is easy to imagine making this a configurable flag for the default renderer, and for the upcoming “dynamic”/interactive HTML server we would emit links back to the local server rather than to the canonical URLs.</p> 57 + <p>Similarly, there may be projects where there is no intention at all of online publication. In such cases, the scheme for assigning canonical URLs can be arbitrary.</p> 58 + </section> 59 + <section> 60 + <header> 61 + <h2>What about access control?</h2> 62 + </header> 63 + <p>Forester does not currently support any kind of access control, but this is indeed an important area that we are considering carefully in order to enable institutional use of Forester, and ease the burden of collaboration in the usual case of a forest that contains a mixture of data with varying levels of confidentiality. I believe that the current design is compatible with essentially any approach to access control that we might adopt, but I am interested in feedback to the contrary.</p> 64 + </section> 65 + </div> 66 + </content> 67 + </entry> 68 + <entry> 69 + <title>Towards Forester 5.0: a design for global identity</title> 70 + <published>2025-03-08T12:17:14Z</published> 71 + <updated>2025-03-08T12:17:14Z</updated> 72 + <author> 73 + <name>Jon Sterling</name> 74 + <uri>https://www.forester-notes.org/jonmsterling/</uri> 75 + </author> 76 + <link rel="alternate" type="text/html" href="https://www.forester-notes.org/OYOJ/" /> 77 + <id>https://www.forester-notes.org/OYOJ/</id> 78 + <content type="xhtml"> 79 + <div xmlns="http://www.w3.org/1999/xhtml"> 80 + <p>As we move closer to <a href="https://www.forester-notes.org/jms-011P/">Forester 5.0</a>, which <a href="https://www.forester-notes.org/30FN/">introduces</a> rudimentary federation capabilities, we must address new problems that did not arise in the days when no two <a href="https://www.forester-notes.org/tfmt-000R/">forests</a> interacted or linked to each other. The most immediate issue is that trees describing entities with “global” identity (including actual people as well as bibliographic references) will naturally be duplicated across many forests. For example, this happens when one person authors trees in multiple forests, and it happens even more often with bibliographic entries (both for the entries themselves and their author attributions). It is very important to handle this problem properly <em>now</em> in a way that (1) minimises friction and (2) enables us to quietly evolve toward <a href="https://www.forester-notes.org/klepmann-et-al-atproto-2024/">more Web-centric approaches to identity as they emerge</a>.</p> 81 + <p>Below, I survey some existing approaches to identity that we would hope to be compatible with at some level. If you want to skip to my concrete proposal, see <a href="https://www.forester-notes.org/OYOR/">§ OYOR/</a>.</p> 82 + <section> 83 + <header> 84 + <h2>Survey of global identification schemes</h2> 85 + </header> 86 + <p>There are several extant schemes for identifying individuals, organisations, and artefacts. Some are centralised, and others are decentralised. Centralisation of identity is not necessarily a bad thing, but it is most viable when nearly everyone agrees on the central authority; on the other hand, decentralisation can help in situations where a single central authority has not accumulated enough trust or prestige to be viable.</p> 87 + <section> 88 + <header> 89 + <h3>Centralised identification via DOIs and ORCIDs</h3> 90 + </header> 91 + <p>Nearly every scholarly paper and book published has a <em>Digital Object Identifier</em> (DOI) assigned to it, which are managed by a single authority (<a href="https://www.doi.org/">The DOI Foundation</a>); this applies to both traditional publishers and eprint servers like the arXiv. Services like <a href="https://zenodo.org/">Zenodo</a> allow individuals to mint their own DOIs and pin resources and artefacts to them. Due to their widespread adoption, DOIs are a completely viable way to identify published papers and books—and I would argue that any attempt to replace DOIs with a decentralised identifier is likely to be counterproductive as the goal should not not be decentralisation <em>per se</em> but rather to have a reliable, universal way to refer to scholarly content and artefacts.</p> 92 + <p>What DOIs do for artefacts, the <em>Open Researcher and Contributor ID</em> (ORCID) aims to do for <em>people</em> acting within the framework of open science. ORCIDs seem to do their job well, but not everyone has or should have an ORCID—nor would every person who does have one voluntarily choose to pin their entire identity to it. Therefore, although I happily use them, I think ORCIDs are likely to face more of an uphill battle than the DOI—which needed buy-in only from major publishers and eprint servers to reach hegemony.</p> 93 + </section> 94 + <section> 95 + <header> 96 + <h3>Informal decentralised identification via web addresses</h3> 97 + </header> 98 + <p>A particularly simple way to identify a single person or organisation is by means of a web domain or an email address. Although not everyone has a domain name, many people have email addresses. On the other hand, people often have many domain names and their email address may change over time; and when people die, their presence on the web is often erased or lost. Therefore, although widespread, this approach may create difficulties with longevity and stability.</p> 99 + </section> 100 + <section> 101 + <header> 102 + <h3>General-purpose decentralised identification via DIDs</h3> 103 + </header> 104 + <p>When reading about the paper of <a href="https://www.forester-notes.org/klepmann-et-al-atproto-2024/">Klepmann et al.</a> outlining Bluesky’s AT Protocol, I learned of <a href="https://www.w3.org/TR/did-1.0/"><em>Decentralised Identifiers</em> (DIDs)</a>. In essence, DIDs are URIs of the form <code>did:method:path</code> where <code>method</code> identifies <em>how</em> the DID is intended to be resolved and <code>path</code> is a colon-separated path that should be resolved by means of that method. In either case, a DID is intended to be resolved to a JSON document that contains information about the resource or entity being described, as well as various methods (like public keys) for verifying the integrity of that information. The <em>methods</em> are somewhat open-ended, but two important methods have emerged.</p> 105 + <section> 106 + <header> 107 + <h4>W3C’s <code>did:web</code> method</h4> 108 + </header> 109 + <p><a href="https://w3c-ccg.github.io/did-method-web/">W3C have specified the <code>did:web</code> method</a>, which in which the <code>path</code> is intended to be a web domain. Simplifying somewhat, a DID like <code>did:web:jonmsterling.com</code> would be substantiated by responding to the HTTPS request <code>https://jonmsterling.com/.well-known/did.json</code> with a document in the appropriate format. The upside is that the owner of a web domain is <em>their own</em> identity authority; in this sense <code>did:web</code> is a truly decentralised identification scheme. The downside is that you have to have a web domain, and you also can never change it ever—the <a href="https://www.forester-notes.org/OYOK/">same disadvantage of informal decentralised identification which we have discussed</a>.</p> 110 + </section> 111 + <section> 112 + <header> 113 + <h4>Bluesky’s <code>did:plc</code> method</h4> 114 + </header> 115 + <p>For a social network like Bluesky, it is critical that users be able to migrate their identity from one domain to another. Obviously users may change or lose their domain over time, but it is important to keep in mind that the vast majority of users will <em>never</em> have their own domain and so they will over the course of their lives jump from one subsidiary domain that they don’t control to the next—just as Mastodon users are constantly migrating from instance to instance, driven to wander endlessly by either the petty tyranny of instance maintainers who think they know best, or by the natural quiescence of instances caused by lack of funds or time, or (in many cases) a combination of the two.</p> 116 + <p>It seems that there is no way to address this problem without introducing some central authority—a <em>directory</em> of permanent identifiers that are then resolved to documents that establish cryptographically verified bidirectional links with more ephemeral and human-readable forms of identification (such as web domains). This is essentially the design of Bluesky’s <code>did:plc</code> method, as explained by <a href="https://www.forester-notes.org/klepmann-et-al-atproto-2024/">Klepmann et al.</a>:</p> 117 + <ol><li>On your own domain, which serves as your (ephemeral) handle, you place a DNS TXT record or file that contains a DID like <code>did:plc:asdlkfh9q8034baliufhbcailurb</code>.</li> 118 + <li>Someone resolves this DID to a document by querying a central <em>directory server</em> (such as <a href="https://web.plc.directory/">Bluesky’s own</a>). This document contains a link back to the domain; signatures are used to ensure that every update to the document has been authorised by whoever signed it when it was first minted.</li></ol> 119 + <p>Although some centralisation is required here, the use of cryptographic proof ensures that the central authority does not need to be trusted (to a certain extent).</p> 120 + </section> 121 + </section> 122 + </section> 123 + <section> 124 + <header> 125 + <h2>Analysis of global identity in Forester</h2> 126 + </header> 127 + <p>Although Forester aims to become a better citizen of the Web and integrate with emerging protocols, it is a non-negotiable design constraint that Forester still remain usable by people who don’t control a domain name, cannot run software on their web host, cannot set DNS records, and could not care less what a <a href="https://www.forester-notes.org/OYON/">DID</a> is. I also have a feeling that there will not be a single protocol that fits all use cases; what I am noticing, however, is that there are commonalities to all the protocols, and that we ought to be informed by these commonalities. For example, in every case an identity is resolved from a URI of some kind—for example, DIDs and DOIs and ORCIDs all have canonical URIs.</p> 128 + <p>Therefore, it strikes me that Forester’s approach to global identity must rest on the axiom that an identity is nothing more or less than a URI; we can place no constraints whatsoever on what form this URI takes, and we should also remain flexible as to compatibility with future replacements of URIs (whether in the form of IRIs, or the <a href="https://www.forester-notes.org/jms-011R/">URL non-“standard”</a>, etc.).</p> 129 + <p>If we start from that point of view, there some problems to address:</p> 130 + <ol><li>Even if an identity is not canonically addressed by a tree in a forest, an identity still often needs to <em>have</em> a tree in the forest. One wants to store biographical and bibliographic information, and maybe even personal notes, etc., and at the very least it is very important to be able to browse backlinks on a biographical page even if the page itself has no content of its own.</li> 131 + <li>Not only must we be able to attach a tree to an identity: we must be able to attach <em>many</em> trees to an identity. This is a requirement of federation.</li></ol> 132 + </section> 133 + <section> 134 + <header> 135 + <h2>A plan for global identity in Forester</h2> 136 + </header> 137 + <p>Building on my <a href="https://www.forester-notes.org/OYOQ/">analysis</a>, I propose that Forester allow any tree to declare that it “describes” a given global identity in the form of a URI. At a first cut this can be done via datalog (but we would probably hide this behind something):</p> 138 + <pre><![CDATA[\execute\datalog{ 139 + -: {\rel/describes \current-tree @{https://orcid.org/0000-0002-0585-5564}} 140 + }]]></pre> 141 + <p>Now it remains to explain how we shall surface the fact that a given entity is described by some tree.</p> 142 + <ol><li>For any identity in this relation, we should automatically create a “disambiguation page” that transcludes all the attached trees.</li> 143 + <li>When a hyperlink points to a URI that lies in this relation, it should be directed to the disambiguation page.</li></ol> 144 + <p>There are further implications for such a feature—for instance, in the future we might automatically populate bibliographic information, etc. (But we have to be careful due to the near-universal unusably low quality of bibliographic databases keyed by DOIs, etc.)</p> 145 + <p>We will need to provide guidance as to how identities should be assigned to (e.g.) people who don’t control an online identity, etc. The rule of thumb should be that we always defer to the preferences of the described person, and to the version of record in the case of an artefact. When there is no canonical choice, users of Forester should do what they like, but they should be willing to update their references in the future should a canonical global entity emerge.</p> 146 + </section> 147 + <section> 148 + <header> 149 + <h2>Request for comment</h2> 150 + </header> 151 + <p>I am hoping to hear other people’s thoughts on this proposal, including any constructive criticisms or suggestions for how we might go about implementing it. You can write to me or the <a href="mailto:~jonsterling/forester-discuss@lists.sr.ht">mailing list</a> with your feedback.</p> 152 + </section> 153 + </div> 154 + </content> 155 + </entry> 156 + <entry> 157 + <title>Build your own Stacks Project in 10 minutes</title> 158 + <published>2023-05-14T00:00:00Z</published> 159 + <updated>2024-04-25T00:00:00Z</updated> 160 + <author> 161 + <name>Jon Sterling</name> 162 + <uri>https://www.forester-notes.org/jonmsterling/</uri> 163 + </author> 164 + <link rel="alternate" type="text/html" href="https://www.forester-notes.org/jms-0052/" /> 165 + <id>https://www.forester-notes.org/jms-0052/</id> 166 + <content type="xhtml"> 167 + <div xmlns="http://www.w3.org/1999/xhtml"> 168 + <p><a href="https://www.forester-notes.org/stacks-project/">The Stacks project</a> is the most successful scientific hypertext project in history. Its goal is to lay the foundations for the theory of algebraic stacks; to facilitate its scalable and sustainable development, several important innovations have been introduced, with the <em>tags</em> system being the most striking.</p> 169 + <blockquote>Each tag refers to a unique item (section, lemma, theorem, etc.) in order for this project to be referenceable. These tags don't change even if the item moves within the text. (<a href="https://stacks.math.columbia.edu/tags">Tags explained</a>, <a href="https://www.forester-notes.org/stacks-project/">The Stacks Project</a>).</blockquote> 170 + <p>Many working scientists, students, and hobbyists have wished to create their own tag-based hypertext knowledge base, but the combination of tools historically required to make this happen are extremely daunting. Both the <a href="https://www.forester-notes.org/stacks-project/">Stacks project</a> and <a href="https://www.forester-notes.org/kerodon/">Kerodon</a> use a cluster of software called <a href="https://www.forester-notes.org/gerby/">Gerby</a>, but bitrot has set in and it is <a href="https://github.com/gerby-project/plastex/issues/60">no longer possible</a> to build its dependencies on a modern environment without significant difficulty, raising questions of longevity.</p> 171 + <p>Moreover, <a href="https://www.forester-notes.org/gerby/">Gerby</a>’s deployment involves running a database on a server (in spite of the fact that almost the entire functionality is static HTML), an architecture that is incompatible with the constraints of the everyday working scientist or student who knows <em>at most</em> how to upload static files to their university-provided public storage. The recent experience of the <a href="https://ncatlab.org/nlab/show/HomePage">nLab</a>’s pandemic-era hiatus and near death experience has demonstrated with some urgency the pracarity faced by any project relying heavily on volunteer system administrators.</p> 172 + <section> 173 + <header> 174 + <h2>Introducing <a href="https://www.forester-notes.org/index/"><em>Forester</em></a>: a <a href="https://www.forester-notes.org/tfmt-0002/">tool for scientific thought</a></h2> 175 + </header> 176 + <p>After spending two years exploring the <a href="https://www.forester-notes.org/tfmt-0001/">design of tools for scientific thought</a> that meet the unique needs of real, scalable scientific writing in hypertext, I have created a tool called <strong><a href="https://www.forester-notes.org/index/">Forester</a></strong> which has the following benefits:</p> 177 + <ol><li><a href="https://www.forester-notes.org/index/">Forester</a> is tag-based like <a href="https://www.forester-notes.org/gerby/">Gerby</a>, and can therefore power large-scale generational projects like <a href="https://www.forester-notes.org/stacks-project/">Stacks</a> and <a href="https://www.forester-notes.org/kerodon/">Kerodon</a>.</li> 178 + <li><a href="https://www.forester-notes.org/index/">Forester</a> produces static content that <a href="https://www.forester-notes.org/jms-007R/">can be uploaded to any web hosting service</a> without needing to run or install any serverside software.</li> 179 + <li><a href="https://www.forester-notes.org/index/">Forester</a> is <a href="https://www.forester-notes.org/jms-006W/">easy to install</a> on your own machine.</li> 180 + <li>To prevent bitrot, <a href="https://www.forester-notes.org/index/">Forester</a> is a single tool rather than a composition of several tools.</li> 181 + <li><a href="https://www.forester-notes.org/index/">Forester</a> satisfies <em>all</em> the <a href="https://www.forester-notes.org/tfmt-000E/">requirements of serious scientific writing</a>, including sophisticated notational macros, typesetting of diagrams, etc.</li></ol> 182 + <p><a href="https://www.forester-notes.org/index/">Forester</a> combines <a href="https://www.forester-notes.org/tfmt-0005/">associative and hierarchical</a> networks of <a href="https://www.forester-notes.org/tfmt-0003/">evergreen notes</a> (called “trees”) into hypertext sites called “forests”.</p> 183 + <section> 184 + <header> 185 + <h3>Forests and trees of evergreen notes</h3> 186 + </header> 187 + <p>A <em>forest of <a href="https://www.forester-notes.org/tfmt-0003/">evergreen notes</a></em> (or a <em>forest</em> for short) is loosely defined to be a collection of <a href="https://www.forester-notes.org/tfmt-0003/">evergreen notes</a> in which multiple <a href="https://www.forester-notes.org/tfmt-0005/">hierarchical structures</a> are allowed to emerge and evolve over time. Concretely, one note may contextualize several other notes via transclusion within its textual structure; in the context of a forest, we refer to an individual note as a <em>tree</em>. Of course, a tree can be viewed as a forest that has a root node.</p> 188 + </section> 189 + <p>Trees correspond roughly to what are referred to as “tags” in the <a href="https://www.forester-notes.org/stacks-project/">Stacks Project</a>.</p> 190 + </section> 191 + <p>In this article, I will show you how to set up your own <a href="https://www.forester-notes.org/tfmt-000R/">forest</a> using the <a href="https://www.forester-notes.org/index/">Forester</a> software. <mark>These instructions pertain to the <a href="https://www.forester-notes.org/jms-00S9/">Forester 4.1</a> version.</mark></p> 192 + <section> 193 + <header> 194 + <h2>Preparing to run the <a href="https://www.forester-notes.org/index/">Forester</a> software</h2> 195 + </header> 196 + <p>In this section, we will walk through the installation of the <a href="https://www.forester-notes.org/index/">Forester</a> software.</p> 197 + <section> 198 + <header> 199 + <h3>System requirements of <a href="https://www.forester-notes.org/index/">Forester</a></h3> 200 + </header> 201 + <section> 202 + <header> 203 + <h4>A unix-based system</h4> 204 + </header> 205 + <p><a href="https://www.forester-notes.org/index/">Forester</a> requires a unix-based system to run; it has been tested on both macOS and Linux. Windows support is <a href="https://todo.sr.ht/~jonsterling/forester/6">desirable</a>, but there are no concrete plans to implement it at this time.</p> 206 + </section> 207 + <section> 208 + <header> 209 + <h4>A working OCaml 5 installation</h4> 210 + </header> 211 + <p><a href="https://www.forester-notes.org/index/">Forester</a> is written in the <a href="https://www.ocaml.org/">OCaml</a> programming language, and makes use of the latest features of OCaml 5. Most users should install <a href="https://www.forester-notes.org/index/">Forester</a> through OCaml's <a href="https://opam.ocaml.org/">opam</a> package manager; instructions to install opam and OCaml simultaneously can be found <a href="https://opam.ocaml.org/">here</a>.</p> 212 + </section> 213 + <section> 214 + <header> 215 + <h4>A working <code>\LaTeX </code> installation</h4> 216 + </header> 217 + <p>If you intend to <a href="https://www.forester-notes.org/tfmt-000L/">embed <code>\LaTeX </code>-rendered diagrams</a> in your <a href="https://www.forester-notes.org/tfmt-000R/">forest</a>, you will need to have a working installation of <code>\LaTeX </code> installed, such as <a href="https://tug.org/texlive/">TeX Live</a>. If all your mathematical expressions are supported by <a href="https://katex.org/"><code>\KaTeX </code></a>, this is not necessary.</p> 218 + </section> 219 + <section> 220 + <header> 221 + <h4>The <a href="https://git-scm.com/">git</a> distributed version control system</h4> 222 + </header> 223 + <p>It is best practice to maintain your <a href="https://www.forester-notes.org/tfmt-000R/">forest</a> inside of <a href="https://en.wikipedia.org/wiki/Distributed_version_control">distributed version control</a>. This serves not only as a way to prevent data loss (because you will be pushing frequently to a remote repository); it also allows you to easily roll back to an earlier version of your <a href="https://www.forester-notes.org/tfmt-000R/">forest</a>, or to create “branches” in which you prepare <a href="https://www.forester-notes.org/tfmt-000R/">trees</a> that are not yet ready to be integrated into the <a href="https://www.forester-notes.org/tfmt-000R/">forest</a>.</p> 224 + <p>The recommended <a href="https://en.wikipedia.org/wiki/Distributed_version_control">distributed version control</a> system is <a href="https://git-scm.com/">git</a>, which comes preinstalled on many unix-based systems and is easy to install otherwise. <a href="https://git-scm.com/">Git</a> is not the most user-friendly piece of software, unfortunately, but it is ubiquitous. It is possible (but not recommended) to use <a href="https://www.forester-notes.org/index/">Forester</a> without version control, but note that the simplest way to <a href="https://www.forester-notes.org/jms-006X/">initialize your own <a href="https://www.forester-notes.org/tfmt-000R/">forest</a></a> involves cloning a <a href="https://git-scm.com/">git</a> repository.</p> 225 + </section> 226 + </section> 227 + <section> 228 + <header> 229 + <h3>Installing the <a href="https://www.forester-notes.org/index/">Forester</a> software</h3> 230 + </header> 231 + <p>Once you have met the <a href="https://www.forester-notes.org/jms-006S/">system requirements</a>, installing <a href="https://www.forester-notes.org/index/">Forester</a> requires only a single shell command:</p> 232 + <pre><![CDATA[opam install forester]]></pre> 233 + <p>To verify that <a href="https://www.forester-notes.org/index/">Forester</a> is installed, please run <code>forester --version</code> in your shell.</p> 234 + </section> 235 + </section> 236 + <section> 237 + <header> 238 + <h2>Setting up your <a href="https://www.forester-notes.org/tfmt-000R/">forest</a> from the template</h2> 239 + </header> 240 + <p>Now that you have <a href="https://www.forester-notes.org/jms-006W/">installed</a> the <a href="https://www.forester-notes.org/index/">Forester</a> software, it is time to set up your <a href="https://www.forester-notes.org/tfmt-000R/">forest</a>. <a href="https://www.forester-notes.org/index/">Forester</a> provides a simple command to initialise a fresh <a href="https://www.forester-notes.org/tfmt-000R/">forest</a> within a folder. We’ll call our folder <code>forest</code>, but you can call it anything you want.</p> 241 + <pre><![CDATA[mkdir forest 242 + cd forest]]></pre> 243 + <p>Now that we are inside our new directory, we can instruct <a href="https://www.forester-notes.org/index/">Forester</a> to initialise a <a href="https://www.forester-notes.org/tfmt-000R/">forest</a>.</p> 244 + <pre>forester init</pre> 245 + <p>This command initialises a <a href="https://git-scm.com/">git</a> repository with the skeleton of a <a href="https://www.forester-notes.org/tfmt-000R/">forest</a>, which contains a configuration file named <code>forest.toml</code>; this file specifies the locations of your trees, assets, etc. There is also a <a href="https://git-scm.com/">git</a> submodule bound to the <code>theme/</code> directory (pointing to the <a href="https://git.sr.ht/~jonsterling/forester-base-theme">base theme</a> repository) that contains the stylesheets that web browsers will need in order to render your <a href="https://www.forester-notes.org/tfmt-000R/">forest</a> as HTML.</p> 246 + <section> 247 + <header> 248 + <h3>Tree addresses in a <a href="https://www.forester-notes.org/tfmt-000R/">forest</a></h3> 249 + </header> 250 + <p>A <a href="https://www.forester-notes.org/tfmt-000R/">tree</a> in <a href="https://www.forester-notes.org/index/">Forester</a> is usually associated to an address of the form <code>xxx-NNNN</code> where <code>xxx</code> is <em>your</em> chosen “namespace” (most likely your initials) and <code>NNNN</code> is a four-digit <a href="https://www.forester-notes.org/jms-0074/">base-36 number</a>. The purpose of the namespace and the <a href="https://www.forester-notes.org/jms-0074/">base-36</a> code is to uniquely identify a <a href="https://www.forester-notes.org/tfmt-000R/">tree</a>, not only within your <a href="https://www.forester-notes.org/tfmt-000R/">forest</a> but across all <a href="https://www.forester-notes.org/tfmt-000R/">forests</a>. A tree with address <code>xxx-NNNN</code> is stored in a file named <code>xxx-NNNN.tree</code> (unless it is emitted from inside another tree by means of the <a href="https://www.forester-notes.org/jms-00O4/">inline subtrees</a> feature).</p> 251 + <p>Note that the format of <a href="https://www.forester-notes.org/tfmt-000R/">tree</a> addresses is purely a matter of convention, and is not forced by the <a href="https://www.forester-notes.org/index/">Forester</a> tool. Users are free to use their own format for tree addresses, and in some cases alternative (human-readable) formats may be desirable: this includes trees representing bibliographic references, as well as biographical trees.</p> 252 + </section> 253 + <section> 254 + <header> 255 + <h3>Building and viewing your <a href="https://www.forester-notes.org/tfmt-000R/">forest</a> for the first time</h3> 256 + </header> 257 + <p>To build your <a href="https://www.forester-notes.org/tfmt-000R/">forest</a>, you can run the following command of <a href="https://www.forester-notes.org/index/">Forester</a>'s executable in your shell:</p> 258 + <pre>forester build forest.toml</pre> 259 + <p>The <code>--dev</code> flag is optional, and when activated supplies metadata to the generated website to support an “edit button” on each tree; this flag is meant to be used when developing your <a href="https://www.forester-notes.org/tfmt-000R/">forest</a> locally, and should not be used when building the <a href="https://www.forester-notes.org/tfmt-000R/">forest</a> to be uploaded to your public web host.</p> 260 + <section> 261 + <header> 262 + <h4><a href="https://www.forester-notes.org/index/">Forester</a> renders each tree to an XML document</h4> 263 + </header> 264 + <p><a href="https://www.forester-notes.org/index/">Forester</a> renders your <a href="https://www.forester-notes.org/tfmt-000R/">forest</a> to some XML files in the <code>output/</code> directory; XML is, like HTML, a format for structured documents that can be displayed by web browsers. The <a href="https://www.forester-notes.org/tfmt-000R/">forest</a> template comes equipped with a built-in XSLT stylesheet (<code>theme/default.xsl</code>) which is used to instruct web browsers how to render your <a href="https://www.forester-notes.org/tfmt-000R/">forest</a> into a pleasing and readable format.</p> 265 + </section> 266 + <section> 267 + <header> 268 + <h4>Serving and viewing your <a href="https://www.forester-notes.org/tfmt-000R/">forest</a> from a local web server</h4> 269 + </header> 270 + <p>The recommended and most secure way to view your <a href="https://www.forester-notes.org/tfmt-000R/">forest</a> while editing it is to <em>serve</em> it from a local web server. To do this, first ensure that you have <a href="https://www.python.org/downloads/">Python 3</a> correctly installed. Then run the following command from the root directory of your <a href="https://www.forester-notes.org/tfmt-000R/">forest</a>:</p> 271 + <pre>python3 -m http.server 1313 -d output</pre> 272 + <p>While this command is running, you will be able to access your <a href="https://www.forester-notes.org/tfmt-000R/">forest</a> by navigating to <a href="http://localhost:1313/index.xml"><code>localhost:1313/index.xml</code></a> in your preferred web browser.</p> 273 + <p>In the future, <a href="https://www.forester-notes.org/index/">Forester</a> <a href="https://todo.sr.ht/~jonsterling/forester/15">may be able to run its own local server</a> to avoid the dependency on external tools like Python.</p> 274 + </section> 275 + <section> 276 + <header> 277 + <h4>Viewing your <a href="https://www.forester-notes.org/tfmt-000R/">forest</a> locally without a local web server</h4> 278 + </header> 279 + <p>It is also possible to open the generated file <code>output/index.xml</code> <em>directly</em> in your web browser. Unfortunately, modern web browsers by default prevent the use of XSLT stylesheets on the local file system for security reasons. Because <a href="https://www.forester-notes.org/jms-007G/"><a href="https://www.forester-notes.org/index/">Forester</a>'s output format is XML</a>, the output cannot be viewed directly in your web browser without disabling this security feature (at your own risk). Users who do not understand the risks involved should <a href="https://www.forester-notes.org/jms-007I/">turn back and use a local web server instead</a>, which is more secure; if you understand and are willing to accept the risks, you may proceed as follows depending on your browser.</p> 280 + <section> 281 + <header> 282 + <h5>Configuring Firefox for viewing a local <a href="https://www.forester-notes.org/tfmt-000R/">forest</a></h5> 283 + </header> 284 + <p>To configure Firefox for viewing your local <a href="https://www.forester-notes.org/tfmt-000R/">forest</a>, navigate to <code>about:config</code> in your address bar.</p> 285 + <ol><li>Firefox will present a page warning you to <em>“Proceed with Caution”</em>: you must <em>“Accept the Risk and Continue”</em>.</li> 286 + <li>In the “Search preference name” box, search for <code>security.fileuri.strict_origin_policy</code>.</li> 287 + <li>Most likely, the <code>security.fileuri.strict_origin_policy</code> will appear set to <code>true</code>. Double click on the word <code>true</code> to toggle it to <code>false</code>.</li></ol> 288 + </section> 289 + <section> 290 + <header> 291 + <h5>Configuring Safari for viewing a local <a href="https://www.forester-notes.org/tfmt-000R/">forest</a></h5> 292 + </header> 293 + <p>To configure Safari for viewing your local <a href="https://www.forester-notes.org/tfmt-000R/">forest</a>, you must activate the <em>Develop</em> menu and then toggle one setting.</p> 294 + <ol><li>Open Safari's settings window.</li> 295 + <li>In the <em>Advanced</em> tab, check the <em>“Show Develop menu in menu bar”</em> checkbox at the bottom.</li> 296 + <li>Open the <em>Develop</em> menu in the menubar, and select <em>“Disable Local File Restrictions”</em>.</li></ol> 297 + </section> 298 + </section> 299 + </section> 300 + </section> 301 + <section> 302 + <header> 303 + <h2>Creating your personal biographical <a href="https://www.forester-notes.org/tfmt-000R/">tree</a></h2> 304 + </header> 305 + <p>The first <a href="https://www.forester-notes.org/tfmt-000R/">tree</a> that you should create is a biographical tree to represent your own identity; ultimately you will link to this tree when you set the <a href="https://www.forester-notes.org/tfmt-000S/">authors</a> of other <a href="https://www.forester-notes.org/tfmt-000R/">trees</a> that you create later on. Although most <a href="https://www.forester-notes.org/tfmt-000R/">trees</a> will be addressed by <a href="https://www.forester-notes.org/jms-0073/">identifiers of the form <code>xxx-NNNN</code></a>, it is convenient to simply use a person’s full name to address a biographical <a href="https://www.forester-notes.org/tfmt-000R/">tree</a>. <a href="https://www.forester-notes.org/jonmsterling/">My own biographical tree</a> is located at <code>trees/people/jonmsterling.tree</code> and contains the following source code:</p> 306 + <pre><![CDATA[\title{Jon Sterling} 307 + \taxon{Person} 308 + \meta{external}{https://www.jonmsterling.com/} 309 + \meta{institution}{[[ucam]]} 310 + \meta{orcid}{0000-0002-0585-5564} 311 + \meta{position}{Associate Professor} 312 + 313 + \p{Associate Professor in Logical Foundations and Formal Methods at University of Cambridge. Formerly a [Marie Skłodowska-Curie Postdoctoral Fellow](jms-0061) hosted at Aarhus University by [Lars Birkedal](larsbirkedal), and before this a PhD student of [Robert Harper](robertharper).}]]></pre> 314 + <p>Let’s break this code down to understand what it does.</p> 315 + <ol><li>The declaration <code><![CDATA[\title{Jon Sterling}]]></code> sets the title of the <a href="https://www.forester-notes.org/tfmt-000R/">tree</a> to my name.</li> 316 + <li>The <code><![CDATA[\taxon{Person}]]></code> declaration informs <a href="https://www.forester-notes.org/index/">Forester</a> that the <a href="https://www.forester-notes.org/tfmt-000R/">tree</a> is biographical. Not ever <a href="https://www.forester-notes.org/tfmt-000R/">tree</a> needs to have a taxon; common taxa include <code>person</code>, <code>theorem</code>, <code>definition</code>, <code>lemma</code>, <code>blog</code>, etc. You are free to use whatever you want, but some taxa are treated specially by <a href="https://www.forester-notes.org/index/">Forester</a>.</li> 317 + <li>The subsequent <code><![CDATA[\meta]]></code> declarations attach additional information to the tree that can be used during rendering. These declarations are optional, and you are free to put whatever metadata you want.</li> 318 + <li>Like in HTML, paragraphs must be wrapped in <code><![CDATA[\p{...}]]></code>.</li></ol> 319 + <p>Do not hard-wrap your text, as this can have visible impact on how <a href="https://www.forester-notes.org/tfmt-000R/">trees</a> are rendered; it is recommended that you use a text editor with good support for soft-wrapping, like <a href="https://code.visualstudio.com/">Visual Studio Code</a>.</p> 320 + <p>You can see that the <a href="https://www.forester-notes.org/jms-007N/">concrete syntax of <a href="https://www.forester-notes.org/index/">Forester</a>'s <a href="https://www.forester-notes.org/tfmt-000R/">trees</a></a> looks superficially like a combination of <code>\LaTeX </code> and Markdown; Markdown-style links are used both for links to other trees <em>and</em> for links to external URLs. <a href="https://www.forester-notes.org/index/">Forester</a>'s concrete syntax is not fully documented, but it is less ambiguous than both <code>\LaTeX </code> and Markdown.</p> 321 + </section> 322 + <section> 323 + <header> 324 + <h2>Creating a new <a href="https://www.forester-notes.org/tfmt-000R/">tree</a> using <code>forester new</code></h2> 325 + </header> 326 + <p>Creating a new <a href="https://www.forester-notes.org/tfmt-000R/">tree</a> in your <a href="https://www.forester-notes.org/tfmt-000R/">forest</a> is as simple as adding a <code>.tree</code> file to the <code>trees</code> folder. Because it is hard to manually choose the <a href="https://www.forester-notes.org/jms-0073/">next incremental tree address</a>, <a href="https://www.forester-notes.org/index/">Forester</a> provides a command to do this automatically. If your chosen <a href="https://www.forester-notes.org/jms-0073/">namespace prefix</a> is <code>xxx</code>, then you should use the following command in your shell to create a new tree:</p> 327 + <pre>forester new forest.toml --dest=trees --prefix=xxx</pre> 328 + <p>In return, <a href="https://www.forester-notes.org/index/">Forester</a> should output the location of the new tree, e.g. <code>trees/xxx-0002.tree</code>. If we look at the contents of this new file, we will see that it is empty except for metadata assigning a date to the tree:</p> 329 + <pre><![CDATA[\date{2023-08-15}]]></pre> 330 + <p><strong>Most <a href="https://www.forester-notes.org/tfmt-000R/">trees</a> should have a <code><![CDATA[\date]]></code> annotation;</strong> this date is meant to be the date of the <a href="https://www.forester-notes.org/tfmt-000R/">tree</a>'s creation. You should proceed by adding further metadata: the title and the <a href="https://www.forester-notes.org/tfmt-000S/">author</a>; for the latter, you will use the address of your <a href="https://www.forester-notes.org/jms-007K/">personal biographical tree</a>.</p> 331 + <pre><![CDATA[\title{My first tree} 332 + \author{jonmsterling}]]></pre> 333 + <p><a href="https://www.forester-notes.org/tfmt-000R/">Tree</a> titles should be given in lower case (except for proper names, etc.); these titles will be <em>rendered</em> by <a href="https://www.forester-notes.org/index/">Forester</a> in sentence case. A <a href="https://www.forester-notes.org/tfmt-000R/">tree</a> can have as many <code><![CDATA[\author]]></code> declarations as it has <a href="https://www.forester-notes.org/tfmt-000S/">authors</a>; these will be rendered in their order of appearance.</p> 334 + <p>Now you can begin to populate the tree with its content, written in the <a href="https://www.forester-notes.org/jms-007N/"><a href="https://www.forester-notes.org/index/">Forester</a> markup language</a>. <a href="https://www.forester-notes.org/tfmt-0007/">Think carefully about keeping each tree relatively independent and atomic</a>.</p> 335 + </section> 336 + <section> 337 + <header> 338 + <h2>Bottom-up hierarchy via <em>transclusion</em></h2> 339 + </header> 340 + <p>You may be used to writing <code>\LaTeX </code> documents, where you work from the top down: you create some section headings, put some text under those headings, make some deeper section headings, put more text, etc. <a href="https://www.forester-notes.org/tfmt-000R/">Forests</a> work in the opposite way, from the bottom up: you start by writing independent, <a href="https://www.forester-notes.org/tfmt-0007/">atomic</a> notes/<a href="https://www.forester-notes.org/tfmt-000R/">trees</a> and then only later start to (sparingly) assemble these into a hierarchy in order to reify the emerging structure.</p> 341 + <p><a href="https://www.forester-notes.org/index/">Forester</a>’s bottom-up approach to section hierarchy works via something called <em>transclusion</em>. The idea is that at any time, you can include (“transclude”) the full contents of another <a href="https://www.forester-notes.org/tfmt-000R/">tree</a> into the current tree as a subsection by adding the following code:</p> 342 + <pre><![CDATA[\transclude{xxx-NNNN}]]></pre> 343 + <p>This is kind of like <code>\LaTeX </code>’s <code><![CDATA[\input]]></code> command, but much better behaved: for instance, section levels are computed on the fly depending on the position in the hierarchy. This <a href="https://www.forester-notes.org/jms-0052/">entire tutorial</a> is cobbled together by transcluding many smaller <a href="https://www.forester-notes.org/tfmt-000R/">trees</a>, each with their own independent existence. For example, the following two sections are transcluded from an <a href="https://www.forester-notes.org/tfmt-0001/">entirely different part</a> of my forest:</p> 344 + <section> 345 + <header> 346 + <h3>The best structure to impose is relatively flat</h3> 347 + </header> 348 + <p>It is easy to make the mistake of prematurely imposing a complex hierarchical structure on a network of notes, which leads to excessive refactoring. Hierarchy should be used sparingly, and its strength is for the large-scale organization of ideas. The best structure to impose on a network of many small related ideas is a relatively flat one. I believe that this is one of the mistakes made in the writing of the <em>foundations of relative category theory</em>, whose hierarchical nesting was too complex and quite beholden to my experience with pre-hypertext media.</p> 349 + </section> 350 + <p>One of the immediate impacts and strengths of <a href="https://www.forester-notes.org/index/">Forester</a>’s transclusion model is that a given <a href="https://www.forester-notes.org/tfmt-000R/">tree</a> has no canonical “geographic” location in the <a href="https://www.forester-notes.org/tfmt-000R/">forest</a>. One <a href="https://www.forester-notes.org/tfmt-000R/">tree</a> can appear as a child of many other <a href="https://www.forester-notes.org/tfmt-000R/">trees</a>, which allows the same content to be incorporated into different textual and intellectual narratives.</p> 351 + <section> 352 + <header> 353 + <h3>Hierarchical structure as non-unique narrative</h3> 354 + </header> 355 + <p>Multiple hierarchical structures can be imposed on the same associative network of nodes; a hierarchical structure amounts to a “narrative” that contextualizes a given subgraph of the network. One example could be the construction of lecture notes; another example could be a homework sheet; a further example could be a book chapter or scientific article. Although these may draw from the same body of definitions, theorems, examples, and exercises, these objects are contextualized within a different narrative, often toward fundamentally different ends.</p> 356 + <p>As a result, any interface for navigating the neighbor-relation in hierarchically organized notes would need to take account of the multiplicity of parent nodes. Most hypertext tools assume that the position of a node in the hierarchy is unique, and therefore have a single “next/previous” navigation interface; we must investigate the design of interfaces that surface all parent/neighbor relations.</p> 357 + </section> 358 + </section> 359 + <section> 360 + <header> 361 + <h2>The <a href="https://www.forester-notes.org/index/">Forester</a> markup language</h2> 362 + </header> 363 + <p>A <a href="https://www.forester-notes.org/tfmt-000R/">tree</a> in <a href="https://www.forester-notes.org/index/">Forester</a> is a single file written in a markup language designed specifically for scientific writing <a href="https://www.forester-notes.org/jms-007L/">with bottom-up hierarchy via transclusion</a>. A <a href="https://www.forester-notes.org/tfmt-000R/">tree</a> has two components: the <a href="https://www.forester-notes.org/jms-007P/">frontmatter</a> and the <a href="https://www.forester-notes.org/jms-007O/">mainmatter</a>.</p> 364 + <section> 365 + <header> 366 + <h3><a href="https://www.forester-notes.org/index/">Forester</a> markup: frontmatter</h3> 367 + </header> 368 + <p>The frontmatter of a <a href="https://www.forester-notes.org/index/">Forester</a> <a href="https://www.forester-notes.org/tfmt-000R/">tree</a> is a sequence of declarations that we summarize below.</p> 369 + <table> 370 + 371 + <tr> 372 + 373 + <th>Declaration</th> 374 + 375 + 376 + <th>Meaning</th> 377 + 378 + </tr> 379 + 380 + 381 + <tr> 382 + <td style="white-space:nowrap"> 383 + <code><![CDATA[\title{...}]]></code> 384 + </td> 385 + <td> 386 + sets the title of the <a href="https://www.forester-notes.org/tfmt-000R/">tree</a>; can contain <a href="https://www.forester-notes.org/jms-007O/">mainmatter</a> markup 387 + </td> 388 + </tr> 389 + 390 + 391 + <tr> 392 + <td style="white-space:nowrap"> 393 + <code><![CDATA[\author{name}]]></code> 394 + </td> 395 + <td> 396 + sets the <a href="https://www.forester-notes.org/tfmt-000S/">author</a> of the <a href="https://www.forester-notes.org/tfmt-000R/">tree</a> to be the biographical tree at address <code>name</code> 397 + </td> 398 + </tr> 399 + 400 + 401 + <tr> 402 + <td style="white-space:nowrap"> 403 + <code><![CDATA[\date{YYYY-MM-DD}]]></code> 404 + </td> 405 + <td>sets the creation date of the <a href="https://www.forester-notes.org/tfmt-000R/">tree</a></td> 406 + </tr> 407 + 408 + 409 + <tr> 410 + <td style="white-space:nowrap"> 411 + <code><![CDATA[\taxon{Taxon}]]></code> 412 + </td> 413 + <td>sets the taxon of the <a href="https://www.forester-notes.org/tfmt-000R/">tree</a>; example taxa include <code>lemma</code>, <code>theorem</code>, <code>person</code>, <code>reference</code>; the latter two taxa are treated specially by <a href="https://www.forester-notes.org/index/">Forester</a> for tracking biographical and bibliographical <a href="https://www.forester-notes.org/tfmt-000R/">trees</a> respectively</td> 414 + </tr> 415 + 416 + 417 + <tr> 418 + <td style="white-space:nowrap"> 419 + <code><![CDATA[\def\ident[x][y]{body}]]></code> 420 + </td> 421 + <td> 422 + defines and exports from the current <a href="https://www.forester-notes.org/tfmt-000R/">tree</a> a function named <code><![CDATA[\ident]]></code> with two arguments; subsequently, the expression <code><![CDATA[\ident{u}{v}]]></code> would expand to <code>body</code> with the values of <code>u,v</code> substituted for <code><![CDATA[\x,\y]]></code> 423 + </td> 424 + </tr> 425 + 426 + 427 + <tr> 428 + <td style="white-space:nowrap"> 429 + <code><![CDATA[\import{xxx-NNNN}]]></code> 430 + </td> 431 + <td> 432 + brings the functions exported by the tree <code>xxx-NNNN</code> into scope 433 + </td> 434 + </tr> 435 + 436 + 437 + <tr> 438 + <td style="white-space:nowrap"> 439 + <code><![CDATA[\export{xxx-NNNN}]]></code> 440 + </td> 441 + <td> 442 + brings the functions exported by the tree <code>xxx-NNNN</code> into scope, and exports them from the current tree 443 + </td> 444 + </tr> 445 + 446 + </table> 447 + </section> 448 + <section> 449 + <header> 450 + <h3><a href="https://www.forester-notes.org/index/">Forester</a> markup: mainmatter</h3> 451 + </header> 452 + <p>Below we summarize the concrete syntax of the <em>mainmatter</em> in a <a href="https://www.forester-notes.org/index/">Forester</a> <a href="https://www.forester-notes.org/tfmt-000R/">tree</a>.</p> 453 + <table> 454 + 455 + <tr> 456 + 457 + <th>Function</th> 458 + 459 + 460 + <th>Meaning</th> 461 + 462 + </tr> 463 + 464 + 465 + <tr> 466 + <td style="white-space:nowrap"> 467 + <code><![CDATA[\p{...}]]></code> 468 + </td> 469 + <td>creates a paragraph containing <code>...</code>; unlike Markdown, it is mandatory to annotate paragraphs explicitly</td> 470 + </tr> 471 + 472 + 473 + <tr> 474 + <td style="white-space:nowrap"> 475 + <code><![CDATA[\em{...}]]></code> 476 + </td> 477 + <td>typesets the content in italics</td> 478 + </tr> 479 + 480 + 481 + <tr> 482 + <td style="white-space:nowrap"> 483 + <code><![CDATA[\strong{...}]]></code> 484 + </td> 485 + <td>typesets the content in boldface</td> 486 + </tr> 487 + 488 + 489 + 490 + <tr> 491 + <td style="white-space:nowrap"> 492 + <code><![CDATA[\ol{...}]]></code> 493 + </td> 494 + <td>creates an ordered list</td> 495 + </tr> 496 + 497 + 498 + 499 + <tr> 500 + <td style="white-space:nowrap"> 501 + <code><![CDATA[\ul{...}]]></code> 502 + </td> 503 + <td>creates an unordered list</td> 504 + </tr> 505 + 506 + 507 + 508 + <tr> 509 + <td style="white-space:nowrap"> 510 + <code><![CDATA[\li{...}]]></code> 511 + </td> 512 + <td>creates a list item</td> 513 + </tr> 514 + 515 + 516 + 517 + <tr> 518 + <td style="white-space:nowrap"> 519 + <code><![CDATA[ #{...}]]></code> 520 + </td> 521 + <td>typesets the content in (inline) math mode using <code>\KaTeX </code>; note that math mode is idempotent in <a href="https://www.forester-notes.org/index/">Forester</a></td> 522 + </tr> 523 + 524 + 525 + <tr> 526 + <td style="white-space:nowrap"> 527 + <code><![CDATA[ ##{...}]]></code> 528 + </td> 529 + <td>typesets the content in (display) math mode using <code>\KaTeX </code></td> 530 + </tr> 531 + 532 + 533 + <tr> 534 + <td style="white-space:nowrap"> 535 + <code><![CDATA[\transclude{xxx-NNNN}]]></code> 536 + </td> 537 + <td><a href="https://www.forester-notes.org/jms-007L/">transcludes</a> the <a href="https://www.forester-notes.org/tfmt-000R/">tree</a> at address <code>xxx-NNNN</code> as a subsection</td> 538 + </tr> 539 + 540 + 541 + <tr> 542 + <td style="white-space:nowrap"> 543 + <code><]]></code> 544 + </td> 545 + <td> 546 + formats the text <code>title</code> as a hyperlink to address <code>address</code>; if <code>address</code> is the address of a <a href="https://www.forester-notes.org/tfmt-000R/">tree</a>, the link will point to that tree, and otherwise it is treated as a URL 547 + </td> 548 + </tr> 549 + 550 + 551 + <tr> 552 + <td style="white-space:nowrap"> 553 + <code><![CDATA[\let\ident[x][y]{body}]]></code> 554 + </td> 555 + <td> 556 + defines a local function named <code><![CDATA[\ident]]></code> with two arguments; subsequently, the expression <code><![CDATA[\ident{u}{v}]]></code> would expand to <code>body</code> with the values of <code>u,v</code> substituted for <code><![CDATA[\x,\y]]></code>. 557 + </td> 558 + </tr> 559 + 560 + 561 + <tr> 562 + <td style="white-space:nowrap"> 563 + <code><![CDATA[\code{...}]]></code> 564 + </td> 565 + <td>typesets the content in monospace</td> 566 + </tr> 567 + 568 + 569 + <tr> 570 + <td style="white-space:nowrap"> 571 + <code><![CDATA[\tex{preamble}{body}]]></code> 572 + </td> 573 + <td>typesets the <code>body</code> externally using <code>\LaTeX </code> using <code>preamble</code> as preamble code (e.g. to set up tikz packages, etc.). It can be useful to wrap this in your own macro in order to insert your preamble code automatically.</td> 574 + </tr> 575 + 576 + </table> 577 + </section> 578 + <section> 579 + <header> 580 + <h3>An complete worked example <a href="https://www.forester-notes.org/tfmt-000R/">tree</a> in <a href="https://www.forester-notes.org/index/">Forester</a></h3> 581 + </header> 582 + <p>An example of a complete <a href="https://www.forester-notes.org/tfmt-000R/">tree</a> in the <a href="https://www.forester-notes.org/index/">Forester</a> markup language can be seen below.</p> 583 + <pre><![CDATA[\title{Creation of (co)limits} 584 + \date{2023-02-11} 585 + \taxon{Definition} 586 + \author{jonmsterling} 587 + 588 + \def\CCat{#{\mathcal{C}}} 589 + \def\DCat{#{\mathcal{D}}} 590 + \def\ICat{#{\mathcal{I}}} 591 + \def\Mor[arg1][arg2][arg3]{#{{\arg2}\xrightarrow{\arg1}{\arg3}}} 592 + 593 + \p{Let \Mor{U}{\CCat}{\DCat} be a functor and let \ICat be a category. The functor #{U} is said to \em{create (co)limits of #{\ICat}-figures} when for any diagram \Mor{C_\bullet}{\ICat}{\CCat} such that #{\ICat\xrightarrow{C_\bullet}\CCat\xrightarrow{F}\DCat} has a (co)limit, then #{C_\bullet} has a (co)limit that is both preserved and reflected by #{F}.}]]></pre> 594 + <p>The code above results in the following tree:</p> 595 + <section> 596 + <header> 597 + <h4>Creation of (co)limits</h4> 598 + </header> 599 + <p>Let <code>{<code>\mathcal {C}</code>}\xrightarrow {U}{<code>\mathcal {D}</code>}</code> be a functor and let <code>\mathcal {I}</code> be a category. The functor <code>U</code> is said to <em>create (co)limits of <code><code>\mathcal {I}</code></code>-figures</em> when for any diagram <code>{<code>\mathcal {I}</code>}\xrightarrow {C_\bullet }{<code>\mathcal {C}</code>}</code> such that <code><code>\mathcal {I}</code>\xrightarrow {C_\bullet }<code>\mathcal {C}</code>\xrightarrow {F}<code>\mathcal {D}</code></code> has a (co)limit, then <code>C_\bullet </code> has a (co)limit that is both preserved and reflected by <code>F</code>.</p> 600 + </section> 601 + </section> 602 + </section> 603 + <section> 604 + <header> 605 + <h2>Deploying your <a href="https://www.forester-notes.org/tfmt-000R/">forest</a> to a web host</h2> 606 + </header> 607 + <p>Now that you have <a href="https://www.forester-notes.org/jms-006X/">created your <a href="https://www.forester-notes.org/tfmt-000R/">forest</a></a> and <a href="https://www.forester-notes.org/jms-007K/">added a few</a> <a href="https://www.forester-notes.org/jms-007H/"><a href="https://www.forester-notes.org/tfmt-000R/">trees</a> of your own</a>, it is time to upload it to your web host. Many users of <a href="https://www.forester-notes.org/index/">Forester</a> will have university-supplied static web hosting, and others may prefer to use GitHub pages; deploying a <a href="https://www.forester-notes.org/tfmt-000R/">forest</a> works the same way in either case.</p> 608 + <ol><li>First, make sure your <a href="https://www.forester-notes.org/tfmt-000R/">forest</a> is built using <a href="https://www.forester-notes.org/jms-007D/">the earlier instructions</a>.</li> 609 + <li>Then take the entire contents of your <code>output</code> directory and upload them to your preferred web host.</li></ol> 610 + </section> 611 + <section> 612 + <header> 613 + <h2>Let a hundred <a href="https://www.forester-notes.org/tfmt-000R/">forests</a> bloom!</h2> 614 + </header> 615 + <p>I am eager to see the new <a href="https://www.forester-notes.org/tfmt-000R/">forests</a> that people create using <a href="https://www.forester-notes.org/index/">Forester</a>. I am happy to offer personal assistance via the <a href="https://lists.sr.ht/~jonsterling/forester-discuss">mailing list</a>.</p> 616 + <p>Many aspects of <a href="https://www.forester-notes.org/index/">Forester</a> are in flux and not fully documented; it will often be instructive to consult the source of existings <a href="https://www.forester-notes.org/tfmt-000R/">forests</a>, such as <a href="https://git.sr.ht/~jonsterling/forester-notes.org">this one</a>.</p> 617 + <p>Have fun, and be sure to send me links to your <a href="https://www.forester-notes.org/tfmt-000R/">forests</a> when you have made them!</p> 618 + </section> 619 + </div> 620 + </content> 621 + </entry> 622 + </feed>

+179

scripts/gen_atom.ml

··· 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

··· 1 + (* Generate an atom feed *) 2 + 3 + let id = Uri.of_string "https://jon.recoil.org/atom.xml" 4 + let title : Syndic.Atom.text_construct = Syndic.Atom.Text "Jon's blog" 5 + 6 + let author = 7 + Syndic.Atom.author "Jon Ludlam" ~uri:(Uri.of_string "https://jon.recoil.org/") 8 + 9 + let updated = Unix.gettimeofday () |> Ptime.of_float_s |> Option.get 10 + 11 + let entry_of_mld odoc_file = 12 + let report_error during msg = 13 + Format.eprintf "Error processing file '%s' while %s: %s\n%!" 14 + (Fpath.to_string odoc_file) 15 + during msg; 16 + None 17 + in 18 + let unit = 19 + match Odoc_odoc.Odoc_file.load odoc_file with 20 + | Ok f -> Some f 21 + | Error (`Msg m) -> 22 + ignore (report_error "loading file" m); 23 + None 24 + in 25 + match unit with 26 + | None -> None 27 + | Some unit -> ( 28 + let page = 29 + match unit.content with 30 + | Odoc_odoc.Odoc_file.Page_content page -> Some page 31 + | _ -> None 32 + in 33 + match page with 34 + | None -> None 35 + | Some page -> ( 36 + let document = 37 + Odoc_document.Renderer.document_of_page ~syntax:OCaml page 38 + in 39 + let frontmatter = page.frontmatter.Odoc_model.Frontmatter.other_config in 40 + let published = 41 + try Some (List.assoc "published" frontmatter) with Not_found -> None 42 + in 43 + match published with 44 + | None -> None (* Skip posts without published date *) 45 + | Some published -> ( 46 + match document with 47 + | Odoc_document.Types.Document.Source_page _ -> None 48 + | Odoc_document.Types.Document.Page p -> 49 + let first_heading = 50 + List.find_map 51 + (function 52 + | Odoc_document.Types.Item.Heading h -> Some h 53 + | _ -> None) 54 + p.preamble 55 + in 56 + match first_heading with 57 + | None -> 58 + ignore (report_error "parsing title" "No heading found"); 59 + None 60 + | Some first_heading -> 61 + let title = 62 + List.filter_map 63 + (function 64 + | Odoc_document.Types.Inline.{ desc = Text t; _ } -> Some t 65 + | _ -> None) 66 + first_heading.title 67 + in 68 + let title = String.concat "" title in 69 + if title = "" then None 70 + else 71 + let resolve = Odoc_html.Link.Current p.url in 72 + let config = 73 + Odoc_html.Config.v ~semantic_uris:false ~indent:false 74 + ~flat:false ~open_details:false ~as_json:false ~remap:[] () 75 + in 76 + let url = Odoc_html.Generator.filepath p.url ~config in 77 + let url = 78 + Format.asprintf "https://jon.recoil.org/%s" 79 + (Fpath.to_string url) 80 + in 81 + (* Generate full content: preamble + items *) 82 + let all_items = p.preamble @ p.items in 83 + let html = Odoc_html.Generator.items ~config ~resolve all_items in 84 + let content_fmt = Fmt.list (Tyxml.Html.pp_elt ()) in 85 + let content = Format.asprintf "%a" content_fmt html in 86 + (* Extract first paragraph for summary *) 87 + let summary = 88 + let first_text = 89 + List.find_map 90 + (function 91 + | Odoc_document.Types.Item.Text blocks -> 92 + List.find_map 93 + (function 94 + | { Odoc_document.Types.Block.desc = 95 + Odoc_document.Types.Block.Paragraph inline; 96 + _ 97 + } -> 98 + let text = 99 + List.filter_map 100 + (function 101 + | Odoc_document.Types.Inline. 102 + { desc = Text t; _ } -> 103 + Some t 104 + | _ -> None) 105 + inline 106 + in 107 + if text = [] then None 108 + else Some (String.concat "" text) 109 + | _ -> None) 110 + blocks 111 + | _ -> None) 112 + p.preamble 113 + in 114 + match first_text with 115 + | Some t -> 116 + if String.length t > 200 then 117 + String.sub t 0 200 ^ "..." 118 + else t 119 + | None -> title 120 + in 121 + let published = 122 + try 123 + ISO8601.Permissive.date published |> Ptime.of_float_s 124 + with _ -> 125 + Format.eprintf "Error parsing date '%s' for %s\n%!" 126 + published (Fpath.to_string odoc_file); 127 + None 128 + in 129 + match published with 130 + | None -> None 131 + | Some published -> 132 + Some 133 + (Syndic.Atom.entry ~id:(Uri.of_string url) 134 + ~title:(Syndic.Atom.Text title) 135 + ~published ~updated:published 136 + ~summary:(Syndic.Atom.Text summary) 137 + ~content:(Syndic.Atom.Html (None, content)) 138 + ~links: 139 + [ 140 + Syndic.Atom.link ~rel:Syndic.Atom.Alternate 141 + (Uri.of_string url); 142 + ] 143 + ~authors:(author, []) ())))) 144 + 145 + let is_blog_post path = 146 + let basename = Fpath.basename path in 147 + Fpath.has_ext "odocl" path 148 + && String.length basename > 5 149 + && String.sub basename 0 5 = "page-" 150 + && basename <> "page-index.odocl" 151 + 152 + let entries = 153 + let mlds = 154 + Bos.OS.Dir.fold_contents 155 + (fun path acc -> if is_blog_post path then path :: acc else acc) 156 + [] 157 + (Fpath.v "_tmp/_odoc/blog") 158 + in 159 + match mlds with 160 + | Ok mlds -> 161 + let entries = List.filter_map entry_of_mld mlds in 162 + (* Sort by published date, newest first *) 163 + List.sort Syndic.Atom.descending entries 164 + | Error (`Msg m) -> 165 + Format.eprintf "Error finding blog posts: %s\n%!" m; 166 + [] 167 + 168 + let self_link = 169 + Syndic.Atom.link ~rel:Self (Uri.of_string "https://jon.recoil.org/atom.xml") 170 + 171 + let alt_link = 172 + Syndic.Atom.link ~rel:Alternate (Uri.of_string "https://jon.recoil.org/blog/") 173 + 174 + let feed = 175 + Syndic.Atom.feed ~id ~title ~updated ~links:[ self_link; alt_link ] entries 176 + 177 + let _ = 178 + Syndic.Atom.write feed "atom.xml"; 179 + Format.printf "Generated atom.xml with %d entries\n%!" (List.length entries)

+197

scripts/gen_blog_index.ml

··· 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

··· 1 + (* Generate blog index.mld files and recent entries list *) 2 + 3 + let month_name = function 4 + | 1 -> "January" | 2 -> "February" | 3 -> "March" | 4 -> "April" 5 + | 5 -> "May" | 6 -> "June" | 7 -> "July" | 8 -> "August" 6 + | 9 -> "September" | 10 -> "October" | 11 -> "November" | 12 -> "December" 7 + | _ -> "Unknown" 8 + 9 + type post = { 10 + year : int; 11 + month : int; 12 + slug : string; (* e.g., "claude-and-dune" *) 13 + title : string; 14 + published : string; (* e.g., "2025-12-18" *) 15 + } 16 + 17 + let parse_post path = 18 + let ic = open_in path in 19 + let content = really_input_string ic (in_channel_length ic) in 20 + close_in ic; 21 + 22 + (* Extract title from {0 ...} *) 23 + let title = 24 + let re = Str.regexp "{0 \\([^}]+\\)}" in 25 + if Str.string_match re content 0 then 26 + Str.matched_group 1 content 27 + else 28 + Filename.basename path |> Filename.remove_extension 29 + in 30 + 31 + (* Extract published date from @published ... *) 32 + let published = 33 + let re = Str.regexp "@published \\([0-9-]+\\)" in 34 + try 35 + ignore (Str.search_forward re content 0); 36 + Str.matched_group 1 content 37 + with Not_found -> "" 38 + in 39 + 40 + (* Parse path to get year/month/slug *) 41 + let parts = String.split_on_char '/' path in 42 + match parts with 43 + | _ :: year_s :: month_s :: filename :: [] -> 44 + let year = int_of_string year_s in 45 + let month = int_of_string month_s in 46 + let slug = Filename.remove_extension filename in 47 + Some { year; month; slug; title; published } 48 + | _ -> None 49 + 50 + let find_posts () = 51 + let posts = ref [] in 52 + let rec scan_dir dir = 53 + let entries = Sys.readdir dir in 54 + Array.iter (fun entry -> 55 + let path = Filename.concat dir entry in 56 + if Sys.is_directory path then 57 + scan_dir path 58 + else if Filename.extension entry = ".mld" && entry <> "index.mld" then 59 + match parse_post path with 60 + | Some post when post.published <> "" -> posts := post :: !posts 61 + | _ -> () 62 + ) entries 63 + in 64 + scan_dir "blog"; 65 + (* Sort by published date descending *) 66 + List.sort (fun a b -> String.compare b.published a.published) !posts 67 + 68 + let post_link post = 69 + let slug_needs_quotes = 70 + String.exists (fun c -> c = '-' || c = '_') post.slug 71 + in 72 + let slug_fmt = 73 + if slug_needs_quotes then 74 + Printf.sprintf "page-\"%s\"" post.slug 75 + else 76 + Printf.sprintf "page-%s" post.slug 77 + in 78 + Printf.sprintf "{{!/site/blog/%04d/%02d/%s}%s}" 79 + post.year post.month slug_fmt post.title 80 + 81 + let generate_month_index year month posts = 82 + let month_posts = List.filter (fun p -> p.year = year && p.month = month) posts in 83 + if month_posts = [] then None 84 + else 85 + let children = 86 + month_posts 87 + |> List.map (fun p -> p.slug) 88 + |> String.concat " " 89 + in 90 + let content = Printf.sprintf "{0 %s}\n\n@children_order %s\n\n" 91 + (month_name month) children 92 + in 93 + Some content 94 + 95 + let generate_year_index year posts = 96 + let year_posts = List.filter (fun p -> p.year = year) posts in 97 + if year_posts = [] then None 98 + else 99 + (* Get unique months, sorted descending *) 100 + let months = 101 + year_posts 102 + |> List.map (fun p -> p.month) 103 + |> List.sort_uniq (fun a b -> compare b a) 104 + in 105 + let children_order = 106 + months 107 + |> List.map (Printf.sprintf "%02d/") 108 + |> String.concat " " 109 + in 110 + let post_links = 111 + year_posts 112 + |> List.map (fun p -> "- " ^ post_link p) 113 + |> String.concat "\n" 114 + in 115 + let content = Printf.sprintf "{0 %d}\n\n@children_order %s\n\n%s\n" 116 + year children_order post_links 117 + in 118 + Some content 119 + 120 + let generate_blog_index posts = 121 + (* Get unique years, sorted descending *) 122 + let years = 123 + posts 124 + |> List.map (fun p -> p.year) 125 + |> List.sort_uniq (fun a b -> compare b a) 126 + in 127 + let children_order = 128 + years 129 + |> List.map (fun y -> Printf.sprintf "%d/" y) 130 + |> String.concat " " 131 + in 132 + Printf.sprintf "{0 Blog}\n\n@children_order %s\n\nUse the sidebar to navigate the blog posts. The most recent posts are listed first.\n" 133 + children_order 134 + 135 + let generate_recent_entries posts n = 136 + let recent = 137 + if List.length posts > n then 138 + List.filteri (fun i _ -> i < n) posts 139 + else 140 + posts 141 + in 142 + recent 143 + |> List.map (fun p -> "- " ^ post_link p) 144 + |> String.concat "\n" 145 + 146 + let write_file path content = 147 + let oc = open_out path in 148 + output_string oc content; 149 + close_out oc; 150 + Printf.printf "Wrote %s\n" path 151 + 152 + let () = 153 + let posts = find_posts () in 154 + Printf.printf "Found %d posts\n\n" (List.length posts); 155 + 156 + (* Get unique year/month combinations *) 157 + let year_months = 158 + posts 159 + |> List.map (fun p -> (p.year, p.month)) 160 + |> List.sort_uniq compare 161 + in 162 + 163 + (* Generate month indexes *) 164 + List.iter (fun (year, month) -> 165 + match generate_month_index year month posts with 166 + | Some content -> 167 + let dir = Printf.sprintf "blog/%04d/%02d" year month in 168 + let path = Filename.concat dir "index.mld" in 169 + write_file path content 170 + | None -> () 171 + ) year_months; 172 + 173 + (* Get unique years *) 174 + let years = 175 + posts 176 + |> List.map (fun p -> p.year) 177 + |> List.sort_uniq (fun a b -> compare b a) 178 + in 179 + 180 + (* Generate year indexes *) 181 + List.iter (fun year -> 182 + match generate_year_index year posts with 183 + | Some content -> 184 + let path = Printf.sprintf "blog/%04d/index.mld" year in 185 + write_file path content 186 + | None -> () 187 + ) years; 188 + 189 + (* Generate main blog index *) 190 + let blog_index = generate_blog_index posts in 191 + write_file "blog/index.mld" blog_index; 192 + 193 + (* Output recent entries for main index *) 194 + Printf.printf "\n=== Recent entries (copy to index.mld) ===\n\n"; 195 + Printf.printf "{1 Recent entries}\n\n"; 196 + print_endline (generate_recent_entries posts 10); 197 + print_endline ""

+49

scripts/mld.ml

··· 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

··· 1 + type t = { raw : string; parsed : Odoc_parser.t } 2 + 3 + type meta = { 4 + libs : string list; [@default []] 5 + html_scripts : string list; [@default []] 6 + other_config : Yojson.Safe.t; [@default `Null] 7 + } 8 + [@@deriving yojson] 9 + 10 + let parse_mld mld_file = 11 + let text = In_channel.(with_open_bin mld_file input_all) in 12 + let location = 13 + { Lexing.pos_fname = mld_file; pos_lnum = 1; pos_bol = 0; pos_cnum = 0 } 14 + in 15 + { raw = text; parsed = Odoc_parser.parse_comment ~location ~text } 16 + 17 + let meta_of_parsed parsed = 18 + let meta_block = 19 + List.filter_map 20 + (fun block -> 21 + let open Odoc_parser in 22 + match block with 23 + | { Loc.value = `Code_block c; _ } -> ( 24 + match c.Ast.meta with 25 + | None -> None 26 + | Some m -> 27 + if m.Ast.language.value = "meta" then Some c.content.value 28 + else None) 29 + | _ -> None) 30 + (Odoc_parser.ast parsed) 31 + in 32 + let meta = 33 + match meta_block with 34 + | [] -> None 35 + | x :: _ :: _ -> 36 + Format.eprintf 37 + "Warning, more than one meta block found. Ignoring all but the first."; 38 + Some x 39 + | [ meta ] -> Some meta 40 + in 41 + match meta with 42 + | None -> None 43 + | Some meta -> 44 + let y = Yojson.Safe.from_string meta in 45 + Some (meta_of_yojson y) 46 + 47 + let meta_of_mld mld_file = 48 + let { parsed; _ } = parse_mld mld_file in 49 + meta_of_parsed parsed

+1 -1

site/blog/2026/02/index.mld

··· 1 - @children_order weeknotes-2026-06 2 {0 February} 3 4

··· 1 + @children_order weeknotes-2026-08 odoc-js-notebooks-fun weeknotes-2026-06 2 {0 February} 3 4

+2

site/blog/2026/02/odoc-js-notebooks-fun.mld

··· 0 0

··· 1 + {0 Fun with odoc, plugins and javascript} 2 +

+53

site/blog/2026/02/weeknotes-2026-08.mld

··· 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

··· 1 + {0 Weeknotes weeks 7-8} 2 + 3 + @notanotebook 4 + @published 2026-02-24 5 + 6 + A combination one again as I took some time off due to school half term. 7 + 8 + {1 Finished off my exam questions} 9 + This was a lot of fun! Obviously I can't talk about it, but while it was stressful and worrying and anxiety inducing and scary, 10 + it was also engaging and interesting and thought-provoking. Having some ideas come together to make a nice coherent whole was 11 + very cool. 12 + 13 + {1 Testing LLMs on past paper questions} 14 + Similar to our work on the ticks that {{:https://www.youtube.com/watch?v=Ub8k1BcSRLQ}Sadiq, I and others did last year}, I wanted to try to see how well LLMs could answer tripos questions. 15 + Partly I wanted to do this so I could check that my own questions were of the right sort of level, and partly it was just a 16 + displacement activity while I wasn't making progress on the actual exam questions! I've not done a useful analysis of the 17 + results yet, but seemed in line with our experience with the ticks, though the pass rate was lower for the same models (qwen). 18 + 19 + {1 Claude from a sunbed} 20 + I went away for a vitamin-D boosting bit of sun. Before I went, I got Claude to spin me up a little Telegram bridge so that I could tell it what to do, 21 + while it's still running in safeties-off mode on my sacrificial VM. This was kind of fun - I got to just indulge thoughts as they came to me, and 22 + off it would go and do stuff. It was a bit limited in how it talked back to me, which wasn't by design but turned out to be nice for this sort of 23 + workflow. The downside is that I've now got a load of stuff to sift through - much of which is a 'good start', but none of it is likely to be usable 24 + without a good deal more effort. Here's a short-list of things I had it do: 25 + 26 + - Resurrect Fay Carson's work on the {{:https://github.com/ocaml/odoc/pull/1295}Menhir parser for odoc}, pushed {{:https://github.com/jonludlam/odoc/tree/menhir-parser-rebased}here} 27 + - Added some instrumentation to Odoc to do some performance experiments 28 + - Ran some simple experiments to measure the impact of various pre-existing performance knobs/switches 29 + - Resurrected an old patch of mine to {{:https://github.com/jonludlam/odoc/tree/parameterised-paths}unify the two path representations} in odoc to measure its effect on performance. 30 + - Tested aggresively reuse of records if their fields don't change during compile/link 31 + - Mixed up the {{:https://tangled.org/jon.recoil.org/odoc-scrollycode-extension}scrollycode backend} and the x-ocaml backend and stuck a playground on at each step 32 + - Unified the oxcaml/ocaml branches of {{:https://tangled.org/jon.recoil.org/js_top_worker}js_top_worker} and x-ocaml via cppo 33 + - Added oxcaml mode/layout annotations to odoc 34 + 35 + {1 OxCaml} 36 + I investigated the oxcaml docs build, which I had got working last week. Anil reported that it wasn't working for him, so I looked at the 37 + build I had and it definitely {i was} working. However, I was building on our machine Monteverde, which is a bit of a beast, so I checked the 38 + memory usage and it was enormous! I tried the build again on my 64 gig VM and it OOM'd. I'd noticed before that the [cmti] files for 39 + base, in particular [base__Container.cmti] were absolutely massive, and so had just assumed that the problem was that. Luke had also 40 + mentioned that some of the output from the template machinery was hidden. However, I had Claude look into this and it couldn't see any 41 + doc stop comments. So I asked it to look a little closer and figure out what was using all the memory. It took an unexpectedly large number 42 + of prods from me to finally figure out what was going on - it was to do with how odoc processes [includes] - specifically an [include sig ... end]. 43 + Essentially an include of that type ends up doubling the storage required of the signature. As the ppx_template extension does quite a lot 44 + of this, and in particular nests them, this ends up going exponential and this turned out to be the cause of most of the memory usage. 45 + With a fair bit more prodding by me, Claude and I eventually got to a solution, which I'll be upstreaming soon - the fix applies to OCaml as 46 + well as OxCaml, but it's this particularly pathalogical usage of includes that ppx_template uses where it'll make the most difference. 47 + 48 + {1 Odoc, plugins, JS and more} 49 + Teaser... I have a blog post coming soon with more on this. It's been a lot of fun, and 50 + should provide a decent inspiration for a roadmap for Odoc and online notebooks! 51 + 52 + 53 +

+3

site/blog/2026/03/index.mld

··· 0 0 0

··· 1 + {0 March} 2 + 3 + @children_order weeknotes-2026-09

site/blog/2026/03/mapdemo.mov

This is a binary file and will not be displayed.

site/blog/2026/03/search.png

This is a binary file and will not be displayed.

+29

site/blog/2026/03/weeknotes-2026-09.mld

··· 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

··· 1 + {0 Weeknotes 2026 week 9} 2 + 3 + @published 2026-03-02 4 + @notanotebook 5 + 6 + Let's make this really terse! 7 + 8 + {1 What did I do?} 9 + - Got docs working with github actions on Anil's oxmono monorepo. Results are {{:https://jonludlam.github.io/oxmono/}here}. 10 + This includes experimental support for oxcaml modes/layouts. 11 + - Got markdown mode output into Sherlodoc's db so you can query it - great for agents! 12 + {image!./search.png} 13 + - Widgets in the JS OCaml toplevels - using FRP for the interactions. The neat thing here is that using FRP via 14 + Daniel Bunzli's {{:}note} library is that all the interactions are all purely functional, no refs or mutables 15 + in sight. You provide a little wrapper scripts that's run in the frontend and the interactions and send back and 16 + forth with the worker running the code where it's translated into Events and Signals. My proof-of-concept of this 17 + is a widget that works with the {{:https://leafletjs.com/}leaflet.js} library: 18 + {video!mapdemo.mov} 19 + Demo coming soon! 20 + - Consolidating all of the Odoc toplevel bits and pieces into the one monorepo. Again, demo of this coming soon! 21 + 22 + {1 What am I going to do?} 23 + - New website! 24 + - Odoc plugins showcase 25 + - Writing writing writing writing 26 + 27 + 28 + 29 +

+4 -1

site/blog/2026/index.mld

··· 1 - @children_order 02/ 01/ 2 {0 2026} 3 4 0 0 0 5 - {{!/jon-site/blog/2026/02/page-"weeknotes-2026-06"}Weeknotes for week 6} 6 - {{!/jon-site/blog/2026/01/page-"weeknotes-2026-04-05"}Weeknotes for weeks 4-5} 7 - {{!/jon-site/blog/2026/01/page-"weeknotes-2026-03"}Weeknotes for week 3}

··· 1 + @children_order 03/ 02/ 01/ 2 {0 2026} 3 4 5 + - {{!/jon-site/blog/2026/03/page-"weeknotes-2026-09"}Weeknotes for week 9} 6 + - {{!/jon-site/blog/2026/02/page-"weeknotes-2026-08"}Weeknotes for weeks 7-8} 7 + - {{!/jon-site/blog/2026/02/page-"odoc-js-notebooks-fun"}Fun with odoc, plugins and javascript} 8 - {{!/jon-site/blog/2026/02/page-"weeknotes-2026-06"}Weeknotes for week 6} 9 - {{!/jon-site/blog/2026/01/page-"weeknotes-2026-04-05"}Weeknotes for weeks 4-5} 10 - {{!/jon-site/blog/2026/01/page-"weeknotes-2026-03"}Weeknotes for week 3}

+287

site/notebooks/interactive_map.mld

··· 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

··· 1 + {0 TESSERA Interactive Map} 2 + 3 + Explore geospatial embeddings from the 4 + {{:https://geotessera.org}GeoTessera} foundation model 5 + directly in the browser. This notebook walks through a complete 6 + land-cover classification workflow: 7 + 8 + {ol 9 + {- Draw a bounding box to select your region of interest} 10 + {- Fetch and visualise GeoTessera embeddings with PCA} 11 + {- Click on the map to place labelled training points} 12 + {- Run k-nearest-neighbours classification} 13 + {- View the classification overlay}} 14 + 15 + {@ocaml kind=setup[ 16 + #require "tessera-geotessera-jsoo";; 17 + #require "tessera-viz-jsoo";; 18 + #require "js_top_worker-widget-leaflet";; 19 + Widget_leaflet.register ();; 20 + ]} 21 + 22 + {1 Select region of interest} 23 + 24 + Draw a rectangle on the map to select the area you want to classify. 25 + The map is centred on Cambridge, UK — navigate to any area of interest 26 + before drawing. 27 + 28 + {@ocaml[ 29 + (* Shared state *) 30 + let bbox : Geotessera.bbox option ref = ref None 31 + let mosaic : (Linalg.mat * int * int) option ref = ref None 32 + let projected : Linalg.mat option ref = ref None 33 + let training_points : (float * float * int) list ref = ref [] 34 + let current_class = ref 0 35 + let class_names = ref [| "water"; "land" |] 36 + let class_colors = [| "#2196F3"; "#4CAF50"; "#FF9800"; "#9C27B0"; "#F44336"; 37 + "#00BCD4"; "#795548"; "#607D8B" |] 38 + 39 + (* Downsample a mosaic to at most max_pixels, preserving aspect ratio *) 40 + let downsample_mosaic mat ~h ~w ~max_pixels = 41 + let n_pixels = h * w in 42 + if n_pixels <= max_pixels then (mat, h, w) 43 + else 44 + let stride = int_of_float (ceil (sqrt (float_of_int n_pixels /. float_of_int max_pixels))) in 45 + let h' = (h + stride - 1) / stride in 46 + let w' = (w + stride - 1) / stride in 47 + let out = Linalg.create_mat ~rows:(h' * w') ~cols:mat.Linalg.cols in 48 + for i = 0 to h' - 1 do 49 + for j = 0 to w' - 1 do 50 + let si = min (i * stride) (h - 1) in 51 + let sj = min (j * stride) (w - 1) in 52 + let src = si * w + sj in 53 + let dst = i * w' + j in 54 + for f = 0 to mat.Linalg.cols - 1 do 55 + Linalg.mat_set out dst f (Linalg.mat_get mat src f) 56 + done 57 + done 58 + done; 59 + (out, h', w') 60 + 61 + let map_id = "tessera-map" 62 + 63 + let status_view text = 64 + let open Widget.View in 65 + Element { tag = "div"; attrs = [Style ("padding", "8px"); Style ("font-family", "monospace")]; 66 + children = [Text text] } 67 + 68 + let () = Widget.display ~id:"status" ~handlers:[] (status_view "Draw a rectangle on the map.") 69 + 70 + let () = 71 + Widget.display_managed ~id:map_id 72 + ~kind:"leaflet-map" 73 + ~config:{| {"center": [52.2, 0.12], "zoom": 13, "height": "500px"} |} 74 + ~handlers:[ 75 + "bbox_drawn", (fun v -> 76 + match v with 77 + | Some json -> 78 + (* Parse bbox JSON: {"south":..,"west":..,"north":..,"east":..} *) 79 + let s = Scanf.sscanf json 80 + {| {"south":%f,"west":%f,"north":%f,"east":%f}|} 81 + (fun s w n e -> Geotessera.{ min_lat = s; min_lon = w; max_lat = n; max_lon = e }) 82 + in 83 + bbox := Some s; 84 + Widget.update ~id:"status" 85 + (status_view (Printf.sprintf "Selected: %.4f,%.4f to %.4f,%.4f — run next cell to fetch embeddings." 86 + s.min_lat s.min_lon s.max_lat s.max_lon)) 87 + | None -> ()); 88 + "click", (fun v -> 89 + match v with 90 + | Some json -> 91 + let lat, lng = Scanf.sscanf json {| {"lat":%f,"lng":%f}|} (fun a b -> (a, b)) in 92 + (match !mosaic with 93 + | None -> () (* ignore clicks before embeddings loaded *) 94 + | Some _ -> 95 + let cls = !current_class in 96 + training_points := (lat, lng, cls) :: !training_points; 97 + let color = class_colors.(cls mod Array.length class_colors) in 98 + let label = (!class_names).(cls) in 99 + Widget.command ~id:map_id "addMarker" 100 + (Printf.sprintf {|{"lat":%f,"lng":%f,"color":"%s","label":"%s"}|} 101 + lat lng color label); 102 + Widget.update ~id:"status" 103 + (status_view (Printf.sprintf "Added %s point at %.4f, %.4f (%d points total)" 104 + label lat lng (List.length !training_points)))) 105 + | None -> ()); 106 + ] 107 + 108 + let () = Widget.command ~id:map_id "enableBboxDraw" "" 109 + ]} 110 + 111 + {1 Fetch embeddings and visualise} 112 + 113 + After drawing your bounding box, run this cell to fetch GeoTessera 114 + embeddings and display a PCA false-colour visualisation. 115 + 116 + {@ocaml[ 117 + let () = 118 + match !bbox with 119 + | None -> Widget.update ~id:"status" (status_view "Error: draw a bounding box first!") 120 + | Some b -> 121 + Widget.update ~id:"status" (status_view "Fetching embeddings..."); 122 + let mat_full, h_full, w_full = Geotessera_jsoo.fetch_mosaic ~year:2024 b in 123 + Widget.update ~id:"status" 124 + (status_view (Printf.sprintf "Fetched %d×%d mosaic. Downsampling..." h_full w_full)); 125 + let mat, h, w = downsample_mosaic mat_full ~h:h_full ~w:w_full ~max_pixels:50_000 in 126 + mosaic := Some (mat, h, w); 127 + (* Compute actual mosaic bounds from tile grid (not user bbox) *) 128 + let snap = Geotessera.snap_to_grid in 129 + let mosaic_south = snap b.min_lat -. 0.05 in 130 + let mosaic_north = snap b.max_lat +. 0.05 in 131 + let mosaic_west = snap b.min_lon -. 0.05 in 132 + let mosaic_east = snap b.max_lon +. 0.05 in 133 + Widget.update ~id:"status" 134 + (status_view (Printf.sprintf "Working at %d×%d (%d pixels). Computing PCA..." h w (h * w))); 135 + (* PCA to 3 components for RGB visualisation *) 136 + let pca = Linalg.pca_fit ~max_samples:5000 mat ~n_components:3 in 137 + let proj = Linalg.pca_transform pca mat in 138 + projected := Some proj; 139 + let pca_img = Viz.pca_to_rgba ~width:w ~height:h proj in 140 + let url = Viz_jsoo.to_data_url pca_img in 141 + Widget.command ~id:map_id "addImageOverlay" 142 + (Printf.sprintf {|{"url":"%s","bounds":[[%f,%f],[%f,%f]],"opacity":0.7}|} 143 + url mosaic_south mosaic_west mosaic_north mosaic_east); 144 + Widget.update ~id:"status" 145 + (status_view "PCA overlay added. Click on the map to place training points, then run the classification cell.") 146 + ]} 147 + 148 + {1 Label training points} 149 + 150 + Click on the map to add training points. Use the buttons below to 151 + switch between classes before clicking. 152 + 153 + {@ocaml[ 154 + let make_class_buttons () = 155 + let open Widget.View in 156 + let buttons = Array.to_list (Array.mapi (fun i name -> 157 + let color = class_colors.(i mod Array.length class_colors) in 158 + let border = if i = !current_class then "3px solid black" else "1px solid #ccc" in 159 + Element { tag = "button"; 160 + attrs = [ 161 + Handler ("click", "class_" ^ string_of_int i); 162 + Style ("margin", "4px"); 163 + Style ("padding", "8px 16px"); 164 + Style ("background", color); 165 + Style ("color", "white"); 166 + Style ("border", border); 167 + Style ("border-radius", "4px"); 168 + Style ("cursor", "pointer"); 169 + ]; 170 + children = [Text name] } 171 + ) !class_names) in 172 + Element { tag = "div"; 173 + attrs = [Style ("padding", "8px")]; 174 + children = 175 + Element { tag = "b"; attrs = []; children = [Text "Active class: "] } 176 + :: buttons 177 + @ [Element { tag = "button"; 178 + attrs = [ 179 + Handler ("click", "add_class"); 180 + Style ("margin", "4px"); 181 + Style ("padding", "8px 16px"); 182 + Style ("border", "1px dashed #999"); 183 + Style ("border-radius", "4px"); 184 + Style ("cursor", "pointer"); 185 + ]; 186 + children = [Text "+ Add class"] }; 187 + Element { tag = "button"; 188 + attrs = [ 189 + Handler ("click", "clear_points"); 190 + Style ("margin", "4px"); 191 + Style ("padding", "8px 16px"); 192 + Style ("border", "1px solid #ccc"); 193 + Style ("border-radius", "4px"); 194 + Style ("cursor", "pointer"); 195 + ]; 196 + children = [Text "Clear all"] }] 197 + } 198 + 199 + let class_handler_list () = 200 + let base = Array.to_list (Array.mapi (fun i _name -> 201 + "class_" ^ string_of_int i, (fun (_v : string option) -> 202 + current_class := i; 203 + Widget.update ~id:"class-buttons" (make_class_buttons ())) 204 + ) !class_names) in 205 + base @ [ 206 + "add_class", (fun _v -> 207 + let n = Array.length !class_names in 208 + let name = "class_" ^ string_of_int n in 209 + class_names := Array.append !class_names [| name |]; 210 + current_class := n; 211 + Widget.update ~id:"class-buttons" (make_class_buttons ())); 212 + "clear_points", (fun _v -> 213 + training_points := []; 214 + Widget.command ~id:map_id "clearMarkers" ""; 215 + Widget.update ~id:"status" (status_view "Cleared all training points.")); 216 + ] 217 + 218 + let () = 219 + Widget.display ~id:"class-buttons" 220 + ~handlers:(class_handler_list ()) 221 + (make_class_buttons ()) 222 + ]} 223 + 224 + {1 Classify and display} 225 + 226 + Run this cell after placing training points to classify the entire 227 + region using k-nearest neighbours. 228 + 229 + {@ocaml[ 230 + let () = 231 + match !mosaic, !projected, !bbox with 232 + | Some (mat, h, w), Some _proj, Some b -> 233 + let points = !training_points in 234 + if points = [] then 235 + Widget.update ~id:"status" (status_view "Error: place some training points first!") 236 + else begin 237 + Widget.update ~id:"status" 238 + (status_view (Printf.sprintf "Classifying with %d training points..." (List.length points))); 239 + (* Compute actual mosaic bounds from tile grid *) 240 + let snap = Geotessera.snap_to_grid in 241 + let mosaic_south = snap b.min_lat -. 0.05 in 242 + let mosaic_north = snap b.max_lat +. 0.05 in 243 + let mosaic_west = snap b.min_lon -. 0.05 in 244 + let mosaic_east = snap b.max_lon +. 0.05 in 245 + (* Convert geo coords to pixel coords and extract embeddings *) 246 + let n_train = List.length points in 247 + let train_mat = Linalg.create_mat ~rows:n_train ~cols:mat.Linalg.cols in 248 + let train_labels = Array.make n_train 0 in 249 + List.iteri (fun i (lat, lng, cls) -> 250 + (* Map lat/lng to pixel row/col using mosaic bounds *) 251 + let row = int_of_float ((mosaic_north -. lat) /. (mosaic_north -. mosaic_south) *. float_of_int h) in 252 + let col = int_of_float ((lng -. mosaic_west) /. (mosaic_east -. mosaic_west) *. float_of_int w) in 253 + let row = max 0 (min (h - 1) row) in 254 + let col = max 0 (min (w - 1) col) in 255 + (* Copy embedding row *) 256 + let src_offset = (row * w + col) * mat.Linalg.cols in 257 + let dst_offset = i * mat.Linalg.cols in 258 + for j = 0 to mat.Linalg.cols - 1 do 259 + Bigarray.Array1.set train_mat.Linalg.data (dst_offset + j) 260 + (Bigarray.Array1.get mat.Linalg.data (src_offset + j)) 261 + done; 262 + train_labels.(i) <- cls 263 + ) points; 264 + (* kNN classification *) 265 + let model = Linalg.knn_fit ~embeddings:train_mat ~labels:train_labels in 266 + let k = min 5 n_train in 267 + let result = Linalg.knn_predict model ~k mat in 268 + (* Build color map *) 269 + let n_classes = Array.fold_left max 0 train_labels + 1 in 270 + let colors = List.init n_classes (fun i -> 271 + let hex = class_colors.(i mod Array.length class_colors) in 272 + (i, Viz.color_of_hex hex) 273 + ) in 274 + let class_img = Viz.classification_to_rgba 275 + ~predictions:result.Linalg.predictions 276 + ~colors ~width:w ~height:h () in 277 + let class_url = Viz_jsoo.to_data_url class_img in 278 + Widget.command ~id:map_id "addImageOverlay" 279 + (Printf.sprintf {|{"url":"%s","bounds":[[%f,%f],[%f,%f]],"opacity":0.7}|} 280 + class_url mosaic_south mosaic_west mosaic_north mosaic_east); 281 + Widget.update ~id:"status" 282 + (status_view (Printf.sprintf "Classification complete! %d classes, %d training points." 283 + n_classes n_train)) 284 + end 285 + | _ -> 286 + Widget.update ~id:"status" (status_view "Error: fetch embeddings first!") 287 + ]}