+2

scripts/atom.xml

··· 1 1 + <?xml version="1.0" encoding="UTF-8"?> 2 2 + <feed xmlns="http://www.w3.org/2005/Atom"><id>https://jon.recoil.org/atom.xml</id><title type="text">Jon's blog</title><updated>2025-04-24T15:41:17-00:00</updated></feed>

+9

scripts/dune

··· 1 1 + (executable 2 2 + (name gen_atom) 3 3 + (preprocess 4 4 + (pps ppx_deriving_yojson)) 5 5 + (libraries syndic uri unix ptime bos odoc.odoc yojson ISO8601)) 6 6 + 7 7 + (executable 8 8 + (name gen_blog_index) 9 9 + (libraries str))

+622

scripts/feed

··· 1 1 + <?xml version="1.0" encoding="UTF-8"?> 2 2 + <feed xmlns="http://www.w3.org/2005/Atom"> 3 3 + <contributor> 4 4 + <name>Jon Sterling</name> 5 5 + <uri>https://www.forester-notes.org/jonmsterling/</uri> 6 6 + </contributor> 7 7 + <updated>2025-03-25T13:44:56Z</updated> 8 8 + <title>Forester Blog</title> 9 9 + <id>https://www.forester-notes.org/30FM/</id> 10 10 + <link rel="alternate" href="https://www.forester-notes.org/30FM/" /> 11 11 + <link rel="self" href="https://www.forester-notes.org/30FM/atom.xml" /> 12 12 + <entry> 13 13 + <title>Towards Forester 5.0 II: a design for canonical URLs</title> 14 14 + <published>2025-03-25T13:44:56Z</published> 15 15 + <updated>2025-03-25T13:44:56Z</updated> 16 16 + <author> 17 17 + <name>Jon Sterling</name> 18 18 + <uri>https://www.forester-notes.org/jonmsterling/</uri> 19 19 + </author> 20 20 + <link rel="alternate" type="text/html" href="https://www.forester-notes.org/JVIT/" /> 21 21 + <id>https://www.forester-notes.org/JVIT/</id> 22 22 + <content type="xhtml"> 23 23 + <div xmlns="http://www.w3.org/1999/xhtml"> 24 24 + <p>One of the goals of <a href="https://www.forester-notes.org/jms-011P/">Forester 5.0</a> is <em>lightweight federation</em>—the ability to have two forests participate in the same graph and therefore provide backlinks, etc. In a previous post (<a href="https://www.forester-notes.org/OYOJ/">Towards <a href="https://www.forester-notes.org/jms-011P/">Forester 5.0</a>: a design for global identity</a>), I talked about some of the difficulties that arise when dealing with identities of people and references that have global scope but could nonetheless be described by trees in many forests. I proposed that such things should be addressed by canonical URIs (e.g. DIDs, DOIs, etc.) and that Forester should grow the ability to bind a canonical URI to multiple trees, which are then gathered into a disambiguation page.</p> 25 25 + <p>Today I want to broaden the discussion to cover the difficulties of addressing trees themselves (as opposed to the global entities they may describe). <mark>This is a proposal and I welcome feedback.</mark></p> 26 26 + <section> 27 27 + <header> 28 28 + <h2>Forester must become part of the Web</h2> 29 29 + </header> 30 30 + <p>I have been working on developing the prerequisites for Forester to emit RSS and Atom feeds for blogs, and I realised that the problem I was trying to solve <a href="https://www.forester-notes.org/OYOJ/">earlier this month</a> is a more multifaceted than I originally thought. It comes down to analysing what is needed for Forester to be a good citizen of the World Wide Web: in particular, <mark>if we emit an RSS feed that has hyperlinks to some trees in it, those links <em>must</em> refer to an actual page on the actual web rather than something specific to Forester’s ontology.</mark></p> 31 31 + <p>This may seem downright obvious in hindsight, but you must understand that for the longest time I was not thinking of Forester as a tool for progressively enhancing the Web, but rather as a tool for building fully-local life-wikis or Zettelkästen; I no longer believe that my former viewpoint is reasonable, and I have concluded that we must integrate Forester into the Web or else we will be buried under friction. This post is the start of a design for how to do this.</p> 32 32 + <p>Forget what you know about how either <a href="https://www.forester-notes.org/jms-011P/">Forester 5.0</a> or previous versions currently work; in order to solve these problems in a reasonable way, we cannot be bound by the past versions of an experimental tool. What we <em>are</em> bound by is the architecture of the World Wide Web, and that will be reflected in the design.</p> 33 33 + </section> 34 34 + <section> 35 35 + <header> 36 36 + <h2>What is the proposal?</h2> 37 37 + </header> 38 38 + <p>Here is the essence of the proposal:</p> 39 39 + <ol><li>We get rid of the <code>forest://host/addr</code> scheme. Instead, trees are globally addressed by a canonical URL.</li> 40 40 + <li>The canonical URL of a tree can in principle be arbitrary, but in practice you will want it to be a place where that tree can be viewed — e.g. the place to which it will be uploaded and served via HTTP(S). Indeed, a default scheme will be provided so as to enable files to be rendered with names and relative locations consistent with the intended global addressing scheme; it is also possible to imagine customisation of this without disturbing the overall design.</li> 41 41 + <li>The canonical URLs are now the vertices of the graph.</li> 42 42 + <li>In Forester source code, a hyperlink like <code><]]></code> would be resolved right away to <code><]]></code> or something, using information supplied in the user’s forest; the same goes for transclusion.</li> 43 43 + 44 44 + <li>Links to trees in foreign forests must, for now, be totally explicit (but we can imagine relaxing this in the future). Importantly, this approach does not require knowing what is in the forest at evaluation-time.</li></ol> 45 45 + </section> 46 46 + <section> 47 47 + <header> 48 48 + <h2>What about replication and mirroring?</h2> 49 49 + </header> 50 50 + <p>It may seem annoying to have canonical URLs. For example, a forest that contains vital information might need to be published in multiple places. That much is true, but the fact that the physical publication of a forest is replicated should not allowed to impact the graph or fill it with redundant vertices and edges (e.g. should two mirrors become federated). So the only problem with replication is that hyperlinks might take you to the original forest instead of keeping you in the mirror, but I think this should be resolved by some kind of middleware that rewrites links, just as the <a href="https://web.archive.org/">Wayback machine</a> rewrites links in its snapshots. That can be handled outside of Forester.</p> 51 51 + </section> 52 52 + <section> 53 53 + <header> 54 54 + <h2>What about viewing my forest locally?</h2> 55 55 + </header> 56 56 + <p>Most of the time, an author is working with their forest on their own machine rather than on the web. It is important that links and transclusions point to the local content rather than whatever (if anything) is stored in the “global” canonical URL. I believe this is not actually a problem: although things like RSS feeds and perhaps even published websites would have all the hyperlinks point to the canonical URLs, there is no reason that this should be required for all renderers. It is easy to imagine making this a configurable flag for the default renderer, and for the upcoming “dynamic”/interactive HTML server we would emit links back to the local server rather than to the canonical URLs.</p> 57 57 + <p>Similarly, there may be projects where there is no intention at all of online publication. In such cases, the scheme for assigning canonical URLs can be arbitrary.</p> 58 58 + </section> 59 59 + <section> 60 60 + <header> 61 61 + <h2>What about access control?</h2> 62 62 + </header> 63 63 + <p>Forester does not currently support any kind of access control, but this is indeed an important area that we are considering carefully in order to enable institutional use of Forester, and ease the burden of collaboration in the usual case of a forest that contains a mixture of data with varying levels of confidentiality. I believe that the current design is compatible with essentially any approach to access control that we might adopt, but I am interested in feedback to the contrary.</p> 64 64 + </section> 65 65 + </div> 66 66 + </content> 67 67 + </entry> 68 68 + <entry> 69 69 + <title>Towards Forester 5.0: a design for global identity</title> 70 70 + <published>2025-03-08T12:17:14Z</published> 71 71 + <updated>2025-03-08T12:17:14Z</updated> 72 72 + <author> 73 73 + <name>Jon Sterling</name> 74 74 + <uri>https://www.forester-notes.org/jonmsterling/</uri> 75 75 + </author> 76 76 + <link rel="alternate" type="text/html" href="https://www.forester-notes.org/OYOJ/" /> 77 77 + <id>https://www.forester-notes.org/OYOJ/</id> 78 78 + <content type="xhtml"> 79 79 + <div xmlns="http://www.w3.org/1999/xhtml"> 80 80 + <p>As we move closer to <a href="https://www.forester-notes.org/jms-011P/">Forester 5.0</a>, which <a href="https://www.forester-notes.org/30FN/">introduces</a> rudimentary federation capabilities, we must address new problems that did not arise in the days when no two <a href="https://www.forester-notes.org/tfmt-000R/">forests</a> interacted or linked to each other. The most immediate issue is that trees describing entities with “global” identity (including actual people as well as bibliographic references) will naturally be duplicated across many forests. For example, this happens when one person authors trees in multiple forests, and it happens even more often with bibliographic entries (both for the entries themselves and their author attributions). It is very important to handle this problem properly <em>now</em> in a way that (1) minimises friction and (2) enables us to quietly evolve toward <a href="https://www.forester-notes.org/klepmann-et-al-atproto-2024/">more Web-centric approaches to identity as they emerge</a>.</p> 81 81 + <p>Below, I survey some existing approaches to identity that we would hope to be compatible with at some level. If you want to skip to my concrete proposal, see <a href="https://www.forester-notes.org/OYOR/">§ OYOR/</a>.</p> 82 82 + <section> 83 83 + <header> 84 84 + <h2>Survey of global identification schemes</h2> 85 85 + </header> 86 86 + <p>There are several extant schemes for identifying individuals, organisations, and artefacts. Some are centralised, and others are decentralised. Centralisation of identity is not necessarily a bad thing, but it is most viable when nearly everyone agrees on the central authority; on the other hand, decentralisation can help in situations where a single central authority has not accumulated enough trust or prestige to be viable.</p> 87 87 + <section> 88 88 + <header> 89 89 + <h3>Centralised identification via DOIs and ORCIDs</h3> 90 90 + </header> 91 91 + <p>Nearly every scholarly paper and book published has a <em>Digital Object Identifier</em> (DOI) assigned to it, which are managed by a single authority (<a href="https://www.doi.org/">The DOI Foundation</a>); this applies to both traditional publishers and eprint servers like the arXiv. Services like <a href="https://zenodo.org/">Zenodo</a> allow individuals to mint their own DOIs and pin resources and artefacts to them. Due to their widespread adoption, DOIs are a completely viable way to identify published papers and books—and I would argue that any attempt to replace DOIs with a decentralised identifier is likely to be counterproductive as the goal should not not be decentralisation <em>per se</em> but rather to have a reliable, universal way to refer to scholarly content and artefacts.</p> 92 92 + <p>What DOIs do for artefacts, the <em>Open Researcher and Contributor ID</em> (ORCID) aims to do for <em>people</em> acting within the framework of open science. ORCIDs seem to do their job well, but not everyone has or should have an ORCID—nor would every person who does have one voluntarily choose to pin their entire identity to it. Therefore, although I happily use them, I think ORCIDs are likely to face more of an uphill battle than the DOI—which needed buy-in only from major publishers and eprint servers to reach hegemony.</p> 93 93 + </section> 94 94 + <section> 95 95 + <header> 96 96 + <h3>Informal decentralised identification via web addresses</h3> 97 97 + </header> 98 98 + <p>A particularly simple way to identify a single person or organisation is by means of a web domain or an email address. Although not everyone has a domain name, many people have email addresses. On the other hand, people often have many domain names and their email address may change over time; and when people die, their presence on the web is often erased or lost. Therefore, although widespread, this approach may create difficulties with longevity and stability.</p> 99 99 + </section> 100 100 + <section> 101 101 + <header> 102 102 + <h3>General-purpose decentralised identification via DIDs</h3> 103 103 + </header> 104 104 + <p>When reading about the paper of <a href="https://www.forester-notes.org/klepmann-et-al-atproto-2024/">Klepmann et al.</a> outlining Bluesky’s AT Protocol, I learned of <a href="https://www.w3.org/TR/did-1.0/"><em>Decentralised Identifiers</em> (DIDs)</a>. In essence, DIDs are URIs of the form <code>did:method:path</code> where <code>method</code> identifies <em>how</em> the DID is intended to be resolved and <code>path</code> is a colon-separated path that should be resolved by means of that method. In either case, a DID is intended to be resolved to a JSON document that contains information about the resource or entity being described, as well as various methods (like public keys) for verifying the integrity of that information. The <em>methods</em> are somewhat open-ended, but two important methods have emerged.</p> 105 105 + <section> 106 106 + <header> 107 107 + <h4>W3C’s <code>did:web</code> method</h4> 108 108 + </header> 109 109 + <p><a href="https://w3c-ccg.github.io/did-method-web/">W3C have specified the <code>did:web</code> method</a>, which in which the <code>path</code> is intended to be a web domain. Simplifying somewhat, a DID like <code>did:web:jonmsterling.com</code> would be substantiated by responding to the HTTPS request <code>https://jonmsterling.com/.well-known/did.json</code> with a document in the appropriate format. The upside is that the owner of a web domain is <em>their own</em> identity authority; in this sense <code>did:web</code> is a truly decentralised identification scheme. The downside is that you have to have a web domain, and you also can never change it ever—the <a href="https://www.forester-notes.org/OYOK/">same disadvantage of informal decentralised identification which we have discussed</a>.</p> 110 110 + </section> 111 111 + <section> 112 112 + <header> 113 113 + <h4>Bluesky’s <code>did:plc</code> method</h4> 114 114 + </header> 115 115 + <p>For a social network like Bluesky, it is critical that users be able to migrate their identity from one domain to another. Obviously users may change or lose their domain over time, but it is important to keep in mind that the vast majority of users will <em>never</em> have their own domain and so they will over the course of their lives jump from one subsidiary domain that they don’t control to the next—just as Mastodon users are constantly migrating from instance to instance, driven to wander endlessly by either the petty tyranny of instance maintainers who think they know best, or by the natural quiescence of instances caused by lack of funds or time, or (in many cases) a combination of the two.</p> 116 116 + <p>It seems that there is no way to address this problem without introducing some central authority—a <em>directory</em> of permanent identifiers that are then resolved to documents that establish cryptographically verified bidirectional links with more ephemeral and human-readable forms of identification (such as web domains). This is essentially the design of Bluesky’s <code>did:plc</code> method, as explained by <a href="https://www.forester-notes.org/klepmann-et-al-atproto-2024/">Klepmann et al.</a>:</p> 117 117 + <ol><li>On your own domain, which serves as your (ephemeral) handle, you place a DNS TXT record or file that contains a DID like <code>did:plc:asdlkfh9q8034baliufhbcailurb</code>.</li> 118 118 + <li>Someone resolves this DID to a document by querying a central <em>directory server</em> (such as <a href="https://web.plc.directory/">Bluesky’s own</a>). This document contains a link back to the domain; signatures are used to ensure that every update to the document has been authorised by whoever signed it when it was first minted.</li></ol> 119 119 + <p>Although some centralisation is required here, the use of cryptographic proof ensures that the central authority does not need to be trusted (to a certain extent).</p> 120 120 + </section> 121 121 + </section> 122 122 + </section> 123 123 + <section> 124 124 + <header> 125 125 + <h2>Analysis of global identity in Forester</h2> 126 126 + </header> 127 127 + <p>Although Forester aims to become a better citizen of the Web and integrate with emerging protocols, it is a non-negotiable design constraint that Forester still remain usable by people who don’t control a domain name, cannot run software on their web host, cannot set DNS records, and could not care less what a <a href="https://www.forester-notes.org/OYON/">DID</a> is. I also have a feeling that there will not be a single protocol that fits all use cases; what I am noticing, however, is that there are commonalities to all the protocols, and that we ought to be informed by these commonalities. For example, in every case an identity is resolved from a URI of some kind—for example, DIDs and DOIs and ORCIDs all have canonical URIs.</p> 128 128 + <p>Therefore, it strikes me that Forester’s approach to global identity must rest on the axiom that an identity is nothing more or less than a URI; we can place no constraints whatsoever on what form this URI takes, and we should also remain flexible as to compatibility with future replacements of URIs (whether in the form of IRIs, or the <a href="https://www.forester-notes.org/jms-011R/">URL non-“standard”</a>, etc.).</p> 129 129 + <p>If we start from that point of view, there some problems to address:</p> 130 130 + <ol><li>Even if an identity is not canonically addressed by a tree in a forest, an identity still often needs to <em>have</em> a tree in the forest. One wants to store biographical and bibliographic information, and maybe even personal notes, etc., and at the very least it is very important to be able to browse backlinks on a biographical page even if the page itself has no content of its own.</li> 131 131 + <li>Not only must we be able to attach a tree to an identity: we must be able to attach <em>many</em> trees to an identity. This is a requirement of federation.</li></ol> 132 132 + </section> 133 133 + <section> 134 134 + <header> 135 135 + <h2>A plan for global identity in Forester</h2> 136 136 + </header> 137 137 + <p>Building on my <a href="https://www.forester-notes.org/OYOQ/">analysis</a>, I propose that Forester allow any tree to declare that it “describes” a given global identity in the form of a URI. At a first cut this can be done via datalog (but we would probably hide this behind something):</p> 138 138 + <pre><![CDATA[\execute\datalog{ 139 139 + -: {\rel/describes \current-tree @{https://orcid.org/0000-0002-0585-5564}} 140 140 + }]]></pre> 141 141 + <p>Now it remains to explain how we shall surface the fact that a given entity is described by some tree.</p> 142 142 + <ol><li>For any identity in this relation, we should automatically create a “disambiguation page” that transcludes all the attached trees.</li> 143 143 + <li>When a hyperlink points to a URI that lies in this relation, it should be directed to the disambiguation page.</li></ol> 144 144 + <p>There are further implications for such a feature—for instance, in the future we might automatically populate bibliographic information, etc. (But we have to be careful due to the near-universal unusably low quality of bibliographic databases keyed by DOIs, etc.)</p> 145 145 + <p>We will need to provide guidance as to how identities should be assigned to (e.g.) people who don’t control an online identity, etc. The rule of thumb should be that we always defer to the preferences of the described person, and to the version of record in the case of an artefact. When there is no canonical choice, users of Forester should do what they like, but they should be willing to update their references in the future should a canonical global entity emerge.</p> 146 146 + </section> 147 147 + <section> 148 148 + <header> 149 149 + <h2>Request for comment</h2> 150 150 + </header> 151 151 + <p>I am hoping to hear other people’s thoughts on this proposal, including any constructive criticisms or suggestions for how we might go about implementing it. You can write to me or the <a href="mailto:~jonsterling/forester-discuss@lists.sr.ht">mailing list</a> with your feedback.</p> 152 152 + </section> 153 153 + </div> 154 154 + </content> 155 155 + </entry> 156 156 + <entry> 157 157 + <title>Build your own Stacks Project in 10 minutes</title> 158 158 + <published>2023-05-14T00:00:00Z</published> 159 159 + <updated>2024-04-25T00:00:00Z</updated> 160 160 + <author> 161 161 + <name>Jon Sterling</name> 162 162 + <uri>https://www.forester-notes.org/jonmsterling/</uri> 163 163 + </author> 164 164 + <link rel="alternate" type="text/html" href="https://www.forester-notes.org/jms-0052/" /> 165 165 + <id>https://www.forester-notes.org/jms-0052/</id> 166 166 + <content type="xhtml"> 167 167 + <div xmlns="http://www.w3.org/1999/xhtml"> 168 168 + <p><a href="https://www.forester-notes.org/stacks-project/">The Stacks project</a> is the most successful scientific hypertext project in history. Its goal is to lay the foundations for the theory of algebraic stacks; to facilitate its scalable and sustainable development, several important innovations have been introduced, with the <em>tags</em> system being the most striking.</p> 169 169 + <blockquote>Each tag refers to a unique item (section, lemma, theorem, etc.) in order for this project to be referenceable. These tags don't change even if the item moves within the text. (<a href="https://stacks.math.columbia.edu/tags">Tags explained</a>, <a href="https://www.forester-notes.org/stacks-project/">The Stacks Project</a>).</blockquote> 170 170 + <p>Many working scientists, students, and hobbyists have wished to create their own tag-based hypertext knowledge base, but the combination of tools historically required to make this happen are extremely daunting. Both the <a href="https://www.forester-notes.org/stacks-project/">Stacks project</a> and <a href="https://www.forester-notes.org/kerodon/">Kerodon</a> use a cluster of software called <a href="https://www.forester-notes.org/gerby/">Gerby</a>, but bitrot has set in and it is <a href="https://github.com/gerby-project/plastex/issues/60">no longer possible</a> to build its dependencies on a modern environment without significant difficulty, raising questions of longevity.</p> 171 171 + <p>Moreover, <a href="https://www.forester-notes.org/gerby/">Gerby</a>’s deployment involves running a database on a server (in spite of the fact that almost the entire functionality is static HTML), an architecture that is incompatible with the constraints of the everyday working scientist or student who knows <em>at most</em> how to upload static files to their university-provided public storage. The recent experience of the <a href="https://ncatlab.org/nlab/show/HomePage">nLab</a>’s pandemic-era hiatus and near death experience has demonstrated with some urgency the pracarity faced by any project relying heavily on volunteer system administrators.</p> 172 172 + <section> 173 173 + <header> 174 174 + <h2>Introducing <a href="https://www.forester-notes.org/index/"><em>Forester</em></a>: a <a href="https://www.forester-notes.org/tfmt-0002/">tool for scientific thought</a></h2> 175 175 + </header> 176 176 + <p>After spending two years exploring the <a href="https://www.forester-notes.org/tfmt-0001/">design of tools for scientific thought</a> that meet the unique needs of real, scalable scientific writing in hypertext, I have created a tool called <strong><a href="https://www.forester-notes.org/index/">Forester</a></strong> which has the following benefits:</p> 177 177 + <ol><li><a href="https://www.forester-notes.org/index/">Forester</a> is tag-based like <a href="https://www.forester-notes.org/gerby/">Gerby</a>, and can therefore power large-scale generational projects like <a href="https://www.forester-notes.org/stacks-project/">Stacks</a> and <a href="https://www.forester-notes.org/kerodon/">Kerodon</a>.</li> 178 178 + <li><a href="https://www.forester-notes.org/index/">Forester</a> produces static content that <a href="https://www.forester-notes.org/jms-007R/">can be uploaded to any web hosting service</a> without needing to run or install any serverside software.</li> 179 179 + <li><a href="https://www.forester-notes.org/index/">Forester</a> is <a href="https://www.forester-notes.org/jms-006W/">easy to install</a> on your own machine.</li> 180 180 + <li>To prevent bitrot, <a href="https://www.forester-notes.org/index/">Forester</a> is a single tool rather than a composition of several tools.</li> 181 181 + <li><a href="https://www.forester-notes.org/index/">Forester</a> satisfies <em>all</em> the <a href="https://www.forester-notes.org/tfmt-000E/">requirements of serious scientific writing</a>, including sophisticated notational macros, typesetting of diagrams, etc.</li></ol> 182 182 + <p><a href="https://www.forester-notes.org/index/">Forester</a> combines <a href="https://www.forester-notes.org/tfmt-0005/">associative and hierarchical</a> networks of <a href="https://www.forester-notes.org/tfmt-0003/">evergreen notes</a> (called “trees”) into hypertext sites called “forests”.</p> 183 183 + <section> 184 184 + <header> 185 185 + <h3>Forests and trees of evergreen notes</h3> 186 186 + </header> 187 187 + <p>A <em>forest of <a href="https://www.forester-notes.org/tfmt-0003/">evergreen notes</a></em> (or a <em>forest</em> for short) is loosely defined to be a collection of <a href="https://www.forester-notes.org/tfmt-0003/">evergreen notes</a> in which multiple <a href="https://www.forester-notes.org/tfmt-0005/">hierarchical structures</a> are allowed to emerge and evolve over time. Concretely, one note may contextualize several other notes via transclusion within its textual structure; in the context of a forest, we refer to an individual note as a <em>tree</em>. Of course, a tree can be viewed as a forest that has a root node.</p> 188 188 + </section> 189 189 + <p>Trees correspond roughly to what are referred to as “tags” in the <a href="https://www.forester-notes.org/stacks-project/">Stacks Project</a>.</p> 190 190 + </section> 191 191 + <p>In this article, I will show you how to set up your own <a href="https://www.forester-notes.org/tfmt-000R/">forest</a> using the <a href="https://www.forester-notes.org/index/">Forester</a> software. <mark>These instructions pertain to the <a href="https://www.forester-notes.org/jms-00S9/">Forester 4.1</a> version.</mark></p> 192 192 + <section> 193 193 + <header> 194 194 + <h2>Preparing to run the <a href="https://www.forester-notes.org/index/">Forester</a> software</h2> 195 195 + </header> 196 196 + <p>In this section, we will walk through the installation of the <a href="https://www.forester-notes.org/index/">Forester</a> software.</p> 197 197 + <section> 198 198 + <header> 199 199 + <h3>System requirements of <a href="https://www.forester-notes.org/index/">Forester</a></h3> 200 200 + </header> 201 201 + <section> 202 202 + <header> 203 203 + <h4>A unix-based system</h4> 204 204 + </header> 205 205 + <p><a href="https://www.forester-notes.org/index/">Forester</a> requires a unix-based system to run; it has been tested on both macOS and Linux. Windows support is <a href="https://todo.sr.ht/~jonsterling/forester/6">desirable</a>, but there are no concrete plans to implement it at this time.</p> 206 206 + </section> 207 207 + <section> 208 208 + <header> 209 209 + <h4>A working OCaml 5 installation</h4> 210 210 + </header> 211 211 + <p><a href="https://www.forester-notes.org/index/">Forester</a> is written in the <a href="https://www.ocaml.org/">OCaml</a> programming language, and makes use of the latest features of OCaml 5. Most users should install <a href="https://www.forester-notes.org/index/">Forester</a> through OCaml's <a href="https://opam.ocaml.org/">opam</a> package manager; instructions to install opam and OCaml simultaneously can be found <a href="https://opam.ocaml.org/">here</a>.</p> 212 212 + </section> 213 213 + <section> 214 214 + <header> 215 215 + <h4>A working <code>\LaTeX </code> installation</h4> 216 216 + </header> 217 217 + <p>If you intend to <a href="https://www.forester-notes.org/tfmt-000L/">embed <code>\LaTeX </code>-rendered diagrams</a> in your <a href="https://www.forester-notes.org/tfmt-000R/">forest</a>, you will need to have a working installation of <code>\LaTeX </code> installed, such as <a href="https://tug.org/texlive/">TeX Live</a>. If all your mathematical expressions are supported by <a href="https://katex.org/"><code>\KaTeX </code></a>, this is not necessary.</p> 218 218 + </section> 219 219 + <section> 220 220 + <header> 221 221 + <h4>The <a href="https://git-scm.com/">git</a> distributed version control system</h4> 222 222 + </header> 223 223 + <p>It is best practice to maintain your <a href="https://www.forester-notes.org/tfmt-000R/">forest</a> inside of <a href="https://en.wikipedia.org/wiki/Distributed_version_control">distributed version control</a>. This serves not only as a way to prevent data loss (because you will be pushing frequently to a remote repository); it also allows you to easily roll back to an earlier version of your <a href="https://www.forester-notes.org/tfmt-000R/">forest</a>, or to create “branches” in which you prepare <a href="https://www.forester-notes.org/tfmt-000R/">trees</a> that are not yet ready to be integrated into the <a href="https://www.forester-notes.org/tfmt-000R/">forest</a>.</p> 224 224 + <p>The recommended <a href="https://en.wikipedia.org/wiki/Distributed_version_control">distributed version control</a> system is <a href="https://git-scm.com/">git</a>, which comes preinstalled on many unix-based systems and is easy to install otherwise. <a href="https://git-scm.com/">Git</a> is not the most user-friendly piece of software, unfortunately, but it is ubiquitous. It is possible (but not recommended) to use <a href="https://www.forester-notes.org/index/">Forester</a> without version control, but note that the simplest way to <a href="https://www.forester-notes.org/jms-006X/">initialize your own <a href="https://www.forester-notes.org/tfmt-000R/">forest</a></a> involves cloning a <a href="https://git-scm.com/">git</a> repository.</p> 225 225 + </section> 226 226 + </section> 227 227 + <section> 228 228 + <header> 229 229 + <h3>Installing the <a href="https://www.forester-notes.org/index/">Forester</a> software</h3> 230 230 + </header> 231 231 + <p>Once you have met the <a href="https://www.forester-notes.org/jms-006S/">system requirements</a>, installing <a href="https://www.forester-notes.org/index/">Forester</a> requires only a single shell command:</p> 232 232 + <pre><![CDATA[opam install forester]]></pre> 233 233 + <p>To verify that <a href="https://www.forester-notes.org/index/">Forester</a> is installed, please run <code>forester --version</code> in your shell.</p> 234 234 + </section> 235 235 + </section> 236 236 + <section> 237 237 + <header> 238 238 + <h2>Setting up your <a href="https://www.forester-notes.org/tfmt-000R/">forest</a> from the template</h2> 239 239 + </header> 240 240 + <p>Now that you have <a href="https://www.forester-notes.org/jms-006W/">installed</a> the <a href="https://www.forester-notes.org/index/">Forester</a> software, it is time to set up your <a href="https://www.forester-notes.org/tfmt-000R/">forest</a>. <a href="https://www.forester-notes.org/index/">Forester</a> provides a simple command to initialise a fresh <a href="https://www.forester-notes.org/tfmt-000R/">forest</a> within a folder. We’ll call our folder <code>forest</code>, but you can call it anything you want.</p> 241 241 + <pre><![CDATA[mkdir forest 242 242 + cd forest]]></pre> 243 243 + <p>Now that we are inside our new directory, we can instruct <a href="https://www.forester-notes.org/index/">Forester</a> to initialise a <a href="https://www.forester-notes.org/tfmt-000R/">forest</a>.</p> 244 244 + <pre>forester init</pre> 245 245 + <p>This command initialises a <a href="https://git-scm.com/">git</a> repository with the skeleton of a <a href="https://www.forester-notes.org/tfmt-000R/">forest</a>, which contains a configuration file named <code>forest.toml</code>; this file specifies the locations of your trees, assets, etc. There is also a <a href="https://git-scm.com/">git</a> submodule bound to the <code>theme/</code> directory (pointing to the <a href="https://git.sr.ht/~jonsterling/forester-base-theme">base theme</a> repository) that contains the stylesheets that web browsers will need in order to render your <a href="https://www.forester-notes.org/tfmt-000R/">forest</a> as HTML.</p> 246 246 + <section> 247 247 + <header> 248 248 + <h3>Tree addresses in a <a href="https://www.forester-notes.org/tfmt-000R/">forest</a></h3> 249 249 + </header> 250 250 + <p>A <a href="https://www.forester-notes.org/tfmt-000R/">tree</a> in <a href="https://www.forester-notes.org/index/">Forester</a> is usually associated to an address of the form <code>xxx-NNNN</code> where <code>xxx</code> is <em>your</em> chosen “namespace” (most likely your initials) and <code>NNNN</code> is a four-digit <a href="https://www.forester-notes.org/jms-0074/">base-36 number</a>. The purpose of the namespace and the <a href="https://www.forester-notes.org/jms-0074/">base-36</a> code is to uniquely identify a <a href="https://www.forester-notes.org/tfmt-000R/">tree</a>, not only within your <a href="https://www.forester-notes.org/tfmt-000R/">forest</a> but across all <a href="https://www.forester-notes.org/tfmt-000R/">forests</a>. A tree with address <code>xxx-NNNN</code> is stored in a file named <code>xxx-NNNN.tree</code> (unless it is emitted from inside another tree by means of the <a href="https://www.forester-notes.org/jms-00O4/">inline subtrees</a> feature).</p> 251 251 + <p>Note that the format of <a href="https://www.forester-notes.org/tfmt-000R/">tree</a> addresses is purely a matter of convention, and is not forced by the <a href="https://www.forester-notes.org/index/">Forester</a> tool. Users are free to use their own format for tree addresses, and in some cases alternative (human-readable) formats may be desirable: this includes trees representing bibliographic references, as well as biographical trees.</p> 252 252 + </section> 253 253 + <section> 254 254 + <header> 255 255 + <h3>Building and viewing your <a href="https://www.forester-notes.org/tfmt-000R/">forest</a> for the first time</h3> 256 256 + </header> 257 257 + <p>To build your <a href="https://www.forester-notes.org/tfmt-000R/">forest</a>, you can run the following command of <a href="https://www.forester-notes.org/index/">Forester</a>'s executable in your shell:</p> 258 258 + <pre>forester build forest.toml</pre> 259 259 + <p>The <code>--dev</code> flag is optional, and when activated supplies metadata to the generated website to support an “edit button” on each tree; this flag is meant to be used when developing your <a href="https://www.forester-notes.org/tfmt-000R/">forest</a> locally, and should not be used when building the <a href="https://www.forester-notes.org/tfmt-000R/">forest</a> to be uploaded to your public web host.</p> 260 260 + <section> 261 261 + <header> 262 262 + <h4><a href="https://www.forester-notes.org/index/">Forester</a> renders each tree to an XML document</h4> 263 263 + </header> 264 264 + <p><a href="https://www.forester-notes.org/index/">Forester</a> renders your <a href="https://www.forester-notes.org/tfmt-000R/">forest</a> to some XML files in the <code>output/</code> directory; XML is, like HTML, a format for structured documents that can be displayed by web browsers. The <a href="https://www.forester-notes.org/tfmt-000R/">forest</a> template comes equipped with a built-in XSLT stylesheet (<code>theme/default.xsl</code>) which is used to instruct web browsers how to render your <a href="https://www.forester-notes.org/tfmt-000R/">forest</a> into a pleasing and readable format.</p> 265 265 + </section> 266 266 + <section> 267 267 + <header> 268 268 + <h4>Serving and viewing your <a href="https://www.forester-notes.org/tfmt-000R/">forest</a> from a local web server</h4> 269 269 + </header> 270 270 + <p>The recommended and most secure way to view your <a href="https://www.forester-notes.org/tfmt-000R/">forest</a> while editing it is to <em>serve</em> it from a local web server. To do this, first ensure that you have <a href="https://www.python.org/downloads/">Python 3</a> correctly installed. Then run the following command from the root directory of your <a href="https://www.forester-notes.org/tfmt-000R/">forest</a>:</p> 271 271 + <pre>python3 -m http.server 1313 -d output</pre> 272 272 + <p>While this command is running, you will be able to access your <a href="https://www.forester-notes.org/tfmt-000R/">forest</a> by navigating to <a href="http://localhost:1313/index.xml"><code>localhost:1313/index.xml</code></a> in your preferred web browser.</p> 273 273 + <p>In the future, <a href="https://www.forester-notes.org/index/">Forester</a> <a href="https://todo.sr.ht/~jonsterling/forester/15">may be able to run its own local server</a> to avoid the dependency on external tools like Python.</p> 274 274 + </section> 275 275 + <section> 276 276 + <header> 277 277 + <h4>Viewing your <a href="https://www.forester-notes.org/tfmt-000R/">forest</a> locally without a local web server</h4> 278 278 + </header> 279 279 + <p>It is also possible to open the generated file <code>output/index.xml</code> <em>directly</em> in your web browser. Unfortunately, modern web browsers by default prevent the use of XSLT stylesheets on the local file system for security reasons. Because <a href="https://www.forester-notes.org/jms-007G/"><a href="https://www.forester-notes.org/index/">Forester</a>'s output format is XML</a>, the output cannot be viewed directly in your web browser without disabling this security feature (at your own risk). Users who do not understand the risks involved should <a href="https://www.forester-notes.org/jms-007I/">turn back and use a local web server instead</a>, which is more secure; if you understand and are willing to accept the risks, you may proceed as follows depending on your browser.</p> 280 280 + <section> 281 281 + <header> 282 282 + <h5>Configuring Firefox for viewing a local <a href="https://www.forester-notes.org/tfmt-000R/">forest</a></h5> 283 283 + </header> 284 284 + <p>To configure Firefox for viewing your local <a href="https://www.forester-notes.org/tfmt-000R/">forest</a>, navigate to <code>about:config</code> in your address bar.</p> 285 285 + <ol><li>Firefox will present a page warning you to <em>“Proceed with Caution”</em>: you must <em>“Accept the Risk and Continue”</em>.</li> 286 286 + <li>In the “Search preference name” box, search for <code>security.fileuri.strict_origin_policy</code>.</li> 287 287 + <li>Most likely, the <code>security.fileuri.strict_origin_policy</code> will appear set to <code>true</code>. Double click on the word <code>true</code> to toggle it to <code>false</code>.</li></ol> 288 288 + </section> 289 289 + <section> 290 290 + <header> 291 291 + <h5>Configuring Safari for viewing a local <a href="https://www.forester-notes.org/tfmt-000R/">forest</a></h5> 292 292 + </header> 293 293 + <p>To configure Safari for viewing your local <a href="https://www.forester-notes.org/tfmt-000R/">forest</a>, you must activate the <em>Develop</em> menu and then toggle one setting.</p> 294 294 + <ol><li>Open Safari's settings window.</li> 295 295 + <li>In the <em>Advanced</em> tab, check the <em>“Show Develop menu in menu bar”</em> checkbox at the bottom.</li> 296 296 + <li>Open the <em>Develop</em> menu in the menubar, and select <em>“Disable Local File Restrictions”</em>.</li></ol> 297 297 + </section> 298 298 + </section> 299 299 + </section> 300 300 + </section> 301 301 + <section> 302 302 + <header> 303 303 + <h2>Creating your personal biographical <a href="https://www.forester-notes.org/tfmt-000R/">tree</a></h2> 304 304 + </header> 305 305 + <p>The first <a href="https://www.forester-notes.org/tfmt-000R/">tree</a> that you should create is a biographical tree to represent your own identity; ultimately you will link to this tree when you set the <a href="https://www.forester-notes.org/tfmt-000S/">authors</a> of other <a href="https://www.forester-notes.org/tfmt-000R/">trees</a> that you create later on. Although most <a href="https://www.forester-notes.org/tfmt-000R/">trees</a> will be addressed by <a href="https://www.forester-notes.org/jms-0073/">identifiers of the form <code>xxx-NNNN</code></a>, it is convenient to simply use a person’s full name to address a biographical <a href="https://www.forester-notes.org/tfmt-000R/">tree</a>. <a href="https://www.forester-notes.org/jonmsterling/">My own biographical tree</a> is located at <code>trees/people/jonmsterling.tree</code> and contains the following source code:</p> 306 306 + <pre><![CDATA[\title{Jon Sterling} 307 307 + \taxon{Person} 308 308 + \meta{external}{https://www.jonmsterling.com/} 309 309 + \meta{institution}{[[ucam]]} 310 310 + \meta{orcid}{0000-0002-0585-5564} 311 311 + \meta{position}{Associate Professor} 312 312 + 313 313 + \p{Associate Professor in Logical Foundations and Formal Methods at University of Cambridge. Formerly a [Marie Skłodowska-Curie Postdoctoral Fellow](jms-0061) hosted at Aarhus University by [Lars Birkedal](larsbirkedal), and before this a PhD student of [Robert Harper](robertharper).}]]></pre> 314 314 + <p>Let’s break this code down to understand what it does.</p> 315 315 + <ol><li>The declaration <code><![CDATA[\title{Jon Sterling}]]></code> sets the title of the <a href="https://www.forester-notes.org/tfmt-000R/">tree</a> to my name.</li> 316 316 + <li>The <code><![CDATA[\taxon{Person}]]></code> declaration informs <a href="https://www.forester-notes.org/index/">Forester</a> that the <a href="https://www.forester-notes.org/tfmt-000R/">tree</a> is biographical. Not ever <a href="https://www.forester-notes.org/tfmt-000R/">tree</a> needs to have a taxon; common taxa include <code>person</code>, <code>theorem</code>, <code>definition</code>, <code>lemma</code>, <code>blog</code>, etc. You are free to use whatever you want, but some taxa are treated specially by <a href="https://www.forester-notes.org/index/">Forester</a>.</li> 317 317 + <li>The subsequent <code><![CDATA[\meta]]></code> declarations attach additional information to the tree that can be used during rendering. These declarations are optional, and you are free to put whatever metadata you want.</li> 318 318 + <li>Like in HTML, paragraphs must be wrapped in <code><![CDATA[\p{...}]]></code>.</li></ol> 319 319 + <p>Do not hard-wrap your text, as this can have visible impact on how <a href="https://www.forester-notes.org/tfmt-000R/">trees</a> are rendered; it is recommended that you use a text editor with good support for soft-wrapping, like <a href="https://code.visualstudio.com/">Visual Studio Code</a>.</p> 320 320 + <p>You can see that the <a href="https://www.forester-notes.org/jms-007N/">concrete syntax of <a href="https://www.forester-notes.org/index/">Forester</a>'s <a href="https://www.forester-notes.org/tfmt-000R/">trees</a></a> looks superficially like a combination of <code>\LaTeX </code> and Markdown; Markdown-style links are used both for links to other trees <em>and</em> for links to external URLs. <a href="https://www.forester-notes.org/index/">Forester</a>'s concrete syntax is not fully documented, but it is less ambiguous than both <code>\LaTeX </code> and Markdown.</p> 321 321 + </section> 322 322 + <section> 323 323 + <header> 324 324 + <h2>Creating a new <a href="https://www.forester-notes.org/tfmt-000R/">tree</a> using <code>forester new</code></h2> 325 325 + </header> 326 326 + <p>Creating a new <a href="https://www.forester-notes.org/tfmt-000R/">tree</a> in your <a href="https://www.forester-notes.org/tfmt-000R/">forest</a> is as simple as adding a <code>.tree</code> file to the <code>trees</code> folder. Because it is hard to manually choose the <a href="https://www.forester-notes.org/jms-0073/">next incremental tree address</a>, <a href="https://www.forester-notes.org/index/">Forester</a> provides a command to do this automatically. If your chosen <a href="https://www.forester-notes.org/jms-0073/">namespace prefix</a> is <code>xxx</code>, then you should use the following command in your shell to create a new tree:</p> 327 327 + <pre>forester new forest.toml --dest=trees --prefix=xxx</pre> 328 328 + <p>In return, <a href="https://www.forester-notes.org/index/">Forester</a> should output the location of the new tree, e.g. <code>trees/xxx-0002.tree</code>. If we look at the contents of this new file, we will see that it is empty except for metadata assigning a date to the tree:</p> 329 329 + <pre><![CDATA[\date{2023-08-15}]]></pre> 330 330 + <p><strong>Most <a href="https://www.forester-notes.org/tfmt-000R/">trees</a> should have a <code><![CDATA[\date]]></code> annotation;</strong> this date is meant to be the date of the <a href="https://www.forester-notes.org/tfmt-000R/">tree</a>'s creation. You should proceed by adding further metadata: the title and the <a href="https://www.forester-notes.org/tfmt-000S/">author</a>; for the latter, you will use the address of your <a href="https://www.forester-notes.org/jms-007K/">personal biographical tree</a>.</p> 331 331 + <pre><![CDATA[\title{My first tree} 332 332 + \author{jonmsterling}]]></pre> 333 333 + <p><a href="https://www.forester-notes.org/tfmt-000R/">Tree</a> titles should be given in lower case (except for proper names, etc.); these titles will be <em>rendered</em> by <a href="https://www.forester-notes.org/index/">Forester</a> in sentence case. A <a href="https://www.forester-notes.org/tfmt-000R/">tree</a> can have as many <code><![CDATA[\author]]></code> declarations as it has <a href="https://www.forester-notes.org/tfmt-000S/">authors</a>; these will be rendered in their order of appearance.</p> 334 334 + <p>Now you can begin to populate the tree with its content, written in the <a href="https://www.forester-notes.org/jms-007N/"><a href="https://www.forester-notes.org/index/">Forester</a> markup language</a>. <a href="https://www.forester-notes.org/tfmt-0007/">Think carefully about keeping each tree relatively independent and atomic</a>.</p> 335 335 + </section> 336 336 + <section> 337 337 + <header> 338 338 + <h2>Bottom-up hierarchy via <em>transclusion</em></h2> 339 339 + </header> 340 340 + <p>You may be used to writing <code>\LaTeX </code> documents, where you work from the top down: you create some section headings, put some text under those headings, make some deeper section headings, put more text, etc. <a href="https://www.forester-notes.org/tfmt-000R/">Forests</a> work in the opposite way, from the bottom up: you start by writing independent, <a href="https://www.forester-notes.org/tfmt-0007/">atomic</a> notes/<a href="https://www.forester-notes.org/tfmt-000R/">trees</a> and then only later start to (sparingly) assemble these into a hierarchy in order to reify the emerging structure.</p> 341 341 + <p><a href="https://www.forester-notes.org/index/">Forester</a>’s bottom-up approach to section hierarchy works via something called <em>transclusion</em>. The idea is that at any time, you can include (“transclude”) the full contents of another <a href="https://www.forester-notes.org/tfmt-000R/">tree</a> into the current tree as a subsection by adding the following code:</p> 342 342 + <pre><![CDATA[\transclude{xxx-NNNN}]]></pre> 343 343 + <p>This is kind of like <code>\LaTeX </code>’s <code><![CDATA[\input]]></code> command, but much better behaved: for instance, section levels are computed on the fly depending on the position in the hierarchy. This <a href="https://www.forester-notes.org/jms-0052/">entire tutorial</a> is cobbled together by transcluding many smaller <a href="https://www.forester-notes.org/tfmt-000R/">trees</a>, each with their own independent existence. For example, the following two sections are transcluded from an <a href="https://www.forester-notes.org/tfmt-0001/">entirely different part</a> of my forest:</p> 344 344 + <section> 345 345 + <header> 346 346 + <h3>The best structure to impose is relatively flat</h3> 347 347 + </header> 348 348 + <p>It is easy to make the mistake of prematurely imposing a complex hierarchical structure on a network of notes, which leads to excessive refactoring. Hierarchy should be used sparingly, and its strength is for the large-scale organization of ideas. The best structure to impose on a network of many small related ideas is a relatively flat one. I believe that this is one of the mistakes made in the writing of the <em>foundations of relative category theory</em>, whose hierarchical nesting was too complex and quite beholden to my experience with pre-hypertext media.</p> 349 349 + </section> 350 350 + <p>One of the immediate impacts and strengths of <a href="https://www.forester-notes.org/index/">Forester</a>’s transclusion model is that a given <a href="https://www.forester-notes.org/tfmt-000R/">tree</a> has no canonical “geographic” location in the <a href="https://www.forester-notes.org/tfmt-000R/">forest</a>. One <a href="https://www.forester-notes.org/tfmt-000R/">tree</a> can appear as a child of many other <a href="https://www.forester-notes.org/tfmt-000R/">trees</a>, which allows the same content to be incorporated into different textual and intellectual narratives.</p> 351 351 + <section> 352 352 + <header> 353 353 + <h3>Hierarchical structure as non-unique narrative</h3> 354 354 + </header> 355 355 + <p>Multiple hierarchical structures can be imposed on the same associative network of nodes; a hierarchical structure amounts to a “narrative” that contextualizes a given subgraph of the network. One example could be the construction of lecture notes; another example could be a homework sheet; a further example could be a book chapter or scientific article. Although these may draw from the same body of definitions, theorems, examples, and exercises, these objects are contextualized within a different narrative, often toward fundamentally different ends.</p> 356 356 + <p>As a result, any interface for navigating the neighbor-relation in hierarchically organized notes would need to take account of the multiplicity of parent nodes. Most hypertext tools assume that the position of a node in the hierarchy is unique, and therefore have a single “next/previous” navigation interface; we must investigate the design of interfaces that surface all parent/neighbor relations.</p> 357 357 + </section> 358 358 + </section> 359 359 + <section> 360 360 + <header> 361 361 + <h2>The <a href="https://www.forester-notes.org/index/">Forester</a> markup language</h2> 362 362 + </header> 363 363 + <p>A <a href="https://www.forester-notes.org/tfmt-000R/">tree</a> in <a href="https://www.forester-notes.org/index/">Forester</a> is a single file written in a markup language designed specifically for scientific writing <a href="https://www.forester-notes.org/jms-007L/">with bottom-up hierarchy via transclusion</a>. A <a href="https://www.forester-notes.org/tfmt-000R/">tree</a> has two components: the <a href="https://www.forester-notes.org/jms-007P/">frontmatter</a> and the <a href="https://www.forester-notes.org/jms-007O/">mainmatter</a>.</p> 364 364 + <section> 365 365 + <header> 366 366 + <h3><a href="https://www.forester-notes.org/index/">Forester</a> markup: frontmatter</h3> 367 367 + </header> 368 368 + <p>The frontmatter of a <a href="https://www.forester-notes.org/index/">Forester</a> <a href="https://www.forester-notes.org/tfmt-000R/">tree</a> is a sequence of declarations that we summarize below.</p> 369 369 + <table> 370 370 + 371 371 + <tr> 372 372 + 373 373 + <th>Declaration</th> 374 374 + 375 375 + 376 376 + <th>Meaning</th> 377 377 + 378 378 + </tr> 379 379 + 380 380 + 381 381 + <tr> 382 382 + <td style="white-space:nowrap"> 383 383 + <code><![CDATA[\title{...}]]></code> 384 384 + </td> 385 385 + <td> 386 386 + sets the title of the <a href="https://www.forester-notes.org/tfmt-000R/">tree</a>; can contain <a href="https://www.forester-notes.org/jms-007O/">mainmatter</a> markup 387 387 + </td> 388 388 + </tr> 389 389 + 390 390 + 391 391 + <tr> 392 392 + <td style="white-space:nowrap"> 393 393 + <code><![CDATA[\author{name}]]></code> 394 394 + </td> 395 395 + <td> 396 396 + sets the <a href="https://www.forester-notes.org/tfmt-000S/">author</a> of the <a href="https://www.forester-notes.org/tfmt-000R/">tree</a> to be the biographical tree at address <code>name</code> 397 397 + </td> 398 398 + </tr> 399 399 + 400 400 + 401 401 + <tr> 402 402 + <td style="white-space:nowrap"> 403 403 + <code><![CDATA[\date{YYYY-MM-DD}]]></code> 404 404 + </td> 405 405 + <td>sets the creation date of the <a href="https://www.forester-notes.org/tfmt-000R/">tree</a></td> 406 406 + </tr> 407 407 + 408 408 + 409 409 + <tr> 410 410 + <td style="white-space:nowrap"> 411 411 + <code><![CDATA[\taxon{Taxon}]]></code> 412 412 + </td> 413 413 + <td>sets the taxon of the <a href="https://www.forester-notes.org/tfmt-000R/">tree</a>; example taxa include <code>lemma</code>, <code>theorem</code>, <code>person</code>, <code>reference</code>; the latter two taxa are treated specially by <a href="https://www.forester-notes.org/index/">Forester</a> for tracking biographical and bibliographical <a href="https://www.forester-notes.org/tfmt-000R/">trees</a> respectively</td> 414 414 + </tr> 415 415 + 416 416 + 417 417 + <tr> 418 418 + <td style="white-space:nowrap"> 419 419 + <code><![CDATA[\def\ident[x][y]{body}]]></code> 420 420 + </td> 421 421 + <td> 422 422 + defines and exports from the current <a href="https://www.forester-notes.org/tfmt-000R/">tree</a> a function named <code><![CDATA[\ident]]></code> with two arguments; subsequently, the expression <code><![CDATA[\ident{u}{v}]]></code> would expand to <code>body</code> with the values of <code>u,v</code> substituted for <code><![CDATA[\x,\y]]></code> 423 423 + </td> 424 424 + </tr> 425 425 + 426 426 + 427 427 + <tr> 428 428 + <td style="white-space:nowrap"> 429 429 + <code><![CDATA[\import{xxx-NNNN}]]></code> 430 430 + </td> 431 431 + <td> 432 432 + brings the functions exported by the tree <code>xxx-NNNN</code> into scope 433 433 + </td> 434 434 + </tr> 435 435 + 436 436 + 437 437 + <tr> 438 438 + <td style="white-space:nowrap"> 439 439 + <code><![CDATA[\export{xxx-NNNN}]]></code> 440 440 + </td> 441 441 + <td> 442 442 + brings the functions exported by the tree <code>xxx-NNNN</code> into scope, and exports them from the current tree 443 443 + </td> 444 444 + </tr> 445 445 + 446 446 + </table> 447 447 + </section> 448 448 + <section> 449 449 + <header> 450 450 + <h3><a href="https://www.forester-notes.org/index/">Forester</a> markup: mainmatter</h3> 451 451 + </header> 452 452 + <p>Below we summarize the concrete syntax of the <em>mainmatter</em> in a <a href="https://www.forester-notes.org/index/">Forester</a> <a href="https://www.forester-notes.org/tfmt-000R/">tree</a>.</p> 453 453 + <table> 454 454 + 455 455 + <tr> 456 456 + 457 457 + <th>Function</th> 458 458 + 459 459 + 460 460 + <th>Meaning</th> 461 461 + 462 462 + </tr> 463 463 + 464 464 + 465 465 + <tr> 466 466 + <td style="white-space:nowrap"> 467 467 + <code><![CDATA[\p{...}]]></code> 468 468 + </td> 469 469 + <td>creates a paragraph containing <code>...</code>; unlike Markdown, it is mandatory to annotate paragraphs explicitly</td> 470 470 + </tr> 471 471 + 472 472 + 473 473 + <tr> 474 474 + <td style="white-space:nowrap"> 475 475 + <code><![CDATA[\em{...}]]></code> 476 476 + </td> 477 477 + <td>typesets the content in italics</td> 478 478 + </tr> 479 479 + 480 480 + 481 481 + <tr> 482 482 + <td style="white-space:nowrap"> 483 483 + <code><![CDATA[\strong{...}]]></code> 484 484 + </td> 485 485 + <td>typesets the content in boldface</td> 486 486 + </tr> 487 487 + 488 488 + 489 489 + 490 490 + <tr> 491 491 + <td style="white-space:nowrap"> 492 492 + <code><![CDATA[\ol{...}]]></code> 493 493 + </td> 494 494 + <td>creates an ordered list</td> 495 495 + </tr> 496 496 + 497 497 + 498 498 + 499 499 + <tr> 500 500 + <td style="white-space:nowrap"> 501 501 + <code><![CDATA[\ul{...}]]></code> 502 502 + </td> 503 503 + <td>creates an unordered list</td> 504 504 + </tr> 505 505 + 506 506 + 507 507 + 508 508 + <tr> 509 509 + <td style="white-space:nowrap"> 510 510 + <code><![CDATA[\li{...}]]></code> 511 511 + </td> 512 512 + <td>creates a list item</td> 513 513 + </tr> 514 514 + 515 515 + 516 516 + 517 517 + <tr> 518 518 + <td style="white-space:nowrap"> 519 519 + <code><![CDATA[ #{...}]]></code> 520 520 + </td> 521 521 + <td>typesets the content in (inline) math mode using <code>\KaTeX </code>; note that math mode is idempotent in <a href="https://www.forester-notes.org/index/">Forester</a></td> 522 522 + </tr> 523 523 + 524 524 + 525 525 + <tr> 526 526 + <td style="white-space:nowrap"> 527 527 + <code><![CDATA[ ##{...}]]></code> 528 528 + </td> 529 529 + <td>typesets the content in (display) math mode using <code>\KaTeX </code></td> 530 530 + </tr> 531 531 + 532 532 + 533 533 + <tr> 534 534 + <td style="white-space:nowrap"> 535 535 + <code><![CDATA[\transclude{xxx-NNNN}]]></code> 536 536 + </td> 537 537 + <td><a href="https://www.forester-notes.org/jms-007L/">transcludes</a> the <a href="https://www.forester-notes.org/tfmt-000R/">tree</a> at address <code>xxx-NNNN</code> as a subsection</td> 538 538 + </tr> 539 539 + 540 540 + 541 541 + <tr> 542 542 + <td style="white-space:nowrap"> 543 543 + <code><]]></code> 544 544 + </td> 545 545 + <td> 546 546 + formats the text <code>title</code> as a hyperlink to address <code>address</code>; if <code>address</code> is the address of a <a href="https://www.forester-notes.org/tfmt-000R/">tree</a>, the link will point to that tree, and otherwise it is treated as a URL 547 547 + </td> 548 548 + </tr> 549 549 + 550 550 + 551 551 + <tr> 552 552 + <td style="white-space:nowrap"> 553 553 + <code><![CDATA[\let\ident[x][y]{body}]]></code> 554 554 + </td> 555 555 + <td> 556 556 + defines a local function named <code><![CDATA[\ident]]></code> with two arguments; subsequently, the expression <code><![CDATA[\ident{u}{v}]]></code> would expand to <code>body</code> with the values of <code>u,v</code> substituted for <code><![CDATA[\x,\y]]></code>. 557 557 + </td> 558 558 + </tr> 559 559 + 560 560 + 561 561 + <tr> 562 562 + <td style="white-space:nowrap"> 563 563 + <code><![CDATA[\code{...}]]></code> 564 564 + </td> 565 565 + <td>typesets the content in monospace</td> 566 566 + </tr> 567 567 + 568 568 + 569 569 + <tr> 570 570 + <td style="white-space:nowrap"> 571 571 + <code><![CDATA[\tex{preamble}{body}]]></code> 572 572 + </td> 573 573 + <td>typesets the <code>body</code> externally using <code>\LaTeX </code> using <code>preamble</code> as preamble code (e.g. to set up tikz packages, etc.). It can be useful to wrap this in your own macro in order to insert your preamble code automatically.</td> 574 574 + </tr> 575 575 + 576 576 + </table> 577 577 + </section> 578 578 + <section> 579 579 + <header> 580 580 + <h3>An complete worked example <a href="https://www.forester-notes.org/tfmt-000R/">tree</a> in <a href="https://www.forester-notes.org/index/">Forester</a></h3> 581 581 + </header> 582 582 + <p>An example of a complete <a href="https://www.forester-notes.org/tfmt-000R/">tree</a> in the <a href="https://www.forester-notes.org/index/">Forester</a> markup language can be seen below.</p> 583 583 + <pre><![CDATA[\title{Creation of (co)limits} 584 584 + \date{2023-02-11} 585 585 + \taxon{Definition} 586 586 + \author{jonmsterling} 587 587 + 588 588 + \def\CCat{#{\mathcal{C}}} 589 589 + \def\DCat{#{\mathcal{D}}} 590 590 + \def\ICat{#{\mathcal{I}}} 591 591 + \def\Mor[arg1][arg2][arg3]{#{{\arg2}\xrightarrow{\arg1}{\arg3}}} 592 592 + 593 593 + \p{Let \Mor{U}{\CCat}{\DCat} be a functor and let \ICat be a category. The functor #{U} is said to \em{create (co)limits of #{\ICat}-figures} when for any diagram \Mor{C_\bullet}{\ICat}{\CCat} such that #{\ICat\xrightarrow{C_\bullet}\CCat\xrightarrow{F}\DCat} has a (co)limit, then #{C_\bullet} has a (co)limit that is both preserved and reflected by #{F}.}]]></pre> 594 594 + <p>The code above results in the following tree:</p> 595 595 + <section> 596 596 + <header> 597 597 + <h4>Creation of (co)limits</h4> 598 598 + </header> 599 599 + <p>Let <code>{<code>\mathcal {C}</code>}\xrightarrow {U}{<code>\mathcal {D}</code>}</code> be a functor and let <code>\mathcal {I}</code> be a category. The functor <code>U</code> is said to <em>create (co)limits of <code><code>\mathcal {I}</code></code>-figures</em> when for any diagram <code>{<code>\mathcal {I}</code>}\xrightarrow {C_\bullet }{<code>\mathcal {C}</code>}</code> such that <code><code>\mathcal {I}</code>\xrightarrow {C_\bullet }<code>\mathcal {C}</code>\xrightarrow {F}<code>\mathcal {D}</code></code> has a (co)limit, then <code>C_\bullet </code> has a (co)limit that is both preserved and reflected by <code>F</code>.</p> 600 600 + </section> 601 601 + </section> 602 602 + </section> 603 603 + <section> 604 604 + <header> 605 605 + <h2>Deploying your <a href="https://www.forester-notes.org/tfmt-000R/">forest</a> to a web host</h2> 606 606 + </header> 607 607 + <p>Now that you have <a href="https://www.forester-notes.org/jms-006X/">created your <a href="https://www.forester-notes.org/tfmt-000R/">forest</a></a> and <a href="https://www.forester-notes.org/jms-007K/">added a few</a> <a href="https://www.forester-notes.org/jms-007H/"><a href="https://www.forester-notes.org/tfmt-000R/">trees</a> of your own</a>, it is time to upload it to your web host. Many users of <a href="https://www.forester-notes.org/index/">Forester</a> will have university-supplied static web hosting, and others may prefer to use GitHub pages; deploying a <a href="https://www.forester-notes.org/tfmt-000R/">forest</a> works the same way in either case.</p> 608 608 + <ol><li>First, make sure your <a href="https://www.forester-notes.org/tfmt-000R/">forest</a> is built using <a href="https://www.forester-notes.org/jms-007D/">the earlier instructions</a>.</li> 609 609 + <li>Then take the entire contents of your <code>output</code> directory and upload them to your preferred web host.</li></ol> 610 610 + </section> 611 611 + <section> 612 612 + <header> 613 613 + <h2>Let a hundred <a href="https://www.forester-notes.org/tfmt-000R/">forests</a> bloom!</h2> 614 614 + </header> 615 615 + <p>I am eager to see the new <a href="https://www.forester-notes.org/tfmt-000R/">forests</a> that people create using <a href="https://www.forester-notes.org/index/">Forester</a>. I am happy to offer personal assistance via the <a href="https://lists.sr.ht/~jonsterling/forester-discuss">mailing list</a>.</p> 616 616 + <p>Many aspects of <a href="https://www.forester-notes.org/index/">Forester</a> are in flux and not fully documented; it will often be instructive to consult the source of existings <a href="https://www.forester-notes.org/tfmt-000R/">forests</a>, such as <a href="https://git.sr.ht/~jonsterling/forester-notes.org">this one</a>.</p> 617 617 + <p>Have fun, and be sure to send me links to your <a href="https://www.forester-notes.org/tfmt-000R/">forests</a> when you have made them!</p> 618 618 + </section> 619 619 + </div> 620 620 + </content> 621 621 + </entry> 622 622 + </feed>

+179

scripts/gen_atom.ml

··· 1 1 + (* Generate an atom feed *) 2 2 + 3 3 + let id = Uri.of_string "https://jon.recoil.org/atom.xml" 4 4 + let title : Syndic.Atom.text_construct = Syndic.Atom.Text "Jon's blog" 5 5 + 6 6 + let author = 7 7 + Syndic.Atom.author "Jon Ludlam" ~uri:(Uri.of_string "https://jon.recoil.org/") 8 8 + 9 9 + let updated = Unix.gettimeofday () |> Ptime.of_float_s |> Option.get 10 10 + 11 11 + let entry_of_mld odoc_file = 12 12 + let report_error during msg = 13 13 + Format.eprintf "Error processing file '%s' while %s: %s\n%!" 14 14 + (Fpath.to_string odoc_file) 15 15 + during msg; 16 16 + None 17 17 + in 18 18 + let unit = 19 19 + match Odoc_odoc.Odoc_file.load odoc_file with 20 20 + | Ok f -> Some f 21 21 + | Error (`Msg m) -> 22 22 + ignore (report_error "loading file" m); 23 23 + None 24 24 + in 25 25 + match unit with 26 26 + | None -> None 27 27 + | Some unit -> ( 28 28 + let page = 29 29 + match unit.content with 30 30 + | Odoc_odoc.Odoc_file.Page_content page -> Some page 31 31 + | _ -> None 32 32 + in 33 33 + match page with 34 34 + | None -> None 35 35 + | Some page -> ( 36 36 + let document = 37 37 + Odoc_document.Renderer.document_of_page ~syntax:OCaml page 38 38 + in 39 39 + let frontmatter = page.frontmatter.Odoc_model.Frontmatter.other_config in 40 40 + let published = 41 41 + try Some (List.assoc "published" frontmatter) with Not_found -> None 42 42 + in 43 43 + match published with 44 44 + | None -> None (* Skip posts without published date *) 45 45 + | Some published -> ( 46 46 + match document with 47 47 + | Odoc_document.Types.Document.Source_page _ -> None 48 48 + | Odoc_document.Types.Document.Page p -> 49 49 + let first_heading = 50 50 + List.find_map 51 51 + (function 52 52 + | Odoc_document.Types.Item.Heading h -> Some h 53 53 + | _ -> None) 54 54 + p.preamble 55 55 + in 56 56 + match first_heading with 57 57 + | None -> 58 58 + ignore (report_error "parsing title" "No heading found"); 59 59 + None 60 60 + | Some first_heading -> 61 61 + let title = 62 62 + List.filter_map 63 63 + (function 64 64 + | Odoc_document.Types.Inline.{ desc = Text t; _ } -> Some t 65 65 + | _ -> None) 66 66 + first_heading.title 67 67 + in 68 68 + let title = String.concat "" title in 69 69 + if title = "" then None 70 70 + else 71 71 + let resolve = Odoc_html.Link.Current p.url in 72 72 + let config = 73 73 + Odoc_html.Config.v ~semantic_uris:false ~indent:false 74 74 + ~flat:false ~open_details:false ~as_json:false ~remap:[] () 75 75 + in 76 76 + let url = Odoc_html.Generator.filepath p.url ~config in 77 77 + let url = 78 78 + Format.asprintf "https://jon.recoil.org/%s" 79 79 + (Fpath.to_string url) 80 80 + in 81 81 + (* Generate full content: preamble + items *) 82 82 + let all_items = p.preamble @ p.items in 83 83 + let html = Odoc_html.Generator.items ~config ~resolve all_items in 84 84 + let content_fmt = Fmt.list (Tyxml.Html.pp_elt ()) in 85 85 + let content = Format.asprintf "%a" content_fmt html in 86 86 + (* Extract first paragraph for summary *) 87 87 + let summary = 88 88 + let first_text = 89 89 + List.find_map 90 90 + (function 91 91 + | Odoc_document.Types.Item.Text blocks -> 92 92 + List.find_map 93 93 + (function 94 94 + | { Odoc_document.Types.Block.desc = 95 95 + Odoc_document.Types.Block.Paragraph inline; 96 96 + _ 97 97 + } -> 98 98 + let text = 99 99 + List.filter_map 100 100 + (function 101 101 + | Odoc_document.Types.Inline. 102 102 + { desc = Text t; _ } -> 103 103 + Some t 104 104 + | _ -> None) 105 105 + inline 106 106 + in 107 107 + if text = [] then None 108 108 + else Some (String.concat "" text) 109 109 + | _ -> None) 110 110 + blocks 111 111 + | _ -> None) 112 112 + p.preamble 113 113 + in 114 114 + match first_text with 115 115 + | Some t -> 116 116 + if String.length t > 200 then 117 117 + String.sub t 0 200 ^ "..." 118 118 + else t 119 119 + | None -> title 120 120 + in 121 121 + let published = 122 122 + try 123 123 + ISO8601.Permissive.date published |> Ptime.of_float_s 124 124 + with _ -> 125 125 + Format.eprintf "Error parsing date '%s' for %s\n%!" 126 126 + published (Fpath.to_string odoc_file); 127 127 + None 128 128 + in 129 129 + match published with 130 130 + | None -> None 131 131 + | Some published -> 132 132 + Some 133 133 + (Syndic.Atom.entry ~id:(Uri.of_string url) 134 134 + ~title:(Syndic.Atom.Text title) 135 135 + ~published ~updated:published 136 136 + ~summary:(Syndic.Atom.Text summary) 137 137 + ~content:(Syndic.Atom.Html (None, content)) 138 138 + ~links: 139 139 + [ 140 140 + Syndic.Atom.link ~rel:Syndic.Atom.Alternate 141 141 + (Uri.of_string url); 142 142 + ] 143 143 + ~authors:(author, []) ())))) 144 144 + 145 145 + let is_blog_post path = 146 146 + let basename = Fpath.basename path in 147 147 + Fpath.has_ext "odocl" path 148 148 + && String.length basename > 5 149 149 + && String.sub basename 0 5 = "page-" 150 150 + && basename <> "page-index.odocl" 151 151 + 152 152 + let entries = 153 153 + let mlds = 154 154 + Bos.OS.Dir.fold_contents 155 155 + (fun path acc -> if is_blog_post path then path :: acc else acc) 156 156 + [] 157 157 + (Fpath.v "_tmp/_odoc/blog") 158 158 + in 159 159 + match mlds with 160 160 + | Ok mlds -> 161 161 + let entries = List.filter_map entry_of_mld mlds in 162 162 + (* Sort by published date, newest first *) 163 163 + List.sort Syndic.Atom.descending entries 164 164 + | Error (`Msg m) -> 165 165 + Format.eprintf "Error finding blog posts: %s\n%!" m; 166 166 + [] 167 167 + 168 168 + let self_link = 169 169 + Syndic.Atom.link ~rel:Self (Uri.of_string "https://jon.recoil.org/atom.xml") 170 170 + 171 171 + let alt_link = 172 172 + Syndic.Atom.link ~rel:Alternate (Uri.of_string "https://jon.recoil.org/blog/") 173 173 + 174 174 + let feed = 175 175 + Syndic.Atom.feed ~id ~title ~updated ~links:[ self_link; alt_link ] entries 176 176 + 177 177 + let _ = 178 178 + Syndic.Atom.write feed "atom.xml"; 179 179 + Format.printf "Generated atom.xml with %d entries\n%!" (List.length entries)

+197

scripts/gen_blog_index.ml

··· 1 1 + (* Generate blog index.mld files and recent entries list *) 2 2 + 3 3 + let month_name = function 4 4 + | 1 -> "January" | 2 -> "February" | 3 -> "March" | 4 -> "April" 5 5 + | 5 -> "May" | 6 -> "June" | 7 -> "July" | 8 -> "August" 6 6 + | 9 -> "September" | 10 -> "October" | 11 -> "November" | 12 -> "December" 7 7 + | _ -> "Unknown" 8 8 + 9 9 + type post = { 10 10 + year : int; 11 11 + month : int; 12 12 + slug : string; (* e.g., "claude-and-dune" *) 13 13 + title : string; 14 14 + published : string; (* e.g., "2025-12-18" *) 15 15 + } 16 16 + 17 17 + let parse_post path = 18 18 + let ic = open_in path in 19 19 + let content = really_input_string ic (in_channel_length ic) in 20 20 + close_in ic; 21 21 + 22 22 + (* Extract title from {0 ...} *) 23 23 + let title = 24 24 + let re = Str.regexp "{0 \\([^}]+\\)}" in 25 25 + if Str.string_match re content 0 then 26 26 + Str.matched_group 1 content 27 27 + else 28 28 + Filename.basename path |> Filename.remove_extension 29 29 + in 30 30 + 31 31 + (* Extract published date from @published ... *) 32 32 + let published = 33 33 + let re = Str.regexp "@published \\([0-9-]+\\)" in 34 34 + try 35 35 + ignore (Str.search_forward re content 0); 36 36 + Str.matched_group 1 content 37 37 + with Not_found -> "" 38 38 + in 39 39 + 40 40 + (* Parse path to get year/month/slug *) 41 41 + let parts = String.split_on_char '/' path in 42 42 + match parts with 43 43 + | _ :: year_s :: month_s :: filename :: [] -> 44 44 + let year = int_of_string year_s in 45 45 + let month = int_of_string month_s in 46 46 + let slug = Filename.remove_extension filename in 47 47 + Some { year; month; slug; title; published } 48 48 + | _ -> None 49 49 + 50 50 + let find_posts () = 51 51 + let posts = ref [] in 52 52 + let rec scan_dir dir = 53 53 + let entries = Sys.readdir dir in 54 54 + Array.iter (fun entry -> 55 55 + let path = Filename.concat dir entry in 56 56 + if Sys.is_directory path then 57 57 + scan_dir path 58 58 + else if Filename.extension entry = ".mld" && entry <> "index.mld" then 59 59 + match parse_post path with 60 60 + | Some post when post.published <> "" -> posts := post :: !posts 61 61 + | _ -> () 62 62 + ) entries 63 63 + in 64 64 + scan_dir "blog"; 65 65 + (* Sort by published date descending *) 66 66 + List.sort (fun a b -> String.compare b.published a.published) !posts 67 67 + 68 68 + let post_link post = 69 69 + let slug_needs_quotes = 70 70 + String.exists (fun c -> c = '-' || c = '_') post.slug 71 71 + in 72 72 + let slug_fmt = 73 73 + if slug_needs_quotes then 74 74 + Printf.sprintf "page-\"%s\"" post.slug 75 75 + else 76 76 + Printf.sprintf "page-%s" post.slug 77 77 + in 78 78 + Printf.sprintf "{{!/site/blog/%04d/%02d/%s}%s}" 79 79 + post.year post.month slug_fmt post.title 80 80 + 81 81 + let generate_month_index year month posts = 82 82 + let month_posts = List.filter (fun p -> p.year = year && p.month = month) posts in 83 83 + if month_posts = [] then None 84 84 + else 85 85 + let children = 86 86 + month_posts 87 87 + |> List.map (fun p -> p.slug) 88 88 + |> String.concat " " 89 89 + in 90 90 + let content = Printf.sprintf "{0 %s}\n\n@children_order %s\n\n" 91 91 + (month_name month) children 92 92 + in 93 93 + Some content 94 94 + 95 95 + let generate_year_index year posts = 96 96 + let year_posts = List.filter (fun p -> p.year = year) posts in 97 97 + if year_posts = [] then None 98 98 + else 99 99 + (* Get unique months, sorted descending *) 100 100 + let months = 101 101 + year_posts 102 102 + |> List.map (fun p -> p.month) 103 103 + |> List.sort_uniq (fun a b -> compare b a) 104 104 + in 105 105 + let children_order = 106 106 + months 107 107 + |> List.map (Printf.sprintf "%02d/") 108 108 + |> String.concat " " 109 109 + in 110 110 + let post_links = 111 111 + year_posts 112 112 + |> List.map (fun p -> "- " ^ post_link p) 113 113 + |> String.concat "\n" 114 114 + in 115 115 + let content = Printf.sprintf "{0 %d}\n\n@children_order %s\n\n%s\n" 116 116 + year children_order post_links 117 117 + in 118 118 + Some content 119 119 + 120 120 + let generate_blog_index posts = 121 121 + (* Get unique years, sorted descending *) 122 122 + let years = 123 123 + posts 124 124 + |> List.map (fun p -> p.year) 125 125 + |> List.sort_uniq (fun a b -> compare b a) 126 126 + in 127 127 + let children_order = 128 128 + years 129 129 + |> List.map (fun y -> Printf.sprintf "%d/" y) 130 130 + |> String.concat " " 131 131 + in 132 132 + Printf.sprintf "{0 Blog}\n\n@children_order %s\n\nUse the sidebar to navigate the blog posts. The most recent posts are listed first.\n" 133 133 + children_order 134 134 + 135 135 + let generate_recent_entries posts n = 136 136 + let recent = 137 137 + if List.length posts > n then 138 138 + List.filteri (fun i _ -> i < n) posts 139 139 + else 140 140 + posts 141 141 + in 142 142 + recent 143 143 + |> List.map (fun p -> "- " ^ post_link p) 144 144 + |> String.concat "\n" 145 145 + 146 146 + let write_file path content = 147 147 + let oc = open_out path in 148 148 + output_string oc content; 149 149 + close_out oc; 150 150 + Printf.printf "Wrote %s\n" path 151 151 + 152 152 + let () = 153 153 + let posts = find_posts () in 154 154 + Printf.printf "Found %d posts\n\n" (List.length posts); 155 155 + 156 156 + (* Get unique year/month combinations *) 157 157 + let year_months = 158 158 + posts 159 159 + |> List.map (fun p -> (p.year, p.month)) 160 160 + |> List.sort_uniq compare 161 161 + in 162 162 + 163 163 + (* Generate month indexes *) 164 164 + List.iter (fun (year, month) -> 165 165 + match generate_month_index year month posts with 166 166 + | Some content -> 167 167 + let dir = Printf.sprintf "blog/%04d/%02d" year month in 168 168 + let path = Filename.concat dir "index.mld" in 169 169 + write_file path content 170 170 + | None -> () 171 171 + ) year_months; 172 172 + 173 173 + (* Get unique years *) 174 174 + let years = 175 175 + posts 176 176 + |> List.map (fun p -> p.year) 177 177 + |> List.sort_uniq (fun a b -> compare b a) 178 178 + in 179 179 + 180 180 + (* Generate year indexes *) 181 181 + List.iter (fun year -> 182 182 + match generate_year_index year posts with 183 183 + | Some content -> 184 184 + let path = Printf.sprintf "blog/%04d/index.mld" year in 185 185 + write_file path content 186 186 + | None -> () 187 187 + ) years; 188 188 + 189 189 + (* Generate main blog index *) 190 190 + let blog_index = generate_blog_index posts in 191 191 + write_file "blog/index.mld" blog_index; 192 192 + 193 193 + (* Output recent entries for main index *) 194 194 + Printf.printf "\n=== Recent entries (copy to index.mld) ===\n\n"; 195 195 + Printf.printf "{1 Recent entries}\n\n"; 196 196 + print_endline (generate_recent_entries posts 10); 197 197 + print_endline ""

+49

scripts/mld.ml

··· 1 1 + type t = { raw : string; parsed : Odoc_parser.t } 2 2 + 3 3 + type meta = { 4 4 + libs : string list; [@default []] 5 5 + html_scripts : string list; [@default []] 6 6 + other_config : Yojson.Safe.t; [@default `Null] 7 7 + } 8 8 + [@@deriving yojson] 9 9 + 10 10 + let parse_mld mld_file = 11 11 + let text = In_channel.(with_open_bin mld_file input_all) in 12 12 + let location = 13 13 + { Lexing.pos_fname = mld_file; pos_lnum = 1; pos_bol = 0; pos_cnum = 0 } 14 14 + in 15 15 + { raw = text; parsed = Odoc_parser.parse_comment ~location ~text } 16 16 + 17 17 + let meta_of_parsed parsed = 18 18 + let meta_block = 19 19 + List.filter_map 20 20 + (fun block -> 21 21 + let open Odoc_parser in 22 22 + match block with 23 23 + | { Loc.value = `Code_block c; _ } -> ( 24 24 + match c.Ast.meta with 25 25 + | None -> None 26 26 + | Some m -> 27 27 + if m.Ast.language.value = "meta" then Some c.content.value 28 28 + else None) 29 29 + | _ -> None) 30 30 + (Odoc_parser.ast parsed) 31 31 + in 32 32 + let meta = 33 33 + match meta_block with 34 34 + | [] -> None 35 35 + | x :: _ :: _ -> 36 36 + Format.eprintf 37 37 + "Warning, more than one meta block found. Ignoring all but the first."; 38 38 + Some x 39 39 + | [ meta ] -> Some meta 40 40 + in 41 41 + match meta with 42 42 + | None -> None 43 43 + | Some meta -> 44 44 + let y = Yojson.Safe.from_string meta in 45 45 + Some (meta_of_yojson y) 46 46 + 47 47 + let meta_of_mld mld_file = 48 48 + let { parsed; _ } = parse_mld mld_file in 49 49 + meta_of_parsed parsed

+1 -1

site/blog/2026/02/index.mld

··· 1 1 - @children_order weeknotes-2026-06 1 1 + @children_order weeknotes-2026-08 odoc-js-notebooks-fun weeknotes-2026-06 2 2 {0 February} 3 3 4 4

+2

site/blog/2026/02/odoc-js-notebooks-fun.mld

··· 1 1 + {0 Fun with odoc, plugins and javascript} 2 2 +

+53

site/blog/2026/02/weeknotes-2026-08.mld

··· 1 1 + {0 Weeknotes weeks 7-8} 2 2 + 3 3 + @notanotebook 4 4 + @published 2026-02-24 5 5 + 6 6 + A combination one again as I took some time off due to school half term. 7 7 + 8 8 + {1 Finished off my exam questions} 9 9 + This was a lot of fun! Obviously I can't talk about it, but while it was stressful and worrying and anxiety inducing and scary, 10 10 + it was also engaging and interesting and thought-provoking. Having some ideas come together to make a nice coherent whole was 11 11 + very cool. 12 12 + 13 13 + {1 Testing LLMs on past paper questions} 14 14 + Similar to our work on the ticks that {{:https://www.youtube.com/watch?v=Ub8k1BcSRLQ}Sadiq, I and others did last year}, I wanted to try to see how well LLMs could answer tripos questions. 15 15 + Partly I wanted to do this so I could check that my own questions were of the right sort of level, and partly it was just a 16 16 + displacement activity while I wasn't making progress on the actual exam questions! I've not done a useful analysis of the 17 17 + results yet, but seemed in line with our experience with the ticks, though the pass rate was lower for the same models (qwen). 18 18 + 19 19 + {1 Claude from a sunbed} 20 20 + I went away for a vitamin-D boosting bit of sun. Before I went, I got Claude to spin me up a little Telegram bridge so that I could tell it what to do, 21 21 + while it's still running in safeties-off mode on my sacrificial VM. This was kind of fun - I got to just indulge thoughts as they came to me, and 22 22 + off it would go and do stuff. It was a bit limited in how it talked back to me, which wasn't by design but turned out to be nice for this sort of 23 23 + workflow. The downside is that I've now got a load of stuff to sift through - much of which is a 'good start', but none of it is likely to be usable 24 24 + without a good deal more effort. Here's a short-list of things I had it do: 25 25 + 26 26 + - Resurrect Fay Carson's work on the {{:https://github.com/ocaml/odoc/pull/1295}Menhir parser for odoc}, pushed {{:https://github.com/jonludlam/odoc/tree/menhir-parser-rebased}here} 27 27 + - Added some instrumentation to Odoc to do some performance experiments 28 28 + - Ran some simple experiments to measure the impact of various pre-existing performance knobs/switches 29 29 + - Resurrected an old patch of mine to {{:https://github.com/jonludlam/odoc/tree/parameterised-paths}unify the two path representations} in odoc to measure its effect on performance. 30 30 + - Tested aggresively reuse of records if their fields don't change during compile/link 31 31 + - Mixed up the {{:https://tangled.org/jon.recoil.org/odoc-scrollycode-extension}scrollycode backend} and the x-ocaml backend and stuck a playground on at each step 32 32 + - Unified the oxcaml/ocaml branches of {{:https://tangled.org/jon.recoil.org/js_top_worker}js_top_worker} and x-ocaml via cppo 33 33 + - Added oxcaml mode/layout annotations to odoc 34 34 + 35 35 + {1 OxCaml} 36 36 + I investigated the oxcaml docs build, which I had got working last week. Anil reported that it wasn't working for him, so I looked at the 37 37 + build I had and it definitely {i was} working. However, I was building on our machine Monteverde, which is a bit of a beast, so I checked the 38 38 + memory usage and it was enormous! I tried the build again on my 64 gig VM and it OOM'd. I'd noticed before that the [cmti] files for 39 39 + base, in particular [base__Container.cmti] were absolutely massive, and so had just assumed that the problem was that. Luke had also 40 40 + mentioned that some of the output from the template machinery was hidden. However, I had Claude look into this and it couldn't see any 41 41 + doc stop comments. So I asked it to look a little closer and figure out what was using all the memory. It took an unexpectedly large number 42 42 + of prods from me to finally figure out what was going on - it was to do with how odoc processes [includes] - specifically an [include sig ... end]. 43 43 + Essentially an include of that type ends up doubling the storage required of the signature. As the ppx_template extension does quite a lot 44 44 + of this, and in particular nests them, this ends up going exponential and this turned out to be the cause of most of the memory usage. 45 45 + With a fair bit more prodding by me, Claude and I eventually got to a solution, which I'll be upstreaming soon - the fix applies to OCaml as 46 46 + well as OxCaml, but it's this particularly pathalogical usage of includes that ppx_template uses where it'll make the most difference. 47 47 + 48 48 + {1 Odoc, plugins, JS and more} 49 49 + Teaser... I have a blog post coming soon with more on this. It's been a lot of fun, and 50 50 + should provide a decent inspiration for a roadmap for Odoc and online notebooks! 51 51 + 52 52 + 53 53 +

+3

site/blog/2026/03/index.mld

··· 1 1 + {0 March} 2 2 + 3 3 + @children_order weeknotes-2026-09

site/blog/2026/03/mapdemo.mov

This is a binary file and will not be displayed.

site/blog/2026/03/search.png

This is a binary file and will not be displayed.

+29

site/blog/2026/03/weeknotes-2026-09.mld

··· 1 1 + {0 Weeknotes 2026 week 9} 2 2 + 3 3 + @published 2026-03-02 4 4 + @notanotebook 5 5 + 6 6 + Let's make this really terse! 7 7 + 8 8 + {1 What did I do?} 9 9 + - Got docs working with github actions on Anil's oxmono monorepo. Results are {{:https://jonludlam.github.io/oxmono/}here}. 10 10 + This includes experimental support for oxcaml modes/layouts. 11 11 + - Got markdown mode output into Sherlodoc's db so you can query it - great for agents! 12 12 + {image!./search.png} 13 13 + - Widgets in the JS OCaml toplevels - using FRP for the interactions. The neat thing here is that using FRP via 14 14 + Daniel Bunzli's {{:}note} library is that all the interactions are all purely functional, no refs or mutables 15 15 + in sight. You provide a little wrapper scripts that's run in the frontend and the interactions and send back and 16 16 + forth with the worker running the code where it's translated into Events and Signals. My proof-of-concept of this 17 17 + is a widget that works with the {{:https://leafletjs.com/}leaflet.js} library: 18 18 + {video!mapdemo.mov} 19 19 + Demo coming soon! 20 20 + - Consolidating all of the Odoc toplevel bits and pieces into the one monorepo. Again, demo of this coming soon! 21 21 + 22 22 + {1 What am I going to do?} 23 23 + - New website! 24 24 + - Odoc plugins showcase 25 25 + - Writing writing writing writing 26 26 + 27 27 + 28 28 + 29 29 +

+4 -1

site/blog/2026/index.mld

··· 1 1 - @children_order 02/ 01/ 1 1 + @children_order 03/ 02/ 01/ 2 2 {0 2026} 3 3 4 4 5 5 + - {{!/jon-site/blog/2026/03/page-"weeknotes-2026-09"}Weeknotes for week 9} 6 6 + - {{!/jon-site/blog/2026/02/page-"weeknotes-2026-08"}Weeknotes for weeks 7-8} 7 7 + - {{!/jon-site/blog/2026/02/page-"odoc-js-notebooks-fun"}Fun with odoc, plugins and javascript} 5 8 - {{!/jon-site/blog/2026/02/page-"weeknotes-2026-06"}Weeknotes for week 6} 6 9 - {{!/jon-site/blog/2026/01/page-"weeknotes-2026-04-05"}Weeknotes for weeks 4-5} 7 10 - {{!/jon-site/blog/2026/01/page-"weeknotes-2026-03"}Weeknotes for week 3}

+287

site/notebooks/interactive_map.mld

··· 1 1 + {0 TESSERA Interactive Map} 2 2 + 3 3 + Explore geospatial embeddings from the 4 4 + {{:https://geotessera.org}GeoTessera} foundation model 5 5 + directly in the browser. This notebook walks through a complete 6 6 + land-cover classification workflow: 7 7 + 8 8 + {ol 9 9 + {- Draw a bounding box to select your region of interest} 10 10 + {- Fetch and visualise GeoTessera embeddings with PCA} 11 11 + {- Click on the map to place labelled training points} 12 12 + {- Run k-nearest-neighbours classification} 13 13 + {- View the classification overlay}} 14 14 + 15 15 + {@ocaml kind=setup[ 16 16 + #require "tessera-geotessera-jsoo";; 17 17 + #require "tessera-viz-jsoo";; 18 18 + #require "js_top_worker-widget-leaflet";; 19 19 + Widget_leaflet.register ();; 20 20 + ]} 21 21 + 22 22 + {1 Select region of interest} 23 23 + 24 24 + Draw a rectangle on the map to select the area you want to classify. 25 25 + The map is centred on Cambridge, UK — navigate to any area of interest 26 26 + before drawing. 27 27 + 28 28 + {@ocaml[ 29 29 + (* Shared state *) 30 30 + let bbox : Geotessera.bbox option ref = ref None 31 31 + let mosaic : (Linalg.mat * int * int) option ref = ref None 32 32 + let projected : Linalg.mat option ref = ref None 33 33 + let training_points : (float * float * int) list ref = ref [] 34 34 + let current_class = ref 0 35 35 + let class_names = ref [| "water"; "land" |] 36 36 + let class_colors = [| "#2196F3"; "#4CAF50"; "#FF9800"; "#9C27B0"; "#F44336"; 37 37 + "#00BCD4"; "#795548"; "#607D8B" |] 38 38 + 39 39 + (* Downsample a mosaic to at most max_pixels, preserving aspect ratio *) 40 40 + let downsample_mosaic mat ~h ~w ~max_pixels = 41 41 + let n_pixels = h * w in 42 42 + if n_pixels <= max_pixels then (mat, h, w) 43 43 + else 44 44 + let stride = int_of_float (ceil (sqrt (float_of_int n_pixels /. float_of_int max_pixels))) in 45 45 + let h' = (h + stride - 1) / stride in 46 46 + let w' = (w + stride - 1) / stride in 47 47 + let out = Linalg.create_mat ~rows:(h' * w') ~cols:mat.Linalg.cols in 48 48 + for i = 0 to h' - 1 do 49 49 + for j = 0 to w' - 1 do 50 50 + let si = min (i * stride) (h - 1) in 51 51 + let sj = min (j * stride) (w - 1) in 52 52 + let src = si * w + sj in 53 53 + let dst = i * w' + j in 54 54 + for f = 0 to mat.Linalg.cols - 1 do 55 55 + Linalg.mat_set out dst f (Linalg.mat_get mat src f) 56 56 + done 57 57 + done 58 58 + done; 59 59 + (out, h', w') 60 60 + 61 61 + let map_id = "tessera-map" 62 62 + 63 63 + let status_view text = 64 64 + let open Widget.View in 65 65 + Element { tag = "div"; attrs = [Style ("padding", "8px"); Style ("font-family", "monospace")]; 66 66 + children = [Text text] } 67 67 + 68 68 + let () = Widget.display ~id:"status" ~handlers:[] (status_view "Draw a rectangle on the map.") 69 69 + 70 70 + let () = 71 71 + Widget.display_managed ~id:map_id 72 72 + ~kind:"leaflet-map" 73 73 + ~config:{| {"center": [52.2, 0.12], "zoom": 13, "height": "500px"} |} 74 74 + ~handlers:[ 75 75 + "bbox_drawn", (fun v -> 76 76 + match v with 77 77 + | Some json -> 78 78 + (* Parse bbox JSON: {"south":..,"west":..,"north":..,"east":..} *) 79 79 + let s = Scanf.sscanf json 80 80 + {| {"south":%f,"west":%f,"north":%f,"east":%f}|} 81 81 + (fun s w n e -> Geotessera.{ min_lat = s; min_lon = w; max_lat = n; max_lon = e }) 82 82 + in 83 83 + bbox := Some s; 84 84 + Widget.update ~id:"status" 85 85 + (status_view (Printf.sprintf "Selected: %.4f,%.4f to %.4f,%.4f — run next cell to fetch embeddings." 86 86 + s.min_lat s.min_lon s.max_lat s.max_lon)) 87 87 + | None -> ()); 88 88 + "click", (fun v -> 89 89 + match v with 90 90 + | Some json -> 91 91 + let lat, lng = Scanf.sscanf json {| {"lat":%f,"lng":%f}|} (fun a b -> (a, b)) in 92 92 + (match !mosaic with 93 93 + | None -> () (* ignore clicks before embeddings loaded *) 94 94 + | Some _ -> 95 95 + let cls = !current_class in 96 96 + training_points := (lat, lng, cls) :: !training_points; 97 97 + let color = class_colors.(cls mod Array.length class_colors) in 98 98 + let label = (!class_names).(cls) in 99 99 + Widget.command ~id:map_id "addMarker" 100 100 + (Printf.sprintf {|{"lat":%f,"lng":%f,"color":"%s","label":"%s"}|} 101 101 + lat lng color label); 102 102 + Widget.update ~id:"status" 103 103 + (status_view (Printf.sprintf "Added %s point at %.4f, %.4f (%d points total)" 104 104 + label lat lng (List.length !training_points)))) 105 105 + | None -> ()); 106 106 + ] 107 107 + 108 108 + let () = Widget.command ~id:map_id "enableBboxDraw" "" 109 109 + ]} 110 110 + 111 111 + {1 Fetch embeddings and visualise} 112 112 + 113 113 + After drawing your bounding box, run this cell to fetch GeoTessera 114 114 + embeddings and display a PCA false-colour visualisation. 115 115 + 116 116 + {@ocaml[ 117 117 + let () = 118 118 + match !bbox with 119 119 + | None -> Widget.update ~id:"status" (status_view "Error: draw a bounding box first!") 120 120 + | Some b -> 121 121 + Widget.update ~id:"status" (status_view "Fetching embeddings..."); 122 122 + let mat_full, h_full, w_full = Geotessera_jsoo.fetch_mosaic ~year:2024 b in 123 123 + Widget.update ~id:"status" 124 124 + (status_view (Printf.sprintf "Fetched %d×%d mosaic. Downsampling..." h_full w_full)); 125 125 + let mat, h, w = downsample_mosaic mat_full ~h:h_full ~w:w_full ~max_pixels:50_000 in 126 126 + mosaic := Some (mat, h, w); 127 127 + (* Compute actual mosaic bounds from tile grid (not user bbox) *) 128 128 + let snap = Geotessera.snap_to_grid in 129 129 + let mosaic_south = snap b.min_lat -. 0.05 in 130 130 + let mosaic_north = snap b.max_lat +. 0.05 in 131 131 + let mosaic_west = snap b.min_lon -. 0.05 in 132 132 + let mosaic_east = snap b.max_lon +. 0.05 in 133 133 + Widget.update ~id:"status" 134 134 + (status_view (Printf.sprintf "Working at %d×%d (%d pixels). Computing PCA..." h w (h * w))); 135 135 + (* PCA to 3 components for RGB visualisation *) 136 136 + let pca = Linalg.pca_fit ~max_samples:5000 mat ~n_components:3 in 137 137 + let proj = Linalg.pca_transform pca mat in 138 138 + projected := Some proj; 139 139 + let pca_img = Viz.pca_to_rgba ~width:w ~height:h proj in 140 140 + let url = Viz_jsoo.to_data_url pca_img in 141 141 + Widget.command ~id:map_id "addImageOverlay" 142 142 + (Printf.sprintf {|{"url":"%s","bounds":[[%f,%f],[%f,%f]],"opacity":0.7}|} 143 143 + url mosaic_south mosaic_west mosaic_north mosaic_east); 144 144 + Widget.update ~id:"status" 145 145 + (status_view "PCA overlay added. Click on the map to place training points, then run the classification cell.") 146 146 + ]} 147 147 + 148 148 + {1 Label training points} 149 149 + 150 150 + Click on the map to add training points. Use the buttons below to 151 151 + switch between classes before clicking. 152 152 + 153 153 + {@ocaml[ 154 154 + let make_class_buttons () = 155 155 + let open Widget.View in 156 156 + let buttons = Array.to_list (Array.mapi (fun i name -> 157 157 + let color = class_colors.(i mod Array.length class_colors) in 158 158 + let border = if i = !current_class then "3px solid black" else "1px solid #ccc" in 159 159 + Element { tag = "button"; 160 160 + attrs = [ 161 161 + Handler ("click", "class_" ^ string_of_int i); 162 162 + Style ("margin", "4px"); 163 163 + Style ("padding", "8px 16px"); 164 164 + Style ("background", color); 165 165 + Style ("color", "white"); 166 166 + Style ("border", border); 167 167 + Style ("border-radius", "4px"); 168 168 + Style ("cursor", "pointer"); 169 169 + ]; 170 170 + children = [Text name] } 171 171 + ) !class_names) in 172 172 + Element { tag = "div"; 173 173 + attrs = [Style ("padding", "8px")]; 174 174 + children = 175 175 + Element { tag = "b"; attrs = []; children = [Text "Active class: "] } 176 176 + :: buttons 177 177 + @ [Element { tag = "button"; 178 178 + attrs = [ 179 179 + Handler ("click", "add_class"); 180 180 + Style ("margin", "4px"); 181 181 + Style ("padding", "8px 16px"); 182 182 + Style ("border", "1px dashed #999"); 183 183 + Style ("border-radius", "4px"); 184 184 + Style ("cursor", "pointer"); 185 185 + ]; 186 186 + children = [Text "+ Add class"] }; 187 187 + Element { tag = "button"; 188 188 + attrs = [ 189 189 + Handler ("click", "clear_points"); 190 190 + Style ("margin", "4px"); 191 191 + Style ("padding", "8px 16px"); 192 192 + Style ("border", "1px solid #ccc"); 193 193 + Style ("border-radius", "4px"); 194 194 + Style ("cursor", "pointer"); 195 195 + ]; 196 196 + children = [Text "Clear all"] }] 197 197 + } 198 198 + 199 199 + let class_handler_list () = 200 200 + let base = Array.to_list (Array.mapi (fun i _name -> 201 201 + "class_" ^ string_of_int i, (fun (_v : string option) -> 202 202 + current_class := i; 203 203 + Widget.update ~id:"class-buttons" (make_class_buttons ())) 204 204 + ) !class_names) in 205 205 + base @ [ 206 206 + "add_class", (fun _v -> 207 207 + let n = Array.length !class_names in 208 208 + let name = "class_" ^ string_of_int n in 209 209 + class_names := Array.append !class_names [| name |]; 210 210 + current_class := n; 211 211 + Widget.update ~id:"class-buttons" (make_class_buttons ())); 212 212 + "clear_points", (fun _v -> 213 213 + training_points := []; 214 214 + Widget.command ~id:map_id "clearMarkers" ""; 215 215 + Widget.update ~id:"status" (status_view "Cleared all training points.")); 216 216 + ] 217 217 + 218 218 + let () = 219 219 + Widget.display ~id:"class-buttons" 220 220 + ~handlers:(class_handler_list ()) 221 221 + (make_class_buttons ()) 222 222 + ]} 223 223 + 224 224 + {1 Classify and display} 225 225 + 226 226 + Run this cell after placing training points to classify the entire 227 227 + region using k-nearest neighbours. 228 228 + 229 229 + {@ocaml[ 230 230 + let () = 231 231 + match !mosaic, !projected, !bbox with 232 232 + | Some (mat, h, w), Some _proj, Some b -> 233 233 + let points = !training_points in 234 234 + if points = [] then 235 235 + Widget.update ~id:"status" (status_view "Error: place some training points first!") 236 236 + else begin 237 237 + Widget.update ~id:"status" 238 238 + (status_view (Printf.sprintf "Classifying with %d training points..." (List.length points))); 239 239 + (* Compute actual mosaic bounds from tile grid *) 240 240 + let snap = Geotessera.snap_to_grid in 241 241 + let mosaic_south = snap b.min_lat -. 0.05 in 242 242 + let mosaic_north = snap b.max_lat +. 0.05 in 243 243 + let mosaic_west = snap b.min_lon -. 0.05 in 244 244 + let mosaic_east = snap b.max_lon +. 0.05 in 245 245 + (* Convert geo coords to pixel coords and extract embeddings *) 246 246 + let n_train = List.length points in 247 247 + let train_mat = Linalg.create_mat ~rows:n_train ~cols:mat.Linalg.cols in 248 248 + let train_labels = Array.make n_train 0 in 249 249 + List.iteri (fun i (lat, lng, cls) -> 250 250 + (* Map lat/lng to pixel row/col using mosaic bounds *) 251 251 + let row = int_of_float ((mosaic_north -. lat) /. (mosaic_north -. mosaic_south) *. float_of_int h) in 252 252 + let col = int_of_float ((lng -. mosaic_west) /. (mosaic_east -. mosaic_west) *. float_of_int w) in 253 253 + let row = max 0 (min (h - 1) row) in 254 254 + let col = max 0 (min (w - 1) col) in 255 255 + (* Copy embedding row *) 256 256 + let src_offset = (row * w + col) * mat.Linalg.cols in 257 257 + let dst_offset = i * mat.Linalg.cols in 258 258 + for j = 0 to mat.Linalg.cols - 1 do 259 259 + Bigarray.Array1.set train_mat.Linalg.data (dst_offset + j) 260 260 + (Bigarray.Array1.get mat.Linalg.data (src_offset + j)) 261 261 + done; 262 262 + train_labels.(i) <- cls 263 263 + ) points; 264 264 + (* kNN classification *) 265 265 + let model = Linalg.knn_fit ~embeddings:train_mat ~labels:train_labels in 266 266 + let k = min 5 n_train in 267 267 + let result = Linalg.knn_predict model ~k mat in 268 268 + (* Build color map *) 269 269 + let n_classes = Array.fold_left max 0 train_labels + 1 in 270 270 + let colors = List.init n_classes (fun i -> 271 271 + let hex = class_colors.(i mod Array.length class_colors) in 272 272 + (i, Viz.color_of_hex hex) 273 273 + ) in 274 274 + let class_img = Viz.classification_to_rgba 275 275 + ~predictions:result.Linalg.predictions 276 276 + ~colors ~width:w ~height:h () in 277 277 + let class_url = Viz_jsoo.to_data_url class_img in 278 278 + Widget.command ~id:map_id "addImageOverlay" 279 279 + (Printf.sprintf {|{"url":"%s","bounds":[[%f,%f],[%f,%f]],"opacity":0.7}|} 280 280 + class_url mosaic_south mosaic_west mosaic_north mosaic_east); 281 281 + Widget.update ~id:"status" 282 282 + (status_view (Printf.sprintf "Classification complete! %d classes, %d training points." 283 283 + n_classes n_train)) 284 284 + end 285 285 + | _ -> 286 286 + Widget.update ~id:"status" (status_view "Error: fetch embeddings first!") 287 287 + ]}