ilyagr.bsky.social
just playing with tangled
1# Git submodules
2
3This is an aspirational document that describes how jj _will_ support Git
4submodules. Readers are assumed to have some familiarity with Git and Git
5submodules.
6
7This document is a work in progress; submodules are a big feature, and relevant
8details will be filled in incrementally.
9
10## Objective
11
12This proposal aims to replicate the workflows users are used to with Git
13submodules, e.g.:
14
15- Cloning submodules
16- Making new submodule commits and updating the superproject
17- Fetching and pushing updates to the submodule's remote
18- Viewing submodule history
19
20When it is convenient, this proposal will also aim to make submodules easier to
21use than Git's implementation.
22
23### Non-goals
24
25- Non-Git 'submodules' (e.g. native jj submodules, other VCSes)
26- Non-Git backends (e.g. Google internal backend)
27- Changing how Git submodules are implemented in Git
28
29## Background
30
31We mainly want to support Git submodules for feature parity, since Git
32submodules are a standard feature in Git and are popular enough that we have
33received user requests for them. Secondarily (and distantly so), Git submodules
34are notoriously difficult to use, so there is an opportunity to improve the UX
35over Git's implementation.
36
37### Intro to Git Submodules
38
39[Git submodules](https://git-scm.com/docs/gitsubmodules) are a feature of Git
40that allow a repository (submodule) to be embedded inside another repository
41(the superproject). Notably, a submodule is a full repository, complete with its
42own index, object store and ref store. It can be interacted with like any other
43repository, regardless of the superproject.
44
45In a superproject commit, submodule information is captured in two places:
46
47- A `gitlink` entry in the commit's tree, where the value of the `gitlink` entry
48 is the submodule commit id. This tells Git what to populate in the working
49 tree.
50
51- A top level `.gitmodules` file. This file is in Git's config syntax and
52 entries take the form `submodule.<submodule-name>.*`. These include many
53 settings about the submodules, but most importantly:
54
55 - `submodule<submodule-name>.path` contains the path from the root of the tree
56 to the `gitlink` being described.
57
58 - `submodule<submodule-name>.url` contains the url to clone the submodule
59 from.
60
61In the working tree, Git notices the presence of a submodule by the `.git` entry
62(signifying the root of a Git repository working tree). This is either the
63submodule's actual Git directory (an "old-form" submodule), or a `.git` file
64pointing to `<superproject-git-directory>/modules/<submodule-name>`. The latter
65is sometimes called the "absorbed form", and is Git's preferred mode of
66operation.
67
68## Roadmap
69
70Git submodules should be implemented in an order that supports an increasing set
71of workflows, with the goal of getting feedback early and often. When support is
72incomplete, jj should not crash, but instead provide fallback behavior and warn
73the user where needed.
74
75The goal is to land good support for pure Jujutsu repositories, while colocated
76repositories will be supported when convenient.
77
78This section should be treated as a set of guidelines, not a strict order of
79work.
80
81### Phase 1: Readonly submodules
82
83This includes work that inspects submodule contents but does not create new
84objects in the submodule. This requires a way to store submodules in a jj
85repository that supports readonly operations.
86
87#### Outcomes
88
89- Submodules can be cloned anew
90- New submodule commits can be fetched
91- Submodule history and branches can be viewed
92- Submodule contents are populated in the working copy
93- Superproject gitlink can be updated to an existing submodule commit
94- Conflicts in the superproject gitlink can be resolved to an existing submodule
95 commit
96
97### Phase 2: Snapshotting new changes
98
99This allows a user to write new contents to a submodule and its remote.
100
101#### Outcomes
102
103- Changes in the working copy can be recorded in a submodule commit
104- Submodule branches can be modified
105- Submodules and their branches can be pushed to their remote
106
107### Phase 3: Merging/rebasing/conflicts
108
109This allows merging and rebasing of superproject commits in a content-aware way
110(in contrast to Git, where only the gitlink commit ids are compared), as well as
111workflows that make resolving conflicts easy and sensible.
112
113This can be done in tandem with Phase 2, but will likely require a significant
114amount of design work on its own.
115
116#### Outcomes
117
118- Merged/rebased submodules result in merged/rebased working copy content
119- Merged/rebased working copy content can be committed, possibly by creating
120 sensible merged/rebased submodule commits
121- Merge/rebase between submodule and non-submodule gives a sensible result
122- Merge/rebase between submodule A and submodule B gives a sensible result
123
124### Phase ?: An ideal world
125
126I.e. outcomes we would like to see if there were no constraints whatsoever.
127
128- Rewriting submodule commits rewrites descendants correctly and updates
129 superproject gitlinks.
130- Submodule conflicts automatically resolve to the 'correct' submodule commits,
131 e.g. a merge between superproject commits creating a merge of the submodule
132 commits.
133- Nested submodules are as easy to work with as non-nested submodules.
134- The operation log captures changes in the submodule.
135
136## Design
137
138### Guiding principles
139
140TODO
141
142### Storing submodules
143
144Possible approaches under discussion. See
145[./git-submodule-storage.md](./git-submodule-storage.md).
146
147### Snapshotting new submodule changes
148
149TODO
150
151### Merging/rebasing with submodules
152
153TODO