gearsco.de
A JavaScript lexer and syntax highlighter for Gleam!
Initial release
+51
-18
+42
-8
README.md
+42
-8
README.md
···
1
-
# just
1
+
# Just
2
+
3
+
A JavaScript lexer and syntax highlighter for Gleam!
2
4
3
5
[](https://hex.pm/packages/just)
4
6
[](https://hexdocs.pm/just/)
5
7
8
+
Just is a JavaScript lexer and syntax highlighter written in Gleam.
9
+
The `just` module, based on [`glexer`](https://hexdocs.mp/glexer) exposes a
10
+
standard lexer API, allowing you to convert JavaScript source code into tokens.
11
+
The `just/highlight` module allows you to highlight javascript code using ansi
12
+
colours, html or a custom format. Heavily inspired by [`contour`](https://hexdocs.pm/contour).
13
+
6
14
```sh
7
15
gleam add just@1
8
16
```
17
+
9
18
```gleam
10
19
import just
20
+
import just/highlight
11
21
12
-
pub fn main() -> Nil {
13
-
// TODO: An example of the project in use
22
+
pub fn main() {
23
+
let code = "console.log('Hello, world!');"
24
+
25
+
let lexer = just.new(code) |> just.strict_mode
26
+
// Lex syntax tokens for parsing or other uses
27
+
let #(tokens, errors) = just.tokenise(lexer)
28
+
let assert [] = errors
29
+
parse_js(tokens)
30
+
31
+
// Highlight with ansi codes to print in the terminal
32
+
let highlighted = highlight.ansi(code)
33
+
io.println(highlighted)
34
+
35
+
// Render to html to show in the browser
36
+
let html = highlight.html(code)
37
+
io.println("<pre><code>" <> html <> "</code></pre>")
38
+
39
+
// Convert to "highlighting tokens" to highlight in some other way
40
+
let highlight_tokens = highlight.tokens(code)
41
+
highlight_tokens_some_other_way(highlight_tokens)
14
42
}
15
43
```
16
44
17
45
Further documentation can be found at <https://hexdocs.pm/just>.
18
46
19
-
## Development
47
+
### Missing features
48
+
The Just lexer should be able to accurately lex most valid JavaScript programs,
49
+
but there are a few things it is missing:
20
50
21
-
```sh
22
-
gleam run # Run the project
23
-
gleam test # Run the tests
24
-
```
51
+
- Proper backtracking for Regular Expressions. Currently the lexer uses the previously
52
+
lexed token to determine whether to lex a `/` character as a division operator or a
53
+
regular expression. This means it can fail in some edge-cases.
54
+
- Lexing of full-unicode identifiers. JavaScript supports more than just ASCII characters
55
+
to make up its identifiers. Currently, Just doesn't support non-ASCII characters in
56
+
identifiers.
57
+
- Lexing identifier escape-sequences. Similarly, JavaScript allows unicode escape sequences
58
+
as part of identifiers (e.g. `let \u0065 = 10;`). This is currently not supported by Just.
+4
-10
gleam.toml
+4
-10
gleam.toml
···
1
1
name = "just"
2
2
version = "1.0.0"
3
3
4
-
# Fill out these fields if you intend to generate HTML documentation or publish
5
-
# your project to the Hex package manager.
6
-
#
7
-
# description = ""
8
-
# licences = ["Apache-2.0"]
9
-
# repository = { type = "github", user = "", repo = "" }
10
-
# links = [{ title = "Website", href = "" }]
11
-
#
12
-
# For a full reference of all the available options, you can have a look at
13
-
# https://gleam.run/writing-gleam/gleam-toml/.
4
+
description = "A JavaScript lexer and syntax highlighter for Gleam!"
5
+
licences = ["Apache-2.0"]
6
+
repository = { type = "github", user = "GearsDatapacks", repo = "just" }
7
+
links = []
14
8
15
9
[dependencies]
16
10
gleam_stdlib = ">= 0.44.0 and < 2.0.0"