You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs-language/syntax.md
+19-25Lines changed: 19 additions & 25 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -8,13 +8,11 @@ import TokenScriptCodeBlock from '@site/src/components/TokenScriptCodeBlock';
8
8
9
9
# Lexical Syntax
10
10
11
-
TokenScript uses a lightweight, whitespace-insensitive syntax optimized for expressing token derivations. This chapter outlines the lexical rules enforced by the lexer (`src/interpreter/lexer.ts`) and the structural expectations enforced by the parser (`src/interpreter/parser.ts`).
12
-
13
11
## Character Set & Identifiers
14
12
15
-
-Source code is UTF-8. Identifiers may include standard ASCII letters, digits, hyphen (`-`), underscore (`_`), and extended Unicode characters (e.g., emoji).
13
+
- Identifiers may include standard ASCII letters, digits, hyphen (`-`), underscore (`_`), and extended Unicode characters (e.g., emoji).
16
14
- Identifiers must begin with an alphabetic or non-ASCII character (numbers are not allowed in the first position).
17
-
- Identifiers are case-sensitive (`primaryColor` and `PrimaryColor` are distinct).
15
+
- Identifiers are case-insensitive (`primaryColor` and `PrimaryColor` are treated equally).
18
16
19
17
## Whitespace & Newlines
20
18
@@ -23,7 +21,7 @@ TokenScript uses a lightweight, whitespace-insensitive syntax optimized for expr
23
21
24
22
## Comments
25
23
26
-
- Single-line comments start with `//` and run to the end of the line. Block comments are not currently supported.
24
+
- Single-line comments start with `//` and run to the end of the line.
| Explicit Strings |`"quoted value"` or `'alternate quotes'`| Preserve whitespace and special characters. |
41
39
| Booleans |`true`, `false`| Reserved keywords mapped to `Boolean`. |
42
40
| Null |`null`| Reserved keyword mapped to `Null`. |
43
-
| Undefined |`undefined`| Reserved keyword; the interpreter currently treats it as a literal string when encountered. |
44
41
| Lists |`value1, value2, value3`| Comma-separated sequence; implicit lists use spaces when built by certain operations. |
45
42
46
43
## Units
47
44
48
-
Numbers may be suffixed with a unit from `src/types.ts#SupportedFormats`, including `px`, `em`, `rem`, `vw`, `vh`, `pt`, `in`, `cm`, `mm`, `deg`, `%`. The lexer emits a `FORMAT` token, which the parser combines with the preceding numeric literal to produce an `ElementWithUnitNode`. At runtime the interpreter constructs a `NumberWithUnitSymbol`.
49
-
45
+
Numbers may be suffixed with a units like `px`, `em`, `rem`, `vw`, `vh`, `pt`, `in`, `cm`, `mm`, `deg`, `%`.
0 commit comments