Spec §2 — Lexical structure

Source text is UTF-8. Lexing converts it to a token stream; trivia (whitespace and comments) separates tokens and is otherwise discarded. A file consisting only of trivia lexes to an empty token stream (and parses as an empty module).

Line terminators. A line terminator is \n, and the two-character sequence \r\n is one line terminator. Text built from source (the value of any string literal) is built from the normalized text, so a literal spanning lines carries \n per source line break and never \r\n, whatever the file’s on-disk encoding. A lone \r (no following \n) is not a line terminator: between tokens it is ordinary whitespace, and inside a string literal it is an ordinary character, preserved. A leading U+FEFF byte-order mark is an encoding marker rather than source text: it is ignored, and positions are counted from the byte after it. A U+FEFF anywhere else is content. Together these make a program’s meaning independent of how an editor saved it: the same source is the same program on every platform. (See the design notes §2; the canonical on-disk form is LF with no BOM, and vilan fmt writes it.)

2.1 Comments

A comment begins with // and runs to the end of the line. There are no block comments.

2.2 Identifiers and keywords

IDENT = ascii_letter | "_" , { ascii_letter | digit | "_" } ;
RESERVED = (* a word of the reserved list below: its own token, never IDENT *) ;

Identifiers are ASCII. The following words are reserved; they lex as keyword tokens and are never IDENT:

async     await     const    css     else    enum    export
external  for       fun      if      impl    import  in
is        let       macro    match   mod     mut     null
ret       struct    trait    type    use     true    false

(true/false lex as boolean literals; null as the null literal.)

A reserved word is still a member name (B414 S4): the name after a member . or ?., a field declared in a struct or given with = in a struct literal, a method declared in an impl or a trait, and a segment after :: in an expression path — event.type, Event { type = kind }, fun match(self), Kit::if(). Each of those positions is entered after a token that commits to it, so no keyword reading can begin there (§3.6). A reserved word never names a binding, a parameter, a free function or a type, and the struct-literal shorthand { type } — which reads a binding of that name — stays refused. After a member dot the name is written AGAINST the dot, with nothing between them (§3.6).

The following words are contextual keywords: they lex as IDENT and take meaning only by position, and are ordinary identifiers everywhere else (vilan --print-keywords prints both lists, each word flagged):

as        auto      borrows   context   dyn       jump      lazy
only      own       self      Self      sync      then      void
with

with is the trait list of an impl or trait head (§3.3); borrows the return clause after a declaration’s return type (§3.3); own and lazy a parameter’s prefix when a binder follows (§3.3), and lazy also opens a lazy let (§3.4); dyn the trait-object marker at a type’s head (§3.9 — dyn:: is a path instead); auto the toolchain-kept type at an annotation’s head (§3.9, §5.8 — auto:: is a path instead); jump loop control when its target follows (§3.4); context the clause after a closure type or a return type (§3.9); sync the marker opening a closure type (§3.9); then the infix conditional after a complete operand (§3.8); void the unit value and type; self and Self the receiver and its type (self is also the file’s own module in mod self;, §3.1); as the alias on an import path leaf (§3.2) and the type ascription after a complete operand (§3.6, §5.8), and only the trailing modifier on an import (§3.2). Beside them, two families of words are matched by text in one position and are ordinary identifiers everywhere else: the ATTRIBUTE names in [...] position (derive, service, client_service, extern, must_use, track_caller, rpc, trait_only, doc, expose, platform, deprecated, internal, resource, hint, reactive — resource was a keyword until B413 made it the [resource] attribute), and the jump TARGETS break and continue after jump. All remain usable as ordinary identifiers elsewhere, with one exception: void may not be a BINDER’s name (a let, a for binder, a function parameter, a match capture). An expression void is always the unit, so a binding by that name could never be read back, and it is refused where it is written rather than where it is read (N113). A member — a struct field, a method — may still be called void: it is reached through a receiver, not through the atom production.

Which a new marker becomes (B485, RULED 2026-10-01). A word is a keyword when it changes what may be written after it (export takes (in PATH) or *, external licenses a ; body, macro declares a compile-time function, const and async also prefix an expression), or when a reader of the signature line needs it to know how a use calls, passes, receives, awaits or evaluates the declaration (async, const fun, lazy, own, &, mut, borrows, context, sync, dyn). Everything else about a declaration is an attribute: its labels, its generated companions, its host binding, its platform fence, the class of a declared type — on the proviso that skipping it while reading never misleads silently, so every effect it has on a use is a diagnostic at that use. Attributes stand above the signature, keywords in it, in one order (§3.2).

Words other languages reserve for vilan’s constructs — return, fn, function, func, def — are ordinary identifiers. Where one stands in the place of ret or fun (return value;, fn name(), or -> in the place of a return type’s :, the parser refuses it once with vilan’s spelling and reads on as if that had been written (B520). Where return can only be a name — return;, return (x), return -x, a block’s tail { return } — the same steer is given where the name fails to resolve, unless the program binds one (B523).

2.3 Literals

Numbers

NUMBER = decimal | hex ;
decimal = digits , [ "." , digits ] , [ SUFFIX ] ;
hex     = "0x" , hexdigit , { hexdigit } , [ SUFFIX ] ;
SUFFIX  = IDENT   (* immediately adjacent, no space *)

The suffix names the literal’s type: i8 i16 i32 u8 u16 u32 (that two’s-complement width), i53/u53 (the wide integers; see below), usize (the index type; see below), f (f64), f32, f64, n (BigInt). An unknown suffix is a compile error (the retired i64/u64 suffixes get a rename hint). An unsuffixed literal takes its type from its CONTEXT — the annotation, the parameter, the field, the return type it lands at, the other operand of a binary operator (on either side: 1 + n and n + 1 are both n’s type), the element type of an annotated list literal (or a typed sibling element), the value a match pattern is tested against, and the type a generic call’s expected result binds its parameter to (let n: u53 = identity(5)) — and i32 for an integer literal, f64 for a fractional one, is the DEFAULT where the context states nothing. A binding with no annotation whose initializer is such a literal (let n = 0;, or arithmetic over literals and other such bindings) has no type of its own either: it takes the numeric type its first typed USE states — an argument, an operand beside a typed value (i < xs.len()), an assignment — and the default only when no use states one. That holds for an expression of unsuffixed literals as well as for a lone one, including the operator it is computed with: let ratio: f64 = 7 / 2; is float division and 3.5, where the same expression with no annotation truncates to 3. A FRACTIONAL literal never takes an integer type from its context: to the right of an integer operand (y / 2.0 with y: i32) it is a compile error whose fix is the other operand’s conversion (y.as_f64() / 2.0). Every integer literal is range-checked against its type at compile time, and a negative one at an unsigned type (let n: usize = -1) is out of range — as is a literal-only expression that folds negative there (let end: usize = 0 - 1;). Negating an unsigned VALUE (-u for u: usize) is refused too: convert it to a signed type first (-u.as_i53()).

i53 spans the symmetric range ±2^53 and u53 spans [0, 2^53]: the window in which every integer is exactly representable in an IEEE-754 double (the backing representation). The names deliberately follow JavaScript’s safe-integer convention (53 bits of integer precision) rather than the two’s-complement iN convention. There is no i64; integers beyond the window take BigInt.

usize, the index type, spans [0, 2^53] as u53 does — on every backend, although its native representation is the platform word. Subtracting a usize past zero is unspecified, never memory-unsafe (§7.2a): the JS backend lets the value go negative, a native debug build panics, and every subscript’s bounds check still refuses the result as an index.

In a hex literal the digit run is maximal, so a suffix must begin with a non-hex letter: 0xFFu8 is valid; 0xFFf is a single hex number 0xFFF, not 0xFF with suffix f.

Strings

STRING           = '"' , { string_char } , '"' ;
string_char      = "\" any_char_except_line_terminator
                 | any_char_except_quote_backslash_or_line_terminator ;
MULTILINE_STRING = '"""' , raw_text , '"""' ;

In a plain string a backslash escapes the next character; escape sequences are preserved in the token and interpreted at code generation with JavaScript string-escape semantics (\n, \", \\, …). A multiline string is raw (a backslash is a backslash) and runs to the first """; the whitespace prefix of the line containing the closing delimiter is stripped from every line of the content.

A single-quoted string must close on the line it opens. A raw line break inside "…" is an error, and so is a backslash immediately before one: nothing escapes a line terminator, because the literal has to close on its line either way. Multi-line text is written """…"""; a single line break inside a one-line string is written \n. (The rule buys error locality: a forgotten closing quote is reported at its own line instead of running on to the next " anywhere below it.)

In a multiline string a source line break contributes a single \n to the value per the line-terminator rule above. An escaped \r is unaffected: it is written into the literal, not read off the end of a line.

Interpolated strings

ISTRING          = 'i' , '"' , { istring_part } , '"' ;
istring_part     = hole | istring_char ;
hole             = '{' , expression , '}' ;
istring_char     = "\" any_char_except_line_terminator
                 | any_char_except_brace_quote_backslash_or_line_terminator ;

i"…" is an interpolated string: {expr} holes embed expressions; \{ and \} are literal braces. Every other escape is carried through as the plain twin carries it — \ and the character it precedes, interpreted at code generation — and an unescaped } outside a hole is an error. The construct is defined by desugaring. An interpolated string is exactly equivalent to a parenthesized concatenation:

i"Hello, {name}!"   ≡   ("" + "Hello, " + (name) + "!")

Each hole’s contents are lexed as ordinary tokens (except that {/} delimit the hole; string literals inside a hole may still contain braces) and parsed as a single parenthesized expression. The result of the whole form is str; each part must therefore be valid as a + operand with str (§5’s operator dispatch).

A string literal inside a hole may be written either way — i"{get("k")}" or i"{get(\"k\")}" — and the two are the same token. The hole is lexed from the source bytes, so the enclosing literal’s quotes never reach it and the plain spelling needs no escape; the escaped spelling is accepted because it is the one a writer reaches for, and it closes at the next \". An embedded quote therefore has no escaped spelling: "a\"b", the plain form, is the one that carries it.

i"…" obeys the single-line rule of its plain twin: a raw line break in its body (or a backslash before one) is the same error. Interpolated multi-line text is i"""…""", which is how a macro writes the code it returns.

Interpolated multiline strings

INTERPOLATED_MULTILINE = 'i' , '"""' , raw_text , '"""' ;

i"""…""" is the multiline string with holes. Two rules apply, in this order:

1. Trimming, on the literal’s raw text. The layout rule of a plain """ applies unchanged and applies first: nothing may follow the opening delimiter on its line, the closing delimiter sits alone on its line, and the whitespace preceding it is the indentation prefix stripped from the start of every content line. Holes and \{ / \} count as ordinary characters of that text, so a hole never disturbs its line’s indent accounting: a line opening with a hole is indented like any other, and a hole in the middle of a line has no effect on it. A hole may span lines; its continuation lines carry the prefix like every other line, and stripping is a no-op inside the hole, where whitespace is trivia.
2. Fragmenting, on the trimmed text. Exactly two escapes exist: \{ and \}, each a literal brace. Nothing else is an escape: a backslash before any other character is a literal backslash and that character, the same near-rawness as a plain """ (\n is a backslash and an n). An unescaped } outside a hole is an error, as it is in i"…".

The body is raw and runs to the first """, so a single " and a "" pair are ordinary content. The value is a str, and (this being one of the two forms that may span lines) a source line break in it contributes one \n.

let report = i"""
    {name} scored {score}.
    Braces are written \{like this\}.
    """

Other literals

true, false (type bool); null (the host-boundary null, §5.2); void (the unit value; a contextual identifier, not a keyword).

2.4 Operators and punctuation

Two token classes:

• Operator tokens: a maximal run of the characters - : ! * / + = | & ^ ? %, plus the arrow => (lexed as one token). Maximal munch means ==, !=, +=, ::, ?.’s ?, &&, || each lex as single operator tokens; conversely a+-b lexes as a, +-, b and is a parse error.
• Control tokens: the single characters ( ) [ ] { } < > ; , ..
• #, a token of its own: the import reach marker, import pkg::a::{ #hidden }; (spec §4.3). It is in neither run — it never joins an operator run and never joins a name — so it is always exactly one token wherever it is written.

< and > are control tokens (they delimit generics), not operator characters. Consequently <=/>= lex as </> followed by =, and the shift operators <</>> are two adjacent control tokens; the parser accepts them as shifts only when span-adjacent (no whitespace): a << b is a shift, a < < b is a parse error (§3.7). The element form (grammar spec, atom) reuses the same discipline: </ and /> are span-adjacent pairs, on: joins by adjacency, and a hyphenated element or attribute name (aria-label, my-widget) is a span-adjacent name---name run — data - id is three tokens of arithmetic, not a name. The lexer itself is untouched by the element form: every token it produces already existed. The css block (grammar spec, atom) is the same: a property name is that identical span-adjacent run, and a dimension such as 1px or 1.5rem was already one number token (§2.3), so CSS’s own value shape needed nothing new.

@ is in none of these classes, and lexes as nothing at all. It belongs to no run and opens no literal, so it is a lex error wherever it is written — including inside a css block, which cannot change that, because lexing finishes before any parser exists (§2.5). The consequence is stated rather than worked around and the diagnostic names the spelling: a block has no at-rules, so a media query is a breakpoint condition rule (.md { … }) and a declaration block under a selector of your own is std::web::style::declare.

# lexes, and a css block refuses it. The byte is a token — the import reach marker — so the lexer has nothing to say about it, and the question “is this a colour or a reach?” is answered where the answer exists: inside a css block’s value, a # is the hex-colour mistake and the block’s own parser refuses it, naming the hole that routes the value through the Color type that carries its own :root line (color: {Color::hex("#333")};). That is the rule §2.5 demands, taken seriously in both directions: the LEXER cannot know which of the two it is looking at, so it does not guess, and the parser, which does know, decides. #id selectors are still unwritable inside a block; [id="x"] is the spelling.

2.5 Trivia and token separation

Whitespace (any Unicode whitespace) and comments may appear between any two tokens and are required only where two tokens would otherwise lex as one (fun main needs the space; a + b does not). Lexing is greedy and context-free: no token depends on parse state — which is why a construct whose body reads as another language, such as the css block, still lexes as ordinary vilan tokens and admits not one byte more.