Docs · Language

Lexical structure

This page defines how Vyi source text becomes tokens: the encoding, identifier and keyword rules, every literal form and escape, comments, and how statements are separated. It is the lookup companion to guide 02 — Values and bindings, which introduces literals and bindings by example.

Source encoding

Vyi source is UTF-8 throughout: identifiers, string and char literals, comments, and trivia. Invalid UTF-8 sequences are reported as unexpected characters and skipped so lexing can continue.

con héllo = 1
con 名前 = 2
// comments may contain 任意の文字

Lexing never gives up: an unexpected byte or disallowed code point is reported and skipped, and the rest of the file still lexes, so one stray character produces one diagnostic, not a cascade.

Tokens

The lexer classifies source into:

Trivia is skipped by the parser but not discarded — newlines still delimit statements (below), and /// doc comments attach to declarations (below).

Identifiers

An identifier is a UTF-8 name:

• start: ASCII letter, _, $, @, or a Unicode letter (including letter numbers)
• continue: start characters, ASCII digits, Unicode digits, combining marks, and connector punctuation

The first character may not be a digit. _ and $ are ordinary identifier characters, usable anywhere in a name. Keywords remain the fixed ASCII spellings (con, fn, …); a Unicode spelling is always a plain identifier.

import { stdout } from "@io"

con _hidden = 10
con my_value2 = 20
con $dollar = 12
con 名前 = 42
con héllo = 0

fn main() !u8 {
    stdout.writeInt(i64(_hidden + my_value2 + $dollar + 名前 + héllo))  // prints 84
    stdout.write("\n")
    return Ok(0)
}

A lone _ is not an identifier — it is the inference/discard placeholder, with its own grammar roles (see types and patterns).

Names of built-in types and variants — i32, bool, any, Some, Ok — are not keywords. They are ordinary bindings provided by @core, resolved by the normal scope rules (and therefore shadowable like any other name).

The @ sigil

@ is an identifier character, and a leading @ is a naming convention with two established uses:

• Operator and protocol method names — @add, @eq, @index, @errmsg, … .

Operators desugar to calls of these methods: a + b is a.@add(b). The full mapping lives in the operators reference; guide 13 introduces it.

• Standard-library package paths — the @ in import { stdout } from "@io" is

part of the import string, marking a stdlib package (@core, @io, @mem, …). See guide 14.

There is no standalone @ token — @add lexes as one identifier.

Keywords

The complete keyword set. A keyword can never be used as an identifier.

Notes:

• true and false are the two bool literals, lexed as keywords.
• The c_* group belongs to C interop (guide 15).
• There is no void (a function with no return type returns (), the empty

tuple), no while/loop (all loops are for), no match (switch covers it), no null, and no and/or/not (the boolean operators are &&, ||, !).

Integer literals

Four bases, all accepting _ as a digit separator anywhere between digits:

The base prefix is case-insensitive (0x/0X, 0b/0B, 0o/0O). There is no sign in the literal itself — -5 is the unary - operator applied to 5.

An integer literal has no fixed type of its own: it becomes whatever integer type the context asks for (an annotation, a parameter, a cast target), and defaults to i32 when nothing asks. A literal that does not fit its target type is a compile error, not a silent wrap:

fn takesU8(x: u8) u8 { return x }

fn main() !u8 {
    con a = 7             // no context → i32
    con b: i64 = 7        // annotation → i64
    con c = takesU8(7)    // parameter → u8
    return Ok(0)
}

con b: u8 = 300    // error: integer literal 300 overflows u8

Float literals

A float literal is digits, a ., and digits — both sides required, _ separators allowed:

con pi = 3.141_592     // f64 by default
con half: f32 = 0.5    // context makes it f32

There is no exponent form (1e5 does not lex as a number) and no leading- or trailing-dot form: .5 is not a literal, and 1. is the integer 1 followed by the . operator — which keeps 1..10 (a range) and 1.method() (a call on the literal) unambiguous. A float literal defaults to f64.

Char literals

'…' is a char literal: exactly one Unicode codepoint (or one escape), producing a 32-bit char value:

con a = 'A'
con emoji = '😀'          // one codepoint, even though it is 4 bytes in UTF-8
con nl = '\n'
con hexed = '\x41'        // 'A' by hex value
con uni = '\u{1F600}'     // '😀' by codepoint
con quote = '\''

More than one codepoint between the quotes is an error — text is a string, in double quotes:

con c = 'ab'    // error: char literal holds exactly one codepoint

String literals

"…" is a string literal: a sequence of UTF-8 bytes baked into the compiled program, flowing into [*]u8 (a pointer-plus-length view of bytes). A string literal must open and close on one line — a raw newline inside one is an "unterminated string literal" error; embed line breaks with \n.

The escape sequences, shared by strings and chars:

import { stdout } from "@io"

fn main() !u8 {
    con s = "tab\t nl\n quote\" hex\x41 uni\u{1F600}"
    stdout.writeInt(i64(s.len - s.len))
    stdout.write("\n")
    return Ok(0)
}

\u{…} rejects values above U+10FFFF and the surrogate range. A backslash followed by anything not in the table is not an escape — the backslash and the character pass through as literal bytes.

Strings have no fixed maximum length, no interpolation form, and no multi-line/raw form. Guide 10 covers working with string data.

Comments

Three forms:

import { stdout } from "@io"

// A line comment — runs to the end of the line.

/* A block comment — runs to the closing marker,
   across as many lines as needed. */

/// A doc comment. Contiguous /// lines form one block that
/// attaches to the declaration immediately below them.
fn documented() i32 {
    return 42
}

fn main() !u8 {
    stdout.writeInt(i64(documented()))
    stdout.write("\n")
    return Ok(0)
}

Block comments do not nest: the first */ closes the comment, so commenting out code that itself contains */ breaks:

/* outer /* inner */ still comment? */
fn main() !u8 { return Ok(0) }

/// doc comments attach to the top-level declaration that follows them (with only blank space in between); tooling — the language server, editor hover — surfaces the attached text. A doc block separated from any declaration is treated as an ordinary comment.

Newlines and statement separation

Statements are separated by newlines — there is no required terminator:

con a = 1
con b = 2

Precisely: a statement ends at a newline whenever what came before it is complete. An expression continues onto the next line only when the last token on the line cannot end one — a binary operator, a ., a comma, or an unclosed (/[/{. So break long expressions after the operator, never before it:

import { stdout } from "@io"

fn add3(a: i32, b: i32, c: i32) i32 { return a + b + c }

fn main() !u8 {
    con x = 1 +        // trailing operator: continues
        2
    con y = add3(      // unclosed paren: argument list spans lines
        1,
        2,
        3,
    )
    stdout.writeInt(i64(x + y))
    stdout.write("\n")
    return Ok(0)
}

fn main() !u8 {
    con x = 1
        + 2       // error: unexpected Plus at statement start
    return Ok(0)
}

The flip side: a ( or [ at the start of a line begins a new statement — it is never a call or index of the previous line's expression. A postfix chain crosses lines only via a trailing token that demands more (in practice, the trailing comma and open-delimiter forms above).

A semicolon is accepted as an explicit separator equivalent to a newline, for putting several statements on one line:

import { stdout } from "@io"

fn main() !u8 { con a = 1; con b = 2; stdout.writeInt(i64(a + b)); stdout.write("\n"); return Ok(0) }

It carries no other meaning, and idiomatic Vyi does not use it — one statement per line needs no separator at all.

Commas — in argument lists, aggregate literals, and switch arms — are part of those constructs' grammar, not statement separators; see statements for the switch comma rule.

Labels

#name — a # immediately followed by an identifier — is a label token. A label prefixes a loop or a block, and break #name / continue #name target it:

import { stdout } from "@io"

fn main() !u8 {
    mut hits = 0
    #outer for i in 0..3 {
        for j in 0..3 {
            if j > i { continue #outer }
            hits = hits + 1
        }
    }
    stdout.writeInt(i64(hits))  // prints 6
    stdout.write("\n")
    return Ok(0)
}

The label rules — what may be labeled, which jumps may target what, break with a value — live in statements and expressions; guide 05 introduces them.

Operators and punctuation

The complete operator and punctuation set. Multi-character operators lex greedily (>>= is one token, not > >=).

Several tokens do double duty by position — prefix & is address-of while infix & is bitwise-and; prefix * is dereference while infix * is multiplication; ! is boolean not in expressions and the error-union marker in types. Meaning, precedence, and the operator→method desugaring are the operators reference's territory.