Module token

High-performance token implementation

This module provides token types and related functionality for ECMAScript/TypeScript parser. The implementation is optimized for both memory efficiency and processing speed.

Structs

• Token
  Performance-optimized token structure Optimized for memory layout and data access patterns

Enums

• TokenType
  Performance-optimized token type Represented as u8 to minimize memory usage
• TokenValue
  Token value enum optimized for efficient representation

Functions

• keyword_to_token_type
  Convert a keyword string to TokenType Utilizes the first byte and word length to quickly locate the keyword in the table Optimized with fast-path checks for common keywords