Efficiently modify strings containing ANSI escape codes
Transform stream that tokenizes CSS
Tokenize a string into an array of string parts and format identifier objects.
A tokenzier for Sass' SCSS syntax
transform stream to tokenize html
Simple HTML Tokenizer is a lightweight JavaScript library that can be used to tokenize the kind of HTML normally found in templates.
Tokenize a string.
Parse CSS color values
Tokenize CSS
Provide a high level wrapper for kuromoji.js
Uses snapdragon to tokenize a single JavaScript block comment into an object, with description, tags, and code example sections that can be passed to any other comment parsers for further parsing.
Tokenize Excel formulas
The lexer for Materialize's SQL dialect, with wasm build targets.
Small library that provides functions to tokenize a string into an array of words with or without punctuation
Multilingual tokenizer that automatically tags each token with its type
tokenize a string that includes ansi code
match a tokenized html stream with css selectors
micromark utility to tokenize subtokens
Estimate the number of tokens for Gemini models
Tokenize a text into sentences
Tokenize a string into words and whitespace tokens
Parser for edge template engine
Generate a changelog from git metadata.
Create a string representation of an object or primative