Efficiently modify strings containing ANSI escape codes
Transform stream that tokenizes CSS
Tokenize a string into an array of string parts and format identifier objects.
A tokenzier for Sass' SCSS syntax
transform stream to tokenize html
BitGo SDK coin library for Thorchain
Simple HTML Tokenizer is a lightweight JavaScript library that can be used to tokenize the kind of HTML normally found in templates.
Tokenize a string.
Parse CSS color values
Tokenize CSS
BitGo SDK coin library for Thorchain
Provide a high level wrapper for kuromoji.js
runestones encode and decode
Uses snapdragon to tokenize a single JavaScript block comment into an object, with description, tags, and code example sections that can be passed to any other comment parsers for further parsing.
Tokenize Excel formulas
Rune — runtime script orchestrator. Move package.json scripts into a shareable, sectioned config file.
Build a multiplayer game played by millions! Your game runs inside the Rune app with 10 million installs across [iOS](https://apps.apple.com/app/rune-games-and-voice-chat/id1450358364) and [Android](https://play.google.com/store/apps/details?id=ai.rune.ti
The lexer for Materialize's SQL dialect, with wasm build targets.
A highlight library for Svelte 5 with Rune
TypeScript SDK for rune-style badge/stamp issuance on the Stacks blockchain — mint, verify, enumerate.
Small library that provides functions to tokenize a string into an array of words with or without punctuation
Multilingual tokenizer that automatically tags each token with its type
tokenize a string that includes ansi code
match a tokenized html stream with css selectors
Approximate token counting, budget checking, text truncation, and overlapping chunk splitting for LLM context windows
Text splitters for LLM context windows: character, token, markdown, and recursive strategies
LlmChain — wire a prompt template, an LLM, and optional memory into a single Chain