Tokenize CSS
A promise based streaming tokenizer
Tokenized zip support
TypeScript definition for strtok3 token
Sixel image format for node and browser.
Algorithms to help you parse CSS from an array of tokens.
Parses and stringifies CSS selectors
A tokenzier for Sass' SCSS syntax
Solve CSS math expressions
A pure JavaScript implementation of a BPE tokenizer (Encoder/Decoder) for GPT-2 / GPT-3 / GPT-4 and other OpenAI models
Simple HTML Tokenizer is a lightweight JavaScript library that can be used to tokenize the kind of HTML normally found in templates.
ProseMirror Markdown integration
Common token types for decoding and encoding numeric and string values
Inline image output in xterm.js. Supports SIXEL and iTerm's inline image protocol (IIP).
tokenizer of source code for jscpd
r/w stream of glsl tokens
Tokenizes a string that represents a regular expression.
Claude tokenizer
Detect which terminal graphics protocols are supported (Kitty, iTerm2, Sixel)
Tokenize a shell string into argv array
A faster than tiktoken tokenizer with first-class support for Vercel's AI SDK.
Native Rust bindings for grep, clipboard, image processing, syntax highlighting, PTY, and shell operations via N-API
Parse CSS media query lists.
detector of copy/paste in files
A tokenizer for serialized Sixel bytes
An interface for querying, manipulating and serializing/deserializing Sixel data