HTML and CSS lexer aimed at code with fatal errors, accepts mixed coding languages
Tokenize CSS
A promise based streaming tokenizer
Tokenized zip support
Various utility functions
TypeScript definition for strtok3 token
Algorithms to help you parse CSS from an array of tokens.
Parses and stringifies CSS selectors
Solve CSS math expressions
A pure JavaScript implementation of a BPE tokenizer (Encoder/Decoder) for GPT-2 / GPT-3 / GPT-4 and other OpenAI models
A tokenzier for Sass' SCSS syntax
Simple HTML Tokenizer is a lightweight JavaScript library that can be used to tokenize the kind of HTML normally found in templates.
ProseMirror Markdown integration
Strip HTML tags from strings. No parser, accepts mixed sources.
tokenizer of source code for jscpd
Common token types for decoding and encoding numeric and string values
Is given string a language code (as per IANA)
Looks up the first non-whitespace character to the left/right of a given index
Collapse the leading and trailing whitespace of a string
Command line app to deep sort JSON files, retains package.json special key order
r/w stream of glsl tokens
Gather string index ranges
Claude tokenizer
Like String.trim() but you can choose granularly what to trim