Tests for CSS Tokenizers
The API for building streaming tokenizers and lexers.
The Linux 64-bit binary for lefthook, git hooks manager.
HTTP tokenizer for Node.js and browser
Convert SQL statements into a list of tokens
Add component's name in `data-qa` attributes to React Components Edit
Split text into sentences with Sentence Boundary Detection (SBD).
A fast tokenizer/lexer for JavaScript
WebSocket framework for Ruby on Rails.
Our library `@lenml/llama2-tokenizer` has been deprecated. We are excited to introduce our new library `@lenml/tokenizers` as its replacement, offering a broader set of features and an enhanced experience.
Simple HTML Tokenizer is a lightweight JavaScript library that can be used to tokenize the kind of HTML normally found in templates.
Create atom (⚛) representation for your application, packages and libraries
Simple algorithm to tokenize Chinese texts into words using CC-CEDICT.
CDK Construct Library to automatically instrument Python and Node Lambda functions with Datadog using AWS CDK v2
Pure-TypeScript CSS toolkit for Bun/Node — parser, walker, generator, selector engine, and minifier. Zero runtime deps.
Forked version of kuromoji with better compatibility for browsers
Prettier plugin for .html.erb files
yi tokenizer for NodeJS/Browser
node's net API for react-native
JMESPath implementation in javascript
Serverless plugin to automatically instrument python and node functions with datadog tracing
Simple CSS tokenizer
Generate deterministic filler text with exact token counts.
Ruby head built on WASI