Simple CSS tokenizer
Simple HTML Tokenizer is a lightweight JavaScript library that can be used to tokenize the kind of HTML normally found in templates.
Tokenized zip support
Tokenize CSS
A promise based streaming tokenizer
TypeScript definition for strtok3 token
Algorithms to help you parse CSS from an array of tokens.
A simple tokenizer for the Heluna programming language.
A tokenzier for Sass' SCSS syntax
Parses and stringifies CSS selectors
A pure JavaScript implementation of a BPE tokenizer (Encoder/Decoder) for GPT-2 / GPT-3 / GPT-4 and other OpenAI models
Solve CSS math expressions
My JavaScript parser
ProseMirror Markdown integration
Pratt parser builder (along with simple tokenizer)
A simple tokenizer for encoding/decoding text into numeric tokens.
Tokenize a shell string into argv array
Common token types for decoding and encoding numeric and string values
tokenizer of source code for jscpd
Simple tokenizer for a simple "language"
r/w stream of glsl tokens
Tokenizes a string that represents a regular expression.
Claude tokenizer
Parse CSS media query lists.
A tiny no_std tokenizer with line & column tracking.
This's a run-time loadable extension of SQLite fts5, supports Chinese and pinyin word segmentation and search.
A token-phrase generator
Simple (but safe) token authentication for Rails apps or API with Devise.
Rack::AllowedHosts allows you to whitelist the hostnames allowed to serve the site. This is helpful to protect against Host Header Injection. See: https://acunetix.com/vulnerabilities/web/host-header-attack
Generate simple tokens for short url and other things
Textoken is a Ruby library for text tokenization. This gem extracts words from text with many customizations. It can be used in many fields like Web Crawling and Natural Language Processing.
This gem adds simple token authentication to users.
RToken generates random tokens, of any length, for multiple purposes, ex: web urls, data identification, etc.
simple FreshBooks API wrapper. supports both OAuth and API token authentication
Simple unobtrusive token invitation engine for rails
RubyTokenizer is a simple language processing command-line tool. It performs low-level tokenization and returns the top 10 most frequent words in a body of text. At the moment it's only available for English texts and it segments words by filtering whitespaces, punctuation marks, parantheses and other special characters.
Greeb is a simple yet awesome and Unicode-aware regexp-based tokenizer, written in Ruby.
Simple token authentication designed to work with mongoid. This gem simply provides a simple interface for authentication.
No description provided.
No description provided.
No description provided.
No description provided.
No description provided.