Tokenize a string with regex.
Tokenize CSS
transform stream to tokenize html
micromark utility to tokenize subtokens
Tokenize a string.
Tokenize a string into an array of string parts and format identifier objects.
Tokenize Excel formulas
Tokenize a text into sentences
Tokenize a string into words and whitespace tokens
tokenize a string that includes ansi code
Tokenize a shell string into argv array
Uses snapdragon to tokenize a single JavaScript block comment into an object, with description, tags, and code example sections that can be passed to any other comment parsers for further parsing.
Simple HTML Tokenizer is a lightweight JavaScript library that can be used to tokenize the kind of HTML normally found in templates.
Tokenize HTML into a list of text tokens
Tokenize JavaScript
Tokenize HTML into a list of text tokens
Transform stream that tokenizes CSS
Efficiently modify strings containing ANSI escape codes
Stream Transform class to tokenize RTF, and another to de-encapsulate text or HTML
A primitive to tokenize your solid-components to enable custom parsing.
Tokenize javascript expressions.
basic tokenize function
Simple algorithm to tokenize Chinese texts into words using CC-CEDICT.
Tokenize paragraphs into sentences, and smaller tokens.
Implementation of the Tokenize specification in Rust
Hermit incentive contract
Full-featured JWT authentication middleware for actix-web: login, logout, refresh, token rotation, cookie management, RSA/HMAC, RBAC authorizer, pluggable token store
A parser tool to generate recursive descent top down parser.
A powerful Rust authentication and authorization framework
Secure, typed, async Rust SDK for OpenBao
Auto-generated client library for the Edgegap API, used by the arbctl tool
This is the SDK for Rust. Like all BlockChyp SDKs, it provides a full client for the BlockChyp gateway and BlockChyp payment terminals.
Hessra biscuit token SDK for Rust
Core library for sa-token-rust, a powerful authentication and authorization framework
Rust SDK for Nad.fun
Implementation of the Common Access Token (CAT) specification
Quick Dirty Token generation by Nick Merwin (lemurheavy.com) 11.29.2009
Tokenize attributes in active record object.
Generate unique tokens for your ActiveRecord models
tokenize_attr adds a `tokenize` class macro to ActiveRecord models. When no prefix is needed it transparently delegates to Rails' built-in has_secure_token (Rails 5+). When a prefix is required it installs a before_create callback backed by SecureRandom.base58 with configurable size, prefix, and retry logic.
A simple multilingual tokenizer for NLP tasks. This tool provides a CLI and a library for linguistic tokenization which is an anavoidable step for many HLT (human language technology) tasks in the preprocessing phase for further syntactic, semantic and other higher level processing goals. Use it for tokenization of German, English and French texts.
Fast state-of-the-art tokenizers for Ruby
HTML Tokenizer
A pure ruby implementation of the RFC 7519 OAuth JSON Web Token (JWT) standard.
Google ID Token utilities; currently just a parser/checker
JWT token generator extension for Doorkeeper
A multilingual tokenizer to split a string into tokens.
A simple multilingual tokenizer for NLP tasks. This tool provides a CLI and a library for linguistic tokenization which is an anavoidable step for many HLT (human language technology) tasks in the preprocessing phase for further syntactic, semantic and other higher level processing goals. Use it for tokenization of German, English and French texts.
No description provided.
No description provided.
No description provided.
No description provided.
No description provided.
No description provided.
No description provided.
No description provided.
No description provided.
No description provided.
No description provided.
No description provided.
No description provided.
No description provided.
No description provided.
No description provided.
No description provided.
No description provided.
No description provided.
No description provided.
No description provided.
No description provided.
No description provided.
No description provided.