A package to make easier to count tokens during data analysis
Opt-in token-aware output controls (--token-count, --token-limit, --token-offset) for Liche CLIs.
Custom pi extensions, including a nerd-style powerline footer with context token count
Framework-agnostic context optimization engine — composable compression pipeline that makes every token count.
A free service that offers API endpoints for users to send documents and have them broken down into a token count analysis.
Fast token estimation at 96% accuracy of a full tokenizer in a 2kB bundle
`@kt3k/tku` is a CLI tool that counts the total number of tokens in a git repository. It uses [tiktoken](https://www.npmjs.com/package/tiktoken) to tokenize file contents and reports the token count per file and in total.
Token estimation plugin for Claude Code. Shows estimated token count before submission and optionally blocks prompts exceeding a configurable threshold.
This agent generate a reference string from a sorted array of strings, adding one by one until the token count exceeds the specified limit.
Fast JSON Web Token implementation
Indent each line in a string
Strip redundant indentation and indent the string
An object-oriented command-line parser for TypeScript
CLI to strip LLM output noise and minimize token count — pipe-friendly, clipboard-ready
Pipeline to strip LLM output noise and minimize token count before re-prompting
Node.js object hash library with properties/arrays sorting to provide constant hashes
A package to get the maximum token count for OpenAI models
A collection of token providers
Pluralize a word
Common token types for decoding and encoding numeric and string values
Word and character count feature for CKEditor 5.
This package provides support for the [RedisBloom](https://redis.io/docs/data-types/probabilistic/) module, which adds additional probabilistic data structures to Redis.
Repeat a string - fast
Get the visual width of a string - the number of columns required to display it
Count tokens for LLM models using exact tokenization
VUN token! TWO tokens! Count all the beautiful tokens ... offline! Ah-ah-ah!
A Rust CLI tool for generating AI-friendly code banks from dependencies. Automatically parses Cargo.toml files, resolves versions, and generates searchable documentation while calculating token counts.
Fork of `dust` (du-dust on crates.io) with token counting & plotly treemap support with --tokens option
CLI tool to aggregate directory contents into a single markdown file optimized for LLM consumption
Tokenless byte/char-based token-count estimator for LLM prompts. Per-model-family calibration for Claude, GPT, Gemini, Llama. Zero deps.
A local operations CLI for Codex auth, usage, and limit workflows.
A3S Power — Privacy-preserving LLM inference for TEE environments
Cross Agent Session Resumer — resume AI coding sessions across providers (CLI binary: casr)
AST-aware code chunking and late chunking for RAG
AST-aware code chunking and late chunking for RAG
Link.Assistant.Router — Claude MAX OAuth proxy and token gateway for Anthropic APIs
An unofficial Ruby wrapper for Tiktoken, a BPE tokenizer written by and used by OpenAI. It can be used to count the number of tokens in text before sending it to OpenAI APIs.
TokenEstimator is a Rails gem that allows you to count tokens in Excel, CSV, PDF, TXT, Markdown, and input text files using different tokenizers.
Developer utilities from Mohit Khare including token counting, text analysis, and AI engineering helper functions.
A Ruby gem providing a consistent interface for various AI/ML tokenizers including OpenAI GPT, Anthropic Claude, Google Gemini, Meta Llama, Mistral, Qwen, and embedding models like BERT, BGE, and multilingual-E5. Features caching, truncation, token counting, and error handling across different tokenization libraries.
Saikuro is a Ruby cyclomatic complexity analyzer. When given Ruby source code Saikuro will generate a report listing the cyclomatic complexity of each method found. In addition, Saikuro counts the number of lines per method and can generate a listing of the number of tokens on each line of code.
When given Ruby source code Saikuro will generate a report listing the cyclomatic complexity of each method found. In addition, Saikuro counts the number of lines per method and can generate a listing of the number of tokens on each line of code.
Saikuro is a Ruby cyclomatic complexity analyzer. When given Ruby source code Saikuro will generate a report listing the cyclomatic complexity of each method found. In addition, Saikuro counts the number of lines per method and can generate a listing of the number of tokens on each line of code.
When given Ruby source code Saikuro will generate a report listing the cyclomatic complexity of each method found. In addition, Saikuro counts the number of lines per method and can generate a listing of the number of tokens on each line of code.
Saikuro is a Ruby cyclomatic complexity analyzer. When given Ruby source code Saikuro will generate a report listing the cyclomatic complexity of each method found. In addition, Saikuro counts the number of lines per method and can generate a listing of the number of tokens on each line of code.
When given Ruby source code Saikuro will generate a report listing the cyclomatic complexity of each method found. In addition, Saikuro counts the number of lines per method and can generate a listing of the number of tokens on each line of code.
An unofficial Ruby wrapper for Tiktoken, a BPE tokenizer written by and used by OpenAI. It can be used to count the number of tokens in text before sending it to OpenAI APIs. This is a fork of tiktoken_ruby by IAPark, which has been cross-compiled for multiple platforms. This way compilation with Rust extensions doesn't need to happen wherever you are deploying it.
LLM Conductor provides a clean, unified interface for working with multiple Language Model providers including OpenAI GPT, Anthropic Claude, Google Gemini, Groq, OpenRouter, and Ollama. Features include prompt templating, token counting, and extensible client architecture.
No description provided.
No description provided.
No description provided.
No description provided.
No description provided.
No description provided.
No description provided.
No description provided.
No description provided.
No description provided.
No description provided.
No description provided.