A faster than tiktoken tokenizer with first-class support for Vercel's AI SDK.
Claude tokenizer
01.AI tokenizer families for tokkit.
Tokenized zip support
Tokenize CSS
A promise based streaming tokenizer
A faster than tiktoken tokenizer with first-class support for Vercel's AI SDK.
A pure JavaScript implementation of a BPE tokenizer (Encoder/Decoder) for GPT-2 / GPT-3 / GPT-4 and other OpenAI models
TypeScript definition for strtok3 token
Algorithms to help you parse CSS from an array of tokens.
Parses and stringifies CSS selectors
A tokenzier for Sass' SCSS syntax
Solve CSS math expressions
detector of copy/paste in files
Simple HTML Tokenizer is a lightweight JavaScript library that can be used to tokenize the kind of HTML normally found in templates.
ProseMirror Markdown integration
[](https://github.com/botisan-ai/gpt3-tokenizer/actions/workflows/main.yml) [](https://www.npmjs.com/
Arcee AI tokenizer families for tokkit.
AIDC-AI tokenizer families for tokkit.
swiss-ai tokenizer families for tokkit.
Common token types for decoding and encoding numeric and string values
tokenizer of source code for jscpd
ServiceNow-AI tokenizer families for tokkit.
r/w stream of glsl tokens
A Ruby gem providing a consistent interface for various AI/ML tokenizers including OpenAI GPT, Anthropic Claude, Google Gemini, Meta Llama, Mistral, Qwen, and embedding models like BERT, BGE, and multilingual-E5. Features caching, truncation, token counting, and error handling across different tokenization libraries.
OpenClacky is a 100% open-source (MIT) AI Agent that matches Claude Code on capability at comparable cost. Built on two years of agentic R&D, it ships with a minimal 16-tool core, near-100% prompt cache hit rate, idle-time auto-compression, and BYOK support for any OpenAI-compatible model. A natural-language Skill system lets you create, evolve, and monetize agent capabilities without writing code, plus first-class integrations with Feishu, WeCom, WeChat, Discord, and Telegram.
RSpec formatter optimized for AI agents and CI systems. Outputs compact NDJSON with file references instead of verbose text. Supports log splitting, error deduplication, and context-efficient reporting.
Official Ruby client for Blackman AI - The AI API proxy that optimizes token usage to reduce costs
A Ruby gem that converts T::Struct and T::Enum to BAML (Boundary AI Markup Language) type definitions. BAML uses 60% fewer tokens than JSON Schema while maintaining type safety.
AgentHarness provides a unified interface for CLI-based AI coding agents like Claude Code, Cursor, Gemini CLI, and others. It offers full orchestration with provider switching, circuit breakers, health monitoring, flexible configuration, dynamic provider registration, and token usage tracking.
Developer utilities from Mohit Khare including token counting, text analysis, and AI engineering helper functions.
Self-hosted error observability and LLM tracing. Captures errors, deduplicates via fingerprinting, and traces AI/LLM calls (token usage, costs, latency). Zero external runtime dependencies.
A tool that extracts and combines text from HTML files into a single, streamlined markdown document. It provides a command-line interface for easy usage, removes unnecessary HTML elements to reduce token usage, and creates an easily uploadable format for AI tools like Claude AI or ChatGPT. The tool preserves document structure and includes frontmatter metadata.
Register AI agents, issue scoped tokens, enforce per-tool permissions, and query audit logs via the AgentsID API. Includes MCP middleware for validating tool calls in MCP servers.
Add a usage-based credit system to your Rails app, easily. Let users buy and spend credits on usage-based actions. Refill Stripe subscriptions with credits, sell one-time booster credit packs, implement PAYG (pay-as-you-go) billing, award free credits as bonuses, manage prepaid credits / tokens. Your users will have wallet balances that they can spend on features, API calls, or other usage-based actions. Perfect for SaaS, AI apps, games, and API products with metered pricing / billing.
No description provided.
No description provided.
No description provided.
No description provided.
No description provided.
No description provided.
No description provided.
No description provided.
No description provided.
No description provided.
No description provided.
No description provided.