Results for model_tokenizer

Count tokens for LLM models using exact tokenization

token billing models for tibba (account, recharge, usage, price)

A crate for token expansion, leveraging batch workflows and language model APIs.

Petri net modeling with ODE simulation and token model DSL

Pack and Unpack for all structs

Token model schema, snapshot, and runtime execution

A command-line tool for benchmarking the performance of LLM inference endpoints.

LLM-native image optimization middleware & MCP server. Reduces vision model token consumption by snapping to tile boundaries.

Standalone high-performance Canadian address parsing engine core

Proc macros for pflow token model DSL

An unofficial Rust client for the Anthropic/Claude API.

S-expression DSL for defining token model schemas