Results for wordchipper

Fast BPE tokenizer library with tiktoken and HuggingFace compatibility

HPC Rust LLM Tokenizer Library

A command-line interface for the Wordchipper tokenizer library.

BPE LLM Tokenizer Training for wordchipper

A disk cache for Wordchipper downloads

Utilities for the Wordchipper CLI

Rename stub crate for wordchipper

A command-line interface for the Wordchipper tokenizer library.

bunsen is acceleration tooling for burn