Results for tiktoken-cli

CLI for counting tokens in files and directories using tiktoken

JavaScript port of tiktoken

JS/WASM bindings for tiktoken

JS/WASM bindings for tiktoken

_(inspired by [tiktoken-node](https://github.com/ceifa/tiktoken-node))_

tiktoken is a fast [BPE](https://en.wikipedia.org/wiki/Byte_pair_encoding) tokeniser for use with OpenAI's models.

This is the **x86_64-unknown-linux-gnu** binary for `tiktoken-rs-node`

A pure JavaScript implementation of a BPE tokenizer (Encoder/Decoder) for GPT-2 / GPT-3 / GPT-4 and other OpenAI models

This is the **x86_64-unknown-linux-gnu** binary for `@anysphere/tiktoken-node`

We use our own fork for now because we are making changes that may not be useful to everyone. For example, we add support for special tokens, and we also add support for running tokenization asynchronously with the computation happening on a different thr

Tokenizer for OpenAI large language models.

This is the **aarch64-apple-darwin** binary for `tiktoken-rs-node`

JavaScript port of tiktoken

A faster than tiktoken tokenizer with first-class support for Vercel's AI SDK.

Tiktoken but thicker; A bundled tokenizer with added helper functions

Estimate the number of tokens an OpenAI chat completion request will use

JS tokenizer for LLaMA-based LLMs

Pure helpers shared by the Releases registry — schema, categories, slicing, IDs, slugs, tokens.

Intelligent context window optimization for Claude Code - store content externally via caching and compression, freeing up your context window for what matters

Tokenizer utilities for Tokenlens.

Pretty unicode tables for the command line. Based on the original cli-table.

🚀 CLI tool to analyze projects, concatenate source files, count tokens, and export HTML with syntax highlighting and copy functionality

Codex CLI is a coding agent from OpenAI that runs locally on your computer.

Get stdout window width, with two fallbacks, tty and then a default.