JS/WASM bindings for tiktoken
JS/WASM bindings for tiktoken
JavaScript port of tiktoken
Estimate the number of tokens an OpenAI chat completion request will use
tiktoken is a fast [BPE](https://en.wikipedia.org/wiki/Byte_pair_encoding) tokeniser for use with OpenAI's models.
JS tokenizer for LLaMA-based LLMs
_(inspired by [tiktoken-node](https://github.com/ceifa/tiktoken-node))_
This is the **x86_64-unknown-linux-gnu** binary for `tiktoken-rs-node`
Tokenizer utilities for Tokenlens.
A pure JavaScript implementation of a BPE tokenizer (Encoder/Decoder) for GPT-2 / GPT-3 / GPT-4 and other OpenAI models
This is the **x86_64-unknown-linux-gnu** binary for `@anysphere/tiktoken-node`
Promptwiz-js is a server-side TypeScript/JavaScript library for prompting large language models (LLMs) using a shared interface compatible with the Promptwise service. `npm install @promptwise/promptwiz-js`
We use our own fork for now because we are making changes that may not be useful to everyone. For example, we add support for special tokens, and we also add support for running tokenization asynchronously with the computation happening on a different thr
This is the **aarch64-apple-darwin** binary for `tiktoken-rs-node`
Tokenizer for OpenAI large language models.
JavaScript port of tiktoken
A faster than tiktoken tokenizer with first-class support for Vercel's AI SDK.
Tiktoken but thicker; A bundled tokenizer with added helper functions
CLI tool for packing files into a prompt for LLM usage.
一个专门用来计算```gpt```的tokens的轻量级库
CLI for counting tokens in files and directories using tiktoken
JavaScript/TypeScript Implementation of LLMLingua-2
Estimate the number of tokens an OpenAI chat completion request will use
Intelligent context window optimization for Claude Code - store content externally via caching and compression, freeing up your context window for what matters