tiktoken

v0.13.0PyPI· Python

tiktoken is a fast BPE tokeniser for use with OpenAI's models

The verdict

Safe default. Widely trusted across the ecosystem, actively maintained.

Live from the PyPI registry · derived rules, not AI

How it scores

MaintenanceHealthy

PopularityTop 1%

SecurityClean

LicenseOther

DepsLean

Maintenance

Last published this month.

Popularity

41M downloads / week

Security

No known advisories for this version (OSV).

License

MIT License

Dependencies

3 direct dependencies

Recent releases