smoltoken

v1.0.0crates.io· Rust

A fast library for Byte Pair Encoding (BPE) tokenization.

The verdict

Deprecated. Don't start a new project on this. yanked

yanked

Live from the crates.io registry · derived rules, not AI

How it scores

MaintenanceDeprecated

PopularityNiche

SecurityClean

LicenseOther

DepsZero deps

Maintenance

Package author marked it deprecated.

Popularity

1 downloads / week

Security

No known advisories for this version (OSV).

License

non-standard

Dependencies

No runtime dependencies

Recent releases