tokenizer_py
v0.2.0crates.io· Rustcrate with a tokenizer that works like a Python tokenizer
The verdict
Deprecated. Don't start a new project on this. yanked
yanked
Live from the crates.io registry · derived rules, not AI
How it scores
MaintenanceDeprecated
PopularityNiche
SecurityClean
LicensePermissive
DepsZero deps
Maintenance
Package author marked it deprecated.
Popularity
0 downloads / week
Security
No known advisories for this version (OSV).
License
Apache-2.0 OR MIT
Dependencies
No runtime dependencies
Recent releases
- 0.2.02 years ago
- 0.1.42 years ago
- 0.1.32 years ago
- 0.1.22 years ago
- 0.1.12 years ago
- 0.1.02 years ago