BETAmodules.com is in beta — open to partnerships & joint ventures.Build with us

roketok

v0.3.1crates.io· Rust

This crate provides a way to simply set up a tokenizer and use it. Not recommended for simple tokenizers as this crate adds a bunch of stuff to support many if not all kinds of tokenizers.

The verdict
Aging — last published 11 months ago — check before adopting. Check the repo for activity before adopting.
Check the repo for activity before adopting.
Live from the crates.io registry · derived rules, not AI
How it scores
MaintenanceAging
PopularityNiche
SecurityClean
LicensePermissive
DepsZero deps
Maintenance
Last published 11 months ago — check before adopting.
Popularity
1 downloads / week
Security
No known advisories for this version (OSV).
License
MIT
Dependencies
No runtime dependencies
Recent releases
  • 0.3.111 months ago
  • 0.3.011 months ago
  • 0.3.0-pre111 months ago
  • 0.2.111 months ago
  • 0.2.011 months ago
  • 0.1.511 months ago
  • 0.1.411 months ago
  • 0.1.3-p111 months ago
roketok — This crate provides a way to simply set up a tokenizer and use it. Not recommended for simple tokenizers as this crate adds a bunch of stuff to support many if not all kinds of tokenizers. (Rust / crates.io) — verdict, alternatives & security · Modules