language-tokenizer

v0.3.0crates.io· Rust

Text tokenizer for linguistic purposes, such as text matching. Supports more than 40 languages, including English, French, Russian, Japanese, Thai etc.

The verdict

Maintained. Niche but maintained, actively maintained.

Live from the crates.io registry · derived rules, not AI

How it scores

MaintenanceHealthy

PopularityNiche

SecurityClean

LicenseOther

DepsZero deps

Maintenance

Last published 1 month ago.

Popularity

496 downloads / week

Security

No known advisories for this version (OSV).

License

non-standard

Dependencies

No runtime dependencies

Recent releases

0.3.01 month ago
0.2.01 month ago
0.1.05 months ago