Fast Chinese characters lookup for Cangjie and Sucheng codes.
Binary byte pair encoding (BPE) trainer and CLI compatible with Hugging Face tokenizers
A fast constrained decoding engine based on context free grammar.