Tokenize a string into words and whitespace tokens
Trim the whitespace within an array of GLSL tokens
hast utility to check if a node is inter-element whitespace
Efficiently modify strings containing ANSI escape codes
Tokenize a string.
micromark factory to parse markdown whitespace (found in lots of places)
Strip comments from JSON. Lets you use comments in your JSON files!
Trim whitespace inside and around CSS rules & declarations.
Strip leading whitespace from each line in a string
lexing json
Transform stream that tokenizes CSS
Tokenize a string into an array of string parts and format identifier objects.
Check if a character is a whitespace character
A tokenzier for Sass' SCSS syntax
transform stream to tokenize html
hast utility to get the plain-text value of a node
rehype plugin to collapse whitespace
List of whitespace sensitive HTML tag names
Returns true if the value passed is all whitespace.
Collapse the leading and trailing whitespace of a string
Simple HTML Tokenizer is a lightweight JavaScript library that can be used to tokenize the kind of HTML normally found in templates.
Trim a consecutively repeated substring: foo--bar---baz → foo-bar-baz
Parse CSS color values
Tokenize CSS
RubyTokenizer is a simple language processing command-line tool. It performs low-level tokenization and returns the top 10 most frequent words in a body of text. At the moment it's only available for English texts and it segments words by filtering whitespaces, punctuation marks, parantheses and other special characters.
A configurable lint engine for SAS source files. Walks the token stream produced by the `sas-lexer` gem and applies a set of pluggable rules covering structural defects (malformed `if` conditions, identical `then`/`else` branches, unreachable inner branches), cosmetic issues (trailing whitespace, tab expansion, line endings, encoding gremlins), and source-header conventions. Includes a `bin/sas_lint` CLI and YAML-based config.