Lexes CommonJS modules, returning their named exports metadata
Lexes ES modules returning their import/export metadata
The pug lexer (takes a string and converts it to an array of tokens)
lexing json
A tool set for CSS: fast detailed parser (CSS → AST), walker (AST traversal), generator (AST → CSS) and lexer (validation and matching) based on specs and browser implementations
MermaidJS parser
Optimised tokenizer/lexer generator! 🐄 Much performance. Moo!
@ts-jison/common: ==== ----------------------------------------- A parser/lexer run-time library used by [@ts-jison/parser](../parser) and [@ts-jison/lexer](../lexer).
ES module parser powered by Rust
@ts-jison/lexer: ==== ----------------------------------------- A parser run-time library used by [@ts-jison/parser-generator](../parser-generator).
This is the Linux 64-bit (musl) binary for rs-module-lexer.
This is the Linux 64-bit binary for rs-module-lexer.
Edge parser to convert text markup to lexer tokens
@ts-jison/parser: ==== A lightly-typescriptified API for creating parsers in JavaScript ----------------------------------------- A lexical analyzer generator used by [@ts-jison/lexer-generator](../lexer-generator).
> A lossless Twig lexer
This is the macOS 64-bit binary for rs-module-lexer.
This is the Linux ARM 64-bit binary for rs-module-lexer.
Core components of Traqula
This is the Linux ARM 32-bit binary for rs-module-lexer.
This is the Linux ARM 64-bit (musl) binary for rs-module-lexer.
This is the macOS ARM 64-bit binary for rs-module-lexer.
This is the Windows ARM 64-bit binary for rs-module-lexer.
This is the Windows 64-bit binary for rs-module-lexer.
Lexer / tokenizer
Source code lexer configurable for any programming language that allows to tokenize and abstract a given source file
Implements a simple lexer using a StringScanner
Generate all parts of a compiler, including the lexer, parser, syntax tree, symbol table, type checker, code generator, and optimizer.
Oedipus Lex is a lexer generator in the same family as Rexical and Rex. Oedipus Lex is my independent lexer fork of Rexical. Rexical was in turn a fork of Rex. We've been unable to contact the author of rex in order to take it over, fix it up, extend it, and relicense it to MIT. So, Oedipus was written clean-room in order to bypass licensing constraints (and because bootstrapping is fun). Oedipus brings a lot of extras to the table and at this point is only historically related to rexical. The syntax has changed enough that any rexical lexer will have to be tweaked to work inside of oedipus. At the very least, you need to add slashes to all your regexps. Oedipus, like rexical, is based primarily on generating code much like you would a hand-written lexer. It is _not_ a table or hash driven lexer. It uses StrScanner within a multi-level case statement. As such, Oedipus matches on the _first_ match, not the longest (like lex and its ilk). This documentation is not meant to bypass any prerequisite knowledge on lexing or parsing. If you'd like to study the subject in further detail, please try [TIN321] or the [LLVM Tutorial] or some other good resource for CS learning. Books... books are good. I like books.
RubyLexer is a lexer library for Ruby, written in Ruby. Rubylexer is meant as a lexer for Ruby that's complete and correct; all legal Ruby code should be lexed correctly by RubyLexer as well. Just enough parsing capability is included to give RubyLexer enough context to tokenize correctly in all cases. (This turned out to be more parsing than I had thought or wanted to take on at first.) RubyLexer handles the hard things like complicated strings, the ambiguous nature of some punctuation characters and keywords in ruby, and distinguishing methods and local variables. It should be able to correctly lex 99.9+% of legal ruby code.
A Ruby FFI binding to the `sas-lexer` Rust crate by Misha Perlov (https://github.com/mishamsk/sas-lexer). Tokenizes SAS source code into a stream of typed tokens with full position metadata. Ships prebuilt native libraries for supported platforms; a runtime FFI loader picks the matching one for the host.
Rockit is a potent parser generator and gives you AST's (Abstract Syntax Tree's) which you can pattern match and pretty-print. Rockit does not distinghuish between lexing and parsing so the generated parsers are scanner-/lexer-less. The vision is to extend Rockit with more advanced compiler-related abilities including back-ends and code generation. However, currently the focus is on parsing and AST-related tasks such as transformation.
No description provided.
No description provided.
No description provided.
No description provided.
No description provided.
No description provided.
No description provided.
No description provided.
No description provided.
No description provided.
No description provided.
No description provided.