a simple lexical scanner for parsing text into tokens
A simple lexer which creates over 115+ various tokens based on the rust programming language. This complete Lexer/Lexical Scanner produces tokens for a string or a file path entry.