Refactoring Lexical Structure #65
Description
Lexical Scanning Improvements
Create a new way to do lexical scanning in compiler.
This refactor issue needs impact the project in this way:
- less entities to perform the lexical reading
- remove tokentype entity (not necessary)
- New Automaton to recognize tokens (improved automaton)
Note
An improved automaton consists in an automaton that implements methods to recognize tokens character by character instead token by token (it doesn't recognize entire character sequencies as was developed)