Given the specifications for Pascal that is available on Moodle and on the class web page (https://home.siegfried/cs645/assignments.html), write a scanner that recognizes a function gettoken() that recognizes the lexemes of the language and returns a token corresponding to that lexeme.
Since we do not yet have a symbol table, all words (a letter followed by zero or more alphanumeric characters) have the temporary token tokword, all numbers have the temporary token toknumber and all others have the temporary token tokop.
If your input is the program
PROGRAM Sample (Input, Output); CONST x = 15; BEGIN ... END;
Your initial output should be
PROGRAM | tokword |
SAMPLE | tokword |
( | tokop |
INPUT | tokword |
, | tokop |
OUTPUT | tokword |
) | tokop |
; | tokop |
VAR | tokword |
X | tokword |
= | tokop |
15 | toknumber |
; | tokop |