Given the specifications for Pascal (distributed earlier), write a scanner that recognizes the lexemes of the languages and returns a token corresponding to that lexeme. The main function of that scanner should be gettoken(), which returns a token corresponding to that lexeme.
Since we do not yet have a symbol table, all words (a letter followed by zero or more letters and digits) have the temporary token tokword, all numbers have the temporary token toknumber and all other have the temporary token tokop. This will change after we write the symbol table manager. For now, declare the enumerated type tokentype:
enum tokentype {tokword, toknumber, tokop};
If your input is the program:
PROGRAM Sample; CONST x = 15; ... ... ... BEGIN ... ... ... END.
Your initial output should be:
PROGRAM tokword SAMPLE tokword ; tokop CONST tokword X tokword = tokop 15 toknumber ; tokop ... ... ... BEGIN tokword ... ... ... END tokword . tokop