The ::SHE+ILA:: Lexer and Parser [$6B]

More about The ::SHE+ILA:: Lexer and Parser [$6B]

Considering the Editor/Compiler module, we will look at the Lexer and the Parser. After the user has typed or edited a line of code, we first scan it from left to right, replacing each identifier (dataname) with its "atom", putting the name into the symbol table (the code generation phase will use these symbol table indices to access the local variables). This is the "atomization" or "tokenization" phase. (Optionally we can also do a spelling check at this stage, in case the user has mis-spelled an identifier). Whenever we hit an opening bracket ("(", "[", "{") or their decorated forms (e.g. "{*" or "{!") we spawn another Buf, which will be linked into the current Buf at the current point; this allows LIST to maintain the order within expressions. After we have processed the entire input line in this way, we scan the Buf for groups of operands and operators (bearing in mind the PEMDAS priorities of the operators). Looking at groups of three, for example, there are eight possibilities :
A B C Possibly a vector or parameter list
A B f Here f is an operator, which obviously belongs to the next expression (we aren't dooing Reverse Polish)
A f B This is a standard Dyadic expression, which emits a (f, A, B) node
A f g Two operators here belonging to the next expression
f A B Possibly a Monadic operator f (so emit (f, A) ), and a lurking B belonging to the next expression
f A g Likewise, no smoking thank you :)
f g A Here g is Monadic and so is f, so emit (f, (g, A))
f g h Hey three operators, the switchboard must be busy :)