Month: January 2011

A Lexical Analyizer for C++ tokens

Lexical analysis is step 1 of compiling code down to machine language. The process breaks source code down into a long list of pieces called tokens. This list of tokens is used by a parser algorithm that extracts meaning from the order and arrangement of the tokens. Here is a small example of lex analysis: code: int main(void) { float myvar = 2.5; return 0; } list of tokens: int type main reserved word ( void keyword ) { float keyword myvar identifier = operator 2.5 floating point constant ; end statement return keyword 0 integer constant ; end statemant […]