Designing of Lexical Analyzer

There are two ways for designing a lexical analyzer, they are
          Hand coding
          Lexical analyzer generator
Hand Coding: Programmer has to perform the following task
          Specify the tokens by writing regular expressions
          Construct Finite Automata equivalent to a regular expression
          Recognize the tokens by a constructed Finite Automata
Where as Lexical analyzer generator, programmer will do step1 remaining two steps are done by lexical analyzer generator automatically.

Lexical Analyzer Generator

Lexical Analyzer Generator introduce a tool called Lex, which allows one to specify a lexical analyzer by specifying regular expressions to describe pattern for tokens

The input for the lex tool is lex language

A program which is written in lex language will compile through lex compiler and produce a C Code called lex.yy.c always i.e.,

Now, C code is compiled by C compiler and produce a file called a.out as always i.e.,

The C compiler output is a working lexical analyzer that can take a stream of input character and produce a stream of tokens i.e.,

Structure of Lex Program
A Lex program has the following form
declarations
   %%
transition rules
   %%
auxiliary functions
The declaration section includes declaration of variables, manifest constants (LT, LE, GT, GE) and regular definitions
The general form of transition rule is
pattern{Action}
where pattern is a regular expression and action defines what action should take when it matches with pattern
Auxiliary function second holds whatever additional functions are used in the actions
These functions can be compiled separately and loaded with the lexical analyzer

Share on Google Plus

About Data Sciences by Venu

Hi, My name is Venugopala Chary and I'm Currently working as Associate Professor in Reputed Engineerng College, Hyderabad. I have B.Tech and M.tech in regular from JNTU Hyderabad. I have 11 Years of Teaching Experience for both B.Tech and M.Tech Courses.