山东大学编译原理Chapter3_lexical analyzer.ppt

CHAPTER 3 LEXICAL ANALYSIS3.40 Approaches to implement a lexical analyzer 1、Simple approach Construct a diagram that illustrates the structure of the tokens of the source language , and then to hand-translate the diagram the diagram into a program for finding tokens Notes: Efficient lexical analyzers can be produced in this manner CHAPTER 3 LEXICAL ANALYSIS3.40 Approaches to implement a lexical analyzer 2、Pattern-directed programming approach Pattern Matching technique Specify and design program that execute actions triggered by patterns in strings Introduce a pattern-action language called Lex for specifying lexical analyzers Patterns are specified by regular expressions A compiler for Lex can generate an efficient finite automation recognizer for the regular expressions CHAPTER 3 LEXICAL ANALYSIS3.1 The role of the lexical analyzer First phase of a compiler 1、Main task To read the input characters To produce a sequence of tokens used by the parser for syntax analysis As an assistant of parser CHAPTER 3 LEXICAL ANALYSIS 3.1 The role of the lexical analyzer 2、Interaction of lexical analyzer with parser CHAPTER 3 LEXICAL ANALYSIS 3.1 The role of the lexical analyzer 3、Processes in lexical analyzers Scanning Pre-processing Strip out comments and white space Macro functions Correlating error messages from compiler with source program A line number can be associated with an error message Lexical analysis CHAPTER 3 LEXICAL ANALYSIS 3.1 The role of the lexical analyzer 4、Terms of the lexical analyzer Token Types of words in source program Keywords, operators, identifiers, constants, literal strings, punctuation symbols(such as commas,semicolons) Lexeme Actual words in source program Pattern A rule describing the set of lexemes that can represent a particular token in source program Relation {.=,,=,==,} CHAPTER 3 LEXICAL ANALYSIS 3.1 The role of the lexical analyzer 5、Attributes for Tokens A pointer to the symbol-table entry in which the informati