Token

Tokens represent pieces of text. Each token has:

val

source text of the token. Its "value".

type

a numeric value indicating the tokens type

srcName

Name of the source stream that it came from

lineNum

Line number from which it came.

Token.end is a class variable set to zero. It is used to indicate that the end of the stream has been read. Do not use 0 as a token Id.

Methods

__init__(self, type, val, srcInfo)

   def __init__(self, type, val, srcInfo):

Constructor for Token. Type is one of the types listed above, val is the value of the token (its text), srcName is the name of the source file that the token was tokenized from, and lineNum is the line number in the source file. srcInfo is a tuple indicating the source file name and the line number.

equals(self, type, val)

   def equals(self, type, val):

Returns true if the token is of the indicated type and has the indicated value.

isType(self, type)

   def isType(self, type):

Returns true if the token is of the indicated type.