Tokens represent pieces of text. Each token has:
source text of the token. Its "value".
a numeric value indicating the tokens type
Name of the source stream that it came from
Line number from which it came.
Token.end is a class variable set to zero. It is used to indicate that the end of the stream has been read. Do not use 0 as a token Id.
def __init__(self, type, val, srcInfo):
Constructor for Token. Type is one of the types listed above, val is the
value of the token (its text), srcName is the name of the source file that the
token was tokenized from, and lineNum is the line number in the source
file. srcInfo is a tuple indicating the source file name and the line
number.
Returns true if the token is of the indicated type and has the indicated
value.
Returns true if the token is of the indicated type.
equals(self, type, val)
def equals(self, type, val):
isType(self, type)
def isType(self, type):