tivars.tokenizer.state.ProgramName

class documentation

class ProgramName(Name): (source)

Constructor: ProgramName(mode, accept, length)

View In Hierarchy

Program names

Class Variable max_length The maximum number of tokens to emit before leaving this state

Inherited from Name:

Method next Determines the next tokenizer timelines given a token

Inherited from MinMode (via Name):

Method __init__ No summary

Inherited from TokenizerState (via Name, MinMode, Line):

Method munch Munch the input string and determine the resulting token, tokenizer timelines, and remainder of the string
Instance Variable accept Undocumented
Instance Variable length Undocumented
Instance Variable mode Undocumented
max_length: int = (source) ΒΆ
overrides tivars.tokenizer.state.TokenizerState.max_length

The maximum number of tokens to emit before leaving this state