tivars.tokenizer.state.Name

class documentation

class Name(Line): (source)

Constructor: Name(length)

Valid var identifiers

Method	`next`	Determines the next encode state given a token
Class Variable	`mode`	Whether to munch maximally (`0`) or minimally (`-1`)

Inherited from EncoderState (via Line):

Method	`__init__`	Undocumented
Method	`munch`	Munch the input string and determine the resulting token, encoder state, and remainder of the string
Class Variable	`max_length`	The maximum number of tokens to emit before leaving this state
Instance Variable	`length`	Undocumented

def next(self, token: Token) -> list[EncoderState]: (source) ¶

overrides tivars.tokenizer.state.Line.next

Determines the next encode state given a token

The current state is popped from the stack, and the states returned by this method are pushed.

If the list of returned states is...

empty, then the encoder is exiting the current state.
length one, then the encoder's current state is being replaced by a new state.
length two, then the encoder is entering a new state, able to exit back to this one.

Parameters
token:`Token`	The current token
Returns
`list[EncoderState]`	A list of encoder states to add to the stack

overrides tivars.tokenizer.state.EncoderState.mode

Whether to munch maximally (0) or minimally (-1)