tivars.tokenizer.state.Line

class documentation

Constructor: Line(length)

Encoder state which is always exited after a line break

Method next Determines the next encode state given a token

Inherited from EncoderState:

Method	`__init__`	Undocumented
Method	`munch`	Munch the input string and determine the resulting token, encoder state, and remainder of the string
Class Variable	`max_length`	The maximum number of tokens to emit before leaving this state
Class Variable	`mode`	Whether to munch maximally (`0`) or minimally (`-1`)
Instance Variable	`length`	Undocumented

def next(self, token: Token) -> list[EncoderState]: (source) ¶

overrides tivars.tokenizer.state.EncoderState.next

overridden in tivars.tokenizer.state.InterpolationStart, tivars.tokenizer.state.Name, tivars.tokenizer.state.String

Determines the next encode state given a token

The current state is popped from the stack, and the states returned by this method are pushed.

If the list of returned states is...

empty, then the encoder is exiting the current state.
length one, then the encoder's current state is being replaced by a new state.
length two, then the encoder is entering a new state, able to exit back to this one.

Parameters
token:`Token`	The current token
Returns
`list[EncoderState]`	A list of encoder states to add to the stack