class documentation
class IllegalState(TokenizerState): (source)
Constructor: IllegalState(mode, accept, length)
Tokenizer state which indicates its timeline must be pruned
Inherited from TokenizerState:
| Method | __init__ |
No summary |
| Method | munch |
Munch the input string and determine the resulting token, tokenizer timelines, and remainder of the string |
| Method | next |
Determines the next tokenizer timelines given a token |
| Class Variable | max |
The maximum number of tokens to emit before leaving this state |
| Instance Variable | accept |
Undocumented |
| Instance Variable | length |
Undocumented |
| Instance Variable | mode |
Undocumented |