The Tokenizer type exposes the following members.
Fields
| Name | Description | |
|---|---|---|
|
currentLine | |
|
|
currentLineNumber | |
|
|
currentPos | |
|
EOF |
A token which represents the end of file/input.
A TokenMatcher can return this symbol if it wishes to terminate processing of the input.
|
|
|
EOL |
A token which represents and end of line.
|
|
|
matchers | |
|
|
sr | |
|
|
tokens |
See Also
