Package org.antlr.runtime
Class UnbufferedTokenStream
- All Implemented Interfaces:
IntStream
,TokenStream
A token stream that pulls tokens from the code source on-demand and
without tracking a complete buffer of the tokens. This stream buffers
the minimum number of tokens possible. It's the same as
OnDemandTokenStream except that OnDemandTokenStream buffers all tokens.
You can't use this stream if you pass whitespace or other off-channel
tokens to the parser. The stream can't ignore off-channel tokens.
You can only look backwards 1 token: LT(-1).
Use this when you need to read from a socket or other infinite stream.
- See Also:
-
Field Summary
FieldsModifier and TypeFieldDescriptionprotected int
Skip tokens on any channel but this one; this is how we skip whitespace...protected int
protected TokenSource
Fields inherited from class org.antlr.runtime.misc.LookaheadStream
currentElementIndex, eof, lastMarker, markDepth, prevElement, UNINITIALIZED_EOF_ELEMENT_INDEX
-
Constructor Summary
Constructors -
Method Summary
Modifier and TypeMethodDescriptionget
(int i) Get a token at an absolute index i; 0..n-1.Where are you getting symbols from? Normally, implementations will pass the buck all the way to the lexer who can ask its input stream for the file name or whatever.Where is this stream pulling tokens from? This is not the name, but the object that provides Token objects.boolean
int
LA
(int i) Get int at current input pointer + i ahead where i=1 is next int.Implement nextElement to supply a stream of elements to this lookahead buffer.toString
(int start, int stop) Return the text of all tokens from start to stop, inclusive.Because the user is not required to use a token with an index stored in it, we must provide a means for two token objects themselves to indicate the start/end location.Methods inherited from class org.antlr.runtime.misc.LookaheadStream
consume, fill, index, LB, LT, mark, release, remove, reset, rewind, rewind, seek, size, syncAhead
Methods inherited from class org.antlr.runtime.misc.FastQueue
add, clear, elementAt, head, range, toString
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait
Methods inherited from interface org.antlr.runtime.IntStream
consume, index, mark, release, rewind, rewind, seek, size
Methods inherited from interface org.antlr.runtime.TokenStream
LT, range
-
Field Details
-
tokenSource
-
tokenIndex
protected int tokenIndex -
channel
protected int channelSkip tokens on any channel but this one; this is how we skip whitespace...
-
-
Constructor Details
-
UnbufferedTokenStream
-
-
Method Details
-
nextElement
Description copied from class:LookaheadStream
Implement nextElement to supply a stream of elements to this lookahead buffer. Return EOF upon end of the stream we're pulling from.- Specified by:
nextElement
in classLookaheadStream<Token>
- See Also:
-
isEOF
- Specified by:
isEOF
in classLookaheadStream<Token>
-
getTokenSource
Description copied from interface:TokenStream
Where is this stream pulling tokens from? This is not the name, but the object that provides Token objects.- Specified by:
getTokenSource
in interfaceTokenStream
-
toString
Description copied from interface:TokenStream
Return the text of all tokens from start to stop, inclusive. If the stream does not buffer all the tokens then it can just return "" or null; Users should not access $ruleLabel.text in an action of course in that case.- Specified by:
toString
in interfaceTokenStream
-
toString
Description copied from interface:TokenStream
Because the user is not required to use a token with an index stored in it, we must provide a means for two token objects themselves to indicate the start/end location. Most often this will just delegate to the other toString(int,int). This is also parallel with the TreeNodeStream.toString(Object,Object).- Specified by:
toString
in interfaceTokenStream
-
LA
public int LA(int i) Description copied from interface:IntStream
Get int at current input pointer + i ahead where i=1 is next int. Negative indexes are allowed. LA(-1) is previous token (token just matched). LA(-i) where i is before first token should yield -1, invalid char / EOF. -
get
Description copied from interface:TokenStream
Get a token at an absolute index i; 0..n-1. This is really only needed for profiling and debugging and token stream rewriting. If you don't want to buffer up tokens, then this method makes no sense for you. Naturally you can't use the rewrite stream feature. I believe DebugTokenStream can easily be altered to not use this method, removing the dependency.- Specified by:
get
in interfaceTokenStream
-
getSourceName
Description copied from interface:IntStream
Where are you getting symbols from? Normally, implementations will pass the buck all the way to the lexer who can ask its input stream for the file name or whatever.- Specified by:
getSourceName
in interfaceIntStream
-