beaver

Class Parser.TokenStream

java.lang.Object

beaver.Parser.TokenStream

Enclosing class:

Parser

public class Parser.TokenStream
extends Object

This class wraps a Scanner and provides a token "accumulator" for a parsing simulation.

If a source that is being parsed does not have syntax errors this wrapper only adds one indirection while it delivers the next token. However when parser needs to recover from a syntax error this wrapper accumulates tokens shifted by a forward parsing simulation and later feeds them to the recovered parser.

Constructor Summary

Constructors

Constructor and Description
TokenStream(Scanner scanner)
TokenStream(Scanner scanner, Symbol first_symbol)

Method Summary

Modifier and Type	Method and Description
void	alloc(int size) Prepare a stream to accumulate tokens.
void	insert(Symbol t0, Symbol t1) Insert two tokens at the beginning of a stream.
Symbol	nextToken()
Symbol	remove(int i) Removes a token from the accumulator.
void	rewind() Prepare accumulated tokens to be reread by a next simulation run or by a recovered parser.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

TokenStream

public TokenStream(Scanner scanner)

TokenStream

public TokenStream(Scanner scanner,
                   Symbol first_symbol)

Method Detail

nextToken

public Symbol nextToken()
                 throws IOException

Throws:

IOException

alloc

public void alloc(int size)

Prepare a stream to accumulate tokens.

Parameters:

size - number of shifted tokens to accumulate

rewind

public void rewind()

Prepare accumulated tokens to be reread by a next simulation run or by a recovered parser.

insert

public void insert(Symbol t0,
                   Symbol t1)

Insert two tokens at the beginning of a stream.

Parameters:

t0 - first token to be inserted

t1 - second token to be inserted

remove

public Symbol remove(int i)

Removes a token from the accumulator.

Parameters:

i - index of a token in the accumulator.

Returns:

removed token