Debugging

Parser can be generated with additional features supporting debugging which include:

description of grammar accessible at runtime
log of most recent parser actions (including shifts and reductions)
methods for tracking parser operation

Parser grammar information

Internally parser uses integer identifiers for grammar elements like tokens, nonterminals or productions. Parser can be generated with additional information on these elements which can be used during debug session to generate user-friendly messages.

When Parser.Debug.Infos is set to true parser contains following additional structures:

ParserTokenInfos – array of information on input terminal symbols (tokens)
ParserRuleInfos – array of information on nonterminals
ParserProdInfos – array of information about grammar productions

Methods for accessing these structures are:

TokenInfo GetParserTokenInfo( int tokenId )
ParserRuleInfo GetParserRuleInfo( int ruleId )
ParserProdInfo GetParserProductionInfo( int prodId )

Each method returns record with information on particular grammar element including its name (as defined in grammar). For productions left side and right side elements are provided.

When Parser.Debug.StateInfos option is set to true, parser contains additional structure:

ParserStateInfos –array of information about automaton states.

For each state a list of productions that participated in creation of the state is provided. This information can be used to generate a user-friendly summary of any state and mapping of this state to original grammar. Elements of this structure can be accessed using:

ParserStateInfo GetParserStateInfo( int stateId )

Parser state

Entire parsing session is a single walk of parser automaton. At any moment during this walk current parser configuration is described by current automation state and contents of parser stack. It is often necessary to inspect these elements during run to understand actions taken by parser.

States and actions

When Parser.Debug.Methods option is set to true, additional methods are available for inspecting current parser condition. Note that some of these methods rely on textual information about symbols and states which should be enabled using Parser.Debug.Infos and Parser.Debug.StateInfos.

Following methods provide list of actions available in given state. Additional category parameter can be used to narrow down list of actions to shifts, reductions or gotos:

static ParserActionInfo[] GetParserStateActions( int stateId, [category] )

Returns list of actions in given state

static string GetParserStateActionsString( int stateId, [category] )

Returns a user-readable string with summary of actions in given state

string GetCurrentStateActionsString( [category] )

Returns a user-readable string with summary of actions in current state.

Following methods provide summary information about states:

static string GetParserStateSummaryString( int stateId, [flags] )

Returns summary string for given state

string GetCurrentStateSummaryString( [flags] )

Returns summary string for current state

Reported elements can be customized with flags parameter

Inspecting parser stack

When option Parser.Debug.DebugStack is enabled, following methods for inspecting contents of parser stack are available:

int GetParserStackCount()

Returns count of elements on parser stack

Ste GetParserStackElt( int idx )

Gives access to single element of stack

bool GetParserStackEltData( int idx, out ValueData data )

Returns data associated with given element on stack

string GetParserStackSummaryString( [flags] )

Returns summary string with all elements on stack. Current parser state is added at the end.

string GetParserStackCompactString( [flags] )

Returns compact form of stack summary info.

string GetParserStackCompactStringWithValues()

Returns compact form of stack summary info with token values.

Recent steps

To understand parser activity it is sometimes necessary to consider not just current state but also a sequence of actions that led to this state. Parser can save history of its most recent actions which can be inspected in runtime. History is saved in a cyclic buffer of a fixed length. This structure introduces little overhead in processing.

When Parser.Debug.RecentStepsReport is set to true, lexer saves recent actions in a circular buffer. The depth of history is controlled by Parser.Debug.RecentStepsCount option (with default value of 100 steps).

When Parser.Debug.RecentStepsValueData is also set, saved history contains also values of tokens. Setting this option introduces extra overhead. Saved steps do not contain actual values but summary strings for these values. By default strings are generated using ToString() method of ValueData record. User can provide own custom function for generating this summary string and set its name via Parser.Debug.ValueDataInfoMethod option.

Methods for accessing recent steps are:

int ParserRecentStepsCount()

Returns count of available steps

ParserStep GetParserRecentStep( int off )

Returns data of a single step. Offset argument is counted from current position backwards.

ParserStep[] GetParserRecentSteps( [int maxCount] )

Returns multiple (or all) most recent steps.

string GetParserRecentStepsString( ParserStepSummaryFlags flags, [int maxCount] )

Returns summary string with actions taken in multiple (or all) most recent steps. Flags parameter controls type information reported.

string GetParserRecentStepsString( [int maxCount] )

Returns default summary string with actions taken in multiple (or all) most recent steps.

Tracing

Parser can be added methods for tracing basic activity. Methods are called when parser performs actions like shift, reduction or handles errors.

User code for the methods can be provided in either of three ways (selected by dedicated options):

UserCode – code provided by user is placed inside body of the function
Virtual – method is declared virtual and must be overridden in a subclass
UserCodeInline – no dedicated method is defined. Code is placed directly in parser method. This option gives best performance but must be used with caution.

Below is the list of activities that can be traced and options enabling tracing these activities.

Exact order and types of arguments of methods described in this section are not guaranteed to be the preserved between Alpag versions.

Shift

When Parser.Trace.ParseShift is set to true, parser reports each shift operation to method:

void TraceParseShift( int symbolId, int curStateId, int nextStateId, ref ValueData data )

Implementation of method is controlled by Parser.Trace.ParseShiftImpl.

Custom user code for method body should be placed in trace_shift code block.

Reduce

When Parser.Trace.ParseReduce is set to true, parser reports each reduce operation to method:

void TraceParseReduce(int ruleId, int prodId, int symbolId, int curStateId, int reduceToStateId, int nextStateId, Ste[] stackArr, int stackOff, int stackCnt, ref ValueData resData )

Implementation is controlled by Parser.Trace.ParseReduceImpl.

Custom user code for method body should be placed in trace_reduce code block.

Error

When Parser.Trace.ParseError is set to true, parser reports error detection and recovery actions to method:

void TraceParseError( ParseErrorEvent errorEvent, int curStateId )

Implementation is controlled by Parser.Trace.ParseErrorImpl.

Custom user code for method body should be placed in trace_parse_error code block.

Error events reported to the method correspond to typical error recovery cycle. These are:

Enter – error occurred. Parser will attempt error recovery. This is reported only once for a single error
StackUnwind – parser stepped back up the stack looking for error handler. This can occur multiple times
ErrorNoHandler - parser found no error handler stepping up the stack. Error recovery failed
Recovering – parser found error handler. Error symbol was shifted to stack. Error recovery begins
ShiftDiscard – input symbol did not match any of the actions and was discarded. Executed once for each input symbol.
Recovered – input symbol matches one of actions. Parser recovered from error. Symbol will be shifted to stack and normal operation will continue.
ErrorEOF – end of input stream was reached while in error recovery mode.

< Reading input | Value data >

Alpag Manual