Value data

Values associated with terminals and nonterminals are stored on stack in dedicated structure called ValueData. Elements of this structure are defined automatically from value types declared by user. By default instances of ValueData structure are allocated, copied and deleted by parser automatically. The default behavior may be inappropriate or insufficient if values stored in the structure require special handling. Alpag provides a number of ways to customize both definition and manipulation of ValueData structure.

Value fields

The ValueData structure by default contains a fields for each value type defined in parser.

If declared value types are:

%valtype INTEGER %type { int }
%valtype MYSTR %type { string }

Value data structure looks as follows:

public struct ValueData
{
public int INTEGER;
public string MYSTR;
//...
}

User is free to define additional members of the structure using value_data code section.

Instead of relying on Alpag-generated fields user can provide own fields. To use these fields with value types defined in grammar user must provide code templates for accessing them. Code templates must be provided in %valtype declarations for particular types. Three code access templates can be defined:

%access - general template for getting and setting a field
%get – template for getting value of fields
%set – template for setting field

Here are typical field and access template declarations:

%valtype SOMETYPE %decl { public int MyField; } %access { $$.MyField; }

Own declaration of field using %decl and access template using %access.

%valtype SOMETYPE %access { $$.MyField; }

When field is declared inside value_data code section it is enough to provide just %access template.

%valtype SOMETYPE %get { $$.myGetter() } %set { $$.mySetter( $1 ) }

Separate setter and getter templates can be defined via %set and %get. These replace common %access template.

When distinct getter and setter templates are defined appropriate syntax must be used to access values. Providing that accessed field is $$ then:

$$ is replaced by %get template
$${ expression } is replaced by %set template with value of expression substituted for $1 placeholder in template.

Value allocation

Alpag can store values it two ways:

on parser stack
in dedicated storage.

By default values are stored on parser stack. When stack grows or shrinks storage for values associated with stack elements is allocated and released accordingly. This solution has some drawbacks:

When grammar production is reduced, temporary storage must be used for result of the reduction before it is placed on stack, since top of stack is still occupied by old elements.
When parser fetches next input terminal symbol its value is first stored with lookahead symbol and remains there until parser decides to push it to stack.

In either case additional copying occurs when values are finally placed on stack. To bypass this redundant operation Alpag provides alternative way of storing values in a heap-like structure. Elements of this structure are allocated independently from stack. This eliminates unnecessary copying.

Storage of Value data is controlled by Parser.ValueData.Placement option. Possible settings are:

Stack - values stored on parser stack
Dedicated - values stored in dedicated structure

Parser behavior is the same with either setting. Difference applies to the number and sequence of value data copy and clear operations.

Value lifecycle

New token values are created in two situations:

When parser fetches next input symbol value for the symbol is provided by user.
When parser performs a reduction, value for result of reduction, a nonterminal is calculated and pushed to stack.

Both these scenarios require using intermediate storage and performing extra copy operations:

When parser requests a new input symbol, this symbol is not immediately pushed to stack. It remains as a lookahead symbol until parser finally decides to shift it. Then it is copied to stack.
When parser performs a reduction, result of reduction cannot be immediately pushed to stack since it is still occupied by right-hand-side elements of production being reduced. Only after reduction is complete result value can be copied to stack.

Lifecycle of a value includes three basic operations: allocation, copying and deletion. Allocations often reuse structure previously used for another value. In such case handling a delete-and-allocate sequence can be simplified if it is treated as reuse. Furthermore copying value from one location to another is sometimes followed by clearing its old location and can be perceived as move.

Copying large value data records can be costly. Additional cost may arise if user explicitly demands that value data records were cleared. User must understand lifecycle of value data record in order to decide if and when to clear it:

Possible choices include:

clearing values before use
clearing values after use (which enables both cleanup and preclear before possible reuse)
not clearing values at all (assuming each assignment overrides previous values)

Code for clearing values can be either generated automatically by Alpag or provided by user.

Cleanup options

Options for adding cleanup code generated automatically by Alpag are:

Parser.ValueData.ClearNew

When turned on, newly allocated data structures are precleared. This option is off by default since clearing new copies of value data is usually not necessary.

Parser.ValueData.ClearReused

When turned on, reused (or possibly reused) data structures are cleared. This option is enabled by default. It can be turned off if user is sure that assignment of new value fully overwrites old value.

Parser.ValueData.ClearDeleted

When turned on, discarded values are cleared. It can be turned off if abandoning old values is acceptable. Note that value data record that is being deleted can be in reused in the future. When this option is off it is good to enable ClearReused. Enabled by default

Parser.ValueData.ClearConsumed

In scenarios where value is copied from one location to another (i.e. value is moved) this option controls clearing old location. Enabled by default.

Custom cleanup code

Sometimes types used for value storage may require additional actions on allocation, copying and cleanup. User can provide custom code for these actions.

Below is the list of named %code templates that can be customized by user:

value_clear

Default code executed when value data is cleared.