4.5.2 Creating a Grammar – A Short Introduction

A good way to create a grammar is to use Ascolog Insight's scanning feature. Select the log header without the description in the Details window, right click and select the Copy command to copy the header to the clipboard. Go to the Tools menu and open the Grammar Definition dialog (see figure Defining a grammar). Paste the header into the Text sample edit control (press Ctrl+V) and press the Scan button next to the control. Ascolog Insight will scan the log header and create a grammar definition, i.e. it will create terminals, nonterminals and production rules.

In order to select the header of the log records in example Sample log records you must select one of the texts with the light gray background. These texts are the headers of these log records. A header does describe the logged event. Other characteristics of a header are that it is recurring and that it provides meta information of the logged event. The meta information in the example are the time of logging, followed by an abbreviation for the log type (E for Error, W for Warning) and a hexadecimal number which represents a process ID. The rest of the log record is part of the description since it describes the event that caused logging.

2013-02-27 23:11:22,104 E 0x1D50 Memory usage: 66% (a problem occurred)
2013-02-27 23:11:25,104 W 0x1D50 Memory usage: 22%

Example 1: Sample log records

If you select the header of the first log record the Grammar Definition dialog will look like in the figure Defining a grammar. The selected header is displayed in the Text sample field.

Grammar definition
Defining a grammar

As you can see Ascolog Insight created 4 terminals and 4 production rules in the Nonterminals list.

The first production rule says that a header represented by the nonterminal symbol HEADER consists of a nonterminal called NT_1 followed by a terminal symbol space followed by a nonterminal NT_2, followed by a terminal space and another nonterminal NT_3. The pieces of information of the log record and the nonterminals are related as shown in table Information identified by the nonterminals.

Table 1: Information identified by the nonterminals
Nonterminal Information Example
NT_1 Timestamp 27.02.13 23:11
NT_2 Log type E (for Error), W (for Warning), other log types are not shown in the example but are possible.
NT_3 Process ID 0x1D50

It is useful to change the generic names of the nonterminal symbols to meaningful names. Ascolog Insight cannot do this for you because it cannot know the actual meaning of the nonterminal symbols (e.g. that the nonterminal NT_3 represents a process ID). In order to change the name select the nonterminal you want to modify and press the Edit button. A dialog as shown in figure Layout Definition Nonterminal will be displayed:

Layout Definition Nonterminal
Layout Definition Nonterminal

In the Name field enter the name TYPE instead of NT_2. You can leave the other settings untouched. Press the OK button to save the changes or the Cancel button to keep the old settings. In the same way you can change the other names. If you are finished open the File menu of the dialog and press Save & Apply to save the new grammar definition and to apply it to the currently loaded logs.

Ascolog Insight uses color highlighting to assign the parts of a text sample to the nonterminals and terminals of a grammar so that you can easily see how the grammar and the text sample relate. The assignment works in both directions: from a text sample to the grammar and from the grammar to the text sample. If no matches can be found nothing is highlighted. You can see some examples in figure Defining a grammar.

This was just a quick introduction about grammars in Ascolog Insight but maybe it is sufficient for your purpose. However, for more details please refer to chapter Grammars - Advanced Topics.