10.5.1 Correcting Timestamps (Time Shifting)

The video tutorial Correcting Timestamps on Youtube will explain how to correct timestamps of log records.

The intention is to merge two or more log files by using their timestamps as merge criteria. The order of the log records in the merged log can be incorrect if the timestamps were created by different clocks that were not synchronous. Ascolog Insight offers you the possibility to correct timestamps by the same but configurable value so that the order of the records in the merged log can be corrected. This is called time shifting.

In order to enable time shifting the following steps are necessary:

  1. The TIME_SHIFT macro which implements the time shift functionality has to be added to the layout's custom columns definition by using the Custom Column Definition editor.

  2. Time shifting requires three parameters. They have to be configured in the CCD Data dialog.
    Parameter ts_cor_mul_1: Corrects the unit of the parameter ts_cor_1.
    Parameter ts_cor_1: The timestamps will be shifted by the value of this parameter. The unit (seconds, minutes, etc.) of this parameter depends on the value of ts_cor_mul_1 because the TIME_SHIFT macro multiplies ts_cor_1 with ts_cor_mul_1 and considers the product of these two values to be a time in milliseconds. If you want to set the unit of ts_cor_1 to minutes you have to set ts_cor_mul_1 to 60000 because 1 minute * 60000 = 60000 milliseconds.
    Parameter ts_fn_1: Time shifting will be applied to the log file specified by this parameter.

The first step is to add the TIME_SHIFT macro to the custom columns definition. Go to the Tools menu and open the Custom Column Definition dialog. Select Edit and Defines... to open the Defines dialog. Add USE_ALL_MACROS=1 in the Defines dialog to enable the TIME_SHIFT macro. Following this, go to the Tools menu and select Includes... to open the Includes dialog. Add the folder where the file is located that contains the TIME_SHIFT macro (file “basemacros.cdf”) to the custom columns definition's include directories. How to find the file “basemacros.cdf” is explained in section Working with Custom Columns Definitions.

Moreover, the file “basemacros.cdf” has to be included in the custom columns definition using the statement #include “basemacros.cdf” and the TIME_SHIFT macro has to be added, too. The custom columns definition has to look like in example Shifting timestamps. The macro gets a timestamp column which is created by the layout's grammar definition as input (TIMESTAMP in the example). It shifts all values of the log file which is specified in the CCD Control dialog by the shift value set in the CCD Control dialog. In the example the output is passed to the TS column.

#include "basecolumns.cdf"
#include "basemacros.cdf"

// this custom column will serve as an adjusted timestamp
TS = TIME_SHIFT(TIMESTAMP);

Shifting timestamps

Press Save & Apply in the File menu of the Custom Column Definition dialog to save the changes made and to apply the custom column definition to the currently loaded log records. The Custom Columns Definition dialog will be closed. The TIME_SHIFT macro adds the TS column to the Log window. The next step is to open the Options dialog which is located in the Tools menu. Set the Columns/MergeColumn property to the TS column. Since the TS column will contain the time shifted timestamps it is useful to set TS as the MergeColumn to use the time shifted timestamps for timestamp comparisons. Sort the TS column in ascending order.

In order to configure the parameters mentioned in step 2 open the CCD Data dialog which is shown in figure Defining parameters for custom columns commands. Press the Add button to add a new parameter. Select a parameter and press Edit to edit a parameter. When you press Remove the selected parameter is deleted whereas Clear deletes all parameters.  

Defining parameters for custom columns commands
Defining parameters for custom columns commands

After adding all parameters the dialog should look like in Defining parameters for custom columns commands. The following description will guide you through this process:

First add the parameter ts_cor_1. The parameter ts_cor_1 should be configured as shown in Editing the ts_cor_1 parameter. Assign the Slider control 1 to the parameter. Using this control you can specify the time value that is used to shift timestamps. The slider control will be used in the CCD Controls dialog to control the parameter. The variable %Slider1 will contain the value set by the Slider control 1. The value of %Slider1 will be assigned to the parameter ts_cor_1. The Min and Max fields specify that the time shift values can be set from -3600 to 3600. The initial value (Value field) will be zero so timestamps are not shifted. 

Editing the ts_cor_1 parameter
Editing the ts_cor_1 parameter

The parameter ts_cor_mul_1 must be configured as shown in figure Editing the ts_cor_mul_1 parameter. This parameter just corrects the unit used for time shifting which is why no control is assigned. Since ts_cor_mul_1 is set to 60000 the time shift values of ts_cor_1 are minutes. Thus you can shift timestamps for 3600 minutes back and forth (see option Slider control default setting of figure Editing the ts_cor_1 parameter).

Editing the ts_cor_mul_1 parameter
Editing the ts_cor_mul_1 parameter

There is still another parameter to configure. An edit control of the CCD Controls dialog should be assigned to the parameter ts_fn_1 as shown in figure Editing the ts_fn_1 parameter. Using this control you can specify the log file whose timestamps should be shifted. The value of %Text1 will be passed to the parameter ts_fn_1.

Editing the ts_fn_1 parameter
Editing the ts_fn_1 parameter

Click on the CCD Controls button in the toolbar to open the dialog shown in figure Setting the values for the time shift where the actual values for the time shift can be set.

The time shifting functionality is now ready to use.

Setting the values for the time shift
Setting the values for the time shift

In the Control set selection drop down list select 1. Control Set [ts_cor_1, ts_fn_1] because in the CCD Data dialog the first controls (Slider control 1 and Text control 1) were assigned to the ts_cor_1 and ts_fn_1 parameters. The TIME_SHIFT macro will apply the time shift on the file specified in the Text1 edit control because this edit control is assigned to the ts_fn_1 parameter. Tick the Merge records check box to tell Ascolog Insight to merge the time shifted log records with the other log records. This means that the log records from the different files will be ordered by their timestamps in the Log window.

A time shift will always be applied to the original timestamp as it is logged in the log file. When you enter -60 in the Slider1 edit control and press the Apply button the timestamps of the log records from file “Samplefile5.log” will be shifted back for 60 minutes, i.e. if the hour part of an original timestamp is 10 the shifted timestamp's hour part will be 09. If you change the time shift value from -60 to 60 minutes this will mean that the hour part of the shifted timestamp will be 11. because the original hour timestamp value (=10) will be used and not the time shifted one (=09). Instead of directly specifying the time shift value in the edit control and pressing the Apply button you can also use the slider. Changes to the slider will be immediately applied. You can also use the "Arrow-left" and "Arrow-right" keys to immediately shift the timestamps.