Log Processing Mode.cfg

The configuration file Log Processing Mode.cfg enables you to pause processing of data into a dataset, specify offline sources, or specify the frequency at which data workbench server saves its state files.

Making changes to the Log Processing Mode.cfg file, including adding or removing sources, does not cause reprocessing of the data.

To edit the Log Processing Mode.cfg file for a dataset profile

1. While working in your dataset profile, open the Profile Manager and click Dataset to show its contents.

   NOTE

   If the Log Processing Mode.cfg file is not located in the directory for the desired profile, you need to copy this file from the Base directory on the data workbench server machine into the profile’s directory.

   For information about opening and working with the Profile Manager, see the Data Workbench User Guide.

2. Right-click the check mark next to the configuration file’s name and click Make Local. A check mark for this file appears in the User column.

3. Right-click the newly created check mark and click Open > from the workbench. The configuration window appears.

4. Edit the parameters in the configuration file using the following table as a guide.

   NOTE

   Some of the parameters in the Log Processing Mode.cfg file have names that include Fast Input or Fast Merge. Fast Inputrefers to the log processing phase of dataset construction and is responsible for approximately half of the total dataset processing time. Fast Merge refers to the transformation phase of dataset construction only when preceded by log processing. Fast Merge does not occur during retransformation that results from modifying a Transformation Dataset Configuration file. Like Fast Input, Fast Merge is also responsible for approximately half of the dataset processing time.

   Parameter	Description
   Cloud Bytes	A tuning parameter that affects the efficiency of data transformation. The default value is 128000000. Note: You should not change this value without consulting Adobe.
   Fast Input Decision Ratio	A tuning parameter that specifies the ratio of total to unread log bytes at which the system enters Fast Input mode (and subsequently Fast Merge) instead of processing data in real time. The default value is 200, meaning that the system enters Fast Input mode from real-time mode when the unread log data is at 1/200th of the total data. A higher decision ratio makes the system enter Fast Input mode more readily, while a lower ratio makes it less likely to enter Fast Input mode. Note: Setting the parameter to 0 prevents the system from entering Fast Input mode at all, even for initial processing. Setting the parameter to 1.1 enables the system to enter Fast Input during initial processing but not for subsequent processing. Adobe does not recommend using values between 0 and 1.1. For more information about setting this parameter, contact Adobe.
   Fast Input FIFO Bytes	A tuning parameter that balances memory usage and system performance during data processing. The default value is 120000000. Note: You should not change this value without consulting Adobe.
   Fast Merge Buffer Bytes	A tuning parameter that balances memory usage and system performance during data processing. The default value is 128000000. Note: You should not change this value without consulting Adobe.
   Offline Sources	Mask of the offline log source. To specify an offline source Right-click Offline Sources, then click Add new > Source. In the parameter for the new source, enter the mask of the log sequence. For Sensor log sources with file names of the format YYYYMMDD-SENSORID.vsl, the mask is SENSORID.SENSORID is case-sensitive. For log file log sources, the mask is the string extracted by the Mask Pattern. See Log Files. Adding or removing sources from Offline Sources does not cause reprocessing of the dataset. As Of time measurements are maintained for the processing of the profile's online sources. When the offline source is again online, the processing of incoming log files for that source resumes. Whenever a source comes back online, you should remove it from Offline Sources. If you do not do so, data workbench server treats the source as an online source and updates the As Of time as long as the source is sending data. If the source goes offline again, the As Of time measurements stop.
   Paused	True or false. If true, new data is not processed into the dataset. The default value is false.
   Real Time Delay	The amount of time in seconds that data workbench Server waits between intervals of processing data into the dataset. When this value is set to zero, the system attempts to keep up with incoming data in real time. The default value is zero (0), but you can increase this value to reduce CPU load.
   Real Time FIFO Bytes	The amount of memory in bytes used to store data that is waiting to be processed into the dataset. You may need to change this value based on the number of seconds that you specify for Real Time Delay. The default value is 16000000. Note: You should not change this value without consulting Adobe.
   Save Interval (sec)	Frequency at which the data workbench server saves its state files. The default value is 3600. Note: You should not change this value without consulting Adobe.

   When editing the Log Processing Mode.cfg file within a data workbench window, you can use shortcut keys for basic editing features, including cut (Ctrl+x ), copy (Ctrl+c) , paste (Ctrl+v ), undo (Ctrl+z ), redo (Ctrl+Shift+z ), select section (click+drag), and select all (Ctrl+a ). In addition, you can use the shortcuts to copy and paste text from one configuration file ( .cfg) to another.

5. Right-click (modified) at the top of the window and click Save.

6. In the Profile Manager, right-click the check mark for the file in the User column, then click Save to > datasetprofile name.

   NOTE

   Do not save the modified configuration file to any of the internal profiles provided by Adobe, as your changes are overwritten when you install updates to these profiles.

Business.Adobe.com resources

Audio Analytics Voice Analytics Data Ingestion Mobile Marketing Identity Management Marketing Analytics