10.5.2 Flow of data processing during record extraction

In record extraction, evaluation of extraction target conditions and extraction conditions and joining of extracted records occur before a new record (retrieved record) is generated. The following figure shows the flow of data processing during record extraction.

Figure 10-14 Flow of data processing during record extraction

[Figure]

Details of the processing in 1 through 3 in this figure are provided below. For details about the tags and attributes in the record extraction definition in the adaptor configuration definition file that are discussed below, see 9.11.4 Record extraction definition.

Organization of this subsection
(1) Evaluating the extraction target conditions
(2) Evaluating the extraction conditions
(3) Generating a retrieved record

(1) Evaluating the extraction target conditions

This processing involves evaluating the extraction target conditions in the input records.

An extraction target condition is one of the types of conditions used to select records to be extracted. You specify an extraction target condition in the targetrecord tag in the record extraction definition.

For the extraction target conditions, specify a record condition and a field condition.

For example, to specify the record name R1, field name F1, and field value MIKE as the conditions, specify source="R1" for the record name in the record condition and source="F1" condition="eq" value="MIKE" in the field condition.

Evaluating the extraction target conditions involves evaluating the record condition and field condition according to the extraction target conditions specified in the record extraction definition.

  1. Evaluate the record condition
    The record extraction function determines whether the record name in the input record matches the record name specified in the record condition.
    • Record satisfying the record condition
      Passed on to the field condition evaluation processing.
    • Record that does not satisfy the record condition
      Output as is and passed on to the next callback.
  2. Evaluate the field condition
    The record extraction function determines whether the field value in the record that satisfies the record condition matches the field value specified in the field condition.
    • Record satisfying the field condition
      Passed on to the extraction condition evaluation processing because this record satisfies the extraction target condition. For details about extraction condition evaluation processing, see (2) Evaluating the extraction conditions.
    • Record that does not satisfy the field condition
      Discarded at this point.

(2) Evaluating the extraction conditions

The extraction condition is evaluated for the records that satisfy the extraction target condition.

An extraction condition is one of the types of conditions used to select records to be extracted. You specify an extraction condition in the extraction tag in the record extraction definition.

For the extraction condition, you specify the maximum number of records that can be retained in the record buffer and a condition between records. For the condition between records, you specify the record input order and the field values to be compared between records. The targets tag is used to specify the condition between records.

Evaluating the extraction conditions involves evaluating the condition between records according to the extraction conditions specified in the record extraction definition. The records that satisfy the condition between records are retained in the record buffer until the extraction conditions are satisfied. The flow of evaluating the extraction conditions is described below.

  1. Evaluate the conditions between records
    The record extraction function evaluates the records that satisfied the extraction target conditions to determine whether they also satisfy the following conditions between records:
    • Whether the record input orders match
    • Whether the field values match between records
    The records that satisfy these conditions are retained in the record buffer, while the records that do not satisfy the conditions are discarded.
  2. Retain records in the record buffer
    The records that satisfy the conditions between records are retained in the record buffer. When all applicable records are obtained after the conditions between records are checked, the extraction condition is satisfied and the generation of the retrieved record is then performed. For details about the retrieved record generation processing, see (3) Generating a retrieved record.

The following figure shows an example of the flow of retaining and discarding records when the record input order is specified.

Figure 10-15 Example flow of retaining and discarding records when the record input order is specified

[Figure]

This example specifies the record input order as Record A[Figure] Record B[Figure] Record C. As a result, the input records are retained or discarded as follows:

If records are discarded, the number of discarded records is output to the adaptor trace information. For details about the adaptor trace information, see 6.3.3 Details of adaptor trace information.

During the extraction condition evaluation processing, records are also discarded if their time information is out of sequence or a timeout occurs on a record retained in the record buffer. These situations are discussed below.

(3) Generating a retrieved record

Generation of a retrieved record involves joining the records that satisfy the extraction conditions to generate a retrieved record. The flow of retrieved record generation is described below.

  1. Select the records to be joined
    Based on the extraction condition, the record extraction function selects the records to be joined to generate a retrieved record. You specify the records to be joined in the source attribute in the select tag.
  2. Generate a retrieved record
    The records selected in step 1 are joined to generate a retrieved record.

A field containing the extraction condition name specified in the name attribute in the extraction tag is added to the generated retrieved record.

The following figure shows an example of a retrieved record.

Figure 10-18 Example of a retrieved record

[Figure]

This example joins records A and C to generate a new record E as the retrieved record.