3.5 Extracting records

After you have filtered for the desired records, you use a record extraction callback to collect all of the necessary information from the filtered records into a single record.

Reference note
As explained in 2.3.6 Joining data streams, you can perform record extraction processing using CQL, but we recommend that you use the adaptor to improve performance.

For example, to summarize and analyze the responsiveness between a client and a server, after the HTTP packet input connector is used as the input callback, you could use a record extraction callback as the data editing callback. You could then use the record extraction callback to join an HTTP request and response packet pair into one record, based on the transmission source IP addresses and the transmission destination IP addresses. This would allow you to gain a clear understanding of response times, and to easily summarize and analyze the resulting data.

In the following figure, after records are filtered by record format and record value so that only the desired records are selected, the record extraction callback joins the resulting records, and generates a new record. The following figure shows the positioning and processing of the callback involved in record extraction.

Figure 3-8 Positioning and processing of the callback involved in record extraction

[Figure]

  1. Records passed to the record extraction callback are first filtered by record format. Only records whose record format is R1 or R2 meet the first condition, so only these records are selected for processing by the next condition. Records that do not satisfy this condition are passed to the next callback.
  2. After the records are filtered by record format, they are then filtered by record value. This condition specifies that records are to be passed to the next process only if the source IP of the request matches the destination IP of the response, and the destination IP of the request matches the source IP of the response. This means that only those records that match this condition are passed to the next process.
  3. Records filtered by record format and record value are joined to produce a single record. Records joined in this step are selected for processing by the next callback.