2.5.2 Evaluating the data editing methods

You must evaluate which of the data editing methods provided by the standard adaptors is to be used to edit data.

Data editing is performed by the input or output adaptors. The standard adaptors support data editing, such as mapping and record extraction. The table below lists and describes the data editing methods.

Table 2-4 Data editing methods

No.Data editing methodDescription
1Format conversionYou can use format conversion to convert from input or output records to common records, and vice versa.
  • Input records and output records:
    Each record consists of a row of data separated by commas (such as in CSV files).
  • Common records:
    Each record is a set of data consisting of a name (field name) and a value (field value).
An input adaptor's format conversion involves conversion from an input record to a common record. An output adaptor's format conversion involves conversion from a common record to an output record. For details about each data format, see 10.2.2(1) Data formats handled by an input adaptor and 10.6.2(1) Data formats handled by an output adaptor
Note that format conversion is performed only in the following cases:
  • When the file input method is used to input data
  • When the file output method is used to output data
For details about format conversion, see 10.2.2(3) Format conversion or 10.6.2(4) Format conversion.
2MappingYou can use mapping to associate the common record output in the callback that precedes the mapping and the common record input in the callback that follows the mapping.
The two types of mapping are the mapping between records and the mapping between record and stream.
For example, a mapping between record and stream associates a common record that is input or output by format conversion with a common record based on the input stream or output stream format.
For details about mapping, see 10.2.2(4) Mapping or 10.6.2(3) Mapping.
3Record filteringYou can use a filter to select records. The target of record filtering is common records.
For details about record filtering, see 10.4 Record filtering.
4Record extractionYou can use record extraction to extract records that satisfy a specified condition and then join the extracted records to create a new record. The target of record extraction is common records.
Note that record extraction is applicable only to input adaptors.
For details about record extraction, see 10.5 Record extraction.

The following figure shows the timing of data editing.

Figure 2-11 Timing of data editing

[Figure]

Timing of format conversion
For input adaptors, format conversion is performed after data input by an input connector. For output adaptors, format conversion is performed before data output by an output connector.
Timing of mapping#
Mapping between record and stream is performed before a tuple is sent for input adaptors, and after a tuple is received for output adaptors.
Mapping between records is performed after data input by input connector, or after format conversion for input adaptors and before data conversion or data output by output connector for output adaptors.
Timing of filtering#
For input adaptors, filtering is performed after data input by the input connector or format conversion. For output adaptors, filtering is performed before format conversion or data output by output connector.
Timing of record extraction#
Record extraction is performed after data input by input connector or format conversion.
#
You can perform filtering, record extraction, and mapping in any order.

Note that you specify data editing in the CB definition for editing in the adaptor configuration definition file. For details about the CB definition in the adaptor configuration definition file, see 9. Adaptor Definition Files.