3.1.1 Standard adaptors

The adaptors provided by Stream Data Platform - AF are called the standard adaptors. The actions that a standard adaptor performs in order to correctly pass data are defined in an adaptor definition file. For details about the adaptor definition file, see the uCosminexus Stream Data Platform - Application Framework Setup and Operation Guide.

Using a standard adaptor, you can input the following types of data into the stream data processing engine:

A standard adaptor can also output the results of the stream data processing engine to the following destinations:

To input data from a data source other than the above, or to output results to an output destination other than the above, you need to use a custom adaptor. For details about custom adaptors, see 3.1.2 Custom adaptors.

The standard adaptors function using units of processing called callbacks. Functions in the adaptor definition file are defined in terms of these callback units.

The standard adaptor callbacks are executed on input data one line (or row) at a time (for HTTP packets, they are executed by request message or by response message). This single line or message unit is called a record.

The following figure shows the organization of the callbacks in an input adaptor.

Figure 3-1 Organization of the callbacks in an input adaptor

[Figure]

The following figure shows the organization of the callbacks in an output adaptor.

Figure 3-2 Organization of the callbacks in an output adaptor

[Figure]

The following table lists and describes the functions of the standard adaptors.

Table 3-1 Functions of the standard adaptors

No.CallbackTypeFunctionDescription
1File input connectorInput callbackInputs a fileUsed for inputting a file.
2HTTP packet input connectorInputs HTTP packetsUsed for inputting HTTP packets over a network.
3Format conversionData editing callbackConverts the format of records during file input or outputUsed only when the file input connector or file output connector is specified. There are two types of format conversion callbacks: one for the input adaptor and one for the output adaptor.
For the input adaptor:
This callback converts the records acquired from the input data (input records) to a format that can be processed by the stream data processing engine (common records).
For the output adaptor:
This callback converts the common records processed by the stream data processing engine to output records.
4FilterFilters recordsUsed to selectively extract records.
When you use a filter callback, the adaptor can perform processing like that executed in the CQL code described in 2.3.2 Retrieving data that satisfies a condition. To improve performance, we recommend that you use the adaptor to perform this kind of processing.
5Record extractionExtracts recordsUsed only with the input adaptor.
This callback allows you to extract only the selected records that will then be joined to generate new records.
When you use a record extraction callback, the adaptor can perform processing like that executed in the CQL code described in 2.3.6 Joining data streams. To improve performance, we recommend that you use the adaptor to perform this kind of processing.
6MappingMaps recordsThere are two types of record mapping callbacks: one for the input adaptor and one for the output adaptor.
For the input adaptor:
This callback converts common records to common records that are consistent with the format of the input stream. Also, if necessary, this callback will first convert the common records to common records that are suitable for processing by the next callback.
For the output adaptor:
This callback converts common records that are consistent with the format of the output stream to common records that are suitable for processing by the next callback. This callback can also further convert these common records to common records that are suitable for processing by the next callback.
7Tuple transmissionTuple exchange callbackSends tuples to the stream data processing engineUsed by the input adaptor to send tuples to the stream data processing engine.
8Tuple receptionReceives tuples processed by the stream data processing engineUsed by the output adaptor to receive tuples processed by the stream data processing engine.
9File output connectorOutput callbackOutputs a fileUsed to output the results of the stream data processing engine to a file.
10Dashboard output connectorOutputs to the dashboardUsed to output and display the results of the stream data processing engine to the dashboard.

For more details about callbacks, see the uCosminexus Stream Data Platform - Application Framework Setup and Operation Guide.

Beginning in section 3.2 Inputting files, this manual provides an overview of the following callbacks:

The callbacks listed in the table are used in combination with one another. The following example describes a callback configuration.

Example of a callback configuration
Assume that you are monitoring the temperature at a number of observation sites, and that you want to summarize and analyze the observation results from one particular observation site, and then output those processing results to a file. To do this, you need to code callbacks that perform the following actions:
  • Summarize and analyze only the data sent from observation site 1 (ID: 1).
  • Output the processing results to a file.
The following two figures show the positioning of the adaptor callbacks used to implement this processing for the input adaptor and the output adaptor, respectively.

Figure 3-3 Example of positioning and processing of the input adaptor callbacks

[Figure]

Figure 3-4 Example of positioning and processing of the output adaptor callbacks

[Figure]
The input adaptor uses a filter so only data from observation site 1 (ID: 1) is passed through for summary analysis, and the output adaptor uses a file output connector to send the processing results to a file.