6.4.1 Detecting the trigger for terminating a data reception application (data processing termination notification)

Since a data reception application cannot be terminated by a command in Stream Data Platform - AF, a termination process must be implemented in the data reception application.

This section explains an implementation that enables a data reception application to detect the trigger for terminating itself. The method shown here can be used in the following cases, in which the trigger for terminating stream data transmission can be detected by the stream data processing engine beforehand:

Organization of this subsection
(1) Method of detecting the trigger for terminating a data reception application
(2) Data transmission termination notification based on the data transmission application calling a method (putEnd method)
(3) Query group stop notification based on the execution of the sdpcqlstop command

(1) Method of detecting the trigger for terminating a data reception application

A data reception application keeps receiving result data as long as queries are being executed by the stream data processing engine. For the data reception application to end, it must detect the following states:

The data reception application can detect these states by catching the SDPClientEndOfStreamException or SDPClientQueryGroupStopException exceptions that occur when the data reception API (get or getAll method) is called. Catching these exceptions allows you to terminate the data reception application.

To generate these exceptions, a data processing termination notification must be sent to the output stream. This is done by having the data transmission application call a method or execute a command.

These notification methods are explained in (2) Data transmission termination notification based on the data transmission application calling a method (putEnd method) and (3) Query group stop notification based on the execution of the sdpcqlstop command.

(2) Data transmission termination notification based on the data transmission application calling a method (putEnd method)

When the data transmission application is finished transmitting data, the putEnd method should be called. When the output stream runs out of query result data, the SDPClientEndOfStreamException exception is thrown to the data reception application.

The following figure shows the flow for sending a data transmission termination notification by calling the putEnd method.

Figure 6-5 Flow for sending a data transmission termination notification by the data transmission application

[Figure]

The details of the flow shown in the figure are explained below. The numbers correspond to the numbers in the figure.

  1. When the transmission of stream data to the stream data processing engine is completely finished, the data transmission application calls the putEnd method to send a data transmission termination notification to the applicable input stream.
  2. Once a transmission termination notification is received for all input streams in the query group, the stream data processing engine executes a query on all tuples in the input streams, outputs the query result data to the appropriate output streams, and then sends a processing termination notification to all output streams.
    Even after the processing termination notification is issued, the data reception application continues to receive data as long as there is query result data in the output stream. To receive data, the get or getAll method is used.
  3. When all query result data in the output streams has been received and the output stream runs out of data, the exception SDPClientEndOfStreamException is thrown the next time the data reception application calls the get or getAll method.
    Using this exception as the trigger, the data reception application can be stopped.

If the query group contains an input stream for which the putEnd method has not been called, the exception SDPClientEndOfStreamException does not occur when the data reception application calls the get or getAll method.

(3) Query group stop notification based on the execution of the sdpcqlstop command

The sdpcqlstop command is executed to stop the query group. When the output stream runs out of query result data, the exception SDPClientQueryGroupStopException is thrown to the data reception application.

The following figure shows the flow for using the sdpcqlstop command to stop a query group.

Figure 6-6 Flow for using the sdpcqlstop command to stop a query group

[Figure]

The details of the flow shown in the figure are explained below. The numbers correspond to the numbers in the figure.

  1. When the sdpcqlstop command issues a query group stop notification, the stream data processing engine stops accepting input stream data. It continues to process the data already in the input stream and then terminates processing.
    Even after the command is executed, if query results remain in the output stream, the data reception application continues to receive the data using the get or getAll method.
  2. When the output stream runs out of data, the exception SDPClientQueryGroupStopException is thrown the next time the data reception application calls the get or getAll method. Using this exception as the trigger, the data reception application can be stopped.