6.4.1 Problems during system operation

This subsection discusses problems that might occur during system operation and how to handle them.

Organization of this subsection
(1) Data cannot be sent or received between the stream data processing engine and adaptors
(2) An overflow occurred in the input stream queue for the stream data processing engine
(3) An internal conflict message was output
(4) A query group was shut down
(5) A timeout occurred during query processing
(6) Adaptor processing has slowed
(7) Adaptor processing has stopped due to unmatched data types
(8) Some tuple logs have been lost
(9) The analysis results cannot be displayed on the dashboard

(1) Data cannot be sent or received between the stream data processing engine and adaptors

The following describes the action to be taken when data cannot be sent or received between the stream data processing engine and adaptors:

  1. Check the message log for an error message and correct the cause of the error, if necessary.
    If an error message has been output, take appropriate action to eliminate the cause of the error by referencing the manual uCosminexus Stream Data Platform - Application Framework Messages.
  2. Check the following:
    • Whether the path of the input source or output target is specified correctly
    • Whether the specification method for format conversion is correct
      If you have specified IGNORE in the unmatchedFormat tag in the format conversion definition in the adaptor configuration definition file, specify WARNING in the unmatchedFormat tag to check the operation so that an invalid format can be detected.

(2) An overflow occurred in the input stream queue for the stream data processing engine

If an overflow has occurred in the input stream queue for the stream data processing engine, control the volume of input adaptor processing. By changing the number of records that are read at one time or the interval at which records are sent to the SDP server, you can reduce the amount of memory used per unit of time.

You can change the number of records by using the readUnit attribute of the input tag in file input connector definition in the adaptor configuration definition file.

You can change the transmission interval by using the interval attribute in the CB definition for input.

(3) An internal conflict message was output

If an interval conflict message is output for a query group or the SDP server, collect data by referencing 6.2 Data to be collected in the event of an error, and then contact the customer engineer.

(4) A query group was shut down

If a failure occurs while a query group is executing, Stream Data Platform - AF shuts down the query group. If a query group is shut down, the tuples accumulated in the stream queues are discarded.

The following describes how to handle query group shutdown for each cause.

When the cause is a stream queue overflow:
If an overflow occurs in a query group's stream queue, the following messages are output to the message log:

KFSP42005-E The upper limit value of the stream queue was exceeded. Stream name = ..., number of elements = ..., upper limit value = ...
KFSP82205-I A query group was blocked. Query group name=...

When these messages are output, check the settings in the SDP server definition files, such as the JavaVM heap size, queue maximum value, and maximum number of tuples that can be retained. If necessary, adjust values and restart the query group.
For details about changing settings in the SDP server definition files, see 5.3.1 Changing settings in the property files. For details about restarting query groups, see sdpcqlstart (starts a query group) in 7. Commands.
When the cause is input of invalid data:
If a division-by-zero error or an invalid type conversion error (value obtained after conversion is not a type supported by the target) occurs during query execution, the following message is output to the message log:

KFSP420**-E =...
KFSP82205-I A query group was blocked. Query group name=...

When this message is output, check the query definition files for an error. If there is an error in the query definition files, correct it. If there is no error in the query definition files but there is a problem in the send data, correct the contents of the input adaptor, and then restart the query group.
For details about changing query definition files, see 5.3.2 Changing the query definition file. For details about restarting query groups, see sdpcqlstart (starts a query group) in 7. Commands.
When the cause is that the number of tuples to be retained exceeded the maximum value while the timestamp adjustment function was being used:
If the number of tuples to be retained exceeded the maximum value for the timestamp adjustment function, the following message is output to the message log:

KFSP42301-E The number of tuples held by the timestamp adjustment function exceeded the upper limit value....
KFSP82205-I A query group was blocked. Query group name=...

When this message is output, check and revise (if necessary) the maximum number of tuples that can be retained or the units and range of times to be adjusted. For details of the tuple retention period, see 10.8.6 Tuple retention period.

(5) A timeout occurred during query processing

If query processing required too much time and a timeout resulted, see 6.2 Data to be collected in the event of an error, collect the required data, and determine the cause of the error. If necessary, take appropriate action, such as forcing termination of the SDP server.

For details about forced termination of the SDP server, see sdpstop (stops the SDP server) in 7. Commands.

(6) Adaptor processing has slowed

If adaptor processing has slowed, check to see if full garbage collections have been occurring frequently. All adaptor processing stops while a full garbage collection is underway, resulting in a slowdown of processing speed. You can obtain information about full garbage collection from the following file, as applicable:

In the in-process connection mode:

working-directory\logs\SDPServerVMinteger.log

In the RMI connection mode:

working-directory\logs\SDPClientVMinteger.log

Note that integer in the file name is a number in the range from 01 to 04 (because a maximum of four files can be used). The output destination and names of these files are set to the default values.

The following shows an example of output of a full garbage collection:

[VGC]<Fri Jan 08 17:19:46.159 2010>[Full GC 54330K->45174K(519296K),
0.1150830​ secs][DefNew::Eden: 9156K->0K(164608K)]...

If the underlined Full GC is output frequently or garbage collection takes several seconds, JavaVM assumes that not enough memory is allocated to the heap area. In such a case, consider taking one of the corrective actions described below:

(7) Adaptor processing has stopped due to unmatched data types

If the data type specified in the type attribute of the field tag in the format conversion definition in the adaptor configuration definition file does not match the data type in the corresponding CQL, an error occurs during tuple transmission and the adaptor processing stops. In such a case, check and revise (if necessary) the data type.

For details about the type attribute of the field tag in the format conversion definition, see 9.11.1 Format conversion definition.

(8) Some tuple logs have been lost

If all tuple log buffers become full, some tuple logs will be lost.

If tuple logs have been lost, the result of query re-execution by the sdptplput command might not match the result obtained by executing queries using adaptors. Therefore, you must specify the appropriate number and sizes of buffers so that a sufficient number of tuples can be acquired.

The formula for estimating the tuple log buffers is shown below. Using this formula, determine the number and sizes of tuple log buffers that you need for storing tuples.

Size of one tuple log buffer (bytes) =
size of one tuple x number of tuples to be stored

For the size of one tuple, see 2.7.1(2) Memory required for one tuple.

(9) The analysis results cannot be displayed on the dashboard

If the analysis results cannot be displayed on the dashboard, check the following:

If an error occurs on the dashboard, an error dialog box is displayed in the dashboard window. This dialog box displays the following information:

Displayed nameDescription
"ErrID"Displays an error message ID
"ErrMsg"Displays the error message

For details about the messages, see the manual uCosminexus Stream Data Platform - Application Framework Messages.