This subsection describes the components of a stream data processing system. It also discusses the mode of connection between adaptors and the SDP server and the number of components that can be placed in the stream data processing system.
The following figure shows the components of a stream data processing system.
Figure 2-3 Components of a stream data processing system
The following subsections describe the components shown in the figure:
You use definition files to specify the operation of the adaptors and the SDP server. Files, such as log and trace files, are output by the adaptors and the SDP server. The following figure shows the relationships between the components of a stream data processing system and individual files.
Figure 2-4 Relationships between the components of a stream data processing system and individual files
The following describes the files shown in the figure:
The adaptors and the SDP server operate either all in the same process or in separate processes. The mode of connection that is used when you operate the adaptors and the SDP server in the same process is called an in-process connection, and the mode of connection that is used when you operate them in separate processes is called an RMI connection.
The table below describes the advantages and disadvantages of these two connection modes.
Table 2-1 Advantages and disadvantages of the adaptor and SDP server connection modes
No. | Connection mode | Advantage | Disadvantage |
---|---|---|---|
1 | In-process connection | Cost of communication between SDP server and adaptors is low and the tuple processing latency can be minimized. | Memory usage must be monitored because all processing is performed in a single process. |
2 | RMI connection | The effects of an adaptor failure on the stream data processing engine can be minimized. Also, more memory can be allocated to adaptors. | The communication cost is higher than with in-process connection because process-to-process communication must be performed. |
You must determine how to configure the SDP server and adaptors, taking into account the advantages and disadvantages of the connection modes.
The system configuration in each connection mode depends on whether standard adaptors or custom adaptors are used. For details about these types of system configurations, see 2.2.3 Configuration when standard adaptors are used and 2.2.4 Configuration when custom adaptors are used.
There are limitations to the number of adaptors that can be started by an SDP server and the number of adaptors that can be managed in an adaptor group. The number of components that can be placed in a stream data processing system depends on whether standard adaptors or custom adaptors are used. The table below shows the number of components that can be placed in a stream data processing system.
Table 2-2 Number of components that can be placed in a stream data processing system
No. | Item | Maximum value when standard adaptors are used | Maximum value when custom adaptors are used |
---|---|---|---|
1 | Number of SDP servers that can be run in one working directory#1 | 1 | 1 |
2 | Number of RMI connection adaptors that can be run by one SDP server | -- | 32#2 |
3 | Number of in-process-connection adaptors that can be registered in one SDP server | -- | 16#2 |
4 | Number of adaptor groups that can be executed by one SDP server | 1 | -- |
5 | Number of adaptors that can be managed in one adaptor group | 64#2 | -- |
6 | Number of query groups that can be executed by one SDP server | 8 | 8 |
7 | Number of streams that can be registered in one SDP server | 1,024#3 in total | 1,024#3 in total |
8 | Number of queries that can be registered in one SDP server |