2.8.3 Database retrieval method (out-of-order execution)
HADB utilizes the out-of-order execution method as the database retrieval method. This method is a way of processing that was devised in order to search through a large-scale database at high speeds.
- Organization of this subsection
(1) Concept of out-of-order execution
With relational databases, retrieval processing consists of a set operation executed against rows. There are no rules governing the sequence in which retrieval results are returned by a DBMS, except for the final sorting process. The out-of-order execution method was devised to take advantage of this. The following figure shows the concept of out-of-order execution.
(2) Retrieval using the out-of-order execution method
By effectively utilizing hardware resources, the out-of-order execution method can search through a large-scale database at high speeds. The following figure compares retrieval using the out-of-order execution method and other retrieval methods.
When the out-of-order execution method is not used, only one retrieval process is assigned to a thread. Furthermore, a new I/O instruction is issued only after the previously issued I/O instruction is completed. Therefore, you can only issue the number of I/O instructions that does not exceed the upper limit of the I/O performance of the storage device. As a result, as the volume of data in the database becomes larger, the amount of time the system must wait for I/O completion becomes longer and the retrieval takes longer.
With the out-of-order execution method, retrieval processes are divided by rows and assigned to individual threads. Each assigned retrieval process is executed independently in each thread. Additionally, the individual threads can be executed out-of-order based on the progress of I/O processing. That is, after the threads issue I/O instructions, they are processed in parallel and processing can be switched to another thread without having to wait for the completion of other I/O instructions (asynchronous I/O processing). The result is high-speed data retrieval.
(3) Threads with which the out-of-order execution method is used
HADB executes I/O instructions and other processing in parallel by assigning processes not having dependent relationships to multiple threads of the thread types listed below. The out-of-order execution method uses the following types of threads:
-
Real threads
Real threads are managed by the OS (kernel and library). When the CPU has two or more cores, threads allow multiple processes to be executed in parallel.
-
Pseudo-threads
Pseudo-threads are managed by HADB. Although multiple pseudo-threads can be generated in each real thread, two or more pseudo-threads cannot be active simultaneously. By using multiple pseudo-threads, HADB executes processes and I/O instructions in parallel.
(4) Cases in which the out-of-order execution method is not applied
In the following cases, the out-of-order execution method is not applied; standard SQL execution order is used instead:
-
If an SQL statement other than retrieval (SELECT statement) is being executed
-
If 0 is specified as the number of real threads to be used when executing an SQL statement (adb_sql_exe_max_rthd_num operand)
-
If pseudo-threads cannot be generated (when 0 is specified as the number of pseudo-threads that can be generated in each real thread (adb_sys_uthd_num operand))
(5) Relationship between the out-of-order execution method and wait status
This subsection explains the relationship between the out-of-order execution method and the wait status that occurs if the number of processing real threads required for executing an SQL statement cannot be allocated.
(a) When executing SQL statements
If the required number of processing real threads cannot be allocated when an SQL statement is to be executed, the SQL statement is placed in wait status. The SQL statement that is in wait status is executed after the required number of processing real threads are allocated.
If an SQL statement or command is placed in wait status, all the SQL statements that follow that SQL statement or command will be placed in wait status. (This also applies even if the number of processing real threads required for any of the subsequent SQL statements can be allocated.) If this occurs, the subsequent SQL statements will be released from wait status only after the prior SQL statement or command has been released from wait status.
- ■ When using the client-group facility
-
If an SQL statement that is executed by an HADB client belonging to a group is placed in wait status, all SQL statements that are executed by other HADB clients belonging to the same group are also placed in wait status. If this occurs, the subsequent SQL statements will be released from wait status only after the prior SQL statement has been released from wait status.
Note that the wait status for an SQL statement that is executed by an HADB client belonging to one group has no effect on SQL statements that are executed by HADB clients belonging to other groups.
For details about the processing real threads used when executing SQL statements, see the description of the adb_sql_exe_max_rthd_num operand in 7.2.2 Operands related to performance (set format).
- Note
-
For details about how to check whether a target SQL statement is in wait status, see 10.8.2 Checking whether the process of allocating processing real threads has gone into wait status.
(b) When executing commands
Processing real threads are used to execute some commands, such as the adbimport command. If the required number of processing real threads cannot be allocated, the corresponding command is placed in wait status. For details about the commands to which this applies, see Targeted commands in the description of the adb_sys_rthd_num operand in 7.2.2 Operands related to performance (set format).
If a command or SQL statement is placed in wait status, all the commands that follow that command or SQL statement are also placed in wait status. (This also applies even if the number of processing real threads required for any subsequent command can be allocated). If this occurs, the subsequent commands will be released from wait status only after the prior command or SQL statement has been released from wait status.
- ■ When using the client-group facility
-
If a command belonging to a group is placed in wait status, all other commands belonging to the same group are also placed in wait status. If this occurs, the subsequent commands will be released from wait status only after the prior command has been released from wait status.
For details about the processing real threads used during command execution, see the description of each command in the manual HADB Command Reference.
- Note
-
For details about how to check whether a target command is in wait status, see 10.8.2 Checking whether the process of allocating processing real threads has gone into wait status.