Hitachi

In-Memory Data Grid Hitachi Elastic Application Data Store


9.1.3 Parameters related to memory and buffers

The following table lists the parameters related to memory and buffers.

Table 9‒1: Parameters related to memory and buffers

No.

Property file

Parameter name

Value to be specified

Default value

1

Server property file

eads.server.connection.buffersize

Buffer size (1024 to 16777216 (bytes))

4096 (bytes)

2

eads.replication.connection.buffersize

Buffer size (1024 to 16777216 (bytes))

131071 (bytes)

3

eads.replication.sendQueue.length

Queue size

(10000 to 1000000)

100000

4

eads.replication.sendQueue.datasize

1048576 to 2147483647 (bytes)

16777216 (bytes)

5

eads.transfer.connection.buffersize

Buffer size (1024 to 16777216 (bytes))

131071(bytes)

6

eads.transfer.datasize

10240 to 2147483647 (bytes)

1048576 (bytes)

7

eads.replication.fillgap.copy.datasize

1024 to 16777216 (bytes)

10240 (bytes)

8

eads.admin.operation.resume.send.datasize

0 to 2147483647 (bytes)

1048576 (bytes)

9

Shared property file

eads.cache.key.maxsize

1 to 1024 (bytes)

1024 (bytes)

10

eads.replication.preparations

1 to 100

20

11

eads.replication.external.heapsize

1 to 268435456 (megabytes)

450 (megabytes)

12

eads.java.heapsize

Heap size#1

3072 (megabytes)

13

eads.java.external.heapsize

Heap size (2 to 2147483647 (megabytes))

1024 (megabytes)

14

eads.java.permanent.maxsize

Memory size#2

83 (megabytes)

15

eads.cache.limiter.enable

  • true

  • false

true

16

eads.cache.keyCount

1024 to 1073741824

1048576

17

Client property file

eads.client.connection.buffersize

Buffer size (1024 to 16777216 (bytes))

4096 (bytes)

#1

The specified values are applied to JavaVM's heap size options (-Xmx and -Xms).

#2

The specified values are applied to JavaVM's memory size options (-XX:PermSize and -XX:MaxPermSize).

Organization of this subsection

(1) Server property file

(a) eads.server.connection.buffersize

This parameter specifies the size (in bytes) of the work area buffer that is allocated each time connection is established and used to send and receive data.

(b) eads.replication.connection.buffersize

This parameter specifies the size (in bytes) of the transmit and receive buffer for consensus messages.

We recommend that you specify the TCP window size that is specified in the OS.

Depending on the OS, a buffer of a size that is different from the value of this parameter might be used.

(c) eads.replication.sendQueue.length

This parameter specifies the size of the consensus message send queue.

As many send queues are allocated as there are EADS servers to which consensus messages are to be sent.

Use the following formula to estimate the queue length. If the estimate result is smaller than the minimum value of this parameter, set the minimum value.

2 [Figure] a [Figure] b + 2 [Figure] a [Figure] numbers of caches + (c - 1) [Figure] a [Figure] numbers of caches

Important note

If consensus messages exceeding the specified queue length enter the send queue, a communication error occurs.

(d) eads.replication.sendQueue.datasize

This parameter specifies the maximum amount (in bytes) of data that can be stored in a queue for sending consensus messages.

Use the following formula to obtain the value you can specify:

2 [Figure] a [Figure] b [Figure] (c + numbers of caches) + d [Figure] (e - 1) [Figure] numbers of caches

  • a: eads.replication.preparations parameter value in the shared properties

  • b: Maximum size of the history of update operations

    The following shows the formula for estimating the maximum size (in bytes) of the history of update operations.

    eads.cache.key.maxsize parameter value in the shared properties + maximum value size# [Figure] MAX(2, maximum number of data items that can be updated simultaneously)

    #

    Maximum size that can be specified when put, create, update, or replace processing is performed.

    MAX:

    Choose the largest value in the parentheses that follow MAX.

    Example: For MAX(2, 10), the calculation result is 10.

    Maximum number of data items that can be updated simultaneously:

    If you will be performing batch data operations using a memory cache, specify 10. Otherwise, specify 1.

  • c: eads.replication.factor parameter value in the shared properties minus 1 (the value is 1 if the number of redundant copies of data plus the original is 1)

  • d: eads.replication.fillgap.copy.datasize parameter value in the server properties

  • e: eads.replication.factor parameter value in the shared properties

Important note

If you specify an excessively small value compared to the estimate result, retries might occur frequently due to send queue overflows, and further processing might be disabled.

(e) eads.transfer.connection.buffersize

This parameter specifies the size (in bytes) of the data transmit and receive buffer that is used during restoration processing and scale-out processing.

We recommend that you specify the TCP window size that is specified in the OS.

Depending on the OS, a buffer of a size that is different from the value of this parameter might be used.

(f) eads.transfer.datasize

This parameter specifies the size (in bytes) of data that will be transmitted during restoration processing and scale-out processing.

During restoration processing and scale-out processing, this parameter value is applied to the EADS server subject to restoration processing and to the EADS servers added during scale-out processing.

During restoration processing and scale-out processing, the active EADS servers send data to the EADS server being restored and the EADS servers added by scale-out processing in order to recover data consistency. Data is sent in units of 10 kilobytes at the interval specified in the eads.transfer.interval parameter until this parameter value is reached.

For details about restoration processing, see 9.3.2(5) Cluster recovery processing. For details about scale-out processing, see 9.3.2(6) Cluster scale-out processing (adding EADS servers).

When disk caches and two-way caches are restored, this parameter is ignored and the value of the eads.cache.disk.transfer.datasize cache property parameter is restored.

Important note

Because at least one data item is always sent during restoration processing and scale-out processing, the amount of data to be sent will not be limited if a value that is smaller than the size of the data stored in the EADS server is specified in this parameter.

(g) eads.replication.fillgap.copy.datasize

This parameter specifies the amount (in bytes) of data that is sent each time data is copied to the EADS server during complementary processing of history of update operations.

For details about the complementary processing of history of update operations, see 9.3.2(7) Complementary processing of the history of update operations.

Use the following formula to obtain the value you can specify:

maximum size of the history of update operations [Figure] maximum number of consensus processes that can be executed simultaneously

Maximum size of the history of update operations:

The following shows the formula for estimating the maximum size (in bytes) of the history of update operations.

eads.cache.key.maxsize parameter value in the shared properties + maximum value size# [Figure] MAX(2, maximum number of data items that can be updated simultaneously)

#

Maximum size that can be specified when put, create, update, or replace processing is performed.

MAX:

Choose the largest value in the parentheses that follow MAX.

Example: For MAX(2, 10), the calculation result is 10.

Maximum number of data items that can be updated simultaneously:

If you will be performing batch data operations using a memory cache, specify 10. Otherwise, specify 1.

Maximum number of consensus processes that can be executed simultaneously:

eads.replication.preparations parameter value in the shared properties

Important note
  • Do not specify an excessively large value, as that would adversely affect memory usage and CPU usage rate.

  • A larger amount of data than the amount specified by this parameter might be sent.

  • Even if you specify a small data amount, complementary processing is performed for at least one history item.

(h) eads.admin.operation.resume.send.datasize

This parameter specifies the size (in bytes) of differential data that is transferred at one time when the eztool resume command is executed.

If 0 is specified, one data item is transmitted at a time.

(2) Shared property file

(a) eads.cache.key.maxsize

This parameter specifies the maximum size (in bytes) of any key that can be stored in the cluster.

The key storage area is allocated based on the size specified in this parameter.

By restricting the maximum key size, you can design a smaller Java heap size for storing keys compared to when the default key size is used.

Important note
  • A cluster whose maximum key size is set to a value smaller than that of another cluster might not be able to import store data files that are output from that other cluster.

  • If disk caches or two-way caches are used, cache data files and cache index files that have been output in a cluster in which the maximum key size is set to a large value might not be usable for data relocation in a cluster in which the maximum key size is set to a smaller value.

(b) eads.replication.preparations

This parameter specifies the maximum number of consensus processes that can be performed simultaneously.

Specify the smaller of the values shown below. Note, however, that if the values are different between EADS servers, specify the largest value in the cluster.

(c) eads.replication.external.heapsize

This parameter specifies the size (in megabytes) of the area for storing the history of update operations.

Be aware that if this parameter's value is greater than the eads.java.external.heapsize parameter value in the shared properties [Figure] 0.97 (in units of megabytes; digits to the right of the decimal point are discarded), EADS server startup processing will fail.

(d) eads.java.heapsize

This parameter specifies the size (in megabytes) of the Java heap in which keys are stored.

(e) eads.java.external.heapsize

This parameter specifies the size (in megabytes) of the explicit heap in which values and the history of update operations are stored.

Note that three percent of the specified explicit heap size is used as a management area (the value is rounded up in megabytes).

Be aware that if the eads.replication.external.heapsize parameter value in the shared properties is greater than this parameter value [Figure] 0.97 (in units of megabytes; digits to the right of the decimal point are discarded), EADS server startup processing will fail.

(f) eads.java.permanent.maxsize

This parameter specifies the size (in megabytes) of the Permanent area.

The Permanent area is used to store such information as the loaded EADS servers and the classes of user functions.

(g) eads.cache.limiter.enable

This parameter specifies whether the total data restriction function is to be enabled.

When the total data restriction function is enabled and a shortage of space at the data storage location is foreseen, the EADS server can be protected from being shut down by setting an error in the corresponding processing.

true

Enables the total data restriction function.

false

Disables the total data restriction function.

(h) eads.cache.keyCount

This parameter specifies the number of data items per range that will be monitored by the total data restriction function.

If false is specified in the eads.cache.limiter.enable parameter (total data restriction function is disabled), this parameter's value is ignored.

(3) Client property file

(a) eads.client.connection.buffersize

This parameter specifies the size (in bytes) of the buffer that will be used by the EADS client to send and receive data.