File input connector definition

uCosminexus Stream Data Platform - Application Framework Setup and Operation Guide

(1) Format

(2) Details of definition

FileInputConnectorDefinition tag (all definition information): Defines all file input connector definition information. You specify this definition only once.

input tag (input definition)

Defines information about the input file read processing and monitoring processing. You specify this definition only once.

readType="{BATCH|REAL_TIME}"

Specifies how to read input files. If this attribute is omitted, REAL_TIME is assumed.

The permitted values are as follows:

BATCH
Reads input files in the batch processing mode.
REAL_TIME
Reads input files in the real-time processing mode.

compositionType="{WRAP_AROUND|ANTI_WRAP_AROUND}"

Specifies the structure of input files. If this attribute is omitted, ANTI_WRAP_AROUND is assumed.

The permitted values are as follows:

WRAP_AROUND
Reads input files in the wraparound mode.
Note that if you specify BATCH in the readType attribute, WRAP_AROUND cannot be specified. Similarly, if you specify REAL_TIME in the readType attribute and DEFINED in the readOrder attribute, WRAP_AROUND cannot be specified.
ANTI_WRAP_AROUND
Reads input files in the non-wraparound mode.
Note that if you specify MODIFIED in the readOrder attribute, ANTI_WRAP_AROUND cannot be specified.

interval="monitoring-interval": If you specified REAL_TIME in the readType attribute, this attribute specifies as an integer from 0 to 1000000 the interval (in milliseconds) at which monitoring of the input file storage directory is to be performed in order to determine whether or not a new file to be input has been created. If you specify 0, monitoring is performed continuously. If this attribute is omitted, 1000 is assumed.

retryCount="monitoring-retries-count": Specifies as an integer from 0 to 1000 a retries count for monitoring the input file storage directory. If this attribute is omitted, 100 is assumed.; When the number of retries reaches the value of this attribute, the KFSP46203-E warning message is output indicating that there is no file in the input file storage directory waiting to be input and then monitoring is resumed.; A monitoring retry for each file to be input is performed at that point. When a target file is read, the retries count is reset to 0. Therefore, the retries count threshold is checked for each input file and a warning message is output for each input file.

readOrder="{DEFINED|MODIFIED}"

Specifies the order in which input files are to be read. If this attribute is omitted, DEFINED is assumed.

DEFINED
Reads input files in the order their names are specified in the name attribute of the file tag.
MODIFIED
Reads input files in the order of the input file update times.

readUnit="reading-unit": Specifies as an integer from 1 to 10000 the number of records that are be read by the file input connector at one time. If this attribute is omitted, 1 is assumed.

file tag (input file definition)

Defines information about the input files. You specify this definition only once.

path="input-path-name": Specifies the path name of the input file storage directory, as 1 to 160 single-byte characters. This attribute cannot be omitted.; The following characters are permitted for this attribute:; Single-byte alphanumeric characters, !, #, $, %, &, ', (, ), =, ~, `, {, }, +, -, ^, ., _, @, \, /, :, single-byte space

name="input-files"

Specifies the files to be input, either as file names or as sequence numbers. This attribute cannot be omitted.

Specifying file names
Specify each file name as a path relative to the input file storage directory specified in the path attribute. You specify multiple file names by delimiting the names with the comma (,). Note that no subdirectory can be specified. Express each input file name as 1 to 60 single-byte characters.
A specified file name cannot be a reserved device name (such as NUL). If a reserved device name is specified, operation is not guaranteed.
Specifying sequence numbers
Specify the sequence numbers of files that are stored in the input file storage directory specified in the path attribute. For details about the specification method, see (4) Specifying sequence numbers for input files.

Express an input file name as 1 to 1,024 single-byte characters, excluding \, /, and :.

The following characters are permitted for this attribute:

Single-byte alphanumeric characters, !, #, $, %, &, ', (, ), =, ~, `, {, }, +, -, ^, ., _, @, single-byte space

messageLog="{ON|OFF}"

Specifies whether or not the start time and input file name are to be output to the message log when input read processing begins for a file. If this attribute is omitted, OFF is assumed.

The permitted values are as follows:

ON
Outputs the start time and input file name to the message log.
OFF
Does not output the start time and input file name to the message log.

(3) Example

(4) Specifying sequence numbers for input files

This subsection describes the format used to specify sequence numbers in place of input file names in the name attribute in the file tag.

Sequence number specification format
prefix-string sequence-number-1 intermediate-string sequence-number-2 suffix string
The following explains each value.

prefix-string, intermediate-string, suffix string

Express each of these values as a string of single-byte characters.

sequence-number-1, sequence-number-2

Express these values in the format minimum-value-maximum-value or minimum-value- enclosed in [ and ]. Express minimum-value and maximum-value as 1 to 8 single-byte numeric characters.
Rules for a sequence number specification
- Only one file can be specified in a sequence number specification.
- Neither [ nor ] can be specified as the prefix string, intermediate string, or suffix string.
- The minimum value cannot be omitted from a sequence number specification. If the maximum value is omitted, 99999999 is assumed as the maximum value.
- A specified sequence number must satisfy the conditions minimum-value maximum-value and the number of digits in maximum-value number of digits in minimum-value.
- The files specified by the sequence numbers are input in the following order:
  1. File with the smallest sequence-number-1
  2. File with the smallest sequence-number-2
  For example, the file a1b2.txt is input before the file a1b2.txt,a2b1.txt, and the file a2b1.txt is input before the file a2b2.txt,a2b1.txt.
- There must be at least one non-numeric character in the intermediate string.
- The file input connector reads file names sequentially from the minimum value to the maximum value of the sequence numbers even when the input files' sequence numbers are not consecutive. For example, when the sequence numbers are 1, 2, 4, where 3 is missing, the file input connector reads 1, 2, and 4 in this order.

Examples of sequence number specification
The table below shows examples of sequence number specification.

Example	Processing result	Order of files to be input during normal operation
a[1-100].txt	Y	a1.txt, a2.txt, ... (omitted) ..., a9.txt, a10.txt, a11.txt, a12.txt, ... (omitted) ..., a99.txt, a100.txt
a[01-100].txt	Y	a01.txt, a02.txt, ... (omitted) ..., a09.txt, a10.txt, a11.txt, a12.txt, ... (omitted) ..., a99.txt, a100.txt
a[001-100].txt	Y	a001.txt, a002.txt, ... (omitted) ..., a009.txt, a010.txt, a011.txt, a012.txt, ... (omitted) ..., a099.txt, a100.txt
a[01-10]_[1-100].txt	Y	a01_1.txt, a01_2.txt, ... (omitted) ..., a01_9.txt, a01_10.txt, a01_a11.txt, ... (omitted) ..., a01_99.txt, a01_100.txt, a02_1.txt, a02_2.txt, ... (omitted) ..., a02_9.txt, a01_10.txt, a02_a11.txt, ... (omitted) ..., a02_99.txt, a02_100.txt, ... (omitted) ..., a10_1.txt, a10_2.txt, ... (omitted) ..., a10_9.txt, a10_10.txt, a10_a11.txt, ... (omitted) ..., a10_99.txt, a10_100.txt
a[5-3].txt	N^#1	--
a[001-9].txt	N^#2	--
a[1-005].txt	Y	a1.txt, a2.txt, a3.txt, a4.txt, a5.txt
a.txt, b[1-3].txt	N^#3	--
a[1-3].txt,b[4-6].txt	N^#3	--

Legend:: Y: Processed normally; N: Results in an error; --: Not applicable

#1: This results in an error because the minimum value is greater than the maximum value.

#2: This results in an error because the number of digits in the minimum value is greater than the number of digits in the maximum value.

#3: This results in an error because only one file can be specified in a sequence number specification.

9.10.1 File input connector definition

(1) Format

(2) Details of definition

(3) Example

(4) Specifying sequence numbers for input files