awk command (performs text processing and pattern matching)

Format

awk [-F input-field-separator] [-v variable-name=variable-value]... [-f script-file-path-name|script]
   [[target-path-name...]|[built-in-variable-name=variable-value...]]...

To Page Top

Description

This command retrieves lines (referred to hereafter as records) in a text file that match a particular pattern and performs specified processing on the retrieved lines.

To Page Top

Arguments

-F input-field-separator

Specifies the value to be used as the input field separator. The specified value becomes the value of the awk command's FS built-in variable.

-v variable-name=variable-value

Specifies a variable name and its value. The variable name and its value are passed to the script that is specified in a script file or in the argument specified in the -f option. Multiple variables can be specified. If you specify the same variable name more than once, the last specification takes effect.

-f script-file-path-name

Specifies the path name of a file (script file) that contains the patterns to be matched in the input files and the processing instructions for the records that match the patterns.

• If - is specified as the path name, the standard input is assumed for the input.

• Up to 19 -f options can be specified.

script

Specifies a pattern to be matched in the input files and the processing instructions for the records that match the pattern.

target-path-name

Specifies the path name of a file to be processed. Multiple path names can be specified.

If no path name is specified or - is specified as the path name, input is read from the standard input. Note that if only the BEGIN pattern is executed, no records are retrieved from the specified file or from the standard input.

built-in-variable-name=variable-value

Specifies the name of a built-in variable and its value. The variable name and its value are passed to the scripts that are specified in the script file or in the script argument specified in the -f option.

• If you specify a name that is not for a listed built-in variable, it is treated the same as the -v option.

• A built-in variable that is specified before all the target path names is enabled for all file processing, except for BEGIN pattern processing, and is also enabled for END pattern processing.

• A built-in variable that is specified after all the target path names is enabled for END pattern processing only.

• A built-in variable that is specified between target path names is enabled for processing of the path names specified after the variable specification and for END pattern processing.

To Page Top

Scripts (patterns and actions)

The following is the descriptive format of a script executed by the awk command:

[pattern] [{[action]}]

A pattern to be searched for in the input files is defined in pattern. For details about the pattern specification, see Types of patterns below. Processing instructions for records that match the pattern are defined in action.

Each successive record from the input file is compared to each specified pattern, and the action specified for a pattern is executed when a match is found for that pattern. A specified action can be performed on all records by not specifying a pattern (omitting pattern and specifying action only).

You specify for the action control statements and functions that perform desired processing on the records that match the specified pattern. The action operation can include control statements, built-in functions, user-defined functions, variables, and operators. Multiple statements, separated by an end-of-line code or semicolon, are permitted. When the entire {action} portion is omitted, including the braces, the matching records are output to the standard output. If you specify the empty braces without action ({ }), no processing is performed.

To include a comment, specify a hash mark (#) and then the comment string. Everything from the hash mark to the end of the line is treated as a comment.

To Page Top

Records and fields

A record is a unit that is obtained by using the input record separator to split up the input. In awk, an end-of-line code serves as the input record separator. In Windows, an end-of-line code is denoted by [CR] + [LF] or by [LF]. In UNIX, an end-of-line code is denoted by [LF]. Note that in UNIX, if [CR] + [LF] is used for the end-of-line code, the [CR] part will be included in the resulting record.

The input record separator can be changed by setting in the RS built-in variable any single-byte character to serve as the new record separator. If a character string is specified, only the first character in the character string becomes the input record separator.

Records are divided by field separators into units called fields. The default field separator is the space. The field separator can be changed by specifying in the -F option or by setting in the FS built-in variable any character string to serve as the new field separator.

The input passed to the specified action consists of the contents of the record currently being read from the input file and the value of each field in the record. The entire contents of the record are stored in field variable $0. The first field of the record is stored in field variable $1, the second field is stored in field variable $2, and so on.

To Page Top

Types of patterns

The following types of patterns can be specified.

• Character strings

  To search for a character string in a field or record, enclose it in slashes (/). Character strings can be specified using regular expressions. To search for a forward slash (/) by itself, you must use the escape character (\), so the specification becomes /\//.

  In the following example, hitachi is specified as the search string:

  /hitachi/{
  (action)
  }

• Relational expressions

  Relational operators (>, >=, <, <=, ==, and !=) can be used to perform comparisons on fields. In the following example, the specified action is performed if the second field of a record is hitachi:

  $2 == "hitachi"{
  (action)
  }

• Combinations of patterns

  Multiple patterns can be combined to express conditions for executing an action. The following table shows the combinations that can be used:

  Format	Description
  pattern1 && pattern2	Using the logical AND operator, execute the action on records that are a match for both pattern1 and pattern2.
  pattern1 || pattern2	Using the logical OR operator, execute the action on records that are a match for either pattern1 or pattern2.
  pattern1 ? pattern2 : pattern3	Using the ternary operator, execute the action on records that are a match for both pattern1 and pattern2 or are a match for pattern3.
  ! pattern	Using the NOT operator, execute the action on records that are not a match for pattern.
  (pattern)	Group multiple conditions into pattern.
  pattern1, pattern2	Execute the action on the range of input records beginning with the record that is a match for pattern1 and concluding with the record that is a match for pattern2. Note that if the search reaches the end of the input file without finding a record that matches pattern2, the last record in the input file will be the end of the range. However, if multiple input files are specified, the successive input files are searched for a record that matches pattern2. This format can be specified up to 50 times.

• BEGIN

  This pattern is for an action that is to be performed before processing of the input file begins. For this pattern, action cannot be omitted. This pattern cannot be combined with other patterns. If multiple input files are specified, the specified action is executed before input starts from the first input file.

• END

  This pattern is for an action that is to be performed after the last record in the file has been processed or when the exit control statement is entered. For this pattern, action cannot be omitted. This pattern cannot be combined with other patterns. If multiple input files are specified, the specified action is not executed until the last record in the last file has been processed.

To Page Top

Control statements

The control statements that can be used are described in the table below. The if, while, for, do, break, continue, and return statements are subject to the C language syntax rules. An exception is the for statement, which is limited to a single initialization expression and a single increment expression.

To Page Top

Built-in functions

The following built-in functions can be used.

• Mathematical functions

  The mathematical functions that can be used are described in the following table:

  Function name	Description
  atan2(y,x)	Returns the arctangent of y/x. The unit is radians. If an argument is missing, 1 is returned and a warning message is output.
  cos(x)	Returns the cosine of x. The unit is radians.
  exp(x)	Returns the exponential function of x. If the result produces an overflow or underflow, 1 is returned and a warning message is output.
  int(x)	Returns an integer by truncating at the decimal point of x.
  log(x)	Returns the natural logarithm of x. If x is zero or negative, 1 is returned and a warning message is output.
  rand()	Returns random number n (in the range 0 n < 1). If you do not use the srand function to set a seed value, the same series of values will be generated each time awk is run.
  sin(x)	Returns the sine of x. The unit is radians.
  sqrt(x)	Returns the square root of x. If x is negative, 1 is returned and a warning message is output.
  srand([expr])	Sets the expr expression as the seed value for the rand function and then returns the seed value that has been set. If expr is omitted, the seed value is set based on the time.

• Character string functions

  The character string functions that can be used are described in the table below. Multibyte characters are treated as single characters.

  Function name	Description
  gsub(r,t[,s])	Replaces all matches for regular expression r in character string s with t. If s is omitted, the assumed replacement target is $0 (the field variable that stores the entire record). If & is specified in t, & is replaced by the matched character string. The number of replacements is returned as the return value.
  index(s,t)	Returns the position of character string t in character string s. If character string t is not found, 0 is returned.
  length[([s])]	Returns the number of characters in character string s. If s is not specified, the number of characters in $0 (the field variable that contains the entire record) is returned.
  match(s,r)	Returns the position at which regular expression r occurs in character string s. If regular expression r is not found, 0 is returned. In addition, the RSTART built-in variable is set to the position at which regular expression r matches the character string, or is set to 0 if there is no match. The RLENGTH built-in variable is set to the length of the character string that matches regular expression r, or to -1 if there is no match.
  sprintf(format, expression[, ... ])	Returns the character string in which expression is formatted according to format. For details about format, see the description under Output format.
  split(s, array[,fs])	Splits character string s into fields using field separator fs and stores the fields in array array. The number of elements in the array is returned as the return code. The values of the fields that were split are stored in array array in the order array[1], array[2], ..., array[return-value]. If field separator fs is not specified, the value of the FS built-in variable is used as the field separator. The fs field separator can be specified as a character string or regular expression. If you specify "" for the fs field separator to indicate no characters, s is split into single characters.
  sub(r, t[,s])	Replaces the first match for regular expression r in character string s with t. If s is omitted, the assumed replacement target is $0 (the field variable that contains the entire record). If & is specified in t, & is replaced by the matched character string. The return code is 1 if regular expression r is found and 0 if regular expression r is not found.
  substr(s,m[,n])	Returns a substring of up to n characters, beginning with the mth character of character string s. If n is omitted, the entire character string beginning with position m is returned.
  tolower(str)	Returns a character string in which all uppercase characters in character string str have been changed to lowercase.
  toupper(str)	Returns a character string in which all lowercase characters in character string str have been changed to uppercase.

• Bit manipulation functions

  The bit manipulation functions that can be used are described in the table below. Both x and y are treated as signed four-byte numeric values.

  Function name	Description
  compl(x)	Returns the one's complement of integer x.
  and(x,y)	Returns the bitwise logical AND of integer x and integer y.
  or(x,y)	Returns the bitwise logical OR of integer x and integer y.
  xor(x,y)	Returns the bitwise logical XOR of integer x and integer y.
  lshift(x,n)	Returns the value of integer x shifted n bits to the left. Because an arithmetic bit shift is performed, the sign bit is also shifted.
  rshift(x,n)	Returns the value of integer x shifted n bits to the right. Because an arithmetic bit shift is performed, the sign bit is reintroduced after the right shift, preserving the sign.

• Input/output functions

  The following input/output functions can be used.

  getline [variable-name]

  Reads the next record from the current input file. If variable-name is specified, the record is read into the variable specified in variable-name and is set as the value of the NR and FNR built-in variables. If variable-name is omitted, the record is read into field variable $0 and set as the value of the NF, NR, and FNR built-in variables. The function returns 1 if the record is successfully read, 0 if the end of the file was reached, or -1 if an error occurred.

  getline [variable-name] < path-name

  Reads the next record from the file specified in path-name. The path name must be enclosed in double quotation marks ("). You can also specify in place of the path name the name of a variable assigned to the path name. If - is specified for path-name, input is read from the standard input.

  The following is an example:

  getline line < "file001.txt"

  If variable-name is specified, the record is read into the variable specified in variable-name. If variable-name is omitted, the record is read into field variable $0 and set as the value of the NF built-in variable.

  The specified file is opened when the first record is read by the getline function and remains open until the awk command terminates. For this reason, you must execute the close function if you want to restart input from first record of the same file.

  command-name | getline [variable-name]

  Reads a record being output to a pipe by the program specified in command-name. The command name must be expressed in command line format, with the name of the program to be executed and the values of its arguments enclosed in double quotation marks ("). You can also specify in place of the command name the name of a variable that has been assigned to the command name.

  If variable-name is specified, the record is read into the variable specified in variable-name. If variable-name is omitted, the record is read into field variable $0 and set as the value of the NF built-in variable. The following are examples.

  Using the output from command-name to the pipe as the input, read one record and assign it to field variable $0:

  "cat -n file01.txt" | getline

  Assign command-name to a variable, and then execute the program by connecting the variable to the getline function:

  rtxt = "cat -n file01.txt"

  rtxt | getline

  Creation of a pipe to receive the output from the specified program and execution of the program are performed when you execute command-name | getline [variable-name]. If you execute the same command-name multiple times, pipe creation and program execution are performed only the first time the specified command-name is executed. The pipe that is created exists only until the awk command terminates. For this reason, if you want to re-run the program specified in command-name, you must execute the close function. This is illustrated in the following example:

  "cat -n file01.txt" | getline rec -->1.

  "cat -n file01.txt" | getline rec -->2.

  close("cat -n file01.txt") -->3.

  "cat -n file01.txt" | getline rec -->4.

  1. The pipe is created and the cat command is executed. The contents of the first record that is output from the cat command to the pipe are set in variable rec.

  2. The contents of the second record that is output from the cat command to the pipe are set in variable rec.

  3. The pipe is closed.

  4. The pipe is created and the cat command is executed. The contents of the first record that is output from the cat command to the pipe are set in variable rec.

  Note that even if this command-name is identical to the command-name specified in a print or printf function, they are considered to be different commands.

  print [expression[, ... ]]

  Prints expression to the standard output. If expression is omitted, the current input record is printed to the standard output. If you specify multiple expressions delimited by the comma (,), the expressions will be separated by the value of the OFS built-in variable. The value of the ORS built-in variable is output at the end of the output record.

  print [expression[, ... ]] > path-name

  Outputs expression to the file specified in path-name. path-name must be enclosed in double quotation marks ("). You can also specify in place of the path name the name of a variable assigned to the path name. If expression is omitted, the current input record is output to the file specified in path-name. If you specify multiple expressions delimited by the comma (,), the expressions will be separated by the value of the OFS built-in variable. The value of the ORS built-in variable is output at the end of the output record. To append the output to an existing file, specify >>path-name.

  The specified file will be opened when you first produce output using the print function and will remain open until the awk command terminates. For this reason, if you want to output from the beginning of the same file, you must execute the close function.

  print [expression[, ... ]] | command-name

  Outputs expression to a pipe and passes it to the program specified in command-name.

  The command name must be expressed in command line format, with the name of the program to be executed and the values of its arguments enclosed in double quotation marks ("). You can also specify in place of the command name the name of a variable assigned to the command name.

  If expression is omitted, the current input record is output to the pipe. If you specify multiple expressions delimited by the comma (,), the expressions will be separated by the value of the OFS built-in variable. The value of the ORS built-in variable is output at the end of the output record.

  The creation of a pipe to receive the output from the specified program and execution of the program are performed when you execute print [expression[, ... ]] | command-name. If you execute the same command multiple times, pipe creation and program execution are performed only the first time the specified command-name is executed. The pipe that is created exists only until the awk command terminates. For this reason, if you want to re-run the program specified in command-name, you must execute the close function. Note that even if this command-name is identical to the command-name specified in a getline function, they are considered to be different commands.

  printf format[, expression[, ... ]]

  Prints to the standard output according to the format in format. For details about format, see the description under Output format. For details about the other parts of the command, see print [expression[, ... ]].

  In Windows, when you use \n to represent a new line, it is output as [CR] + [LF].

  printf format[, expression[, ... ]] >path-name

  Prints to a file according to the format in format. For details about format, see the description under Output format. For details about the other parts of the command, see print [expression[, ... ]] >path-name.

  printf format[, expression[, ... ]] | command-name

  Outputs to a pipe according to the format in format. For details about format, see the description under Output format. For details about the other parts of the command, see print [expression[, ... ]] | command-name. If this command-name is identical to the command-name specified in a print function, their output is considered to be from the same program.

  close(path-name|command-name)

  Closes a file that was used by the getline, print, or printf function, or a pipe that was created when the getline, print, or printf function was executed.

  When the close function is successful, 0 is returned. When the close function fails, the return value of the OS's close function is returned in the case of a file, or the return value of the pclose function is returned in the case of a pipe.

  The argument specifies a path name or command that was specified in a getline, print, or printf function. path-name must be enclosed in double quotation marks ("). command-name must be expressed in command line format, with the name of the program to be executed and the values of its arguments enclosed in double quotation marks ("). Alternatively, you can specify a variable that contains the path name or command that was specified in the getline, print, or printf function. Note that the path-name or command-name argument to the close function must be identical, including the number of characters, to the path name or command used in the getline, print, or printf function. If command-name or path-name specifies a different character string, it will be considered a different path name or command. Note that in addition to closing command-name, the close function also closes any pipes to command-name from the getline, print, or printf functions. The following is an example:

  print "hitachi" | "cat -n"

  close("cat -n")

  fflush([path-name|command-name])

  Flushes a file used by the getline, print, or printf function, or a pipe created when a getline, print, or printf function is executed. When the flush is successful, 0 is returned. When the flush fails, the return value of the OS's fflush function is returned.

  The argument specifies a path name or command that was specified in a getline, print, or printf function. path-name must be enclosed in double quotation marks ("). command-name must be expressed in command line format, with the name of the program to be executed and the values of its arguments enclosed in double quotation marks ("). Alternatively, you can specify a variable that contains the path name or command that was specified in the getline, print, or printf function. Note that the path-name or command-name argument to the fflush function must be identical, including the number of characters, to the path name or command used in the getline, print, or printf function. If command-name or path-name specifies a different character string, it will be considered a different path name or command.

  If no argument is specified, all files and pipes are flushed.

• General functions

  The general function that can be used is described in the following table:

  Function name

  Description

  system(command-name)

  Executes the program specified in command-name and returns the status of the executed program. command-name must be expressed in command line format, with the name of the program to be executed and the values of its arguments enclosed in double quotation marks ("). You can also specify in place of the command name the name of a variable assigned to the command name. In UNIX, the status of the executed program will be the value of the low-order 8 bits of the return code that is returned by the program.

To Page Top

User-defined functions

In addition to the built-in functions, you can also define your own functions. The syntax of a user-defined function is as follows:

function | func name([param[, ...]]) { statements }

The function name name must be specified in alphanumeric characters and the underscore (_), and the first character must be non-numeric.

You can specify in param arguments to the function using the names of user-defined variables or arrays. An argument is passed to the function by its value in the case of a user-defined variable or as a reference in the case of an array.

A parsing error does not occur if the number of arguments specified in the function definition differs from the number of arguments specified when the function is called. However, if the number of arguments specified when the function is called is greater than the number of arguments specified in the function definition, a warning message is output. The arguments specified in the function definition are considered local variables, but if the number of arguments specified when the function is called is greater than the number of arguments specified in the function definition, the extra arguments at the time of the function call are considered global variables.

A maximum of 50 arguments can be specified in a function definition. Similarly, a maximum of 50 arguments can be specified when the function is called. A check is performed at the time the function is called to confirm that the number of arguments does not exceed 50.

To Page Top

Variables

The types of variables used in scripts include user-defined variables, field variables, built-in variables, and arrays. User-defined variables and arrays are generated the first time they are used in a script. Note that the initial value stored in an uninitialized variable (one that has not been used in an arithmetic or assignment expression) is 0 in the case of a numeric value and NULL in the case of a character string.

The type of the value of a variable changes to numeric or character string depending on the situation in which it is used. However, non-numeric character strings have a numeric value of 0. For example, in the following example, the two print functions both produce 7 as the output:

x = "3" + "4"
y = 3 + 4
print x
print y

The following describes each type of variable.

• User-defined variable

  A variable name must consist of only alphanumeric characters and the underscore (_). The first character must be non-numeric.

• Array

  An array name is represented with an index enclosed in square brackets ([]) after the variable name. An index can be a character string enclosed in double quotation marks (") instead of a number. Indexes written as a comma-separated list represent a multidimensional array.

  Multidimensional arrays are implemented in awk by connecting the index values at each dimension with the value of the SUBSEP built-in variable, in order to treat them as a single character string index. In other words, a multidimensional array is handled internally as a one-dimensional array. In the example below, the value of the SUBSEP built-in variable is set to #, and then an array is created as shown. The two print functions both output the same value, which in this case is Hitachi.

  SUBSEP = "#"
  arry["a", "b", "c"] = "Hitachi"
  print arry["a", "b", "c"]
  print arry["a#b#c"]

• Field variables

  The field variables for referring to the contents of input records are described in the following table:

  Variable	Description
  $0	Set to the entire contents of the record currently being read from the input file.
  $1,$2,...	Set sequentially to the field contents of the record that has just been read. The record is split into fields by the value of the FS built-in variable, and then variable $1 is set to the record's first field, variable $2 is set to the record's second field, and so on.
  $variable-name	By assigning field numbers to variables, you can refer to fields in the same way as if you had written the field numbers directly as $0, $1, and so on.# When you do this in a function or control statement, the variable must be set to the field number before it is used.
  $(expression)	By setting expression to an expression that evaluates to a field number (such as variable-name+1), you can refer to fields in the same way as if you had written the field numbers directly as $0, $1, and so on.

  #

  For example, in the following case, print $a and print $1 produce the same output:

    a = 1
    print $a
    print $1

• Built-in variables

  The built-in variables are described in the following table:

  Variable	Description
  ARGC	Stores the number of command line arguments. This number does not include option values and script specification values. The value of ARGC can be changed by a script or by the -v option. If you set ARGC to 0 with the -v option, target path names specified in arguments will be ignored.
  ARGV	Stores an array of command line arguments. The array elements can be modified by a script. For example, if you set to NULL an element storing one of the target path names specified in the arguments, that input file will be ignored and no records will be read from it. This will overwrite values specified with the -v option.
  CONVFMT#	Stores the conversion format to use when converting numeric values. The default is %.6g. If the numeric value has a fractional part, it is also included.
  ENVIRON	Stores an array of run-time environment variables. The indexes are the names of environment variables. The name of an environment variable must be enclosed in double quotation marks ("). You can also specify in place of the name of an environment variable the name of a variable that has been assigned the name of the environment variable.
  FILENAME	Stores the name of the current input file. The file name is set to - when the input is from the standard input.
  FNR	Stores the number of input records that have been read from the current input file. This value is updated each time a record is read from the file specified in the target path name. It is also updated when a record is read by getline [variable-name]. When more than one target path name is specified, it is initialized to 0 when input starts to be read from the next input file.
  FS	Stores the field separator. The default is a one-byte space. FS can be set to a regular expression. If you wish to change the field separator, specify a character string of no more than 99 bytes. If the value of the field separator is a one-byte space, it will split fields at both spaces and tabs (\t). If no value is set for the field separator, each individual character will become a field. If you specify more than one character, the backslash (\) will be regarded as the regular expression escape character. If the -F option is also specified, this variable takes precedence.
  NF	Stores the number of fields in the current input record. NF is set each time an input record is read from the file specified in the target path name. It is also set when an input record is read into the $0 field variable by the getline function. You can reference the value of the last field of the current input record by specifying $NF.
  NR	Stores the number of input records detected so far. The value is updated each time a record is read from a file specified by the target path name. The value is also updated when a record is read by getline [variable-name]. If more than one target path name is specified, the number of records includes the records from all files from which input has been performed.
  OFMT#	Stores the output format for numeric values. The default is %.6g. If the numeric value has a fractional part, it is also included.
  OFS	Stores the output field separator. The default is a one-byte space.
  ORS	Stores the output record separator. The default is the newline character (\n). In Windows, the newline character (\n) is output as [CR] + [LF].
  RLENGTH	Stores the length of a substring matched by the match function. The value is set to -1 if no match was found.
  RS	Stores the input record separator. The default is the end-of-line character. If you change its value, use single-byte characters only. If you specify a character string or a multibyte character, the initial byte only will be used. When RS is set to the newline character (\n), this will match either [CR] + [LF] or [LF] in an input file in Windows, and it will match [LF] in an input file in UNIX. If the value of RS is set to something other than the newline character, the end-of-line character included in the input records will be [LF] in the case of Windows. On a UNIX system, if the input file uses [CR] + [LF] as the end-of-line code, the [CR] part only will be included with the input records.
  RSTART	Stores the starting position of a substring matched by the match function. The value is set to 0 if no match was found.
  SUBSEP	Stores the separator used in multidimensional arrays. The default is 0x1C.

  #

  The conversion specifiers f, e, g, E, and G are supported. Do not use any other conversion specifiers.

To Page Top

Operators

The operators that can be used are listed and described in the table below, in order of lowest to highest priority. For an expression with operators at the same priority level, the operators are listed from left to right in order of highest to lowest priority.

To Page Top

Output format

The following table lists and describes the conversion specifiers that follow % to indicate conversion specifications in the printf and sprintf functions:

To Page Top

Escape characters

You can use escape characters as follows:

• In the input field separator in the -F option

• As a variable value in the -v option

• As a built-in variable value specified in an argument

• Within character strings that are enclosed in double quotation marks (") in an assignment to a pattern or variable

The following table shows the escape characters that can be used:

A backslash (\) specified in variable values in the -v option and in built-in variable values in arguments is treated as an escape character (except when it is enclosed in single quotation marks (')). Specify path names carefully. The following shows examples.

Example 1: The correct path name, c:\a\b\c, cannot be passed to the awk command's scripts.

In this example, \ is deleted because it is processed as an escape character. As a result, c:\a\b\c is set in VAR001.

After that, \ is processed as an escape character again when the value is set in the variable in the awk command. As a result, c:abc is set in VAR001:

CCC01="c:\\a\\b\\c"
awk -v VAR001="${CCC01}"  -f prog01.awk

Similarly, the following two examples also cannot pass the correct path name:

awk -v VAR001=c:\\a\\b\\c  -f prog01.awk
awk -v VAR001='c:\a\b\c'  -f prog01.awk

Example 2: The correct path name, c:\a\b\c, is passed to the awk command's scripts.

In this example, c:\\a\\b\\c is stored in both CCC01 and CCC02.

After that, when the value is stored in VAR001 in the awk command, it becomes c:\a\b\c, thereby processing correctly:

CCC01='c:\\a\\b\\c'
CCC02="c:\\\\a\\\\b\\\\c"
awk -v VAR001="${CCC01}"  -f prog01.awk
awk -v VAR001="${CCC02}"  -f prog01.awk

Similarly, the following two examples can also pass the correct path name:

awk -v VAR001=c:\\\\a\\\\b\\\\c  -f prog01.awk
awk -v VAR001='c:\\a\\b\\c'  -f prog01.awk

To Page Top

Special file names

Special file names can be used to represent the input source and output destination when you use the getline function to read from the standard input or the print or printf function to output to the standard output or standard error output. The table below lists the special file names that are available. Note that attempting to apply the close function to a special file name will have no effect.

To Page Top

Return codes

To Page Top

Notes

• The awk command treats numeric values as double-precision floating point numbers (8 bytes). When you output or convert a numeric value by specifying the conversion specifier d, i, o, x, X, or u in the output format of the printf or sprintf function, it is rounded to a four-byte signed integer. For this reason, an error occurs when you output or convert a numeric value that is outside the range of a four-byte signed integer by specifying the conversion specifier d, i, o, x, X, or u. The particular error depends on the OS.

• The maximum number of files that can be opened at the same time by the getline, print, or printf function is 256. Note that the number of files that can be open at the same time includes pipes generated by the commands that invoke these functions. In UNIX, the maximum number will be less than 256, depending on OS settings, such as the maximum number of files that can be open at the same time on the system as a whole, or restrictions imposed by ulimit on the number of open file descriptors for those processes.

• Input from binary files and output of binary data are not guaranteed to work.

• Execution of external programs, for example by the system function, is implemented in terms of arguments to other programs, as described below. For this reason, the execution specifications for the external program, such as the maximum length of path names, depend on the specifications of those executing programs.

  • Windows

    External programs are executed as arguments to the command processor specified in the COMSPEC environment variable, the default value for which is cmd.exe. The command processor that is used is determined by the COMSPEC and PATH environment variables.

  • UNIX

    External programs are executed as arguments to the shell. Different executing programs might be launched depending on OS specifications.

• In Windows, at the start of input or output from a command that is specified in a getline, print, or printf function, an application error might occur in the execution of the specified command due to insufficient desktop heap space. For this reason, either use the close function close the command when command input/output is no longer required or adjust the value of the desktop heap.

• In Windows, if you specify a file or directory name that contains wildcards in an argument to a command that executes the system function, the wildcards are not expanded.

• In Windows, if you specify file and directory names that contain wildcards as arguments to a command that is connected by a pipe to a getline, print, or printf function, the wildcards are not expanded.

• In Windows, if you specify the path name of a linked file in a place where you specify an external function, such as in the system function, the program that it has been linked to will launch. Take note of this when executing batch jobs.

• In Windows, when you specify a path name in an input/output function or a general function, observe the following constraints concerning the backslash (\) character:

  • The directory separator character \ must be specified as \\.

  • If you specify a path name that includes spaces in the place where you specify an external function, such as in the system function, the entire path must be enclosed in \".

  • The \ character is treated as representing the escape character when it is specified as the value of a built-in variable specified in the input field separator in the -F option, or as a variable value specified in an argument or in the -v option.

• In Windows, file descriptors are closed without being inherited by processes that were generated by the system function or similar means. For example, an error results if you attempt to perform input or output on a file descriptor opened by the parent process without reopening it. This does not apply to the standard input, standard output, and standard error output, which do not need to be reopened.

• In Windows, if the command name specified in the system function contains no path, the command found in the path search order of a command processor such as the command prompt is executed.

• When you use the exit statement to return a return code for a command, depending on the value specified in the exit statement, the return code might sometimes be different from the return code that is returned to the caller of the command. See the description of the exit statement for details about specifying a command return code in the exit statement. Note that in the awk command, the value specified in the exit statement is rounded to a four-byte signed integer. For this reason, an error occurs if you specify a numeric value that is outside the range of a four-byte signed integer. The specific error is OS-dependent.

To Page Top

Usage examples

• In an argument to the command, specify that the action is to convert from lowercase to uppercase the records that match the search pattern. The input file is file01.txt.

  Contents of file01.txt:

  hitachi group01 Tokyo
  HITACHI group02 Yokohama
  hitachi group03 Fukuoka
  HITACHI group04 Hokkaido

  The results of executing the command are as follows:

  C:\TEMP>%ADSH_OSCMD_DIR%\awk "/hitachi/{print toupper($0)}" file01.txt
  HITACHI GROUP01 TOKYO
  HITACHI GROUP03 FUKUOKA

• Output the records where the second field matches the pattern in a regular expression. The input file is file01a.txt.

  Contents of file01a.txt:

  hitachi group01 Tokyo
  hitachi group02 Yokohama
  hitachi grp0000 Fukuoka
  hitachi group04 Hokkaido

  The results of executing the command are as follows:

  C:\TEMP>%ADSH_OSCMD_DIR%\awk "$2 ~ /group/" file01a.txt
  hitachi group01 Tokyo
  hitachi group02 Yokohama
  hitachi group04 Hokkaido

• Output the contents of the third field, using the -F option to specify # as the input field separator. The input file is file02.txt.

  Contents of file02.txt:

  hitachi#group01#Nagoya
  HITACHI#group02#Hiroshima
  hitachi#group03#Ooita

  The results of executing the command are as follows:

  C:\TEMP>%ADSH_OSCMD_DIR%\awk -F"#" "{print $3}" file02.txt
  Nagoya
  Hiroshima
  Ooita

• Specify the variable padstr and its value in an argument, in order to pass them to the action, which is specified in the prog01.awk script file. The input file is file03.txt.

  Contents of prog01.awk:

  # program name : prog01
  {
  count++
  print padstr " " $0#
  }
  END{
      print "total record : " count
  }

  #

  Outputs the value of the variable specified in the -v option, a space, and the contents of the input record.

  Contents of file03.txt:

  group01 Tokyo
  group02 Yokohama
  group03 Fukuoka

  The results of executing the command are as follows:

  C:\TEMP>%ADSH_OSCMD_DIR%\awk -v padstr="hitachi" -f prog01.awk file03.txt
  hitachi group01 Tokyo
  hitachi group02 Yokohama
  hitachi group03 Fukuoka
  total record : 3

• In a command argument, specify # as the value of the FS built-in variable for input file file02.txt. The input files are file01.txt and file02.txt.

  Contents of file01.txt:

  hitachi group01 Tokyo
  HITACHI group02 Yokohama
  hitachi group03 Fukuoka
  HITACHI group04 Hokkaido

  Contents of file02.txt:

  hitachi#group01#Nagoya
  HITACHI#group02#Hiroshima
  hitachi#group03#Ooita

  The results of executing the command are as follows:

  C:\TEMP>%ADSH_OSCMD_DIR%\awk "{print $3}" file01.txt FS="#" file02.txt
  Tokyo
  Yokohama
  Fukuoka
  Hokkaido
  Nagoya
  Hiroshima
  Ooita

• Retrieve records beginning with the record that contains group03 through the record that contains group06, and output them to the file file06.txt. The script file is prog02.awk. The input files are file04.txt and file05.txt.

  Contents of prog02.awk:

  BEGIN{
      print "Extract record : group03 - group06" > "file06.txt"
  }
  /group03/, /group06/{#
      count++;
      print >> "file06.txt";
   
  }
  END{
      printf "total record : %03d\n",  count >> "file06.txt"
  }

  #

  The processing target begins with the record that matches group03 and extends through the record that matches group06.

  Contents of file04.txt:

  hitachi group01 Tokyo
  HITACHI group02 Yokohama
  hitachi group03 Fukuoka
  HITACHI group04 Hokkaido

  Contents of file05.txt:

  hitachi group05 Nagoya
  HITACHI group06 Hiroshima
  hitachi group07 Ooita

  The results of executing the command are as follows:

  C:\TEMP>%ADSH_OSCMD_DIR%\awk -f prog02.awk file04.txt file05.txt
  C:\TEMP>%ADSH_OSCMD_DIR%\cat file06.txt
  Extract record : group03 - group06
  hitachi group03 Fukuoka
  HITACHI group04 Hokkaido
  hitachi group05 Nagoya
  HITACHI group06 Hiroshima
  total record : 004

• Output beginning with the first record in file file04.txt through the record that contains group02. In addition, output all records from the file file05.txt. The script file is prog03.awk. The input files are the previous file04.txt and file05.txt.

  Contents of prog03.awk:

  {
      count++; print
      if ($2 == "group02") {
          nextfile#
      }
  }
  END{
      printf("total record : %03d\n",  count)
  }

  #

  If the second field is group02, begin processing the next input file.

  The results of executing the command are as follows:

  C:\TEMP>%ADSH_OSCMD_DIR%\awk -f prog03.awk file04.txt file05.txt
  hitachi group01 Tokyo
  HITACHI group02 Yokohama
  hitachi group05 Nagoya
  HITACHI group06 Hiroshima
  hitachi group07 Ooita
  total record : 005

• If the second field of the record is group02, terminate execution of the awk command. In addition, return as the return code of the command the number of records that were read before the command terminated. The script file is prog04.awk. The input files are file04.txt and file07.txt.

  Contents of prog04.awk:

  {
      print
  if ($2 == "group02") {
          exit(NR)#
      }
  }
  END{
      printf("total record : %03d\n",  NR)
  }

  #

  Terminates the command and sets as the return code the value of the NR built-in variable, which contains the number of records read so far.

  Contents of file04.txt:

  hitachi group01 Tokyo
  HITACHI group02 Yokohama
  hitachi group03 Fukuoka
  HITACHI group04 Hokkaido

  Contents of file07.txt:

  hitachi group05 Nagoya
  HITACHI group06 Hiroshima
  hitachi group07 Ooita
  hitachi group03 Okinawa

  The results of executing the command are as follows:

  C:\TEMP>%ADSH_OSCMD_DIR%\awk -f prog04.awk file04.txt file07.txt
  hitachi group01 Tokyo
  HITACHI group02 Yokohama
  total record : 002

• Use the sub function to change the first occurrence of a specific character string in each record. The script file is prog05.awk. The input file is file08.txt.

  Contents of prog05.awk:

  {
      if (sub(/Hitachi/,  "& Corporation")#) {
          print
      } else {
             print "Unconverted record : " $0
      }
  }

  #

  Replaces Hitachi with Hitachi Corporation in the records.

  Contents of file08.txt:

  Hitachi Yokohama Office Hitachi Group
  Hitachi Tokyo Office Hitachi Group
  Tanaka Okinawa Office Tanaka Group
  Hitachi Fukuoka Office Hitachi Group

  The results of executing the command are as follows:

  C:\TEMP>%ADSH_OSCMD_DIR%\awk -f prog05.awk file08.txt
  Hitachi Corporation Yokohama Office Hitachi Group
  Hitachi Corporation Tokyo Office Hitachi Group
  Unconverted record : Tanaka Okinawa Office Tanaka Group
  Hitachi Corporation Fukuoka Office Hitachi Group

• Use the gsub function to change every occurrence of a specific character string in each record. The script file is prog06.awk. The input file is file09.txt.

  Contents of prog06.awk:

  {
      if (gsub(/Hitachi/,  "& Corporation")#) {
          print
      } else {
             print "Unconverted record : " $0
      }
  }

  #

  Replaces every instance of Hitachi with Hitachi Corporation in the records.

  Contents of file09.txt:

  Hitachi Yokohama Office Hitachi Group
  Hitachi Tokyo Office Hitachi Group
  Tanaka Okinawa Office Tanaka Group
  Hitachi Fukuoka Office Hitachi Group

  The results of executing the command are as follows:

  C:\TEMP>%ADSH_OSCMD_DIR%\awk -f prog06.awk file09.txt
  Hitachi Corporation Yokohama Office Hitachi Corporation Group
  Hitachi Corporation Tokyo Office Hitachi Corporation Group
  Unconverted record : Tanaka Okinawa Office Tanaka Group
  Hitachi Corporation Fukuoka Office Hitachi Corporation Group

• Find the position of a specific character string using the index function. The script file is prog07.awk.

  Contents of prog07.awk:

  BEGIN{
      str = "HI:hitachi"
      print "Column = " index(str,  "hitachi")
  }

  The results of executing the command are as follows:

  C:\TEMP>%ADSH_OSCMD_DIR%\awk -f prog07.awk
  Column = 4

• Find the length of a character string using the length function. The script file is prog08.awk.

  Contents of prog08.awk:

  BEGIN{
      str = "HI:hitachi"
      print "Length = " length(str)
  }

  The results of executing the command are as follows:

  C:\TEMP>%ADSH_OSCMD_DIR%\awk -f prog08.awk
  Length = 10

• Use the match function to find the position and length of a specific character string. The script file is prog09.awk.

  Contents of prog09.awk:

  BEGIN{
      str = "hitachi:MOUSE"
      print "Column  = " match(str,  /U.E/)
      print "RSTART  = " RSTART
      print "RLENGTH = " RLENGTH
  }

  The results of executing the command are as follows:

  C:\TEMP>%ADSH_OSCMD_DIR%\awk -f prog09.awk
  Column  = 11
  RSTART  = 11
  RLENGTH = 3

• Split a character string using a specific character as the delimiter, and store the results in an array. The script file is prog10.awk.

  Contents of prog10.awk:

  BEGIN{
      str = "Hitachi#Yokohama Office#Hitachi Group"
      num = split(str,  arry, "#")
      for (i = 1; i <= num; i++ ) {
          print arry[i]
      }
  }

  The results of executing the command are as follows:

  C:\TEMP>%ADSH_OSCMD_DIR%\awk -f prog10.awk
  Hitachi
  Yokohama Office
  Hitachi Group

• Find the substring at a specific position. The script file is prog11.awk.

  Contents of prog11.awk:

  BEGIN{
      str = "hitachi:MOUSE"
      rtnstr = substr(str,  11, 2)
      print "SUBSTR = " rtnstr
  }

  The results of executing the command are as follows:

  C:\TEMP>%ADSH_OSCMD_DIR%\awk -f prog11.awk
  SUBSTR = US

• Read records from an input file that is not specified as an argument. The script file is prog12.awk. The input file is file10.txt.

  Contents of prog12.awk:

  BEGIN{
      while ((getline rec < "file10.txt"#) > 0) {
          print rec
      }
  }

  #

  Reads a record from the specified input file file10.txt and stores the contents of the record in the rec variable.

  Contents of file10.txt:

  hitachi group01 Tokyo
  HITACHI group02 Yokohama
  hitachi group03 Fukuoka
  HITACHI group04 Hokkaido

  The results of executing the command are as follows:

  C:\TEMP>%ADSH_OSCMD_DIR%\awk -f prog12.awk
  hitachi group01 Tokyo
  HITACHI group02 Yokohama
  hitachi group03 Fukuoka
  HITACHI group04 Hokkaido

• Pass output from the print function to a command through a pipe. The script file is prog13.awk. The input file is file11.txt.

  Contents of prog13.awk:

  BEGIN{
      cmd = "sort "
  }
  {
      if (sub(/group01/,  $2)) {
          count++
          print | cmd#1
      }
  }
  END{
      close(cmd)#2
      print "total record : " count
  }

  #1

  Passes records to the sort command specified in the cmd variable.

  #2

  Closes the pipe and terminates execution of the sort command by executing the close function.

  Contents of file11.txt:

  hitachi group01 003 tokyo
  hitachi group02 001 yokohama
  hitachi group03 001 fukuoka
  hitachi group01 004 hokkaido
  hitachi group01 001 nagoya
  hitachi group02 001 hiroshima
  hitachi group01 002 ooita

  The results of executing the command are as follows:

  C:\TEMP>%ADSH_OSCMD_DIR%\awk -f prog13.awk file11.txt
  hitachi group01 001 nagoya
  hitachi group01 002 ooita
  hitachi group01 003 tokyo
  hitachi group01 004 hokkaido
  total record : 4

• Delete an element of an array. The script file is prog14.awk.

  Contents of prog14.awk:

  BEGIN{
      arry["Fukuoka"]   = "Fukuoka"
      arry["Hokkaido"]   = "Sapporo"
      arry["Kanagawa"] = "Yokohama"
      arry["Shimane"]   = "Matsue"
      for (key in arry) {
          printf("  %*s : %s\n",  6, key,   arry[key])
      }
      print "Deletes result of the array element"
      delete arry["Kanagawa"]
      for (key in arry) {
          printf("  %*s : %s\n",  6, key,   arry[key])
      }
  }

  The results of executing the command are as follows:

  C:\TEMP>%ADSH_OSCMD_DIR%\awk -f prog14.awk
     Fukuoka : Fukuoka
    Hokkaido : Sapporo
    Kanagawa : Yokohama
     Shimane : Matsue
  Deletes result of the array element
     Fukuoka : Fukuoka
    Hokkaido : Sapporo
     Shimane : Matsue

• When the program starts, create a directory and output the contents of the records to a file in the directory that was created. The script file is prog15.awk. The input file is file12.txt.

  Contents of prog15.awk:

  BEGIN{
      if ((rc = system("mkdir dir001")#1)) {
          printf("system func error rc = %x\n",  rc) > "/dev/stderr"#2
          exit(1)
      }
  }
  {
      print >> "dir001\\outfile.txt"
  }

  #1

  Creates a directory by executing the mkdir command from the system function.

  #2

  Outputs the return code of the system function to the standard error output.

  Contents of file12.txt:

  hitachi group01 Tokyo
  HITACHI group02 Yokohama
  hitachi group03 Fukuoka
  HITACHI group04 Hokkaido

  The results of executing the command are as follows:

  C:\TEMP>%ADSH_OSCMD_DIR%\awk -f prog15.awk file12.txt
  C:\TEMP>%ADSH_OSCMD_DIR%\cat dir001\\outfile.txt
  hitachi group01 Tokyo
  HITACHI group02 Yokohama
  hitachi group03 Fukuoka
  HITACHI group04 Hokkaido

• Call a user-defined function and output its results. The script file is prog16.awk.

  Contents of prog16.awk:

  BEGIN{
  a = 3
  b = 4
  result = func01(a,  b,  c)
  print "func01 = " result
  }
  function func01(x, y){
  x *= x
  y *= y
  return x + y
  }

  The results of executing the command are as follows:

  C:\TEMP>%ADSH_OSCMD_DIR%\awk -f prog16.awk
  awk: warning: function func01 called with 3 args,  uses only 2#
   source line number 4
  func01 = 25

  #

  A warning message is output because the number of arguments in the call to the user-defined function exceeds the number of arguments in the function definition.

• Display a message when a syntax error is detected in a control statement. The script file is prog17.awk.

  Contents of prog17.awk:

  BEGIN{
      while ((getline rec < "file10.txt") > 0)) {#
          print rec
      }
  }

  #

  The number of opening and closing parentheses is mismatched in the while statement.

  The results of executing the command are as follows:

  C:\DIR>%ADSH_OSCMD_DIR%\awk -f prog17.awk
  awk: extra ) at source line 2 source file prog17.awk
   context is
              while ((getline rec < "file10.txt") > >>>  0)) <<<
  awk: syntax error at source line 2 source file prog17.awk
  awk: illegal statement at source line 2 source file prog17.awk
          extra )

• Display a message when the syntax of a built-in function is invalid. The script file is prog18.awk.

  Contents of prog18.awk:

  BEGIN{
      str = "Hitachi:hitachi"
      print "Column = " index(str)#
  }

  #

  No character string is specified as an argument to the index function.

  The results of executing the command are as follows:

  C:\DIR>%ADSH_OSCMD_DIR%\awk -f prog18.awk
  awk: syntax error at source line 3 source file prog18.awk
   context is
              print "Column = " >>>  index(str) <<<
  awk: illegal statement at source line 3 source file prog18.awk

• Use the temporary file functionality and path conversion functionality of JP1/Advanced Shell to perform input and output on a file. The job definition script is adsh001.ash. The script file is prog19.awk. The input file is file12.txt.

  Specification of the temporary file functionality and path conversion functionality for the environment files:

  #-adsh_conf TEMP_FILE_DIR       "C:\\TEMP\\ADSH"
  #-adsh_conf PATH_CONV_ENABLE / :
  #-adsh_conf PATH_CONV "/var/hitachi/jp1as/perm" "C:\\hitachi\\JP1AS\\perm"

  Contents of adsh001.ash:

  #-adsh_file_temp TEMP
  #-adsh_step_start adsh001 -onError stop
  "$ADSH_OSCMD_DIR/awk" -f prog19.awk "/var/hitachi/jp1as/perm/file12.txt"
  #-adsh_step_error
  exit 100
  #-adsh_step_end

  Contents of prog19.awk:

  {
      print FILENAME, ":", $0 > ENVIRON["TEMP"]
  }
  END{
  while(getline var < ENVIRON["TEMP"])
      print var
  }

  Contents of file12.txt:

  001 abc
  002 efgh
  003 ijklmnop

  Output of the awk command (sent to the standard output by the job):

  C:\hitachi\JP1AS\perm\file12.txt : 001 abc
  C:\hitachi\JP1AS\perm\file12.txt : 002 efgh
  C:\hitachi\JP1AS\perm\file12.txt : 003 ijklmnop

• Display a message when no arguments are specified:

  C:\DIR>%ADSH_OSCMD_DIR%\awk
  usage: awk [-F fs] [-v var=value] [-f progfile | prog]
         [[file ...] | [built-in-var=value ...]] ...

• Display an option error message.

  This message might vary depending on the platform on which the command is executed. The following shows an example in Windows:

  C:\TEMP>%ADSH_OSCMD_DIR%\awk -z
  awk: illegal option -- z
  usage: awk [-F fs] [-v var=value] [-f progfile | prog]
       [[file ...] | [built-in-var=value ...]] ...

To Page Top