uniq command (removes duplicated lines from a sorted file)

Format

uniq [-c] [-d] [-u] [input-path-name [output-path-name]]

To Page Top

Description

This command outputs the results of consolidating duplicated lines in a file into single lines. Note that lines with identical content are considered to be duplicates only if they are consecutive.

To Page Top

Arguments

If no options are specified, the processing is the same as when the -d and -u options are both specified. That is, the command outputs duplicate lines as a single line, and it also outputs all non-duplicate lines.

-c

Specifies that each output line is to be preceded by a count of the number of times the line occurred, followed by a single space. A count is displayed as a four-digit number, but the number of digits will be increased if necessary to accommodate values that exceed four digits. A single space is displayed after each count.

-d

Specifies that only duplicate lines are to be output.

-u

Specifies that only lines that had no duplicates are to be displayed.

input-path-name

Specifies the input file. If input-path-name is not specified or is specified as -, the standard input is read.

output-path-name

Specifies the output file for the results. If output-path-name is not specified or is specified as -, the standard output is assumed.

To Page Top

Return codes

Return code	Meaning
0	Normal termination
1 or greater	Error termination

To Page Top

Notes

• If you specify the same file for input-path-name and output-path-name, the file will be empty.

• The maximum number of bytes that can be compared in a single line is 8,192.

• Input from binary files and output of binary data are not guaranteed to work.

To Page Top

Usage examples

The following shows the format of the file used in the examples below to illustrate the results of executing the uniq command.

• file1.txt

  aaaa
  aaaaaaa                       duplicate
  aaaaaaa                       duplicate
  bbbbbbb
  bbbbbbbbbbb                   duplicate
  bbbbbbbbbbb                   duplicate
  bbbbbbbbbbb                   duplicate
  bbbbbbbbbbb                   duplicate
  bcbcbcbcbcb
  ddddddddddddddddddd
  dddddddddddddddddddddd
  dddddddddddddddddddddddd
  ddddddddddddddddddddddddeee    duplicate
  ddddddddddddddddddddddddeee    duplicate

The file listed above is used as the input file in the following examples.

• Display the default with no options specified.

  C:\TEMP>%ADSH_OSCMD_DIR%\uniq file1.txt
  aaaa
  aaaaaaa
  bbbbbbb
  bbbbbbbbbbb
  bcbcbcbcbcb
  ddddddddddddddddddd
  dddddddddddddddddddddd
  dddddddddddddddddddddddd
  ddddddddddddddddddddddddeee
   
  C:\TEMP>

• Specify the -c option to precede each output line with a count of the number of times the line occurred.

  C:\TEMP>%ADSH_OSCMD_DIR%\uniq -c file1.txt
     1 aaaa
     2 aaaaaaa
     1 bbbbbbb
     4 bbbbbbbbbbb
     1 bcbcbcbcbcb
     1 ddddddddddddddddddd
     1 dddddddddddddddddddddd
     1 dddddddddddddddddddddddd
     2 ddddddddddddddddddddddddeee
   
  C:\TEMP>

• Specify the -d option to display only the duplicate lines (one instance of each set of lines that were duplicated).

  C:\TEMP>%ADSH_OSCMD_DIR%\uniq -d file1.txt
  aaaaaaa
  bbbbbbbbbbb
  ddddddddddddddddddddddddeee
   
  C:\TEMP>

• Specify the -u option to display only the lines that had no duplicates.

  C:\TEMP>%ADSH_OSCMD_DIR%\uniq -u file1.txt
  aaaa
  bbbbbbb
  bcbcbcbcbcb
  ddddddddddddddddddd
  dddddddddddddddddddddd
  dddddddddddddddddddddddd
   
  C:\TEMP>

• Display an option error message.

  This message might vary depending on the platform on which the command is executed. The following shows an example in Windows:

  C:\TEMP>%ADSH_OSCMD_DIR%\uniq -w
  uniq: illegal option -- w
  usage: uniq [-cdu] [input_file [output_file]]

To Page Top