uniq command (removes duplicated lines from a sorted file)
- Organization of this page
Format
uniq [-c] [-d] [-u] [input-path-name [output-path-name]]
Description
This command outputs the results of consolidating duplicated lines in a file into single lines. Note that lines with identical content are considered to be duplicates only if they are consecutive.
Arguments
If no options are specified, the processing is the same as when the -d and -u options are both specified. That is, the command outputs duplicate lines as a single line, and it also outputs all non-duplicate lines.
- -c
-
Specifies that each output line is to be preceded by a count of the number of times the line occurred, followed by a single space. A count is displayed as a four-digit number, but the number of digits will be increased if necessary to accommodate values that exceed four digits. A single space is displayed after each count.
- -d
-
Specifies that only duplicate lines are to be output.
- -u
-
Specifies that only lines that had no duplicates are to be displayed.
- input-path-name
-
Specifies the input file. If input-path-name is not specified or is specified as -, the standard input is read.
- output-path-name
-
Specifies the output file for the results. If output-path-name is not specified or is specified as -, the standard output is assumed.
Return codes
Return code |
Meaning |
---|---|
0 |
Normal termination |
1 or greater |
Error termination |
Notes
-
If you specify the same file for input-path-name and output-path-name, the file will be empty.
-
The maximum number of bytes that can be compared in a single line is 8,192.
-
Input from binary files and output of binary data are not guaranteed to work.
Usage examples
The following shows the format of the file used in the examples below to illustrate the results of executing the uniq command.
-
file1.txt
aaaa aaaaaaa duplicate aaaaaaa duplicate bbbbbbb bbbbbbbbbbb duplicate bbbbbbbbbbb duplicate bbbbbbbbbbb duplicate bbbbbbbbbbb duplicate bcbcbcbcbcb ddddddddddddddddddd dddddddddddddddddddddd dddddddddddddddddddddddd ddddddddddddddddddddddddeee duplicate ddddddddddddddddddddddddeee duplicate
The file listed above is used as the input file in the following examples.
-
Display the default with no options specified.
C:\TEMP>%ADSH_OSCMD_DIR%\uniq file1.txt aaaa aaaaaaa bbbbbbb bbbbbbbbbbb bcbcbcbcbcb ddddddddddddddddddd dddddddddddddddddddddd dddddddddddddddddddddddd ddddddddddddddddddddddddeee C:\TEMP>
-
Specify the -c option to precede each output line with a count of the number of times the line occurred.
C:\TEMP>%ADSH_OSCMD_DIR%\uniq -c file1.txt 1 aaaa 2 aaaaaaa 1 bbbbbbb 4 bbbbbbbbbbb 1 bcbcbcbcbcb 1 ddddddddddddddddddd 1 dddddddddddddddddddddd 1 dddddddddddddddddddddddd 2 ddddddddddddddddddddddddeee C:\TEMP>
-
Specify the -d option to display only the duplicate lines (one instance of each set of lines that were duplicated).
C:\TEMP>%ADSH_OSCMD_DIR%\uniq -d file1.txt aaaaaaa bbbbbbbbbbb ddddddddddddddddddddddddeee C:\TEMP>
-
Specify the -u option to display only the lines that had no duplicates.
C:\TEMP>%ADSH_OSCMD_DIR%\uniq -u file1.txt aaaa bbbbbbb bcbcbcbcbcb ddddddddddddddddddd dddddddddddddddddddddd dddddddddddddddddddddddd C:\TEMP>
-
Display an option error message.
This message might vary depending on the platform on which the command is executed. The following shows an example in Windows:
C:\TEMP>%ADSH_OSCMD_DIR%\uniq -w uniq: illegal option -- w usage: uniq [-cdu] [input_file [output_file]]