8.4.38 uniq command (removes duplicated lines from a sorted file)
Syntax
uniq [-c] [-d] [-u] [input-path-name [output-path-name]]
Description
This command outputs the results of consolidating duplicated lines in a file into single lines. Note that lines with identical content are considered to be duplicates only if they are consecutive.
Arguments
If no options are specified, the processing is the same as when the -d and -u options are both specified. That is, the command outputs duplicate lines as a single line, and it also outputs all non-duplicate lines.
- -c
Specifies that each output line is to be preceded by a count of the number of times the line occurred, followed by a single space. A count is displayed as a four-digit number, but the number of digits will be increased if necessary to accommodate values that exceed four digits. A single space is displayed after each count.
- -d
Specifies that only duplicate lines are to be output.
- -u
Specifies that only lines that had no duplicates are to be displayed.
- input-path-name
Specifies the input file. If input-path-name is not specified or is specified as -, the standard input is read.
- output-path-name
Specifies the output file for the results. If output-path-name is not specified or is specified as -, the standard output is assumed.
Return codes
Return code |
Meaning |
---|---|
0 |
Normal termination |
1 or greater |
Error termination |
Notes
If you specify the same file for input-path-name and output-path-name, the file will be empty.
The maximum number of bytes that can be compared in a single line is 8,192.
Input from binary files and output of binary data are not guaranteed to work.
Examples
The following shows the format of the file used in the examples below to illustrate the results of executing the uniq command.
file1.txt
aaaa aaaaaaa
duplicate aaaaaaa
duplicate bbbbbbb bbbbbbbbbbb
duplicate bbbbbbbbbbb
duplicate bbbbbbbbbbb
duplicate bbbbbbbbbbb
duplicate bcbcbcbcbcb ddddddddddddddddddd dddddddddddddddddddddd dddddddddddddddddddddddd ddddddddddddddddddddddddeee
duplicate ddddddddddddddddddddddddeee
duplicate
The file listed above is used as the input file in the following examples.
Display the default with no options specified.
C:\TEMP>%ADSH_OSCMD_DIR%\uniq file1.txt aaaa aaaaaaa bbbbbbb bbbbbbbbbbb bcbcbcbcbcb ddddddddddddddddddd dddddddddddddddddddddd dddddddddddddddddddddddd ddddddddddddddddddddddddeee C:\TEMP>
Specify the -c option to precede each output line with a count of the number of times the line occurred.
C:\TEMP>%ADSH_OSCMD_DIR%\uniq -c file1.txt 1 aaaa 2 aaaaaaa 1 bbbbbbb 4 bbbbbbbbbbb 1 bcbcbcbcbcb 1 ddddddddddddddddddd 1 dddddddddddddddddddddd 1 dddddddddddddddddddddddd 2 ddddddddddddddddddddddddeee C:\TEMP>
Specify the -d option to display only the duplicate lines (one instance of each set of lines that were duplicated).
C:\TEMP>%ADSH_OSCMD_DIR%\uniq -d file1.txt aaaaaaa bbbbbbbbbbb ddddddddddddddddddddddddeee C:\TEMP>
Specify the -u option to display only the lines that had no duplicates.
C:\TEMP>%ADSH_OSCMD_DIR%\uniq -u file1.txt aaaa bbbbbbb bcbcbcbcbcb ddddddddddddddddddd dddddddddddddddddddddd dddddddddddddddddddddddd C:\TEMP>
Display an option error message.
This message might vary depending on the platform on which the command is executed. The following shows an example in Windows:
C:\TEMP>%ADSH_OSCMD_DIR%\uniq -w uniq: illegal option -- w usage: uniq [-cdu] [input_file [output_file]]