removing redundancy from output columns What is a *NIX way of removing redundancy in a case where I have pairwise comparisons like these in two columns A B B A A C A D C A D A B C C B `A B` and `B A` represent the same comparison and I would like to remove such redundancy from the dataset. The final result should be A B A C A D B C

doit ()
{
awk '{
key=$1<=$2? $1 FS $2 : $2 FS $1;
if (!seen[key]) print $1,$2
seen[key]=1
}'
}



$ doit A B
A C
A D
B C
$


(or, getting terser with it 'cause Chris Down's answer's so sweet)


awk '!seen[$1<=$2? $1 FS $2: $2 FS $1]++ {print $1,$2}'


which could be further reduced if you don't care about the spaces in your data


awk '!seen[$1<=$2? $1 FS $2: $2 FS $1]++'


)

The `FS` is awk's "field separator" variable, used here to guarantee the boundaries between key fields will be properly identified. My original had them run together, `$1$2`, which as Stephane Chazelas pointed out would have treated `A BC` and `AB C` as duplicates.