Showing posts with label compare. Show all posts

Column wise comaprision using awk

There was a requirement once where i need to compare two files not with thier rows instead  i needed to do the comparision with columns.
I wanted only those rows where any of the columns in the lines differ.

for eg:
File1
1 A B C D
2 E F G H
File2
1 A Z C D
2 E F Y H
3 M N O P
Below is the Output I need:
file1 1 col2 B
file2 1 col2 Z
file1 2 col3 G
file2 2 col3 Y

Below is the solution in awk that i have written.
awk 'FNR==NR{a[FNR]=$0;next} {
if(a[FNR])
{split(a[FNR],b);
for(i=1;i<=NF;i++)
{
if($i!=b[i])
{
printf "file1 "b[1]" col"b[i-1]" "b[i]"\n";
printf "file2 "$1" col"b[i-1]" "$i"\n";
}
}
}
}'
Below is the test i made on my solaris server:


> nawk 'FNR==NR{a[FNR]=$0;next}{if(a[FNR]){split(a[FNR],b);for(i=1;i<=NF;i++){if($i!=b[i]){printf "file1 "b[1]" col"i-1" "b[i]"\n";printf "file2 "$1" col"i-1" "$i"\n";}}}}' file1 file2
file1 1 col2 B
file2 1 col2 Z
file1 2 col3 G
file2 2 col3 Y
>
Read More

AWK one-liner for multi-column comparision two unsorted files

This awk one-liner works for multi-column on unsorted files. Comparision is done based on 1st,2nd,3rd,4th of the first file and 1st,3rd,6th,7th of the second file.
File1 
7000,2,1,6
7001,2,1,7
7002,2,1,6
7003,1,2,1
File2
7000,john,2,0,0,1,6
7000,john,2,0,0,1,7
7000,john,2,0,0,1,8
7000,john,2,0,0,1,9
7001,elen,2,0,0,1,7
7002,sami,2,0,0,1,6
7003,mike,1,0,0,2,1
7003,mike,1,0,0,2,2
7003,mike,1,0,0,2,3
7003,mike,1,0,0,2,4
8001,nike,1,2,4,1,8
8002,paul,2,0,0,2,7
Output
awk -F, 'NR==FNR{a[$1,$2,$3,$4]++;next} (a[$1,$3,$6,$7])' File1 File2
7000,john,2,0,0,1,6
7001,elen,2,0,0,1,7
7002,sami,2,0,0,1,6
7003,mike,1,0,0,2,1
Read More

Comparing two files using awk


I have two files
File 1 contains 3 fields
File 2 contains 4 fields

The number of rows of File 1 is much smaller than that of File 2

I would like to compare between two files based on 1st field with the following operation
If the first field in any row of file 1 appears in the first field of a row in file 2,
don't print that row for file 2.

Input File 1
S13109 3739 31082
S45002 3800 31873
S43722 3313 26638

Input File 2
S13109 3738 31081 0
S13109 3737 31080 0
S00033 3008 29985 0
S00033 3007 29984 0
S00022 4130 31838 0
S00022 4129 31837 0
S00188 3317 27372 0
S45002 3759 31832 0
S45002 3758 31831 0
S45002 3757 31830 0
S43722 3020 26345 0
S43722 3019 26344 0
S00371 3737 33636 0
S00371 3736 33635 0

Desired Output
S00033 3008 29985 0
S00033 3007 29984 0
S00022 4130 31838 0
S00022 4129 31837 0
S00188 3317 27372 0
S00371 3737 33636 0
S00371 3736 33635 0

solution
---------
awk 'FNR==NR{a[$1]++;next}!a[$1]' file1 file2

How it works:

FNR==NR

When you have two (or more) input files to awk, FNR will reset back to 1
on the first line of the next file whereas NR will continuing incrementing
from where it left off. By checking FNR==NR we are essentially checking
to see if we are currently parsing the first file.

a[$1]++

If we are parsing the first file (see above) then create an associative
array with the first field $1 as the key and post increment the value by 1.
This essentially lets us create a 'seen' list.

 
next

This command tells awk not to process any further commands and to read in
the next record and start over.
We do this because file1 is only meant to set the associative array

!a[$1]

This line only executes when FNR==NR is false, i.e. we are not parsing
file1 and thus must be parsing file2. We then use the first field $1 of
file2 as the key to index into our 'seen' list created earlier. If the
value returned is 0 it means we didn't see it in file1 and therefore we
should print this line. Conversely, if the value is non-zero then we
did see it in file1 and thus we should not print its value.
Note that !a[$1] is equivalent to !a[$1]{print} because the default action
when one is not given is to print the entire line.
Read More
Subscribe to: Posts (Atom)