Showing posts with label sed. Show all posts

Converting single column to multiple columns

I have a file which contains all the entries in a single column like: 
0
SYSCATSPACE
16384
13432
2948
1
1
TEMPSPACE1
1
1
applicable
1
2
USERSPACE1
4096
1888
2176
1
If I want to convert this in a tabular form of 3*6: 
0 SYSCATSPACE 16384 13432 2948       1
1 TEMPSPACE1  1     1     applicable 1
2 USERSPACE1  4096  1888  2176       1
Below is the command that I will use: 
perl -lne '$a.="$_ ";
           if($.%6==0){push(@x,$a);$a=""}
           END{for(@x){print $_}}' your_file
output would be : 
> perl -lne '$a.="$_ ";if($.%6==0){push(@x,$a);$a=""}END{for(@x){print $_}}' temp
0 SYSCATSPACE 16384 13432 2948 1 
1 TEMPSPACE1 1 1 applicable 1 
2 USERSPACE1 4096 1888 2176 1
Read More

Delete empty lines in a file

Some times there are some empty lines which we feel are redundant in the file and want to remove them.Below is the command in unix to do that. 
sed -i '/^$/d' your_file
But there is also another way to do this:
grep . your_file > dest_file
In perl also we can acheive this as below:
perl -pi -e 's/^$//g' your_file
the above mentioned perl and sed solutions will do an inplace replacement in the file
If in case the lines have some spaces then:
perl -pi -e 's/^\s*$//g' your_file
Read More

Print every character in a new line


I have a file like this:

    This is a sentence.
    This is another sentence.

I need to put a new line after each character, such that only one character appears on every line, e.g.:
    T
    h
    i
    s
    
    i
    s
    
    a
    
    s
    e
    n
    t
    e
    n
    c
    e
    .
    T
    h
    i
    s
    
    i
    s
    
    a
    n
    o
    t
    h
    e
    r
    
    s
    e
    n
    t
    e
    n
    c
    e
    .

Solution for this is:
sed:
sed 's/\(.\)/\1\n/g' -i filename
perl:
perl -F// -lane 'print join "\n", @F' filename
awk:
awk -F '' -v 'OFS=\n' '{$1=$1}1' filename
Read More

Print limited characters per a line

Printing first 80 characters in a line.Below are the different ways to do it.
Cut
cut -c1-80 your_file
Awk
awk '{print substr($0,0,80)}' your_file
Sed
sed -e 's/^\(.\{80\}\).*/\1/' your_file
Perl
perl -lne 'print substr($_,0,80)' your_file
perl -lpe 's/.{80}\K.*//s' your_file

Grep
grep -o "^.\{80\}" your_file
Read More

Find and replace a string in all the files recursively

Below are some useful commands to  find and replace a string in all the files recursively in unix:

find . -type f|xargs perl -pi -e 's/source/target/g'

or
find . -type f -exec perl -pi -e 's/source/target/g' {} \;
or
find . -type f -exec sed -i 's/source/target/g' {} \;

All the three are logically similar
Read More

Join lines based on a pattern

Lets say we have a text file like: 
ESP Client,ESP Engagement,Misc_Projects_120101,DEFAULT,HA,Unknown,No,Unknown,201704,4.1,Unknown,AAA,Collected-Done,"she,joy.",200111,Unknown,Full Time,,Delivery_DONE AMO,Approved,2012-12-03,2012-12-06,2012-12-06,"Occupied Hours 
(0)",0,"Approved Hours 
(112)",8,"Pending Hours 
(0)",0,"Pending and Approved Hours 
(112)",8,

ESP Client,ESP Engagement,Misc Projects_120101,DEFAULT,HR,Unknown,No,Unknown,201704,4.1,Unknown,AAA,Collected - Pending,"she, aj v.",200111,Unknown,Full Time,,Delivery_Pending AMO,Approved,2012-12-04,2012-12-14,2012-12-14,"Occupied Hours 
(0)",0,"Approved Hours 
(112)",8,"Pending Hours 
(0)",0,"Pending and Approved Hours 
(112)",8,
And if we want the expected output to be: 
 ESP Client,ESP Engagement,Misc_Projects_120101,DEFAULT,HA,Unknown,No,Unknown,201704,4.1,Unknown,AAA,Collected-Done,"she,joy.",200111,Unknown,Full Time,,Delivery_DONE AMO,Approved,2012-12-03,2012-12-06,2012-12-06,"Occupied Hours  (0)",0,"Approved Hours  (112)",8,"Pending Hours  (0)",0,"Pending and Approved Hours  (112)",8, 
 ESP Client,ESP Engagement,Misc Projects_120101,DEFAULT,HR,Unknown,No,Unknown,201704,4.1,Unknown,AAA,Collected - Pending,"she, aj v.",200111,Unknown,Full Time,,Delivery_Pending AMO,Approved,2012-12-04,2012-12-14,2012-12-14,"Occupied Hours  (0)",0,"Approved Hours  (112)",8,"Pending Hours  (0)",0,"Pending and Approved Hours  (112)",8,
Below is the perl one liner for this: 
perl -lne '$line=$line." ".$_;if(/^$/ or eof){print $line;undef $line;}' your_file
Read More

Join lines based upon the first field

Let's say I have a file with data like below: 
ENST000001.1 + 67208778 67210057
ENST000001.1 + 67208778 67210768
ENST000001.1 + 67208778 67208882
ENST000002.5 + 67208778 67213982
ENST000003.1 - 57463571 57463801
ENST000003.1 - 57476352 57476463
ENST000003.1 - 57476817 57476945
I want to join some lines based on the first field and follow certain pattern for joining the lines. Expected output is:
ENST000001.1 + 67208778_67210057  67208778_67210768  67208778_67208882 
ENST000002.5 + 67208778_67213982
ENST000003.1 - 57463571_57463801  57476352_57476463  57476817_57476945
I actually have two different solutions to achieve the same. Below they are one in Awk and one in Perl: 
awk '{a[$1" "$2]=a[$1" "$2]" "$3" "$4;}
     END{
         for(i in a)print i,a[i]
        }' your_file
In Perl:
perl -F -lane '$H{$F[0]." ".$F[1]}=
               $H{$F[0]." ".$F[1]}." ".$F[2]."_".$F[3];
               if(eof){
                      foreach(keys %H){print $_,$H{$_}}
                      }' your_file
Read More

Join consequent lines in unix

I have a file like this. 
John
30
Mike
0.0786268
Tyson
0.114889
Gabriel
0.176072
Fiona
0.101895
I need to shift every second row to a new column so it should look like this 
John   30
Mike   0.0786268
Tyson  0.114889
Gabriel 0.176072
Fiona   0.101895
There can be many solutions for this:
sed: 
sed 'N;s/\n/ /g' yourfile
Awk: 
awk '{if(NR%2!=0){p=""}else{p="\n"};printf $0" "p}' your_file
Paste: 
paste - - file new_file
My pick is: 
xargs -n2
Read More
Subscribe to: Posts (Atom)