> The Unix shell: Delete duplicate lines in a file in Unix

Sunday, December 30, 2012

Delete duplicate lines in a file in Unix

awk '!x[$0]++' file.txt

Posted by vijay sarathi at 9:23 AM

Email This BlogThis!Share to Twitter Share to Facebook

Labels: awk, file

No comments:

Useful Links

C++ Reference
All about C++
Sed
Awk User Guide
Perl Blog's
Learning Perl
Perlfaq5

Labels

abs (1)
array (2)
arrays (1)
awk (21)
awk fnr (3)
background (1)
bash (2)
bitwise operators (1)
books (1)
bulk (1)
c (5)
c malloc (1)
c++ (9)
calloc (1)
capitalize (1)
child (1)
coloum (1)
column (2)
columns (2)
command (1)
compare (2)
conditional statements (1)
copy (1)
cpp (1)
create (2)
cut (1)
debugger (1)
delete (2)
difference (1)
editor (1)
empty (1)
empty lines (1)
env (1)
exec (2)
explicit (1)
expr (1)
fg (1)
file (5)
files (2)
filter (1)
find (3)
firsline (1)
fnr (2)
foreground (1)
fork (1)
format (1)
free (1)
ftp (2)
grep (3)
grep -f (1)
guide (1)
hash (3)
head (1)
if (1)
indentation (1)
interview-question (3)
isql (1)
java (1)
jobs (1)
join (4)
ksh (1)
last (1)
lc (1)
line number (2)
list (4)
malloc (1)
match (3)
mv (2)
nawk (2)
nr (1)
number (1)
numeric sort (3)
ors (1)
output (1)
paste (1)
pattern (2)
pattern match (2)
perl (31)
perl script (3)
perldoc (1)
popen (1)
pow (1)
process (1)
programming (1)
ps (1)
regex (2)
remote server (1)
rename (2)
replace (2)
reverse (1)
search (2)
sed (8)
semicolon (1)
separator (1)
shebang (1)
shell (3)
shell variable (2)
shell-script (3)
shell-scripting (1)
sort (3)
split (2)
standard libraries (1)
store (2)
string (2)
sum (1)
sybase (2)
table (1)
tail (1)
tar (1)
tolower (1)
toupper (1)
type-casting (1)
typecast (1)
uc (1)
unique (1)
unix (4)
variable (1)
vector (1)
vi (2)
vim (1)
why (1)
xargs (3)

Popular Posts

comparing two files with awk

I have two files File 1 contains 3 fields File 2 contains 4 fields The number of rows of File 1 is much smaller than that of File 2 I w...
Find and replace a string in all the files recursively

How to do a Find and replace a string in all the files recursively in unix: Its very simple: find . -type f|xargs perl -pi -e 's/sour...
How does free/delete in C/C++ know how much memory to free/delete

This is probably very important for every C/C++ programmer to know. And most of the interviewers too ask this question.I myself had asked i...
List of Bash Conditional statements

At times you need to specify different courses of action to be taken in a shell script, depending on the success or failure of a command. Th...
Running a background process and foreground process

In unix, if we run a process(a shell script).It will not return to the terminal untill the script ends. lets say there is a script temp.sh ...
Splitting a string in C++

Once I writing a tool for processing some csv files wherin i needed to split strings and store them in a vector.I wrote one function for it...
Retrieving a List of files from Remote Server

Below is the perl script for retrieving the list of files from Remote server: #!/usr/bin/perl use strict; use warnings; ...
Semicolon is optional in perl

I guess most of the people have noticed about this and have just ignored it. Or they must not have noticed it at all. Do you know this fac...
C++ overcome the definition inthe standard libraries

I have a function abs in one of the libraries and i wanted to use that function by linking to that library. But i face this error below: ...
Bulk renaming of files in unix

Most of the time it is required that w eneed to rename the files in bulk. this can be done in many ways,Mostly people do it by writing a s...

Google+ Followers

Blog Archive

► 2013 (39)
- ► May (6)
- ► April (4)
- ► March (4)
- ► February (4)
- ► January (21)

▼ 2012 (51)
- ▼ December (51)

About Me

vijay sarathi

View my complete profile

Simple template. Template images by luoman. Powered by Blogger.

Sunday, December 30, 2012

Delete duplicate lines in a file in Unix

No comments:

Post a Comment