Tested & verified 50 useful AWK One Liners

1. How to print the line which matches /regex/?

   awk '/regex/{print $0}'

2. How to print the line immediately above the line containing /regex/?

awk '{x=y"\n"$0;y=$0};x~/regex/{gsub(/\n.*/,"",x);print x}'

3. How to print the line immediately after the line containing /regex/?

awk '{x=NR+1};NR==x{print $0}'

4. How to get the uniq lines from a file & the count the line is repeated?

awk '{arr[$0]++;};END { for (i in arr) { print i "\t" arr[i];} }'

5. Precede each lines with its line number.

awk ‘{print NR “\t” $0}’

6. Precede line numbers only on the non-blank lines.

awk 'NF{++c; print  c "t" $0};NF==0{print $0}'

7. Count the Lines.

awk ‘END{print NR}’

8. Printing words of a line in reverse order

awk '{for (i=NF;i>0;i--){ printf  "%s%s",$i,FS  } print "" }'

9. Printing lines of a file in Reverse order.

awk ‘{ arr[NR]=$0};END{for(i=NR;i> 0;i--){print arr[i]}}’

10. How to double space a file?

awk 'NF{print $0;print ""}'

11. Precede each lines by its line number for multiple files.

awk ‘{print FNR “\t” $0}’ files*

12. Print the sum of all the fields in all the lines.

awk '{ for(i=1;i<=NF;i++)sum=sum+$i };END{print sum}'

13.  Print sum of fileds of everylines.

awk '{ sum=0;for(i=1;i<=NF;i++)sum=sum+$i;print sum}’

14. Print the absolute values of all the fields.

awk '{for(i=1;i<=NF;i++){ if($i < 0) $i=-$i; }; print $0}'

15. Delete the lines containing a pattern only if next line contains a specific pattern

awk '/regex1/{print $0;getline;if(/regex2/)$0=""};{print $0}'

16. Print the sum of all the elements in every columns of a file.

awk '{ for(i=1;i<=NF;i++) sum[i]=sum[i]+$i;if(max < NF)max=NF;};END { for(j=1;j<=max;j++) printf "%d\t",sum[j];}'

17.  How to get the longest line of a file by fields?

awk 'max < NF{max=NF;maxline=$0};END{print maxline}'

18.  How to print the total number of words in a file?

awk ‘{total=total+NF};END{print total}’

19. How to get the largest number in the 2^nd field?

awk '{if(NR==1)max=$2; if(max < $2){max=$2}};END{print max}'

20.  Print the total number of lines which contains pattern  /Wipro/.

awk ‘/Wipro/{C++;};END{print C}’

21. Print the number of words in each line.

awk ‘{print NF “\t” $0}’

22. Print the number of fields in last line of the file.

awk ‘END{print NF “\t” $0}’

23. Print the last field of each line.

awk ‘{print $NF}’

24. Print the longest line of the file by characters.

awk '{if(max < length($0)){max=length($0); maxline=$0}}; END{ print maxline}'

25.  Print every line which contains atleast 3 fields.

awk ‘NF >2 {print $0}’

26.  Delete the leading & trailing tabs & white spaces.

awk '{sub(/^[ \t]*| [ \t]*$/,"");print}'

27. How to insert 2 spaces at the beginning of every line?

awk ‘{sub(/^/,”  ”); print}’

28.  Substitute bar with baz in all the lines only on the first instance.

awk ‘{sub(/bar/,”baz”);print }’

29. Substitute bar with baz in everyline on all the instances.

awk ‘{gsub(/bar/,”baz”);print }’

30. Substitute bar with baz only on 3^rd instance.

awk ‘{$0=gensub(/bar/,”baz”,3); print $0}’

31. Substitute bar with baz only if first field is foo.

awk ‘$1~/foo/{gsub(/bar/,”baz”)};{print $0}’

32. Exchange first & second fields in a datafile.

awk  ‘{temp=$2;$2=$1;$1=temp;print $0}’

33. Delete the last word from every line.

awk ‘{$NF=””;print $0}’

34. Concatenate every Two lines using colon separator.

awk '{ if(NR%2!=0) ORS=":"; else ORS="\n"; print $0}'

35. Print the First line of the file.

awk ‘NR<2 p="" print="">

36. Print Last line of the file.

awk ‘END{print $0}’

37. Print the last two Lines of a file.

awk ‘{y=x”\n”$0;x=$0}END{print y}’

39. Print the lines which contains 65 or more characters.

awk ‘length >64{print $0}’

40. Print the lines between 5^th & 15^th line number.

awk ‘NR>4 && NR<16 or="" p="" print="">

awk ‘NR==5,NR==15’

41. Print the lines between pattern regex1 & regex2.

awk ‘/regex1/,/regex2/{print $0}’

42. Print the lines from a pattern to the end of file.

awk ‘/regex/,0’

43. Delete the blank lines.

awk 'NF'

44. Separate each lines by only one blank line.

 awk 'NF{print "";print}'

45.  Replace 3^rd & 4^th occurrence of a pattern by a string in every lines of a file.

awk ‘{$0=gensub(/regex/,”string”,3, gensub(/regex/,”string”,4))};{print}’

46. Print only the valid IP addresses & hosts from a file which contains host & IP address columns.

awk  '$1~/^([01]?[0-9]?[0-9]?|2[0-4][0-9]|25[0-5])\.([01]?[0-9]?[0-9]?|2[0-4][0-9]|25[0-5])\.([01]?[0-9]?[0-9]?|2[0-4][0-9]|25[0-5])\.([01]?[0-9]?[0-9]?|2[0-4][0-9]|25[0-5])$/'

47. Print the valid phone numbers from a file & format them like (dddd)-ddd-ddd

awk --re-interval '$1~/^[0-9]{10}$/{ $0=gensub(/([0-9]{4})([0-9]{3})([0-9]{3})/,"\(\\1\)-\\2-\\3",g,$1);print $0}' 2> /dev/null

48. Print the valid e-mail address from a file which contains e-mail address & username information.

awk --re-interval '$1~/[a-z_.A-Z0-9]+@[a-zA-Z]+(.[a-zA-Z]+){1,}/'

49.  How to check if a string is palindrome.

echo ""| awk '{ split($1,a,"");for(i=1;i<=length($1)/2;i++) if (a[length($1)-i+1] == a[i]) flag=0;else {flag=1;exit 1};if(flag) print "Not a palindrome"; else print "Its a palindrome"}'

50. How to get the last two word of each line.

awk 'NF<2 n="" nbsp="" printf="" s="" t="">
51. How to replace each fields with its equivalent md5sum ?
echo "xyc|jkjk|jkjkj" | awk -F\| 'BEGIN{OFS=FS}{ for(i=1;i<=NF;i++){  $i="'"`echo $i|md5sum`"'";gsub(/-|[ \t]+/,"",$i) };print }'    

52. Renaing the file names:
Example: Move all the file names having  'Hourly-Daily' to 'Hourly_Daily'
ls -t | grep -i 'Hourly-Daily' |  awk  '{orig=$NF;gsub("Hourly-Daily","Hourly_Daily",$NF);system("mv" orig “ “ $NF) }'