LINUX HACK 25

Hack 25. Cut Command

Cut command can be used to display only specific columns from a text file or other command outputs.

Following are some of the examples.

Display the 1^st  field (employee name) from a colon delimited file

$  cut  -d:  -f  1  names.txt

Emma  Thomas

Alex  Jason

Madison  Randy

Sanjay  Gupta

Nisha  Singh

Display 1^st  and 3^rd  field from a colon delimited file

$  cut  -d:  -f  1,3  names.txt

Emma  Thomas:Marketing

Alex  Jason:Sales

Madison  Randy:Product  Development

Sanjay  Gupta:Support

Nisha  Singh:Sales

Display only the first 8 characters of every line in a file

$  cut  -c  1-8  names.txt

Emma  Tho

Alex  Jas

Madison

Sanjay  G

Nisha  Si

Misc Cut command examples

o    cut -d: -f1 /etc/passwd Displays the unix login names for all the users in the system.

o    free | tr -s ' ' | sed '/^Mem/!d' | cut -d" " -f2 Displays the total memory available on the system.

LINUX HACK 24

Hack 24. Uniq Command

Uniq command is mostly used in combination with sort command, as uniq removes duplicates only from a sorted file. i.e In order for uniq to work, all the duplicate entries should be in the adjacent lines. Following are some common examples.

1. When you have an employee file with duplicate entries, you can do the following to remove duplicates.

$  sort  namesd.txt  |  uniq

$  sort  –u  namesd.txt

2. If you want to know how many lines are duplicates, do the following. The first field in the following examples indicates how many duplicates where found for that particular line. So, in this example the lines beginning with Alex and Emma were found twice in the namesd.txt file.

$  sort  namesd.txt  |  uniq  –c

2  Alex  Jason:200:Sales

2  Emma  Thomas:100:Marketing

1  Madison  Randy:300:Product  Development

1  Nisha  Singh:500:Sales

1  Sanjay  Gupta:400:Support

3. The following displays only the entries that are duplicates. $ sort namesd.txt | uniq –cd

2  Alex  Jason:200:Sales

2  Emma  Thomas:100:Marketing

LINUX HACK 23

Hack 23. Sort Command



Sort command sorts the lines of a text file. Following are several practical examples on how to use the sort command based on the following sample text file that has employee information in the format:


employee_name:employee_id:department_name.



$ cat names.txt

Emma Thomas:100:Marketing


Alex Jason:200:Sales


Madison Randy:300:Product Development


Sanjay Gupta:400:Support


Nisha Singh:500:Sales


Sort a text file in ascending order



$ sort names.txt

Alex Jason:200:Sales


Emma Thomas:100:Marketing


Madison Randy:300:Product Development


Nisha Singh:500:Sales


Sanjay Gupta:400:Support



Sort a text file in descending order


$ sort -r names.txt

Sanjay Gupta:400:Support


Nisha Singh:500:Sales


Madison Randy:300:Product Development


Emma Thomas:100:Marketing


Alex Jason:200:Sales



Sort a colon delimited text file on 2nd field (employee_id)


$ sort -t: -k 2 names.txt

Emma Thomas:100:Marketing


Alex Jason:200:Sales


Madison Randy:300:Product Development


Sanjay Gupta:400:Support


Nisha Singh:500:Sales



Sort a tab delimited text file on 3rd field (department_name) and suppress duplicates


$ sort -t: -u -k 3 names.txt

Emma Thomas:100:Marketing

Madison Randy:300:Product Development


Alex Jason:200:Sales


Sanjay Gupta:400:Support


Sort the passwd file by the 3rd field (numeric userid)


$ sort -t: -k 3n /etc/passwd | more

root:x:0:0:root:/root:/bin/bash bin:x:1:1:bin:/bin:/sbin/nologin daemon:x:2:2:daemon:/sbin:/sbin/nologin adm:x:3:4:adm:/var/adm:/sbin/nologin lp:x:4:7:lp:/var/spool/lpd:/sbin/nologin


Sort /etc/hosts file by ip-addres


$ sort -t . -k 1,1n -k 2,2n -k 3,3n -k 4,4n /etc/hosts

127.0.0.1 localhost.localdomain localhost 192.168.100.101 dev-db.thegeekstuff.com dev-db 192.168.100.102 prod-db.thegeekstuff.com prod-db 192.168.101.20 dev-web.thegeekstuff.com dev-web 192.168.101.21 prod-web.thegeekstuff.com prod-web


Combine sort with other commands



o ps –ef | sort : Sort the output of process list


o ls -al | sort +4n : List the files in the ascending order of the file-size. i.e sorted by 5th filed and displaying smallest files first.


ls -al | sort +4nr : List the files in the descending order of the file-size. i.e sorted by 5th filed and displaying largest files first.

LINUX HACK 22

Hack 22. Xargs Command

xargs is a very powerful command that takes output of a command and pass it as argument of another command. Following are some practical examples on how to use xargs effectively.

1. When you are trying to delete too many files using rm, you may get error message: /bin/rm Argument list too long – Linux. Use xargs to avoid this problem.

find  ~  -name  ‘*.log’  -print0  |  xargs  -0  rm  -f

2. Get a list of all the *.conf file under /etc/. There are different ways to get the same result. Following example is only to demonstrate the use of xargs. The output of the find command in this example is passed to the ls –l one by one using xargs.

#  find  /etc  -name  "*.conf"  |  xargs  ls  –l

4.  If you have a file with list of URLs that you would like to download, you can use xargs as shown below.

#  cat  url-list.txt  |  xargs  wget  –c

5.  Find out all the jpg images and archive it.

#  find / -name *.jpg -type f -print | xargs tar -cvzf images.tar.gz

6.  Copy all the images to an external hard-drive.

#  ls *.jpg | xargs -n1 -i cp {} /external-hard-drive/directory

LINUX HACK 21

Hack 21. Change the Case

Convert a file to all upper-case:

$  cat  employee.txt

100  Jason  Smith

200  John  Doe

300  Sanjay  Gupta

400  Ashok  Sharma

$  tr  a-z  A-Z  <  employee.txt

100  JASON  SMITH

200  JOHN  DOE

300  SANJAY  GUPTA

400  ASHOK  SHARMA

Convert a file to all lower-case:

$  cat  department.txt

100  FINANCE

200  MARKETING

300  PRODUCT  DEVELOPMENT

400  SALES

$  tr  A-Z  a-z  <  department.txt

100  finance

200  marketing

300  product  development

400  sales

LINUX HACK 20

Hack 20. Join Command

Join command combines lines from two files based on a common field.

In the example below, we have two files – employee.txt and salary.txt. Both have employee-id as common field. So, we can use join command to combine

the data from these two files using employee-id as shown below.

 

$  cat  employee.txt

100  Jason  Smith

200  John  Doe

300  Sanjay  Gupta

400  Ashok  Sharma

$  cat  bonus.txt

100  $5,000

200  $500

300  $3,000

400  $1,250

$  join  employee.txt  bonus.txt

100  Jason  Smith  $5,000

200  John  Doe  $500

300  Sanjay  Gupta  $3,000

400  Ashok  Sharma  $1,250