Posts

Showing posts with the label UNIX

Remove duplicate successive line using uniq utility

With uniq utility you can remove duplicate line if they are within successive line. For example you have successive identical line in the file, then with uniq you can discard all but one of successive identical lines from the file. Consider you have following lines within with files. # vi student_data.txt Roll number is 024401 His Name is SHAIK His Name is SHAIK He is 24 years old. Roll number is 12345 Then using use uniq utility as below will yield following result. # uniq student_data.txt Roll number is 12345 His Name is SHAIK He is 24 years old. Note that within file there was two duplicate lines. One is, "Roll number is 12345" and another is "His Name is SHAIK". Using the "uniq" output only "His Name is SHAIK" line is omitted because they are successive identical lines. However "Roll number is 12345" text line is not removed because they are not successive though they are identical. So uniq utility is used to remove adjacent identic...

Edit file on linux using sed utility

With sed utility you can edit text inside file directly. It is not needed to open the file using any editor and then do editing task. sed is stream editor for filtering and transforming text. Below is my student_grade.txt # cat student_grade.txt 024401 4.98 024402 4.95 024403 4.95 024404 4.50 024405 4.95 Now using sed utility we will replace first few digits of "student_id" that is 0244 by "Roll:". The syntax of using sed utility is, sed {expression} {file} Now using sed utility we want to replace "0244" with "Roll:" # sed '/0244/s//Roll:/g' student_grade.txt Roll:01 4.98 Roll:02 4.95 Roll:03 4.95 Roll:04 4.50 Roll:05 4.95 Let's now understand about the command. within single quote, /0244 indicates search for string 024434. /s means substitute or replace work. //Roll: means replace the word "0244" by "Roll:" /g means make the changes global

Data manipulation using awk utility in linux

With awk utility you can scan a pattern within file record and then process the record as you want. Suppose you have the student_grade.txt like below. First field is student id and second field is grade. Both field is separated by tab delimiter. # cat student_grade.txt 024401 4.98 024402 4.95 024403 4.95 024404 4.50 024405 4.95 Now we want to find out those student id whose grade is 4.95. With awk utility we can do this. We will search each record for the grade 4.95 and then print the 1st field. The syntax of usage awk utiity is, awk 'pattern_action' {file_name} Now we can extract the student id whose grade is 4.95 by, # awk '/4.95/{print $1}' student_grade.txt 024402 024403 024405 It will search for 4.95 within each record of file student_grade.txt and then by command "print $1" it will print first field. Meta characters used in awk To search for a pattern in awr you can use various meta characters. The list of meta characters along with their meaning is...

Translate or replace characters using tr utility

With tr command you can translate letter from uppercase to lowercase and vice-versa. In other word, with tr command you can replace a letter by another letter. The syntax for using tr command is, tr {source_pattern} {destination_pattern} Each letter in source_pattern is replaced by corresponding letter in destination_pattern. For example if you write tr "a6" "7y" then from the string or file every "a" will be replaced by "7", and every "6" will be replaced by "y". Let's see an example. My names.txt looks like below. # cat names.txt momin arju bony tany azmeri queen tasreen Now we want letter "o" will be replaced by number "0". Letter "i" will be replaced by number "1". Small letter "e" will be replaced by capital letter "E". # tr "oie" "01E" m0m1n arju b0ny tany azmEr1 quEEn tasrEEn We can also capitalize all letters inside names.txt with singl...

Join utility in linux

To merge line by lines(line 1 of file1 is merged to line 1 of file2 and etc) within two files. But the join utility is used to merge if there is common field in both file and if values are identical to each other. Join does not work line by line. It works with all the lines between file to search for identical values. A example will make you more clear. We have id_age.txt and id_dept.txt file and data are shown below. Both join and paste are shown below. # cat id_age.txt 024401 28 024402 26 024434 23 # cat id_dept.txt 024401 CIT 024434 CSE 024438 EEE # paste id_age.txt id_dept.txt 024401 28 024401 CIT 024402 26 024434 CSE 024434 23 024438 EEE # join id_age.txt id_dept.txt 024401 28 CIT 024434 23 CSE Note that the id 024434 is on the 2nd line of the id_dept.txt but the id 024434 is on the 3rd line of the id_age.txt and merge is done successfully.

getopts command in linux shell script

The getopts command in shell script is used to check the valid command line arguments passed into script. The syntax of using getopts inside shell script is, getopts {optsring} {variable_name} From the manual page, "optstring contains the option letters to be recognized; if a letter is followed by a colon, the option is expected to have an argument, which should be separated from it by white space. Each time it is invoked, getopts places the next option in the shell variable variable_name, When an option requires an argument, getopts places that argument into the variable OPTARG. On errors getopts diagnostic messages are printed when illegal options or missing option arguments are encountered. If an illegal option is seen, getopts places ? into variable_name." For example you have a shell script named student_info that would be run by, ./student_info -i 024434 -a 23 -d CIT -s male where student_info is the shell script name -i is used for the student id. -a is used for age. -...
File and Directory Navigation The " pwd " command displays the current directory: root> pwd /u01/app/oracle/product/9.2.0.1.0 The " ls " command lists all files and directories in the specified directory. If no location is defined it acts on the current directory. The "-a" flag lists hidden "." files. The "-l" flag lists file details. root> ls root> ls /u01 root> ls -al More Examples: ls -al | pg do a full directory listing and prompt to stop stuff whizzing off the page. ls | wc -l count the files in the current directory. ls -alt list files in date order ls -alt | head -10 as above but only display the first 10 ls -l $ORACLE_HOME/reports60/printer/admin/spoolcmd.sh Verify that the spoolcmd.sh file has execute permissions ls -s | awk '{if ($1 > 50) print $1 " " $2 }' list all files over 50 blocks in size. ls -alq List files with hidden characters. Very useful when you cannot delete a file f...