[1] Check if a String Starts with Substring or a Wildcard Pattern

Reproducing from Advanced Bash-Scripting Guide -> Chapter 7. Tests.

The == comparison operator behaves differently within a double-brackets test than within single brackets.

[[ $a == z* ]]   # True if $a starts with an "z" (pattern matching).
[[ $a == "z*" ]] # True if $a is equal to z* (literal matching).

eg.,

% cat -n compare.sh
     1	#!/bin/bash
     2	
     3	echo "Wildcard matching"
     4	
     5	for name in rats star
     6	do
     7		[[ ${name} == st* ]] && echo ${name} starts with "st" \
     8	        	             || echo ${name} does not start with "st"
     9	done
    10	
    11	echo
    12	echo "Literal matching"
    13	
    14	for name in st* star
    15	do
    16		[[ ${name} == "st*" ]] && echo ${name} matches with "st*" \
    17	        	             || echo ${name} does not match with "st*"
    18	done

% ./compare.sh
Wildcard matching
rats does not start with st
star starts with st

Literal matching
st* matches with st*
star does not match with st*

[2] Floating-Point Arithmetic in Bash

Bash doesn't understand or support floating-point arithmetic; and assumes numbers containing a decimal point as strings. Following simple calculation shows incorrect result.

% let a=1/2
% echo $a
0

Rely on other tools such as bc, dc (calculators), sed, awk, python, perl, .. for floating-point arithmetic in bash.

eg.,

# bc
% bc <<< 'scale=2; 1/2'
.50

% a=$(bc <<< 'scale=2; 1/2')
% echo $a
.50

# perl
% echo print 1/2 | perl
0.5

# python
% echo 'print(1/2)' | python3
0.5

[3] Floating-Point Comparison in Bash

bash man page makes it clear that arithmetic evaluation is done in fixed-width integers so decimal/floating-point numbers need to rely on other tools such as bc for arithmetic evaluation (see previous tip).

Following examples highlight the problem with floating-point evaluation in bash. bash didn't have a problem with 1 < 2 comparision involving fixed-width integers but failed with an error when decimal numbers are involved in a similar comparison.

% echo $((1 > 2))
0

% echo $((1 < 2))
1

% echo $((1.0 < 2.0))
-bash: 1.0 < 2.0: syntax error: invalid arithmetic operator (error token is ".0 < 2.0")

Workaround or alternative is to rely on other tools. Following example performs the floating-point comparison with the help of bc (basic calculator).

% echo "(1.0 < 2.0)" | bc -l
1

% echo "(1.0 > 2.0)" | bc -l
0

[4] Multiline or Block Comments in Bash

I guess few examples are sufficient for this one.

# Example 1

: <<'ABC'
comment
comment
comment
ABC

# Example 2

: <<'EOF'
comment
comment
EOF

# Example 3

: '
comment
comment
'

# Example 4

<< '////'
comment
comment
////

Note that any line that starts with a colon (:) is a no-op. All the text between the delimiter identifier strings in heredoc will be redirected to a no-op command ("ABC" and "EOF" are delimiter identifiers in above examples 1 and 2). The space between colon and << is important and the string in quotes is equally important to stop the shell from expanding the variables and/or evaluating them.

(Source: stackoverflow)

Labels: Shell Scripting Tips UNIX Linux

Posted by: Giri Mandalika.    # 9:39 PM Tweet

[1] Check if the content of file X is in another file Y

One option is to rely on comm utility to compare common lines in both files. comm requires both files to be sorted.

comm utility produces three text columns as output -> lines found only in file1, lines found only in file2 and lines found in both files. In order to check the existence of file X's content in file Y, simply run comm command with sorted conent of both files and suppress first two columns of output. Then simply match the line count from the output with the line count of file X.

eg.,

$ cat /tmp/fileX
Oh, yes I can make it now the pain is gone
All of the bad feelings have disappeared

$ cat /tmp/fileY
Gone are the dark clouds that had me blind
It's gonna be a bright bright
Bright bright sunshiny day
It's gonna be a bright bright
Bright bright sunshiny day
Oh, yes I can make it now the pain is gone
All of the bad feelings have disappeared
Here is that rainbow I've been praying for
It's gonna be a bright bright

$ comm -12 <(sort -u fileX) <(sort -u fileY)
All of the bad feelings have disappeared
Oh, yes I can make it now the pain is gone

$ comm -12 <(sort -u fileX) <(sort -u fileY) | wc -l
       2	<== number of lines common in both files

$ wc -l fileX
       2 fileX	<== numer of lines in source file

In above example, the numer of lines in source file match with the number of common lines in both files - so, we can assume that the content of file X is found in file Y.

To be clear, this is not really fool-proof but may work in majority of cases.

<(command) syntax invokes process substitution. Perhaps it is a topic for another post.

[2] Checking shell script syntax without running it

Specify -n option. No commands will be executed in this "noexec" mode.

bash -n <script>
sh -n <script>

eg.,

$ cat -n hello.sh 
     1	#!/bin/bash
     2	echo 'Hi Hola
     3	echo "Ni Hao Namaste"

Syntax Check

$ bash -n hello.sh 
hello.sh: line 2: unexpected EOF while looking for matching `''
hello.sh: line 4: syntax error: unexpected end of file

Execute to confirm

$ ./hello.sh 
./hello.sh: line 2: unexpected EOF while looking for matching `''
./hello.sh: line 4: syntax error: unexpected end of file

Labels: Shell Scripting Tips UNIX Linux

Posted by: Giri Mandalika.    # 8:23 PM Tweet

Extracting the filename and extension from an absolute path

eg.,

Input: abspath="/var/tmp/localuserinfo.out"

Filename with extension:

$ basename ${abspath}
localuserinfo.out

Extension:

$ basename ${abspath##*.}
        -OR-
$ echo ${abspath##*.}
out

${variable##pattern} trims the longest match from the beginning of a string. Likewise ${variable#pattern} trims the shortest match from the beginning of a string.

Filename without extension:

$ basename ${abspath%.*}
localuserinfo

${variable%pattern} trims the shortest match from the end of a string. Similarly ${variable%%pattern} trims the longest match from the end of a string.

Ref:

• Bash Reference Manual -> Shell Parameter Expansion -> ${parameter##word}
• bash String Manipulations by Jim Dennis

Replacing commas with white space in a string

eg.,

Input: dummystr="Python,C,Java"

To replace all matches:

$ echo ${dummystr//,/ }
Python C Java

To replace only the first match:

$ echo ${dummystr/,/ }
Python C,Java

The difference is the extra '/' in pattern in first expression. If pattern begins with '/', all matches of pattern are replaced with string. Otherwise, only the first match is replaced.

Ref: Bash Reference Manual -> Shell Parameter Expansion -> ${parameter/pattern/string}

Sending array as argument to a shell function

Expand array before passing as argument to a function in a shell script. Sending the array name only passes the first element of the array.

Since $@ holds all arguments passed to the function, access the array argument with the help of $@ in the function definition. Using variables $1, $2, .., $n gives access to individual elements in the array.

$ cat numbers.sh 

#!/bin/bash

# print_array() .. prints all elements as expected
function print_array() {
 int_array=("$@")
 for elem in "${int_array[@]}"
 do
  echo $elem
 done
}

# print_array2() .. prints only one element .. first one in the array
function print_array2() {
 int_array=($1)
 for elem in "${int_array[@]}"
 do
  echo $elem
 done
}

numbers=(1 2 3 4)
echo "Attempting to print numbers array .."
print_array ${numbers[@]}
echo
echo "Attempt #2 with a different function .."
print_array2 ${numbers[@]}
echo
echo "Attempt #3 with only the array name as argument to print_array function .."
# following call prints only the first element of the array
print_array ${numbers}

$ ./numbers.sh 
Attempting to print numbers array ..
1
2
3
4

Attempt #2 with a different function ..
1

Attempt #3 with only the array name as argument to print_array function ..
1

Labels: Shell Scripting Tips UNIX Linux

Posted by: Giri Mandalika.    # 11:23 PM Tweet

Length of a String Stored in a Variable

${#VARIABLE}

$ FRUIT="Orange"

$ echo ${#FRUIT}
6

In bash, variables that store values are known as parameters.

Remove First n Characters from a String

${VARIABLE:n}

$ PARAM="Quick Brown Fox"

$ echo ${PARAM:6}
Brown Fox

Remove Last n Characters from a String

${VARIABLE::-n}

$ PARAM="Quick Brown Fox"

$ echo ${PARAM::-6}
Quick Bro

Convert a String to Upper Case

${VARIABLE} | tr '[:lower:]' '[:upper:]' eg.,

$ NAME='Giri Mandalika'

$ echo ${NAME} | tr '[:lower:]' '[:upper:]'
GIRI MANDALIKA

Convert a String to Lower Case

${VARIABLE} | tr '[:upper:]' '[:lower:]' eg.,

$ NAME='Giri Mandalika'

$ echo ${NAME} | tr '[:upper:]' '[:lower:]'
giri mandalika

Labels: Shell Scripting Tips UNIX Linux

Posted by: Giri Mandalika.    # 9:22 PM Tweet