#+HTML_HEAD:

Table of Contents

1 Shell Script Examples

Shell script is powerful if you could remember all the weird syntax. Here is the example to show how to use Array in Shell script. Note: readarray only works in Bash 4.x, check your version: bash –version

  1. Array in Shell script 1

    string='Paris, France, Europe';
    echo $string
    readarray -td, array <<<"$string"; declare -p array;
    ## declare -a array=([0]="Paris" [1]=" France" [2]=$' Europe\n')
    
    country='Paris/Frence/Europe/Sweden/Croatia';
    echo $country
    readarray -td/ array <<<"$country"; declare -p array;
    for item in "${array[@]}" 
    do
      echo "$item" 
    done      
    
  2. count number of argument

    echo "$#"
    var="string"
    
  3. array in bash

    arr=(                            
    "dog"                            
    "cat"                            
    "rat"                            
    )                                
    for var in ${arr[*]}             
    do                               
    echo "var=$var"                  
    done                             
    
  4. if statement, one liner

    var="$#" && if [ "$var" -eq 2 ]; then echo "two arguments"; else echo "not two argument"; fi
    
  5. Iterate all files with one line shell command

    for name in $(ls *); do echo $name; done
    
  6. String matches

    for name in $(ls *); do if [[ $name == *"txt"* ]]; then echo $name; else echo ""; fi 
    
  7. Use regex match =~

    if [[ $name =~ "txt" ]]; then echo $name; else echo ""; fi
    
  8. Repeat shell command with watch If you want to get constantly update information or stream data shell commands then you can use watch repeats your commands For Exmple, I want to monitor my system disk usage

    watch df -lah  # default 2 seconds interval
    

    watch shell commands with pipe, e.g. ps aux | grep --color httpd with double quotes

    watch "ps aux | grep --color httpd"
    

2 grep Examples

  1. grep content of file with specified file name extension, show full file name

    grep --include="*.java" -Hnris pattern $PWD
    
  2. cat and grep

    cat /tmp/file.txt | grep string
    
  3. grep with color

    cat /tmp/file.txt | grep --color string  
    
  4. grep print NUM line of trailing context file after matching lines, two line after

    cat file | grep -A 'dog'
    
  5. grep print NUM lines of leading context file before matching lines, two line before

    cat file | grep -B 'dog'
    
  6. grep inversion

    # find all non java file  
    grep -v -E '.java$'       
    
  7. grep string that contains "txt", grep matches string

    if echo $name | grep -q "txt"; then echo $name; else echo ""; fi  
    
  8. grep string its suffix is "txt", e.g. "file.txt"

    if echo $name | grep -q "txt$" then echo $name; else echo ""; fi  
    
  9. grep string its prefix is "file", e.g. "file.txt"

    # grep -q => no output                                           
    if echo $name | grep -q "^file" then echo $name; echo echo ""; fi
    
  10. grep in POSIX basic and extended regular expression

    # basic regex                                                    
    ps aux | grep --color 'wai\|filewatcher'                         
    # extended regex                                                 
    ps aux | grep --color -E 'wai|filewatcher'                       
    

3 sed Example

// list file from line 3 to 9
sed -n 3,9p file.txt 

4 sed does not work on MacOS, sort of

  • sed on MacOS is BSD version, it does not implement regular expression extension: GNU regex Extension
  • following characters classes works for GNU sed

    \w [a-bA-Z_] word echo "dog cat" | sed 's/\w/X/g' => "XXX XXX"
    \s   whitespace echo "dog cat" | sed 's/\s/X/g' => "dogXcat"
    \b   match a word boundary echo "dog cat" | sed 's/\b/X/g' => "XdogX XcatX"
    \W   non whitespace echo "dog cat" | sed 's/\W/X/g' => "dogXcat"
  • gsed can be installed on your MacOS which is GNU sed.
# does not work on MacOS or FreeBSD
echo "dog cat " | sed -E 's/\s*//g'  => "dog cat "
echo "dog cat " | gsed   's/\s*//g'  => "dogcat"
# it seems gsed does not need flag -E

5 rsync examples

  • rysnc file with specified extensions such png, svg etct.

    • include and exclude files.
    # rsync svg, png and jpg files excluding other files.
    rsync -rvzu --include='*'{svg,png,jpf} --exclude "*"  source  dest
    # rsync html file excluding other files.
    rsync -rvzu --include='*.html' --exclude "*"  source  dest
    # rsync folder
    rsync -artv folder  source dest
    
    # rsync local folder with remote folder
    rsync -artv local_folder user@xfido.com:/var/www/html
    

6 If and Else in Shell Script

String comparision Description
Str1 = Str2 Return true if strings are equal.
Str1 != Str2 Return true if string are NOT equal
-n Str Return true if string is NOT null
-z Str Return true if string IS null
Numeric Comparision Description
expr1 -eq expr2 Return true if expressions are equal
expr1 -ne expr2 Return true if expressions are NOT equal
expr1 -gt expr2 Return true if expr1 is greater than expr2
expr1 -lt expr2 Return true if expr1 is less than expr2
expr1 -ge expr2 Return true if expr2 is greater than or equal to expr2
expr1 -le expr2 Return true if expr2 is less than or equal to expr2
File Conditionals Description
-d file Return true if file is a directory
-e file Return true if file exists
-f file Return true if the provided string is a file.
-g file Return true if the group id is set on a file.
-r file Return true if file is readable.
-s file Return true if file is a non-zero size.
-u Return true if the user id is set on a file.
-w Return true if the file is writable.
-x Return true if the file is executable.

7 Execute remote shell script Execute remote shell

wget -O - https://bitbucket.org/zsurface/publicfile/raw/46cc8426ac08e2d575b60312fdc01f6ab6e3656c/test1.sh | bash

8 Check whether a command is in bash

command -v curl
if [ $? ]; then echo "curl exists"; else echo "curl does not exist"; fi

9 FreeBSD install ports without prompt make install clean BATCH=yes

  • ports installation is not working.
cd /usr/ports/redis && make install clean BATCH=yes

10 Install Redis On FreeBSD

pkg install -y redis

11 Enable Redis on FreeBSD

# add redis_enable="YES" to /etc/rc.conf

12 Run Redis on FreeBSD

/usr/local/etc/rc.d/redis start

13 How to avoid all the ssh-agent and ssh-add BS.

  • Add follwoing two lines to your ~/.profile, it works in FreeBSD.

    eval $(ssh-agent -s)
    ssh-add ~/.ssh/myprivate_id_rsa
    

14 Here are all the public keys:

15 Add global shared profile under /etc/profile in FreeBSD and Ubuntu.

16 In Haskell, if check whether a file is modified or not, use modificationTime

  • If the path is directory, the time will be changed if file is added or deleted.
  • See filewatcher.hs or watchDir in AronModule.hs
modificationTime :: FileStatus -> EpochTime

17 Execute remote shell command

ssh user@xfido.com  'cd user; ls;'

Author: cat

Created: 2019-08-04 Sun 13:08

Validate