Unix scripting notes

Collection of everyday snippets of things I find myself googling for way too often.

shell

Shebang

#!/bin/bash

Write something into a file

 echo "some data for the file" >> fileName.txt

Go through every directory in a directory

for d in */ ; do
     echo "$d"
done

Check if file exists

FILE=/etc/resolv.conf
if [ -f "$FILE" ]; then
    echo "$FILE exists."
fi

Go through every line in a file

while IFS="" read -r p || [ -n "$p" ]
do
  printf '%s\n' "$p"
done < peptides.txt

Print out current time:

now=$(date +"%T")
echo "Current time : $now"

Output:

Current time : 13:31:55

For loop one liner

for i in *; do echo $i; done

Increment a variable

var=$((var+1))

If line exists in a file

if grep -q "${tikarray[$vidrandom]}" playlog.txt;
    then
        #something
    else
        #something else
    fi

Bash Shell Find Out If a Variable Is Empty Or Not

if [ -z "$var" ]
then
      echo "\$var is empty"
else
      echo "\$var is NOT empty"
fi

Replace unwanted characters with _ in string

sed -r 's/[<>:"/\|?* ]+/_/g'

wget

Wget output into a file foo.html

wget -O foo.html google.com

Wget user agent

wget -U "User Agent Here" "http://website.url.here"

example google chrome

wget -U "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.36" "http://website.url.here"

List of possible user agents:
https://www.networkinghowtos.com/howto/common-user-agent-list/

Get webpage, save into file and grep that

wget "https://www.nettiauto.com/bmw/i3?id_vehicle_type=1&id_country[]=73" -qO temp.txt && cat temp.txt | grep totAdInList

Output:
<h2 class="ma0">Vaihtoautohaun tulokset: <span class="totAdInList">66</span> vaihtoautoa.</h2>

Output into variable

RESULT=$(wget -qO- http://example.com)
echo $RESULT

grepping and formatting

grep only lines that say FOO

grep FOO

Replace THING with a newline

sed 's/THING/\n/g'

Remove everything before FOO and replace with BAR

sed 's/^.*FOO/BAR/'

Take only the first column

awk '{print $1}'

Remove file extensions

loadfile=${loadfile%%.*}

Remove four first lines

tail -n +4 file.txt

Remove first three characters

cut -c 3-

Only take the first three characters

cut -c -3

Divide by “/” and only take the first field.

cut -f1 -d"/"

Leave a Reply

Your email address will not be published. Required fields are marked *

Post comment