Friday, September 17, 2021

Example of scripts and solution

Practice Exercises:

Exercise 1:

Write a shell script that prints "Shell Scripting is Fun!" on the screen.

Hint 1:  Remember to make the shell script executable with the chmod command.

Hint 2:  Remember to start your script with a shebang !

Exercise 2:

Modify the shell script from exercise 1 to include a variable.  The variable will hold the contents of the message "Shell Scripting is Fun!".

Exercise 3:

Store the output of the command "hostname" in a variable.  Display "This script is running on _______." where "_______" is the output of the "hostname" command.

Hint:  It's a best practice to use the ${VARIABLE} syntax if there is text or characters that directly precede or follow the variable.

Exercise 4:

Write a shell script to check to see if the file "/etc/shadow" exists.  If it does exist, display "Shadow passwords are enabled."  Next, check to see if you can write to the file.  If you can, display "You have permissions to edit /etc/shadow."  If you cannot, display "You do NOT have permissions to edit /etc/shadow."

Exercise 5:

Write a shell script that displays "man", "bear", "pig", "dog", "cat", and "sheep" on the screen with each appearing on[…]”


Excerpt From: Cannon, Jason. “Shell Scripting: How to Automate Command Line Tasks Using Bash Scripting and Shell Programming”. Apple Books. 



“Solutions to the Practice Exercises

Exercise 1:

#!/bin/bash

echo "Shell Scripting is Fun!"

Exercise 2:

#!/bin/bash

MESSAGE="Shell Scripting is Fun!"

echo "$MESSAGE"

Exercise 3:

#!/bin/bash

HOST_NAME=$(hostname)

echo "This script is running on ${HOST_NAME}."

Exercise 4:

#!/bin/bash

FILE="/etc/shadow"

if [ -e "$FILE" ]

then

  echo "Shadow passwords are enabled."

fi

if [ -w "$FILE" ]

then

  echo "You have permissions to edit ${FILE}."

else

  echo "You do NOT have permissions to edit ${FILE}."

fi

Exercise 5:

#!/bin/bash

for ANIMAL in man bear pig dog cat sheep

do

“ echo "$ANIMAL"

done

Exercise 6:

#!/bin/bash

read -p "Enter the path to a file or a directory: " FILE

if [ -f "$FILE" ]

then

  echo "$FILE is a regular file."

elif [ -d "$FILE" ]

then

  echo "$FILE is a directory."

else

  echo "$FILE is something other than a regular file or directory."

fi

ls -l $FILE

Exercise 7:

#!/bin/bash

FILE=$1

if [ -f "$FILE" ]

then

  echo "$FILE is a regular file."

elif [ -d "$FILE" ]

then

  echo "$FILE is a directory."

else

  echo "$FILE is something other than a regular file or directory."

fi

ls -l $FILE

Exercise 8:

#!/bin/bash

for FILE in $@

do

  if [ -f "$FILE" ]

  then

    echo "$FILE is a regular file."

  elif [ -d "$FILE" ]

  then

    echo "$FILE is a directory."

  else

    echo "$FILE is something other than a regular file or directory."

  fi

  ls -l $FILE

done”


Excerpt From: Cannon, Jason. “Shell Scripting: How to Automate Command Line Tasks Using Bash Scripting and Shell Programming”. Apple Books. 




Thursday, September 16, 2021

Why you should do shell scripting and what is The Shebang: ?


Why you should do shell scripting :

Whether you have you perform the same task, again and again, every day and it is boring, right?  “For example, if there is a slight chance that you will have to perform the same set of commands again, to make it easier you should try to create a shell script then and there you have to do it again.  
When I need to do that task again, I could execute that script.  If maintenance needs to be performed on a system at 3:00 in the morning, we can write a script that does the required work and schedule a job to run it.”



SHELL SCRIPTING, in brief:- 


  • A script is a command-line program that contains a series of commands. 
  • The commands contained in the script are executed by an interpreter. 
  • In the case of shell scripts, the shell acts as the interpreter and executes the commands listed in the script one after the other.
  • Anything you can execute at the command line, you can put into a shell script.  
  • Shell scripts are great at automating tasks.  
  • If you find yourself running a series of commands to accomplish a given task and will need to perform that task again in the future, you can—and probably should—create a shell script for that task.

A simple shell script:-


The name of this script is the script.sh  

#!/bin/bash
echo "Hello, this is my first script and it's awesome!"

Before you try to execute the script, make sure that it is executable

$ chmod 755 script1.sh
$ chmod  +x script1.sh

Here is what happens when you execute the script.
$ ./script.sh

The Shebang:

You'll notice that the first line of the script starts with #! followed by the path to the bash shell program, /bin/bash. 

The number sign is very similar to the sharp sign used in music notations - “Also, some people refer to the exclamation mark as a "bang."  So, #! can be spoken as "sharp bang."  The term "Shebang" is an inexact contraction of "sharp bang."
When a script's first line starts with a shebang, what follows is used as the interpreter for the commands listed in the script.  Here are three examples of shell scripts, each using a different shell program as the interpreter.”

- Excerpt From Cannon, Jason. “Shell Scripting: How to Automate Command Line Tasks Using Bash Scripting and Shell Programming”. Apple Books.  

Launch GCP instance for the first time

 


  Launch GCP instance for the first time 





Tuesday, May 19, 2020

uniq Command

uniq removes duplicate consecutive lines in a text file and is useful for simplifying the text display.

Because uniq requires that the duplicate entries must be consecutive, one often runs sort first and then pipes the output into uniq; if sort is used with the -u option, it can do all this in one step.

To remove duplicate entries from multiple files at once, use the following command:

1
sort file1 file2 | uniq > file3

or

1
sort -u file1 file2 > file3

To count the number of duplicate entries, use the following command:

1
uniq -c filename

Sunday, May 17, 2020

zcat, zless, zdiff and zgrep.

When working with compressed files, many standard commands cannot be used directly. For many commonly-used file and text manipulation programs, there is also a version especially designed to work directly with compressed files. These associated utilities have the letter "z" prefixed to their name. For example, we have utility programs such as zcatzlesszdiff and zgrep.

Here is a table listing some z family commands:

CommandDescription
$ zcat compressed-file.txt.gzTo view a compressed file
$ zless somefile.gz or $ zmore somefile.gzTo page through a compressed file
$ zgrep -i less somefile.gzTo search inside a compressed file
$ zdiff file1.txt.gz file2.txt.gzTo compare two compressed files

Note that if you run zless on an uncompressed file, it will still work and ignore the decompression stage. There are also equivalent utility programs for other compression methods besides gzip.

Let's Learn "cat" the most frequently used Linux command line utilities.

cat is short for concatenate and is one of the most frequently used Linux command line utilities. It is often used to read and print files, as well as for simply viewing file contents. To view a file, use the following command:

1
$ cat <filename>

For example, cat readme.txt will display the contents of readme.txt on the terminal. However, the main purpose of cat is often to combine (concatenate) multiple files together. You can perform the actions listed in the table using cat.

The tac command (cat spelled backwards) prints the lines of a file in reverse order. Each line remains the same, but the order of lines is inverted. The syntax of tac is exactly the same as for cat, as in:

1
2
$ tac file
$ tac file1 file2 > newfile
CommandUsage
cat file1 file2Concatenate multiple files and display the output; i.e. the entire content of the first file is followed by that of the second file
cat file1 file2 > newfileCombine multiple files and save the output into a new file
cat file >> existingfileAppend a file to the end of an existing file
cat > fileAny subsequent lines typed will go into the file, until Ctrl-D is typed
cat >> fileAny subsequent lines are appended to the file, until Ctrl-D is typed






Saturday, July 21, 2018

Introduction to Process Management - Linux


Linux, in general, is a fairly stable system. Occasionally, things do go wrong however and sometimes we also wish to tweak the running of the system to better suit our needs. 

In this section, we will take a brief look at how we may manage programs or processes on a Linux system. So what are they?

A program is a series of instructions that tell the computer what to do. When we run a program, those instructions are copied into memory and space is allocated for variables and other stuff required to manage its execution. This running instance of a program is called a process and it's processed which we manage.

What is Currently Running?

Linux, like most modern OS's, is a multitasking operating system. This means that many processes can be running at the same time. As well as the processes we are running, there may be other users on the system also running stuff and the OS itself will usually also be running various processes which it uses to manage everything in general. If we would like to get a snapshot of what is currently happening on the system we may use a program called top.
top

Below is a simplified version of what you should see when you run this program.
1. top
2. Tasks: 174 total, 3 running, 171 sleeping, 0 stopped
3. KiB Mem: 4050604 total, 3114428 used, 936176 free
4. Kib Swap: 2104476 total, 18132 used, 2086344 free
5.  
6. PID USER %CPU %MEM COMMAND
7. 6978 ryan 3.0  21.2 firefox
8.   11 root 0.3   0.0 rcu_preempt
9. 6601 ryan 2.0   2.4 kwin
10. ...


Here are details 

Line 2 Tasks is just another name for processes. It's typical to have quite a few processes running on your system at any given time. Most of them will be system processes. Many of them will typically be sleeping. This is ok. It just means they are waiting until a particular event occurs, which they will then act upon.
Line 3 This is a breakdown of working memory (RAM). Don't worry if a large amount of your memory is used. Linux keeps recently used programs in memory to speed up performance if they are run again. If another process needs that memory, they can easily be cleared to accommodate this.
Line 4 This is a breakdown of Virtual memory on your system. If a large amount of this is in use, you may want to consider increasing its size. For most people with most modern systems having gigabytes of RAM, you shouldn't experience any issues here.
Lines 6 - 10 Finally is a listing of the most resource intensive processes on the system (in order of resource usage). This list will update in real time and so is interesting to watch to get an idea of what is happening on your system. The two important columns to consider are memory and CPU usage. If either of these is high for a particular process over a period of time, it may be worth looking into why this is so. The USER column shows who owns the process and the PID column identifies a process's Process ID which is a unique identifier for that process.
Top will give you a real-time view of the system and only show the number of processes which will fit on the screen. Another program to look at processes is called ps which stands for processes. In it's normal usage it will show you just the processes running in your current terminal (which is usually not very much). If we add the argument aux then it will show a complete system view which is a bit more helpful.

ps [aux]

It does give quite a bit of output so people usually pipe the output to grep to filter out just the data they are after. We will see in the next bit an example of this.
Killing a Crashed Process

It doesn't happen often, but when a program crashes, it can be quite annoying. Let's say we've got our browser running and all of a sudden it locks up. You try and close the window but nothing happens, it has become completely unresponsive. No worries, we can easily kill Firefox and then reopen it. To start off we need to identify the process id.

1. ps aux | grep 'firefox'
2. ryan 6978 8.8 23.5 2344096 945452 ? Sl 08:03 49:53 /usr/lib64/firefox/firefox


It is the number next to the owner of the process that is the PID (Process ID). We will use this to identify which process to kill. To do so we use a program which is appropriately called kill.
kill [signal] <PID>

1. kill 6978
2. ps aux | grep 'firefox'
3. ryan 6978 8.8 23.5 2344096 945452 ? Sl 08:03 49:53 /usr/lib64/firefox/firefox
.
Sometimes you are lucky and just running kill normally will get the process to stop and exit. When you do this kill sends the default signal ( 1 ) to the process which effectively asks the process nicely to quit. We always try this option first as a clean quit is the best option. Sometimes this does not work, however. In the example above we ran ps again and saw that the process was still running. No worries, we can run kill again but this time supply a signal of 9 which effectively means, go in with a sledgehammer and make sure the process is well and truly gone.

1. kill -9 6978
2. ps aux | grep 'firefox'

Normal users may only kill processes which they are the owner for. The root user on the system may kill anyone processes.


When a process crashes and locks up, it can lock up the entire desktop. If this happens there is still hope.

Linux actually runs several virtual consoles. Most of the time we only see console 7 which is the GUI but we can easily get to the others. If the GUI has locked up, and we are in luck, we can get to another console and kill the offending process from there. To switch between consoles you use the keyboard sequence CTRL + ALT + F<Console>. So CTRL + ALT F2 will get you to a console (if all goes well) where you can run the commands as above to identify process ids and kill them. Then CTRL + ALT F7 will get you back to the GUI to see if it has been fixed. The general approach is to keep killing processes until the lock up is fixed. Normally you can look for tell tale signs such as high CPU or Memory usage and start with those processes first. Sometimes this approach works, sometimes it doesn't and you need to restart the computer. Just depends how lucky you are.
Foreground and Background Jobs

Featured Post

Managing CA Certificates on Red Hat Linux 9: Understanding update-ca-trust extract

  Managing CA Certificates on RHEL9 RHEL8 OracleLinux9 OracleLinux8 In today's digital landscape, securing communications and verifying ...