Writing and Running Shell Scripts

A shell script is a simple text file and can be written using any Unix text editor. Some discussion of Unix text editors can be found in the section called “Text Editors”.

Caution

Recall from the section called “Unix vs. Windows Text Files” that Windows uses a slightly different text file format than Unix. Hence, editing Unix shell scripts in a Windows editor can be problematic. Users are advised to do all of their editing on a Unix machine rather than write programs and scripts on Windows and transfer them to Unix.

Shell scripts often contain very complex commands that are wider than a typical terminal window. A command can be continued on the next line by typing a backslash (\) immediately before pressing Enter. A backslash as the very last character on a line (not even white space may follow it) is known as a continuation character. This feature is included in all Unix shells and other languages such as Python.

printf "%s %s\n" "This command is too long to fit in a single 80-column" \
       "terminal window, so we break it up with a backslash.\n"
        

It's a good idea to name the script with a file name extension that matches the shell it uses. This just makes it easier to see which shell each of your script files use. Table 4.1, “Conventional script file name extensions” shows conventional file name extensions for the most common shells. However, if a script is to be installed into the PATH so that it can be used as a regular command, it is usually given a name with no extension. Most users would rather type "cleanup" than "cleanup.bash".

Caution

A common mistake is to use the wrong file name extension on a shell script, such as naming the file script.sh when it uses bash features. On some systems such as Redhat Enterprise Linux, sh is actually a link to bash, so this will work fine. However, this causes problems on systems where sh is not bash, such as BSD and Debian Linux. If your script uses bash features, it should have a ".bash" file name extension, not ".sh".

Table 4.1. Conventional script file name extensions

ShellExtension
Bourne Shell.sh
C shell.csh
Bourne Again Shell.bash
T shell.tcsh
Korn Shell.ksh
Z-shell.zsh

Like all programs, shell scripts should contain comments to explain what the commands in it are doing. In all Unix shells, anything from a '#' character to the end of a line is considered a comment and ignored by the shell.

# Print the name of the host running this script
hostname
        

Practice Break

Using your favorite text editor, enter the following text into a file called hello.sh.

  1. The first step is to create the file containing your script, using any text editor, such as nano:

    shell-prompt: nano hello.sh
                    

    Once in the text editor, add the following text to the file:

    printf "Hello!\n"
    printf "I am a script running on a computer called `hostname`\n"
                    

    After typing the above text into the script, save the file and exit the editor. If you are using nano, the menu at the bottom of the screen tells you how to save (write out, Ctrl+o) and exit (Ctrl+x).

  2. Once we've written a script, we need a way to run it. A shell script is simply input to a shell program. Like many Unix programs, shells take their input from the standard input by default. We could, therefore, use redirection to make it read the file via standard input:

    shell-prompt: sh < hello.sh
                    

    Shells can also take an input file as a data argument:

    shell-prompt: sh hello.sh
                    

However, Unix shells and other scripting languages provide a more convenient method of indicating what program should interpret them. If we add a special comment, called a shebang line to the top of the script file and make the file executable using chmod, the script can be executed like a Unix command. We can then simply type its name at the shell prompt, and another shell process will start up and run the commands in the script. If the directory containing such a script is included in $PATH, then the script can be run from any CWD just like ls, cp, etc.

The shebang line consists of the string "#!" followed by the full path name of the command that should be used to execute the script, or the path /usr/bin/env followed by the name of the command. For example, both of the following are valid ways to indicate a Bourne shell (sh) script, since /bin/sh is the Bourne shell command.

#!/bin/sh
        
#!/usr/bin/env sh
        

When you run a script simply by typing its file name at the Unix command-line, a new shell process is created to interpret the commands in the script. The shebang line specifies which program is invoked for the new shell process that runs the script.

Note

The shebang line must begin at the very first character of the script file. There cannot even be blank lines above it or white space to the left of it. The "#!" is an example of a magic number. Many files begin with a 16-bit (2-character) code to indicate the type of the file. The "#!" indicates that the file contains some sort of interpreted language program, and the characters that follow will indicate where to find the interpreter.

The /usr/bin/env method is used for add-on shells and other interpreters, such as Bourne again shell (bash), Korn shell (ksh), and Perl (perl). These interpreters may be installed in different directories on different Unix systems. For example, bash is typically found in /bin/bash on Linux systems, /usr/local/bin/bash on FreeBSD systems, /usr/pkg/bin/bash on NetBSD, and /usr/bin/bash on SunOS. The T shell is found in /bin/tcsh on FreeBSD and CentOS Linux and in /usr/bin/tcsh on Ubuntu Linux.

In addition, users of Redhat Enterprise Linux (RHEL) and derivatives may want to install a newer version of bash under a different prefix, using pkgsrc or another add-on package manager. RHEL is a special kind of Linux distribution built on an older snapshot of Fedora Linux for the sake of long-term binary compatibility and stability. As such, it comes with older versions of bash and other common tools.

The env command is found in /usr/bin/env on virtually all Unix systems. Hence, this provides a method for writing shell scripts that are portable across Unix systems (i.e. they don't need to be modified to run on different Unix systems).

Note

Every script or program should be tested on more than one platform (e.g. BSD, Cygwin, Linux, Mac OS X, etc.) immediately, in order to shake out bugs before they cause problems.

The fact that a program works fine on one operating system and CPU does not mean that it's free of bugs.

By testing it on other operating systems, other hardware types, and with other compilers or interpreters, you will usually expose bugs that will seem obvious in hindsight.

As a result, the software will be more likely to work properly when time is critical, such as when there is an imminent deadline approaching and no time to start over from the beginning after fixing bugs. Encountering software bugs at times like these is very stressful and usually easily avoided by testing the code on multiple platforms in advance.

Bourne shell (sh) is present and installed in /bin on all Unix-compatible systems, so it's safe to hard-code #!/bin/sh is the shebang line.

C shell (csh) is not included with all systems, but is virtually always in /bin if present, so it is generally safe to use #!/bin/csh as well.

For all other interpreters it's best to use #!/usr/bin/env.

#!/bin/sh           (OK and preferred)
        
#!/bin/csh          (Usually OK)
        
#!/bin/bash         (Bad idea: Not portable)
        
#!/usr/bin/perl     (Bad idea: Not portable)
        
#!/usr/bin/python   (Bad idea: Not portable)
        
#!/bin/tcsh         (Bad idea: Not portable)
        
#!/usr/bin/env bash     (This is portable)
        
#!/usr/bin/env tcsh     (This is portable)
        
#!/usr/bin/env perl     (This is portable)
        
#!/usr/bin/env python   (This is portable)
        

By default, a shell script will continue to run after one of the commands in the script fails. We usually do not want this behavior. We can tell the script to exit on errors by adding a -e to the command line:

#!/bin/sh -e
#!/bin/csh -e
        

Unfortunately, we cannot pass arguments to the interpreter following env in the shebang line.

#!/usr/bin/env bash -e      # This will fail
        

With Bourne family shells, we can add command line options after the fact using the internal set command. We can also disable any command line option by changing the '-' to a '+'. It may seem counterintuitive to enable something with '-' and disable it with '+', but it is what it is.

#!/usr/bin/env bash

# Set exit-on-error as if bash had been run with -e
set -e

# Disable exit-on-error
set +e
        

Example 4.1. A Simple Bourne Shell Script

Suppose we want to write a script that is always executed by bash, the Bourne Shell. We simply need to add a shebang line indicating the path name of the bash executable file.

shell-prompt: nano hello.sh
            

Enter the following text in the editor. Then save the file and exit back to the shell prompt.

#!/bin/sh -e

# A simple command in a shell script
printf "Hello, world!\n"
            

Now, make the file executable and run it:

shell-prompt: chmod a+rx hello.sh   # Make the script executable
shell-prompt: ./hello.sh            # Run the script as a command
            

Example 4.2. A Simple C-shell Script

Similarly, we might want to write a script that is always executed by csh, C Shell. We simply need to add a shebang line indicating the path name of the csh executable file.

shell-prompt: nano hello.csh
            
#!/bin/csh -e

# A simple command in a shell script
printf "Hello, world!\n"
            
shell-prompt: chmod a+rx hello.csh  # Make the script executable
shell-prompt: ./hello.csh           # Run the script as a command
            

Note

The shebang line in a script is ignored when you explicitly run a shell and provide the script name as an argument or via redirection. The content of the script will be interpreted by the shell that you have manually invoked, regardless of what the shebang line says.

# This might not work, since sh will not recognize
# C shell syntax.
shell-prompt: sh hello.csh
        

Scripts that you create and intend to use regularly can be placed in your PATH, so that you can run them from anywhere. A common practice among Unix users is to create a directory called ~/bin, and configure the login environment so that this directory is always in the PATH. Programs and scripts placed in this directory can then be used like any other Unix command, without typing the full path name.

You can greatly speed up the script development process by using an Integrated Development Environment, or IDE, instead of a simple text editor. Using an IDE, such as APE, eliminates the need to exit the editor (or use another shell window) to run the script. In APE, we can simply press F5 or type Esc followed by 'r' to run the script. When it finishes, we are still in the editor at the same spot in the script. An IDE is also specialized for writing programs, and hence provides features such as syntax colorization, edit macros, etc.

shell-prompt: ape hello.sh
        

Figure 4.1. Editing a script in APE

Editing a script in APE

Practice

Note

Be sure to thoroughly review the instructions in Section 2, “Practice Problem Instructions” before doing the practice problems below.
  1. Is it wise to write Unix shell scripts in a Windows editor and then upload them to a Unix system? Why or why not?

  2. How can we keep very long commands tidy in a script?

  3. What rules does Unix enforce regarding file name extensions on shell scripts?

  4. What problems might arise if we write a bash script and give it a ".sh" file name extension?

  5. Why are comments important in shell scripts?

  6. Show three ways to run a Bourne shell script called analysis.sh and any requirements they entail.

  7. What shebang line should be used for Bourne shell scripts? For Bourne again shell scripts? For Python scripts? Explain.

  8. How to we make a shell script exit immediately when any command fails?

  9. What happens if we run a script as an argument to a shell command that does not match the intended shell, such as csh analysis.sh?

  10. What are the advantages of using an IDE instead of a simple text editor like nano?