1.7. Basic Shell Tools

1.7.1. Common Unix Shells

There are many different shells available for Unix systems. This might sound daunting if you're new to Unix, but fortunately, like most Unix tools, all the common shells adhere to certain standards. All of the common shells are derived from one of two early ancestors:

  • Bourne shell (sh) is the de facto basic shell on all Unix systems, and is derived from the original Unix shell developed at AT&T.
  • C shell (csh) offers mostly the same features as Bourne shell, but the two differ in the syntax of their scripting languages, which are discussed in Chapter 2, Unix Shell Scripting. The C shell syntax is designed to be more intuitive and similar to the C language.

Most Unix commands are exactly the same regardless of which shell you are using. Differences will only become apparent when using more advanced command features or writing shell scripts, both of which we will cover later.

Common shells derived from Bourne shell include the following:

  • Almquist shell (ash), used as the Bourne shell on some BSD systems.
  • Korn shell (ksh), an extended Bourne shell with many added features for user-friendliness.
  • Bourne again shell (bash) another extended Bourne shell from the GNU project with many added features for user-friendliness. Used as the Bourne shell on some Linux systems.
  • Debian Almquist shell (dash), a reincarnation of ash which is used as the Bourne shell on Debian based Linux systems.

Common shells derived from C shell include the following:

  • T shell (tcsh), and extended C shell with many added features for user-friendliness.
  • Hamilton C shell, an extended C shell used primarily on Microsoft Windows.

Unix systems differ in which shells are included in the base installation, but most shells can be easily added to any Unix system using the system's package manager.

1.7.2. Command History

Most shells remember a configurable number of recent commands. This command history is saved both in memory and to disk, so that you can still recall this session's commands next time you log in. The exact mechanisms for recalling those commands varies from shell to shell, but some of the features common to all shells are described below.

Most modern shells support scrolling through recent commands by using the up-arrow and down-arrow keys. Only very early shells like lack this capability.

Note

This feature may not work if your TERM variable is not set properly, since the arrow keys send magic sequences that may differ among terminal types.

The history command lists the commands that the shell currently has in memory.

shell-prompt: history
            

A command consisting of an exclamation point (!) followed by any character string causes the shell to search for the most recently executed command that began with that string. This is particularly useful when you want to repeat a complicated command.

shell-prompt: find Programs -name '*.o' -exec rm -i '{}' \;
shell-prompt: !find
            

An exclamation point followed by a number runs the command with that history index:

shell-prompt: history
   385  13;42   more output.txt
   386  13:54   ls
   387  13:54   cat /etc/hosts
shell-prompt: !386
ls
Avi-admin/            Materials-Studio/     iperf-bsd
Backup@               New-cluster/          notes
Books/                Peregrine-admin/      octave-workspace
            

Tantalizing sneak preview: We can check the history for a particular pattern such as "find" as follows:

shell-prompt: history | grep find
            

More on the "| find" in Section 1.13, “Redirection and Pipes”.

1.7.3. Auto-completion

In most Unix shells, you need only type enough of a command or argument filename to uniquely identify it. At that point, pressing the TAB key will automatically fill in the rest for you. Try the following:

shell-prompt: touch sample.txt
shell-prompt: cat sam<Press the TAB key now>
            

If there are other files in your directory that begin with "sam", you may need to type a few additional characters before the TAB, like 'p' and 'l' before auto-completion will work.

1.7.4. Command-line Editing

Modern shells allow extensive editing of the command currently being entered. The key bindings for different editing features depend on the shell you are using and the current settings. Some shells offer a selection of different key bindings that correspond to Unix editors such as vi or Emacs.

See the documentation for your shell for full details. Below are some example default key bindings for shells such as bash and tcsh.

Table 1.3. Default Key Bindings in some Shells

KeyAction
Left arrowMove left
Right arrowMove right
Ctrl+aBeginning of line
Ctrl+eEnd of line
Backspace or Ctrl+hDelete left
Ctrl+dDelete current

1.7.5. Globbing (File Specifications)

There is often a need to specify a large number of files as command line arguments. Typing all of them would be tedious, so Unix shells provide a mechanism called globbing that allows short, simple patterns to match many file names. This allows us to type a brief specification that represents a large number (a glob) of files.

These patterns are built from literal text and/or special symbols called wild cards as shown in Table 1.4, “Globbing Symbols”.

Table 1.4. Globbing Symbols

SymbolMatches
*Any sequence of characters (including none) except a '.' in the first character of the filename.
?Any single character, except a '.' in the first character of the filename.
[string]Any character in string
[c1-c2]Any character from c1 to c2, inclusive
{thing1,thing2}Thing1 or thing2

Normally, the shell handles these special characters, expanding globbing patterns to a list of matching file names before the command is executed.

If you want an argument containing special globbing characters to be sent to a command in its raw form, it must be enclosed in quotes, or each special character must be escaped (preceded by a backslash, \).

Certain commands, such as find need to receive the pattern as an argument and attempt to do the matching themselves rather than have it done for them by the shell. Therefore, patterns to be used by the find command must be enclosed in quotes.

shell-prompt: ls *.txt     # Lists all files ending in ".txt"
shell-prompt: ls "*.txt"   # Fails, unless there is a file called '*.txt'
shell-prompt: ls '*.txt'   # Fails, unless there is a file called '*.txt'
shell-prompt: ls .*.txt    # Lists hidden files ending in ".txt"
shell-prompt: ls [A-Za-z]* # Lists all files and directories
                           # whose name begins with a letter
shell-prompt: find . -name *.txt    # Fails
shell-prompt: find . -name '*.txt'  # List .txt files in all subdirectories
shell-prompt: ls *.{c,c++,f90}
            

Caution

The exact behavior of character ranges such as [A-Z] may be affected by locale environment variables such as LANG, LC_COLLATE, and LC_ALL. See Section 1.16, “Environment Variables” for general information about the environment. Information about locale and collation can be found online. Behavior depends on these settings as well as which Unix shell you are using and the shell's configuration settings. Setting LANG and the LC_ variables to C or C.UTF-8 will usually ensure the behavior described above.

1.7.6. Practice

Instructions

  1. Make sure you are using the latest version of this document.

  2. Carefully read one section of this document and casually read other material (such as corresponding sections in a textbook, if one exists) if needed.

  3. Try to answer the questions from that section. If you do not remember the answer, review the section to find it.

  4. Write the answer in your own words. Do not copy and paste. Verbalizing answers in your own words helps your memory and understanding. Copying does not, and demonstrates a lack of interest in learning.

  5. Check the answer key to make sure your answer is correct and complete.

    DO NOT LOOK AT THE ANSWER KEY BEFORE ANSWERING QUESTIONS TO THE VERY BEST OF YOUR ABILITY. In doing so, you would only cheat yourself out of an opportunity to learn and prepare for the quizzes and exams.

Important notes:

  • Show all your work. This will improve your understanding and ensure full credit for the homework.

  • The practice problems are designed to make you think about the topic, starting from basic concepts and progressing through real problem solving.

  • Try to verify your own results. In the working world, no one will be checking your work. It will be entirely up to you to ensure that it is done right the first time.

  • Start as early as possible to get your mind chewing on the questions, and do a little at a time. Using this approach, many answers will come to you seemingly without effort, while you're showering, walking the dog, etc.

  1. What is the de facto standard shell on Unix systems?

  2. How do most Unix commands differ when run under one shell such as C shell as opposed to running under another such as Bourne shell or Bourne again shell?

  3. What if a shell we like or need is not present on our Unix installation?

  4. How can we quickly rerun the previous command in most Unix shells?

  5. How can we list all recent commands executed from this shell?

  6. How can we run the last command that began with "ls"?

  7. Given the shell history shown below, how can we run the last command used to log into unixdev1?

       984  16:48   vi .ssh/known_hosts
       985  16:50   ssh -X bacon@unixdev1.ceas.uwm.edu
       986  16:52   ssh -X -C bacon@unixdev1.ceas.uwm.edu
       987  16:58   ape
       988  16:59   ssh -X -C bacon@unixdev1.ceas.uwm.edu
            
  8. How can we avoid typing a long file name or command name in most shells?

  9. How can we instantly move to the beginning of the command we are currently typing?

  10. How do we list all the non-hidden files in the current directory ending in ".txt"?

  11. How do we list all the hidden files in the current directory ending in ".txt"?

  12. How do we list all the files in /etc beginning with "hosts"?

  13. How do we list all the files in the current directory starting with a lower case letter and ending in ".txt"?

  14. How do we list all the files in the current directory starting with any letter and ending in ".txt"?

  15. How do we list all the non-hidden files in the current directory ending with ".pdf" or ".txt"?

  16. How do we list all the files in /etc and all other directories under /etc with names ending in ".conf"?

  17. When are globbing patterns normally expanded to a list of files?

  18. How can we include a file name in a command if the file name contains a special character such as '*' or '['?