Chapter 1. Using Unix

Table of Contents

1.1. KISS: Keep It Simple, Stupid
1.1.1. Practice
1.2. What is Unix?
1.2.1. Aw, man... I Have to Learn Another System?
1.2.2. Operating System or Religion?
1.2.3. The Unix Standard API
1.2.4. Shake Out the Bugs
1.2.5. The Unix Standard UI
1.2.6. Fast, Stable and Secure
1.2.7. Sharing Resources
1.2.8. Practice
1.3. Unix User Interfaces
1.3.1. Graphical User Interfaces (GUIs)
1.3.2. X11 on Mac OS X
1.3.3. Command Line Interfaces (CLIs): Unix Shells
1.3.4. Terminals
1.3.5. Basic Shell Use
1.3.6. Practice
1.4. Still Need Windows? Don't Panic!
1.4.1. Cygwin: Try This First
1.4.2. Windows Subsystem for Linux: Another Compatibility Layer
1.4.3. Practice
1.5. Logging In Remotely
1.5.1. Unix to Unix
1.5.2. Windows to Unix
1.5.3. Terminal Types
1.5.4. Practice
1.6. Unix Command Basics
1.6.1. Practice
1.7. Basic Shell Tools
1.7.1. Common Unix Shells
1.7.2. Command History
1.7.3. Auto-completion
1.7.4. Command-line Editing
1.7.5. Globbing (File Specifications)
1.7.6. Practice
1.8. Processes
1.8.1. Practice
1.9. The Unix File System
1.9.1. Unix Files
1.9.2. File system Organization
1.9.3. Ownership and Permissions
1.9.4. Practice
1.10. Unix Commands and the Shell
1.10.1. Internal Commands
1.10.2. External Commands
1.10.3. Getting Help
1.10.4. A Basic Set of Unix Commands
1.10.5. Practice
1.11. POSIX and Extensions
1.11.1. Practice
1.12. Subshells
1.12.1. Practice
1.13. Redirection and Pipes
1.13.1. Device Independence
1.13.2. Redirection
1.13.3. Special Files in /dev
1.13.4. Pipes
1.13.5. Misusing Pipes
1.13.6. Practice
1.14. Power Tools for Data Processing
1.14.1. Introduction
1.14.2. grep
1.14.3. awk
1.14.4. cut
1.14.5. sed
1.14.6. sort
1.14.7. tr
1.14.8. find
1.14.9. xargs
1.14.10. bc
1.14.11. tar
1.14.12. gzip, bzip2, xz
1.14.13. zip, unzip
1.14.14. time
1.14.15. top
1.14.16. iostat
1.14.17. netstat
1.14.18. iftop
1.14.19. curl, fetch, wget
1.14.20. Practice
1.15. File Transfer
1.15.1. File Transfers from Unix
1.16. Environment Variables
1.16.1. Self-test
1.17. Shell Variables
1.17.1. Self-test
1.18. Process Control
1.18.1. External Commands
1.18.2. Special Key Combinations
1.18.3. Internal Shell Commands and Symbols
1.18.4. Self-test
1.19. Remote Graphics
1.19.1. Configuration Steps Common to all Operating Systems
1.19.2. Graphical Programs on Windows with Cygwin
1.20. Where to Learn More

Before You Begin

If you think the word "Unix" refers to Sumerian servants specially "trained" to guard a harem, you've come to the right place. This chapter is designed as a tutorial for users with little or no Unix experience.

If you are following this guide as part of an ungraded workshop, please feel free to work together on the exercises in this text. It would be very helpful if experienced users could assist less experienced users during the "practice breaks" in order to keep the class moving forward and avoid leaving anyone behind.

1.1. KISS: Keep It Simple, Stupid

Most people make most things far more complicated than they need to be. Engineers and scientists, especially so.

Aside

To the engineer, all matter in the universe can be placed into one of two categories:

  1. Things that need to be fixed

  2. Things that will need to be fixed after I've had a few minutes to play with them

Engineers like to solve problems. If there are no problems available, they will create their own problems. Normal people don't understand this concept; they believe that if it ain't broke, don't fix it. Engineers believe that if it ain't broke, it doesn't have enough features yet.

No engineer can look at a television remote control without wondering what it would take to turn it into a stun gun. No engineer can take a shower without wondering whether some sort of Teflon coating would make showering unnecessary. To the engineer, the world is a toy box full of sub-optimized and feature-poor toys.

-- The Engineer Identification Test (Anonymous)

Avoid people who tend to look for the most "sophisticated" solution to a problem. Those who look for the simplest solution are more productive, more rational, and more fun to have a beer with. For more amusement on the subject, look up the story of the king's toaster.

Simplicity is the ultimate sophistication. We achieve more when we make things simple for ourselves. We achieve less when we make things complicated. Most people choose the latter. Complexity is the product of carelessness or ego, and simplicity is the product of a wisdom and clarity of thought.

The original Unix designers were an example of wisdom and clarity. Unix is designed to be as simple and elegant as possible. Some things may not seem intuitive at first, but this is probably because the first idea you came up with is not as elegant as the Unix way. The Unix developers had the wisdom to constantly look for simpler ways to implement solutions instead going with what seemed intuitive at first glance. Learning the Unix way will therefore make you a wiser and happier computer user. I speak from experience.

Unix is not hard to learn. You may have gotten the impression that it's a complicated system meant for geniuses while listening to geniuses talk about it. Don't let them fool you, though. The genius ego compels every genius to make things sound really hard, so you'll think they're smarter than you.

Another challenge with learning anything these days is filtering out the noise on the Internet. Most tutorials on any given subject are incomplete and contain misinformation or bad advice. As a result, new users are often led in the wrong direction and hit a dead end before long. One of the goals of this guide is a show a simple, sustainable, portable, and expandable approach to using Unix systems. This will reduce your learning curve by an order of magnitude.

Most researchers don't know enough about Unix. As a result, their productivity suffers dramatically. Unix has grown immensely since it was created, but the reality is, you don't need to know a lot in order to use Unix effectively. You can become more sophisticated over time if you want, but most Unix users don't really need to. It may be better to stick to the KISS principal (Keep It Simple, Stupid) and focus on learning to use the basic tools well rather than a learning a huge collection of tools and using them poorly. It's quality vs quantity. Knowledge is not wisdom. Wisdom is knowing how to apply it effectively.

Aside

Einstein was once asked how many feet are in a mile. His reply: "I don't know. Why should I fill my brain with facts I can find in two minutes in any standard reference book?"

Many martial arts students like to collect "forms" (choreographed sequences of moves), for the sake of bragging rights. Knowing more forms does not improve one's Kung Fu, however. The term Kung Fu essentially means "skill". A master is someone who can demonstrate mastery of a few forms, not knowledge of many. This depth of understanding does far more for both self-defense capability and personal development than a shallow knowledge of many forms. Develop your Unix Kung Fu in the same way. Aim to become a master rather than an encyclopedia.

Unix is designed to be as simple as possible and to allow you to work as fast as possible, by staying out of your way. Many other systems will slow you down by requiring you to use cumbersome user interfaces or spend time learning new proprietary methods. As you become a master of Unix, your productivity will be limited only by the speed of the hardware and programs you run.

If something is proving difficult to do under Unix, you're probably going about it the wrong way. There is almost always an easier way, and if there isn't, then you probably shouldn't be trying to do what you're trying to do. If it were a wise thing to do, some Unix developer would have invented an elegant solution by now. Adapt to the wisdom of those who traveled this road before you, and life will become simpler.

1.1.1. Practice

  1. What's the engineer's motto regarding things that ain't broke?

  2. Why do many people believe that Unix is hard to learn?

  3. Is it better to accumulate vast amounts of knowledge or to become highly skilled using the fundamentals? Why?

  4. What is the core principle of Unix design? Explain.