UNIX Filters

What is this?

You can think of a UNIX filter as a biological creature. It has three important components: a mouth (standard input), a cloaca (standard output), and an alimentary canal that connects the two. It hoovers up data in its mouth, transforms it in the alimentary canal, and excretes it via it cloaca.

Here is an exmple. The program cat puts a file to the screen. You just say cat some_file and the contents of the file are put to the screen. When you give it a file that file goes in its "mouth" and it by default goes to the screen (the place it gets deposited). This filter actually doesn't filter at all! You can even cat an entire glob of files.

Another is grep, which examines a file line by line and prints out all the lines containing a string you give it as an argument.

These filters can be chained together using the pipe mechanism and the output can easily be dumped into a file. You can use a single file or a whole glob of files as input.

Outline

The prerequisite to this workshop is the workshop on the file system and the BASH shell.

Participants will also get a PDF text which addresses the topics shown below.

• Where do I put this crap? Redirecting stdout into a file. We don't always want 10,000 lines spewing to our terminal screen.
• cat and its options
• You can make heads and tail tails of this.
• You can view this file more or less, but less is more
• Using grep to dig for textual nuggets
• Specifying textual patterns with regular expressions
• Side by Side with Sondheim: Using paste to paste files side by side
• Sorting
  • Alphabetically and Asciicographically
  • Numerically
• I am afraid you are redundant! Use uniq to weed out duplicate lines in a file
• Grab those columns Using awk to extract columular data