LinuxPlanet Blogs

Front-end old-style terminal interface, for web services like those provided by Google and Yahoo.

- Enter green commands without parameters to change default mode.
- Anything that’s not a command will search in current default mode.
- Aliases will expand to commands. Numbers will expand to corresponding search results.
- Use cursor up and down for command history.
- Enter keyword and hit the tab-key for tab-completion.
- Commands marked with * are experimental, use them with care and please report any bugs.

Site: http://goosh.org

Project site: http://code.google.com/p/goosh/

Ever since I started using Linux all those years ago I have always been on the look out for a different shell that would top bash. I had tried zsh and a few others but none of them really topped bash in a massive way. Recently a mate of mine told me about fish so I tried it out on my Gentoo dev box.

There are a lot of interesting features in fish most of them to do with the auto completion aspects.

The auto-completion of ls arguments is just a very basic example of how fish really improves on standard auto-complete functionality like in bash. As well as being able to complete the normal long arguments it can complete short ones as well plus giving an actual explanation to what the command does. I think this is absolutely great personally for two reasons. First of all if your new to Linux then it helps you learn. Second if your like me and your mind just falls apart some times as its to full other junk (mine being full of knowing every episode of The Simpsons in full word for word) its handy to get a bit of a prompt.

As well as the improvement of auto-completing arguments the auto-completion for programs is improved as well telling you what the program is where possible but at least telling you what the type of application or program is. The below screen shot is when listing a lot of programs which is output using a less type application if there is only a few it will be output like with the above ls auto-complete.

Next one of my personal favorite features of fish is the auto-completion of wildcards for example if I am at the top of my web directory and wanted to get the details of every PHP file in all the sub directories, I would need to do something like `find . -iname “*.php” -print0 | xargs -0 ls -la` while with fish you could just do `ls –la **.php` and get the same result.
Another example as follows is auto-completing on wild cards something that’s not possible with bash.

The last feature I would like to point out in this quick coverage of fish is a small one using the % symbol you can autocomplete for process IDs which can be really handy.

There are some more screen shots showing some other cool features on their website http://fishshell.org/screenshots.html and further detail on all the features on the documentation page http://fishshell.org/user_doc/index.html
Also if your interested checkout the possible features list http://fishshell.org/doc2/index.html#todo-possible think there is some cool ones coming up.

That’s it for now. I am going to continue using fish over the next few days then I will be putting together a bit of a cheat list which ill be posting up here, so if your interested check back soon.

GNU screen is a nice utility that allows running multiple interactive shells from the same terminal session and allows you to detach from your terminal while keeping those shells alive. Later on, you can re-attach to your background screen to get back to your shells. It has a lot more features like automatic session logging and terminal window splitting. You can discover them all in the manual.

How many times did you start a long-running task like gcc compilation on a remote server and then suddenly needed to disconnect from your shell? Maybe you just needed to move to some other place with your laptop, but if you disconnected from your LAN, your ssh connection would go down. How many times you thought “Damn, if I had launched screen before this…”?

The trick to save your compile time and not break your schedule is simple: just have your shell .profile script run screen at startup on your remote server. For bash, the syntax is simple, just add the following line at the end of your ˜/.profile script:

if [ ${SHLVL} -eq 1 ]; then
    ((SHLVL+=1)); export SHLVL
    exec screen -R -e "^Ee" ${SHELL} -l
fi

Quick implementation notes:

1. Parameter -R reattaches to an existing detached session, if it exists, otherwise creates a new one.
2. Parameter -e sets a non-standard escape character. This is useful since you don’t want login screen to interfere with other screens you may spawn during your activity. I chose Ctrl-E as it’s not used by other well-known keyboard shortcuts and works on most OSes.

To detach from your server type Ctrl-E d or just close your terminal window. Running processes will remain active in background, without detaching from your shell. When you connect to your remote shell again, you’ll get back to your session.

Do you like Unix tips like this? Follow me on Twitter or subscribe to my RSS feed for more.

Related posts:

1. Gentooize Part 1: colorize console
2. LUKS mermaids of remote unlock
3. Process management roundup/1

Background

Before my wife and I had kids we thought it would be nice to collect the United States quarters that were released during 1999 through 2008 to commemorate each of the 50 states. Seemed like something simple to do and would be a nice gift for the kids when they got older. So we bought a couple of the blue books which you can fill up with quarters as you find them. Each book contains 100 slots, 2 for each state. One slot is for the Philadelphia minted version of the quarter, and the other slot is for the Denver mint.

Problem

Well we ended up having 3 kids so we have to collect 300 quarters. The task of finding the quarters has been more of a dad task so when I have a chance, I’ll put a $5 dollar bill in various soda machines at work and go quarter fishing. This approach has been working fairly well and we’ve collected ~130 of the 300 quarters thus far.

However I’ve started getting to the point where I’m netting a lot of duplicates and the job of having to bring them home to weed through them is starting to get old. Having a list of which quarters we already have would sure be nice, so I could quickly nix any duplicates.

Solution

Of course I wanted a low tech solution, i.e. a piece of paper in my wallet would do the job, but how to do it?

The answer? A text file that I could maintain would suffice. No need for a bloated spreadsheet or some fancy handheld app. So I created a file, quarters.txt, like so:

1
2
3
4
5
6
7
8
9
10
11
12
13
14

State           Year  Cnt (P/D)
=====           ====  =========
Alabama         2003  2/1
Alaska          2008  3/1
Arizona         2008  3/0
Arkansas        2003  3/0
California      2005  3/0
Colorado        2006  3/0
Connecticut     1999  3/2
Delaware        1999  0/0
Florida         2004  2/0
Georgia         1999  2/1
Hawaii          2008  3/0
...

It’s about as simple a text file as you can get. 3 columns, State, Year, and Counts. The 3rd column shows how many P and D quarters I have for a given state. So for example, for Alabama, I’ve got 2 Philadelphia minted quarters, and 1 Denver minted.

So you’re probably wondering, “why the hell is this guy writing up this in a blog post?”

We’ll the interesting bit to this low tech solution is how I print this list out. For this task I make use of a pretty powerful UNIX command called enscript, which lets you do all kinds of nifty things to a text file to augment how it looks when it gets printed.

About the only thing enscript doesn’t do for you, is give you the ability to preview your text file prior to printing. To accomplish this bit, I made use of another powerful UNIX command called ps2pdf. This command will take a postscript file (ps) and convert it to a pdf file.

So putting all the pieces together I came up with the following command:

1

enscript --fancy-header -U 4 quarters.txt -o - | ps2pdf - quarters_sm.pdf

The first part of this command, will call enscript instructing it to convert the file quarters.txt, and print it to standard out -o -. The printout will include some fancy headers and enscript will print the text file out in what is called 4 UP. This means that 4 pages will be printed on a single piece of paper. You could also print the page out in 2 UP, 8 UP, etc. It only needs to be a power of 2. BTW, 2 and 4 are the most commonly used, 8 is pretty hard to read.

The second part of this command passes the postscript generated by enscript through a UNIX pipe which gets picked up by ps2pdf, and converts it into a PDF file, quarters_sm.pdf.

From here you can check what the page would look like using your favorite PDF viewer, such as evince or xpdf. Once you’re comfortable with the page you can actually print it out from the PDF reader, or via the command-line.

Here’s what the resulting PDF file looks like:

PDF file

Here’s a portion of the PDF file at a 150% of it’s original size:

screenshot of quarters_sm.pdf at 150%

Useful Links

• enscript printing guide

NOTE: For further details regarding my one-liner blog posts, check out my one-liner style guide primer.

Background

From time to time I need to move and/or copy a subset of files from one directory to another. I typically would use something like one of these to do the task:

1
2
3
4
5

#-- COPY
find . -type f -ctime -1 | xargs -I {} cp {} /some/other/directory
 
#-- MOVE
find . -type f -ctime -1 | xargs -I {} mv {} /some/other/directory

NOTE: The 1st command finds all the files in the current directory that are less than 24 hours old, and copies them to /some/other/directory. The 2nd command finds all the files in the current directory that are less than 24 hours old, and moves them to /some/other/directory.

But then I realized that by using xargs’ –I switch I was basically limiting xargs to doing a single file at a time. According to xargs’ man page, when you use the –I switch you’re implying the –x switch AND the –L 1 switch. The L switch is what tells xargs how many lines of input to process at a time, so we’re basically telling it to only handle one file at a time. This made no sense. I was unintentially limiting xargs’ ability to optimize the command-line. So I found a better way.

New Approach

By utilizing 2 little used switches (–t | ––target-directory) on cp and mv I could un-tie xargs’ hands.

copy

1
2
3
4
5

# long form
find . -type f -ctime -1 | xargs -0 cp --target-directory=/some/other/directory
 
# short form
find . -type f -ctime -1 | xargs -0 cp -t /some/other/directory

move

1
2
3
4
5

# long form
find . -type f -ctime -1 | xargs -0 mv --target-directory=/some/other/directory
 
# short form
find . -type f -ctime -1 | xargs -0 mv -t /some/other/directory

NOTE: For further details regarding my one-liner blog posts, check out my one-liner style guide primer.

Problem

Recently at my day job I’ve been having to go through some pretty old Bash scripts that I’ve basically inherited. As I’ve been going through them I’ve been seeing a lot of confusion as to the proper use of Bash’s export command. The major offense? Not really understand whether a particular variables needs to be exported, or not. So I thought I’d take a moment just to clarify when and when not to use export.

The export command has really only one true purpose. To mark and/or unmark variables (and functions) that you want to have automatically exported to environments of subsequently executed commands. So if you create a script that calls other commands, and you want to push variables into the environment of these commands, then you’ll want to use export.

Example #1 (without export)

For example, let’s say we have the following 2 scripts:

1
2
3
4
5
6
7

#!/bin/bash
# script #1: parent.bash
 
var1="this was set by the parent shell script"
echo "inside $0 script: $var1"
 
./child.bash

1
2
3
4

#!/bin/bash
# script #2: child.bash
 
echo "inside $0 script: $var1"

And when I run the script parent.bash I get this output:

1
2
3
4
5

# output from parent.bash & child.bash (without export)
 
% ./parent.bash
inside ./parent.bash script: this was set by the parent shell script
inside ./child.bash script:

Notice how the variable $var1, which was set in the parent.bash script, didn’t get displayed by the child.bash script? Now watch this example with the variable $var1 exported in the parent.bash script.

Example #2 (with export)

1
2
3
4
5
6
7

#!/bin/bash
# script #1: parent.bash
 
export var1="this was set by the parent shell script"
echo "inside $0 script: $var1"
 
./child.bash

1
2
3
4

#!/bin/bash
# script #2: child.bash
 
echo "inside $0 script: $var1"

And when we run parent.bash

1
2
3
4
5

# output from parent.bash & child.bash (with export)
 
% ./parent.bash
inside ./parent.bash script: this was set my the parent shell script
inside ./child.bash script: this was set my the parent shell script

Example #3 (un-exporting)

Export isn’t just a one trick pony. It can also unmark a previously exported variable.

1
2
3
4
5
6
7
8
9

#!/bin/bash
# script #1: parent.bash
 
export var1="this was set by the parent shell script"
echo "inside $0 script: $var1"
 
./child.bash
export -n var1
./child.bash

1
2
3
4

#!/bin/bash
# script #2: child.bash
 
echo "inside $0 script: $var1"

1
2
3
4
5
6

# output from parent.bash & child.bash (with a before/after export)
 
% ./parent.bash
inside ./parent.bash script: this was set by the parent shell script
inside ./child.bash script: this was set by the parent shell script
inside ./child.bash script:

Here we see the effects of the export -n on child.bash the 2nd time it’s called.

Which Variables are Flagged for Export?

You can use the command export -p to get a list of all the variables marked for export, like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

% export -p
declare -x CCACHE_DIR="/var/cache/ccache"
declare -x CCACHE_UMASK="002"
declare -x COLORTERM="gnome-terminal"
declare -x CVS_RSH="ssh"
declare -x DESKTOP_SESSION="gnome"
declare -x DISPLAY=":0.0"
...
...
declare -x SHELL="/bin/bash"
declare -x SHLVL="4"
declare -x TERM="xterm"
declare -x USER="root"
declare -x WINDOWID="77021004"
declare -x XMODIFIERS="@im=imsettings"
declare -x var1="this was set by the parent shell script"

What else?

There is one additional trick related to exporting variables, but it doesn’t make use of the export command. It uses Bash’s set command. This command allows you to automatically export ALL the variables that have been modified or created to the environment of subsequent commands.

Here’s a quick example:

1
2
3
4
5
6
7
8
9
10
11
12

#!/bin/bash
# script #1: parent.bash
 
# automatically export EVERYTHING
set -a
 
var1="this was set by the parent shell script"
var2="this was set by the parent shell too"
echo "inside $0 script: $var1"
echo "inside $0 script: $var2"
 
./child.bash

1
2
3
4
5

#!/bin/bash
# script #2: child.bash
 
echo "inside $0 script: $var1"
echo "inside $0 script: $var2"

Useful links

• Daniel Robbin’s 3 part Bash tutorial
• Bash Reference Manual

Background

I recently saw a post over on Linux Journal that discussed how to glean information about a system’s hard drive, such as its serial number, without having to actually open up the case and physically check it. So I thought I’d take the opportunity to write up a blog post with the specifics of how to do this under Fedora & CentOS, just so I’d have this info handy for future use.

BTW, I was able to accomplish this task several different ways, so this post will cover all the different ways that I could get this info.

Command #1: lshw

This is probably the best tool for getting at a system’s internals. First make sure it’s installed.

1

yum install lshw

For our example you would run the command lshw -class disk:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

% lshw -class disk
  *-disk                  
       description: ATA Disk
       product: HTS726060M9AT00
       vendor: Hitachi
       physical id: 0
       bus info: scsi@0:0.0.0
       logical name: /dev/sda
       version: MH4O
       serial: MRH403M4GS551Y
       size: 55GiB (60GB)
       capabilities: partitioned partitioned:dos
       configuration: ansiversion=5 signature=cccdcccd
  *-cdrom
       description: DVD reader
       product: UJDA755yDVD/CDRW
       vendor: MATSHITA
       physical id: 1
       bus info: scsi@1:0.0.0
       logical name: /dev/cdrom
       logical name: /dev/cdrw
       logical name: /dev/dvd
       logical name: /dev/scd0
       logical name: /dev/sr0
       version: 1.71
       capabilities: removable audio cd-r cd-rw dvd
       configuration: ansiversion=5 status=nodisc

The first section that’s returned is called -disk. Here’s you’ll see the vendor: Hitachi, the product number, HTS726060M9AT00, and my serial number: MRH403M4GS551Y.

Command #2: smartctl

The next tool that would give this type of info is called smartctl. It’s a tool that’s part of the smartmontool package. You may be familiar with the acronym S.M.A.R.T.. The acronym stands for: Self-Monitoring, Analysis, and Reporting Technology. This is a standard that most modern disks have in which vital statistics about a disk drive are provided through a standard API. Here’s how to install it.

1

yum install smartmontools

…and once installed you can use the bundled in tool smartctl like so:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

# smartctl example
 
% smartctl -i /dev/sda
smartctl version 5.38 [i386-redhat-linux-gnu] Copyright (C) 2002-8 Bruce Allen
Home page is http://smartmontools.sourceforge.net/
 
=== START OF INFORMATION SECTION ===
Model Family:     Hitachi Travelstar 7K60
Device Model:     HTS726060M9AT00
Serial Number:    MRH403M4GS551Y
Firmware Version: MH4OA6BA
User Capacity:    60,011,642,880 bytes
Device is:        In smartctl database [for details use: -P show]
ATA Version is:   6
ATA Standard is:  ATA/ATAPI-6 T13 1410D revision 3a
Local Time is:    Mon Sep 21 00:03:50 2009 EDT
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

Command #3: hdparm

Another way that I’ve used to get at hard drive meta data is with the command hdparm. This is probably the oldest way, at least that I’m familiar with, for getting at hard drive meta data. To install it:

1

yum install hdparm

Using it is simply:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

# hdparm example
 
% hdparm -i /dev/sda
 
/dev/sda:
 
 Model=HTS726060M9AT00                         , FwRev=MH4OA6BA, SerialNo=      MRH403M4GS551Y
 Config={ HardSect NotMFM HdSw>15uSec Fixed DTR>10Mbs }
 RawCHS=16383/16/63, TrkSize=0, SectSize=0, ECCbytes=4
 BuffType=DualPortCache, BuffSize=7877kB, MaxMultSect=16, MultSect=?0?
 CurCHS=16383/16/63, CurSects=16514064, LBA=yes, LBAsects=117210240
 IORDY=on/off, tPIO={min:240,w/IORDY:120}, tDMA={min:120,rec:120}
 PIO modes:  pio0 pio1 pio2 pio3 pio4 
 DMA modes:  mdma0 mdma1 mdma2 
 UDMA modes: udma0 udma1 udma2 udma3 udma4 *udma5 
 AdvancedPM=yes: mode=0xC0 (192) WriteCache=enabled
 Drive conforms to: ATA/ATAPI-6 T13 1410D revision 3a:  ATA/ATAPI-2,3,4,5,6
 
 * signifies the current active mode

Command #4: lsscsi

Here’s another tool, lsscsi, that I’ve used off and on to get at hard drive meta data. It’s probably the least known of all the tools mentioned here. Installation is the same as the others:

1

yum install lsscsi

It’s usage is pretty much in-line with the other commands too:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

# lsscsi example
 
% lsscsi -lll -vvv
 sysfsroot: /sys
[0:0:0:0]    disk    ATA      HTS726060M9AT00  MH4O  /dev/sda
  device_blocked=0
  iocounterbits=32
  iodone_cnt=0x92983
  ioerr_cnt=0x4f
  iorequest_cnt=0x92983
  queue_depth=1
  queue_type=none
  scsi_level=6
  state=running
  timeout=60
  type=0
  dir: /sys/bus/scsi/devices/0:0:0:0  [/sys/devices/pci0000:00/0000:00:1f.1/host0/target0:0:0/0:0:0:0]
[1:0:0:0]    cd/dvd  MATSHITA UJDA755yDVD/CDRW 1.71  /dev/sr0
  device_blocked=0
  iocounterbits=32
  iodone_cnt=0xa0f7
  ioerr_cnt=0x0
  iorequest_cnt=0x283a7
  queue_depth=1
  queue_type=none
  scsi_level=6
  state=running
  timeout=0
  type=5
  dir: /sys/bus/scsi/devices/1:0:0:0  [/sys/devices/pci0000:00/0000:00:1f.1/host1/target1:0:0/1:0:0:0]

Command #4: /dev/disk/by-id directory

Finally you can get the manufacturer’s product number & serial number can be had by going directly to the /dev directory, specifically here: /dev/disk/by-id. In the resulting output you’ll see 2 important substrings. The first, HTS726060M9AT00, is he product number while the second, MRH403M4GS551Y, is the serial number.

1
2
3
4
5
6
7
8
9

# /dev/disk/by-id example
 
% ls -1 /dev/disk/by-id
ata-HTS726060M9AT00_MRH403M4GS551Y
ata-HTS726060M9AT00_MRH403M4GS551Y-part1
ata-HTS726060M9AT00_MRH403M4GS551Y-part2
scsi-SATA_HTS726060M9AT00_MRH403M4GS551Y
scsi-SATA_HTS726060M9AT00_MRH403M4GS551Y-part1
scsi-SATA_HTS726060M9AT00_MRH403M4GS551Y-part2

NOTE: For further details regarding my one-liner blog posts, check out my one-liner style guide primer.

As it is with UNIX & Linux there is always another way. In my previous article [one-liner]: Filtering ps from ps, one reader, Christoph, mentioned an alternative method to the one I outlined. In this case, I would consider his to be a better way, so I thought I would take a second to demonstrate this alternative method. The alternative? Use the command pgrep.

The Original Approach

My original post offered the following one-liner:

1
2
3
4
5
6
7
8
9
10

% ps -eaf | grep "[h]ttpd"
root      2683     1  0  2008 ?        00:20:31 /usr/sbin/httpd
apache   17146  2683  0 Aug30 ?        00:00:02 /usr/sbin/httpd
apache   17147  2683  0 Aug30 ?        00:00:02 /usr/sbin/httpd
apache   17149  2683  0 Aug30 ?        00:00:02 /usr/sbin/httpd
apache   17150  2683  0 Aug30 ?        00:00:02 /usr/sbin/httpd
apache   17151  2683  0 Aug30 ?        00:00:02 /usr/sbin/httpd
apache   17152  2683  0 Aug30 ?        00:00:02 /usr/sbin/httpd
apache   17153  2683  0 Aug30 ?        00:00:02 /usr/sbin/httpd
apache   17154  2683  0 Aug30 ?        00:00:02 /usr/sbin/httpd

This one-liner provided a list of all the httpd processes running, while filtering out the actual string from the grep httpd command.

The Alternative Approach

By using the command pgrep, the same effect can be achieved and a lot more. For starters, you can get a list of all the httpd PIDs:

1
2
3
4
5
6
7
8
9

# list of httpd PIDs
 
% pgrep httpd
1608
7645
9739
10051
27712
27859

This could be useful in a shell script, if needed, to check for any running httpd processes. For example:

1
2
3
4

# test for httpd processes
 
% [ -z "`pgrep httpd`" ] || echo "running"
running

Here are some other examples:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

# list of PIDs with corresponding command name
 
% pgrep -l httpd
1608 httpd
7645 httpd
9739 httpd
10051 httpd
27712 httpd
27859 httpd
 
# list of PIDs with corresponding command name owned by user root
 
% pgrep -l -u root httpd
1608 httpd
 
# list of PIDs, separated with a comma delimiter
 
% pgrep -d, httpd
1608,7645,9739,14119,14162,27859
 
# detailed list of httpd PIDs via ps
# NOTE: $(...) runs the command above, returning the list of PIDs to ps
 
% ps -fp $(pgrep -d, httpd)
UID        PID  PPID  C STIME TTY          TIME CMD
root      1608     1  0 Aug03 ?        00:00:05 /usr/sbin/httpd
apache    7645  1608  0 Sep04 ?        00:00:47 /usr/sbin/httpd
apache    9739  1608  0 Sep04 ?        00:01:50 /usr/sbin/httpd
apache   14119  1608  0 Sep04 ?        00:00:13 /usr/sbin/httpd
apache   14162  1608  0 Sep04 ?        00:00:13 /usr/sbin/httpd
apache   27859  1608  0 Sep04 ?        00:07:19 /usr/sbin/httpd

Thanks again to Christoph for pointing out this alternative.

NOTE: For further details regarding my one-liner blog posts, check out my one-liner style guide primer.

Background

This is a pretty handy trick to know when you want to filter out the command you’re running, so that it’s not included in ps output. This proves handy when writing a shell script that needs to parse output from ps.

NOTE: The command ps, allows you to see all the processes being run on a UNIX/Linux system. You typically use it with the switches “-eaf” or “-ef”.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

# columns description:
# UID   = user who owns the process
# PID   = process #
# PPID  = parents' process #
# C     = 
# STIME = start time
# TTY   = terminal type (has to do with which shell command ran from)
# TIME  = system time its run
# CMD   = command (aka. program name)
 
% ps -eaf
UID        PID  PPID  C STIME TTY          TIME CMD
root         1     0  0  2008 ?        00:24:46 init [5]
root         2     0  0  2008 ?        00:00:00 [kthreadd]
root         3     2  0  2008 ?        00:01:58 [migration/0]
root         4     2  0  2008 ?        00:23:13 [ksoftirqd/0]
root         5     2  0  2008 ?        00:00:00 [watchdog/0]
root         6     2  0  2008 ?        00:02:35 [migration/1]
root         7     2  0  2008 ?        00:09:55 [ksoftirqd/1]
root         8     2  0  2008 ?        00:00:00 [watchdog/1]
root         9     2  0  2008 ?        00:06:36 [events/0]
root        10     2  0  2008 ?        00:06:16 [events/1]
root        11     2  0  2008 ?        00:00:10 [khelper]
root        54     2  0  2008 ?        00:19:26 [kblockd/0]

Problem

Here’s an example where we want to see if a program is running, so we grep the output of ps like so:

1
2
3
4
5
6
7
8
9
10
11

% ps -eaf | grep httpd
root      2683     1  0  2008 ?        00:20:31 /usr/sbin/httpd
user1    13188  3984  0 12:45 pts/1    00:00:00 grep httpd
apache   17146  2683  0 Aug30 ?        00:00:02 /usr/sbin/httpd
apache   17147  2683  0 Aug30 ?        00:00:02 /usr/sbin/httpd
apache   17149  2683  0 Aug30 ?        00:00:02 /usr/sbin/httpd
apache   17150  2683  0 Aug30 ?        00:00:02 /usr/sbin/httpd
apache   17151  2683  0 Aug30 ?        00:00:02 /usr/sbin/httpd
apache   17152  2683  0 Aug30 ?        00:00:02 /usr/sbin/httpd
apache   17153  2683  0 Aug30 ?        00:00:02 /usr/sbin/httpd
apache   17154  2683  0 Aug30 ?        00:00:02 /usr/sbin/httpd

The problem? Notice that a portion of our command, “grep httpd” is polluting our ps output. How can we get rid of it?

The Trick

You can alter what you are grepping for, without actually altering the results, by using a benign regular expression.

1
2
3
4
5
6
7
8
9
10

% ps -eaf | grep "[h]ttpd"
root      2683     1  0  2008 ?        00:20:31 /usr/sbin/httpd
apache   17146  2683  0 Aug30 ?        00:00:02 /usr/sbin/httpd
apache   17147  2683  0 Aug30 ?        00:00:02 /usr/sbin/httpd
apache   17149  2683  0 Aug30 ?        00:00:02 /usr/sbin/httpd
apache   17150  2683  0 Aug30 ?        00:00:02 /usr/sbin/httpd
apache   17151  2683  0 Aug30 ?        00:00:02 /usr/sbin/httpd
apache   17152  2683  0 Aug30 ?        00:00:02 /usr/sbin/httpd
apache   17153  2683  0 Aug30 ?        00:00:02 /usr/sbin/httpd
apache   17154  2683  0 Aug30 ?        00:00:02 /usr/sbin/httpd

Explanation

By re-encoding what we are looking for from httpd to [h]ttpd we’ve altered it by utilizing grep’s capability to use regular expressions. Regular expressions provide us the ability to use short-handed notations to represent patterns of text. Most people should be somewhat familiar with this capability when using the command ls to list files. The command ls can be used to find all the files that start with the letter a by utilizing a wildcard regular expression, aka. the asterisk character, aka. star, aka. *.

1
2
3
4
5
6
7
8
9
10
11
12

# wildcard examples
 
% ls
afile1  afile2  afile3  bfile1  bfile2  bfile3  cfile1  cfile2  cfile3
 
# example 1
% ls a*
afile1  afile2  afile3
 
# example 2
% ls b*
bfile1  bfile2  bfile3

Now lets say we want to create a single command that provides us with the list of ALL the files that start with an a or b. The bracket notation [ ] regular expression allows you to list sets of characters. So for example:

1
2
3
4
5
6
7
8
9
10
11
12

# set examples
 
% ls
afile1  afile2  afile3  bfile1  bfile2  bfile3  cfile1  cfile2  cfile3
 
# example 1
% ls [ab]*
afile1  afile2  afile3  bfile1  bfile2  bfile3
 
# example 2
% ls [bc]*
bfile1  bfile2  bfile3  cfile1  cfile2  cfile3

So back to our ps example. What does the bracket notation regular expression provide us here? This is the trick. By telling grep that we are looking for [h]ttpd, we’ve encoded our grep query to look like this: [h]ttpd, while what we’re actually grepping for to be: httpd. Since [h]ttpd doesn’t match httpd at the literal string level, grep will effectively ignore itself as being a match in the ps output.

A pretty neat [t]rick.

NOTE: For further details regarding my one-liner blog posts, check out my one-liner style guide primer.

Background

I recently worked a problem in a previous post where it wasn’t clear which version of the command nohup was getting used. The version that was built into the C-shell interpreter, or the executable sitting under /usr/bin/nohup. This brought up an interesting point, how do you know what commands are builtins to the shell itself?

Enter the commands: builtins and enable

These are 2 commands that I’d never even heard of until I started doing research for this post. They both basically do the same thing. They show you all the builtin commands for a particular shell, builtins for C-shell, and enable for Bourne Shell.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

# C-shell builtins
 
#NOTE: run this command within a C-shell (aka. csh or tcsh)
% csh
 
% builtins
:          @          alias      alloc      bg         bindkey    break
breaksw    builtins   case       cd         chdir      complete   continue
default    dirs       echo       echotc     else       end        endif
endsw      eval       exec       exit       fg         filetest   foreach
glob       goto       hashstat   history    hup        if         jobs
kill       limit      log        login      logout     ls-F       nice
nohup      notify     onintr     popd       printenv   pushd      rehash
repeat     sched      set        setenv     settc      setty      shift
source     stop       suspend    switch     telltc     termname   time
umask      unalias    uncomplete unhash     unlimit    unset      unsetenv
wait       where      which      while

1
2
3
4
5
6
7

# Bourne Again Shell builtins
 
#NOTE: run this command within Bourne Again Shell (aka. bash)
% bash
 
% enable | cut -d" " -f2 | sed -e :a -e '$!N;s/\n/ /;ta'
. : [ alias bg bind break builtin caller cd command compgen complete continue declare dirs disown echo enable eval exec exit export false fc fg getopts hash help history jobs kill let local logout popd printf pushd pwd read readonly return set shift shopt source suspend test times trap true type typeset ulimit umask unalias unset wait

NOTE: The command is enable. I’m simply running the output of enable through this cut & sed filter so that the output of enable is more concise for this example.

nohup

As you can see, by looking at the output from builtins, the command nohup is listed there, which means that C-shell has it’s own implementation of nohup. Looking at the output from enable, you can see that Bourne Shell, doesn’t include it’s own builtin for nohup, so it would use /usr/bin/nohup. Pretty simple, eh?

NOTE: For further details regarding my one-liner blog posts, check out my one-liner style guide primer.