So, I have been selected as one of the teaching assistants for Operating systems course and this time we had to give some examples of Named Pipes, demonstrating one form of Inter Process Communication aka IPC to the students. I had never used them before, but its fun knowing and working with them.

I hope all of us have used pipes on Linux systems. Here is a simple pipe example I often use to check if a process is running or not.

[shredder12]$ ps -e | grep firefox

The symbol "|" used here is called pipe(its mostly above the "Enter" key). Just like conventional pipes, UNIX pipes are used to transfer data between processes, aka Inter Process communication. They transfer one program's/process's output to another process as input, ps and grep in our case. The output of 'ps -e' is transfered through pipe as an input to grep. They are called "pipes" because of their FIFO(first in first out) nature. They can be seen as files but with the FIFO property. We will see the reason in a little while.

Difference between Pipes and Named Pipes

In above example, as soon as the message is passed and grep executes, the pipe will no longer exist, i.e. it is a non-persistent thing, gets removed as soon as the task is over. Named pipes, however are persistent, have a name and they exist even after the processes are terminated. I think I might have lost you people here, so lets see a simple named pipe example.

[shredder12]$ mkfifo mypipe

This command creates a pipe. Now, if you do ls -l, you will see something like this

So, as you can see, it has everything similar to a file, but the FIFO property. It can be deleted just like a simple file, using rm.

[shredder12]$ rm mypipe

Lets begin with the usage part. Try running this command in a terminal

[shredder12]$ ps -e > mypipe

This command should have returned to a prompt(thats what you would have expected) but apparently it seems to get hanged in the middle of nowhere, why? The answer is these pipes work exactly like a conventional water-pipe does, i.e. it needs both the end to be open, one serving as an input and the other as output. So, the reason why the above command hasn't returned a prompt is because only one end of the pipe is opened, for the input. Until there is a process at the other end of the pipe to receive the data, the transfer or the communication isn't complete and so the command doesn't terminate.

Now, with the above command running in a terminal, try to run the following in another

[shredder12]$ cat mypipe

Now you will actually see the output of 'ps -e'. And note that this means the communication is done and as a result, both the commands will terminate and return to prompt.

If you have understood the basic working of pipes then you might wonder if we can create an infinite loop using two of them. In a 1997 article of LJ, the writer mentioned this little trick. Using two commands, he hooked up the ouptut of one pipe to other's input and vice versa, thus generating a loop.

However, I couldn't make it work on my system. After running the command, I didn't see any cat processes consuming huge resources. May be I did something wrong or may be the article is too old to work on today's systems, probably some defensive technique in later kernels to escape such accidents.

Anyway, I hope this small introduction to named pipes has given you a new insight into the working of Unix and Linux sytems.

Those of you who have not been in such a situation would never know how it feels when you are stuck opening/deleting a file or directory. The more you have been working on the shell, the more it sucks. When I tried all I could to get it work and still ended up with the same error all the time, I just sat on my chair, staring at the monitor for a while, wondering if this is somekind of bash's loophole :P.

I was trying to grep through some pidgin facebook chats when I was stuck for a while trying to open a directory with the name starting with a hyphen("-"). I tried all I could think of, but I always ended with the same error.

[shredder12]$ cd -directory
bash: cd: -d: invalid option
cd: usage: cd [-L|-P] [dir]

[shredder12]$ cd \-directory

[shredder12]$ cd *directory

[shredder12]$ cd "-directory"

As you can see the trouble is that the alphabets after a "-", hyphen, are being considered as the command attributes/options but they are not, so what we need here is:

• a way to tell the command that its not an option
• or, make the command start with something other than a hyphen

The first method is to use a double-dash "--" before the name. A double-dash means the end of options to that command.

[shredder12]$ cp -- -directory/

For the other way, we need to start the command with something other than a hyphen and still make sure that its the same file.

[shredder12]$ cp ./-directory

While using Wget in scripts or otherwise, mostly when there is a lot of content to be downloaded, we need to set wget's network usage and bandwidth parameters. They both default to unlimited and when there is a lot of download involved, its always a good practice to set the limits. In this article, we will see how to control the bandwidth rate and total usage(quota).

Limit the download bandwidth rate using Wget's --limit-rate option

Just like any other application, wget will try to use the maximum amount of bandwidth available. If you want to control the download rate of wget, then use the --limit-rate flag. Lets say you want to download some video.

[shredder12]$ wget --limit-rate=60k http://foo.bar/video.ogv

This will limit the download rate to 60 KBps. Use "m" for MBps. Wget even accepts a decimal value, e.g. 2.5. This flag is beneficial only for large files. Thats because the implementation of the rate limitation feature takes sometime to achieve the concerned limit. So, it might not appear to be working for small files.

How to Wget: limit network bandwidth usage/quota using Wget's -Q option

The network quota or the total bandwidth usage can also be limited using wget's --quota or -Q attribute.

[shredder12]$ wget --quota 10m http://foo.bar/file.tgz http://foo.bar/otherfile.tgz

Please note that, if quota overflows while downloading a file then further downloading will be stopped once the current file is successfully downloaded. So, if the size of both the files are 8 and 5 MB then it will download both of them. Just like --limit-rate we can use m for Mega Bytes and similarly others. 0 sets the value to be unlimited.