Inter-process communication in RAM on the Raspberry Pi
The Raspberry Pi is a really fun platform for all sorts of fun projects. I have already used several of these puppies in recent projects (including an energy-efficient NAS and to control an actual traffic light) and I have a feeling that it is only the beginning of a long friendship.
One thing that worries me however is that the Raspberry Pi uses a SD card for non-volatile memory. On the plus side, these cards are so cheap nowadays that you can buy a bunch of them, install a different OS and applications on each one and later all you have to to swap the SD card to turn your Raspberry Pi from a media centre to a wireless router. How cool is that?
But on the other hand, if these cards are so cheap, how reliable can they be? I have heard stories of SD cards working after having been thrown in the washer by accident but the Raspberry Pi forums are also full of stories of people having weird problems with their Pi due to corrupted SD cards. It seems that SD cards are resilient but some might wear quickly if you write data too often on them.
There are several solutions to this issue. The usual recommendations in the forums include disabling the swap and the non essential logs. See these articles and threads for more details:
- http://www.ideaheap.com/2013/07/stopping-sd-card-corruption-on-a-raspberry-pi/
- http://raspberrypi.stackexchange.com/questions/169/how-can-i-extend-the-life-of-my-sd-card
- http://www.raspberrypi.org/phpBB3/viewtopic.php?f=29&t=20505
Some go as far as mounting the file system on SD card in read-only mode. I guess it makes sense in some cases (Linux Live CDs have shown how much you can do with a read-only media).
However in my latest project I needed to have two daemons running on the Raspberry Pi with one sending data to the other up to several times per second. A quick and dirty solution would have been to have the sender daemon write the data into one or several files on the SD card and have the receiver daemon monitor the file or folder for incoming data. But is there a better way that does not involve constantly writing on the SD card? Of course there is! There is actually several solutions but the one that caught my attention is called named pipes (also know as FIFO). It works almost like a standard file but the idea is that when a process writes to this special file, the OS keeps the data in memory and gives it back to the next process that reads that same special file. There is no actual data written on the SD card, the file is just an address for the processes to exchange data in memory.
On unix systems, creating a named pipe is done using the command mkfifo:
$ mkfifo test
If you know ls the file that you just created, you will notice the p at the beginning to indicate that this is a special pipe file.
$ ls -l test
prw-r--r-- 1 tom tom 0 Oct 2 22:15 test
Now what happens if you write data into that file:
$ echo foo > test
You will immediately notice that the command does not return. This is normal, named pipes are blocking. It means that when a process writes data into the file, the write operation does not finish until the data has been read by another process. To test this, you can open another command shell and type:
$ cat test
foo
The content of the named pipe is has expected what we have written before but more importantly if you go back to your initial command shell you will see that your command has now returned. So we have successfully sent data from one shell to another and nothing was ever written on the file system. It all happens in the RAM.
The blocking nature of named pipes can be an useful but in my case however it is rather an issue since I don't want to have the "producer" daemon stuck in case the "consumer" daemon was down. But since I am using the start-stop-daemon command to run my daemons, I can use the -m option to ask start-stop-daemon to store the PID of the consumer daemon in a file and then the producer daemon can use start-stop-daemon's -t option to test if the consumer is running before writing to the named pipe. Of course this is not completely foolproof (for example the consumer daemon could die in between the moment the producer daemon runs start-stop-daemon -t command and the moment it writes to the named piped). But that's good enough for my use case since the consumer daemon should always be running.
In this solution I do read the PID of the consumer daemon in a file so there will be some read access to the SD card but these do not wear the SD card as much as the writes so it should not be an issue.
I should also say that there are many other very interesting methods of inter-process communication (IPC) including some much more advanced. Named pipes are just the simplest solution that suited my needs.
I hope that you enjoyed reading this post and that you learned a thing or two along the way. Until next time, have fun!
Short URL for this post: http://lepl.us/1h