Dear helpful readers...
What is the best practice for handling daemons and server processes? Does anyone know of good documentation on this? Especially considering that otherwise "good" software often is crappy at handling this part?
paster serve is, in effect, a little application server. It has a --daemon option and other options. I try to make them nice, but things trip me up.
Like, I enter daemon mode, then typically bind to a port. If something else is already being served up, the startup fails. I might clobber their PID file. The startup script appears to function, because it exits from the master process, not the daemon process, and the master process exited just fine. How should I deal with this stuff? I'd like to stick to Python, and not use other programs (like start-stop-daemon or whatnot), but I'd also like to know how those programs work -- what they call, in what order, with what protections, etc. And which one of the many options out there is best to emulate. All just for Posix, of course -- it's a whole different situation on Windows.
There's a well-ranked recipe at the Python Cookbook, called "Creating a daemon the Python way" that includes a good discusion of the topic.
http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/278731
# Graham Fawcett
You might look at what David Binger has done with qp - it stops and starts multiple servers
http://www.mems-exchange.org/software/qp/qp-1.1.tar.gz/qp-1.1/lib/site.py -- and http://www.mems-exchange.org/software/qp/qp-1.1.tar.gz/qp-1.1/bin/qp
Dulcinea and now qp have always had site management (stop, restart / start, status) type functionality to control various servers including SCGI, Durus and web servers.
You'll find older examples (including driving Apache) in Dulcinea - check the bin dir:
http://www.mems-exchange.org/software/dulcinea/Dulcinea-0.11.tar.gz/Dulcinea-0.11/bin/
Actually I've had a great deal of luck with start-stop-daemon. It keeps track of your PID, writes the pid file, checks it -- everything. Do one thing, do it well -- start-stop-daemon does it well.
At a minimum, you should:
- call setsid(2) so you're the leader of the process group (not a member of the parent's)
- call chdir(2) with your daemon's working directory, or "/" (just not on a network mount or any funky filesystem)
- call umask(2) with the appropriate file creation mask
- close unused FD's (usually 3-FD_MAX, unless you do funny stuff with pipes/IPC)
The search word you're looking for is 'daemonize'. It brought up Graham's link as the first hit, which looks right to me (it's been a couple years ;-).
Here's how Twisted does it: http://svn.twistedmatrix.com/cvs/trunk/twisted/scripts/twistd.py?view=markup
# Nicola Larosa
twistd's approach is pretty correct as far as POSIX-ness and such goes (I believe), but it's a bit of a mess in terms of internal factoring. If you have ideas about how to improve it so it would be more re-usable please feel free to complain.
Before daemonizing I see if the pidfile exists, if it does I open it and try to lock it. If my lock succeeds, there is no other process.. I the close the file and continue. After daemonization I then open the pid file and lock it, flush it and keep it open (might be race in there).. This has worked pretty well for me..
I still have the issue of something failing after daemonization.. I've had good luck creating a pipe before daemonizing.. After the fork my parent has the read end and the daemon has the write end.. The daemon then can write to the pipe which is then displayed to stdout and it sends a keyword when it has started successfully at which point the parent can then exit. Its worked pretty well, but can get a little complex.
# anonymous
Do you (or anyone else) have an example of doing the piping? Live-but-should-be-dead processes are always really painful to debug ("why don't my code edits change anything?!?"), especially since sometimes tracebacks show the current code, not the code that's actually being run, further bending your mind ("how can I get a NameError for x, when x isn't used on that line?!?").
I have added some stuff from QP that tests if a process is still alive, so it should make these detached and untracked servers a little less common (e.g., by testing that a process is dead before deleting the pidfile, and testing that it's not alive before overwriting the pidfile). That might be enough.
# Ian Bicking
Daemontools http://cr.yp.to/daemontools.html is a sure bet if you're on unix. Even if you don't like it, please provide an option that stops your program from forking and redirects it's logging to standard out - this way one can wrap in daemontools.
# anonymous
runit is even nicer; same design, less djb weirdness: http://smarden.org/runit/
I actually tend to run things like twisted servers from under runit, using twistd's keep-in-foreground option and then letting runit do the daemonization...
# Nathaniel Smith
I've been writing a bunch of daemons in Python lately, using most of the techniques described in the comments above. I soon found that all of my daemons needed to do pretty much exactly the same thing at startup. In addition to the fork and detach, I always need to read a configuration file, set up logging, check and write a pidfile, etc. So I refactored all of the common functionality into a single module. You can see it here:
http://hathawaymix.org/Software/Sketches/daemon.py
I feel like something like this belongs in the Python standard library.
Nice sketch, maybe you should add it to sourceforge, or active state
# Anthony
A good code to view is "fetchmail" (http://www.catb.org/~esr/fetchmail/)