How to make a Python script run like a service or daemon in Linux
I have written a Python script that checks a certain e-mail address and passes new e-mails to an external program. How can I get this script to execute 24/7, such as turning it into daemon or service in Linux. Would I also need a loop that never ends in the program, or can it be done by just having the code re executed multiple times?
«checks a certain e-mail address and passes new e-mails to an external program» Isn’t that what sendmail does? You can define mail alias to route a mailbox to a script. Why aren’t you using mail aliases to do this?
On a modern linux which has systemd you can create a systemd service in daemon mode as described here. See also: freedesktop.org/software/systemd/man/systemd.service.html
16 Answers 16
You have two options here.
- Make a proper cron job that calls your script. Cron is a common name for a GNU/Linux daemon that periodically launches scripts according to a schedule you set. You add your script into a crontab or place a symlink to it into a special directory and the daemon handles the job of launching it in the background. You can read more at Wikipedia. There is a variety of different cron daemons, but your GNU/Linux system should have it already installed.
- Use some kind of python approach (a library, for example) for your script to be able to daemonize itself. Yes, it will require a simple event loop (where your events are timer triggering, possibly, provided by sleep function).
I wouldn’t recommend you to choose 2., because you would be, in fact, repeating cron functionality. The Linux system paradigm is to let multiple simple tools interact and solve your problems. Unless there are additional reasons why you should make a daemon (in addition to trigger periodically), choose the other approach.
Also, if you use daemonize with a loop and a crash happens, no one will check the mail after that (as pointed out by Ivan Nevostruev in comments to this answer). While if the script is added as a cron job, it will just trigger again.
+1 to the cronjob. I don’t think the question specifies that it is checking a local mail account, so mail filters do not apply
What happen does use a loop without termination in a Python program and then register it into crontab list will be? If I set up such .py for hourly, will it create many processes that will never be terminated? If so, I think this would quite like daemon.
I can see that cron is an obvious solution if you check check for emails once a minute (which is the lowest time resolution for Cron). But what if I want to check for emails every 10 seconds? Should I write the Python script to run query 60 times, which means it ends after 50 seconds, and then let cron start the script again 10 seconds later?
I have not worked with daemons/services, but I was under the impression that it (OS/init/init.d/upstart or what it is called) takes care of restarting a daemon when/if it ends/crashes.
I’m writing a python script that needs check about 20 sensors about 10 times per minute each 24/7. Is cron still a good use? I suspect a daemon would be better.
Here’s a nice class that is taken from here:
#!/usr/bin/env python import sys, os, time, atexit from signal import SIGTERM class Daemon: """ A generic daemon class. Usage: subclass the Daemon class and override the run() method """ def __init__(self, pidfile, stdin='/dev/null', stdout='/dev/null', stderr='/dev/null'): self.stdin = stdin self.stdout = stdout self.stderr = stderr self.pidfile = pidfile def daemonize(self): """ do the UNIX double-fork magic, see Stevens' "Advanced Programming in the UNIX Environment" for details (ISBN 0201563177) http://www.erlenstar.demon.co.uk/unix/faq_2.html#SEC16 """ try: pid = os.fork() if pid > 0: # exit first parent sys.exit(0) except OSError, e: sys.stderr.write("fork #1 failed: %d (%s)\n" % (e.errno, e.strerror)) sys.exit(1) # decouple from parent environment os.chdir("/") os.setsid() os.umask(0) # do second fork try: pid = os.fork() if pid > 0: # exit from second parent sys.exit(0) except OSError, e: sys.stderr.write("fork #2 failed: %d (%s)\n" % (e.errno, e.strerror)) sys.exit(1) # redirect standard file descriptors sys.stdout.flush() sys.stderr.flush() si = file(self.stdin, 'r') so = file(self.stdout, 'a+') se = file(self.stderr, 'a+', 0) os.dup2(si.fileno(), sys.stdin.fileno()) os.dup2(so.fileno(), sys.stdout.fileno()) os.dup2(se.fileno(), sys.stderr.fileno()) # write pidfile atexit.register(self.delpid) pid = str(os.getpid()) file(self.pidfile,'w+').write("%s\n" % pid) def delpid(self): os.remove(self.pidfile) def start(self): """ Start the daemon """ # Check for a pidfile to see if the daemon already runs try: pf = file(self.pidfile,'r') pid = int(pf.read().strip()) pf.close() except IOError: pid = None if pid: message = "pidfile %s already exist. Daemon already running?\n" sys.stderr.write(message % self.pidfile) sys.exit(1) # Start the daemon self.daemonize() self.run() def stop(self): """ Stop the daemon """ # Get the pid from the pidfile try: pf = file(self.pidfile,'r') pid = int(pf.read().strip()) pf.close() except IOError: pid = None if not pid: message = "pidfile %s does not exist. Daemon not running?\n" sys.stderr.write(message % self.pidfile) return # not an error in a restart # Try killing the daemon process try: while 1: os.kill(pid, SIGTERM) time.sleep(0.1) except OSError, err: err = str(err) if err.find("No such process") > 0: if os.path.exists(self.pidfile): os.remove(self.pidfile) else: print str(err) sys.exit(1) def restart(self): """ Restart the daemon """ self.stop() self.start() def run(self): """ You should override this method when you subclass Daemon. It will be called after the process has been daemonized by start() or restart(). """
How to Run Shell Script as Systemd Service in Linux
Systemd is a service manager that provides you features of managing daemons, process tracking, mount, automount. It uses a Linux control group to manage the task. It replaces the older initd and is available in Debian, Fedora, RedHat, ubuntu, centos, arch Linux distributions.
In this article, I am going to show you how to create a service using systemd. I will make a manual script which will act like a process to find disk utilization of the Linux system.
Let’s get started. Make a bash script that redirects date and disk utilization in a file. You can create files in any location. Here I am going to make in executable directory /usr/bin
Copy and paste the following script and save your file.
#!/bin/bash # Script generates disk utilization by the system and store in a file while true do date >> /var/storage-monitor.txt sudo du -sch / >> /var/storage-monitor.txt sleep 120 done
Make the file executable by running the following command.
Now, let’s make a service for running the script. Just create a file in the following directory. Note you can give any name but it must end with .service extension.
$ vim /etc/systemd/system/monitor-disk.service
[Unit] Description=My disk monitoring service Documentation=https://www.kernel.org/ #After=networking.service [Service] Type=simple User=root Group=root TimeoutStartSec=0 Restart=on-failure RestartSec=30s #ExecStartPre= ExecStart=/usr/bin/script.sh SyslogIdentifier=Diskutilization #ExecStop= [Install] WantedBy=multi-user.target
- The [Unit] section consists of description, documentation details. Here I have mentioned ‘After’ which states the service that we are going to create must be running first.
- [Service] Section defines the service type, username, group, what to do in failure, restart timeout. The main is ‘ExecStart’ which says to start our script file. You can also define ‘ExecStartPre’ to define anything before the actual script file. ‘SyslogIdentifier’ is the keyword to identify our service in syslog. Similarly , ExecStop is the instruction to say what to do to stop the service.
- [Install] section is used to define different levels of target in the system.
Just save the file and start the service using the systemctl command.
$ systemctl start monitor-disk.service
Check the service status using systemctl status command. You can also see service name, file form where systemd service is loaded, documentation, process running and logs.
$ systemctl status monitor-disk.service
Verify that your script is correctly working by looking into the file defined in the script file.
You can also check the enable, disable facility of the systemd manager.
$ systemctl enable monitor-disk.service
$ systemctl disable monitor-disk.service
Also, check by stopping and restarting the service. No error should be thrown.
$ systemctl stop monitor-disk.service
$ systemctl restart monitor-disk.service
Systemd also enables default logging in syslog. So, you can view the live log of the service using the following command. Search the keyword ‘SyslogIdentifier=Diskutilization’
you denied in the above service file.
Conclusion
This article shows you how to run your own shell script as a systemd service. We hope you learnt how to make your own service. Please feel free to send your feedback.
Karim Buzdar holds a degree in telecommunication engineering and holds several sysadmin certifications including CCNA RS, SCP, and ACE. As an IT engineer and technical author, he writes for various websites.
How to make Python script run as service?
Im using $ python flashpolicyd.py and it works fine. The question is: How to keep this script running even after I close the terminal(console)?
Not an exact duplicate — the linked-to question is about a recurring task, this is about a network daemon; the solution to the other was cron, the solution to this is inetd (or equivalent).
5 Answers 5
I offer two recommendations:
supervisord
sudo apt-get install supervisor
2) Create a config file for your daemon at /etc/supervisor/conf.d/flashpolicyd.conf :
[program:flashpolicyd] directory=/path/to/project/root environment=ENV_VARIABLE=example,OTHER_ENV_VARIABLE=example2 command=python flashpolicyd.py autostart=true autorestart=true
3) Restart supervisor to load your new .conf
supervisorctl update supervisorctl restart flashpolicyd
systemd (if currently used by your Linux distro)
[Unit] Description=My Python daemon [Service] Type=simple ExecStart=/usr/bin/python3 /opt/project/main.py WorkingDirectory=/opt/project/ Environment=API_KEY=123456789 Environment=API_PASS=password Restart=always RestartSec=2 [Install] WantedBy=sysinit.target
Place this file into /etc/systemd/system/my_daemon.service and enable it using systemctl daemon-reload && systemctl enable my_daemon && systemctl start my_daemon —no-block .
systemctl status my_daemon
Hi @Freude! I added a systemd daemon config to my answer. Many Linuxes now ship with systemd for init scripts and the config is very similar to supervisord.
Using this systemd approach I could get the daemon working but it can not startup after reboot with the following error: Job active_controller.service/start deleted to break ordering cycle starting with sysinit.target/start
I use this code to daemonize my applications. It allows you start/stop/restart the script using the following commands.
python myscript.py start python myscript.py stop python myscript.py restart
In addition to this I also have an init.d script for controlling my service. This allows you to automatically start the service when your operating system boots-up.
Here is a simple example to get your going. Simply move your code inside a class, and call it from the run function inside MyDeamon .
import sys import time from daemon import Daemon class YourCode(object): def run(self): while True: time.sleep(1) class MyDaemon(Daemon): def run(self): # Or simply merge your code with MyDaemon. your_code = YourCode() your_code.run() if __name__ == "__main__": daemon = MyDaemon('/tmp/daemon-example.pid') if len(sys.argv) == 2: if 'start' == sys.argv[1]: daemon.start() elif 'stop' == sys.argv[1]: daemon.stop() elif 'restart' == sys.argv[1]: daemon.restart() else: print "Unknown command" sys.exit(2) sys.exit(0) else: print "usage: %s start|stop|restart" % sys.argv[0] sys.exit(2)
If you are running an operating system that is using Upstart (e.g. CentOS 6) — you can also use Upstart to manage the service. If you use Upstart you can keep your script as is, and simply add something like this under /etc/init/my-service.conf
start on started sshd stop on runlevel [!2345] exec /usr/bin/python /opt/my_service.py respawn
You can then use start/stop/restart to manage your service.
start my-service stop my-service restart my-service
A more detailed example of working with upstart is available here.
If you are running an operating system that uses Systemd (e.g. CentOS 7) you can take a look at the following Stackoverflow answer.