Service start timeout linux

How to prevent systemd service start operation from timing out

Specify systemd startup timeout option to prevent service start operation from timing out.

Inspect default timeout for start operation.

$ sudo systemctl show elasticsearch | grep ^Timeout TimeoutStartUSec=1min 30s TimeoutStopUSec=infinity

Elasticsearch service will be terminated if it cannot start in 90 seconds (by default).

$ sudo systemctl start elasticsearch Job for elasticsearch.service failed because a timeout was exceeded. See "systemctl status elasticsearch.service" and "journalctl -xe" for details.

Inspect service runtime status information to confirm the findings.

$ sudo systemctl status elasticsearch ● elasticsearch.service - Elasticsearch Loaded: loaded (/etc/systemd/system/elasticsearch.service; enabled; vendor preset: enabled) Drop-In: /etc/systemd/system/elasticsearch.service.d └─startup-timeout.conf Active: failed (Result: timeout) since Thu 2019-11-28 23:14:45 GMT; 23s ago Docs: http://www.elastic.co Process: 3267 ExecStart=/usr/share/elasticsearch/bin/elasticsearch -p $/elasticsearch.pid --quiet (code=killed, signal=TERM) Main PID: 3267 (code=killed, signal=TERM) Nov 28 23:14:44 debian systemd[1]: Starting Elasticsearch. Nov 28 23:14:45 debian elasticsearch[3267]: OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release. Nov 28 23:14:45 debian systemd[1]: elasticsearch.service: Start operation timed out. Terminating. Nov 28 23:14:45 debian systemd[1]: elasticsearch.service: Main process exited, code=killed, status=15/TERM Nov 28 23:14:45 debian systemd[1]: elasticsearch.service: Failed with result 'timeout'. Nov 28 23:14:45 debian systemd[1]: Failed to start Elasticsearch.

Alternatively, inspect the most recent log data from the journal.

$ sudo journalctl -u elasticsearch -- Logs begin at Thu 2019-11-28 23:14:07 GMT, end at Thu 2019-11-28 23:15:26 GMT. -- Nov 28 23:14:44 debian systemd[1]: Starting Elasticsearch. Nov 28 23:14:45 debian elasticsearch[3267]: OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release. Nov 28 23:14:45 debian systemd[1]: elasticsearch.service: Start operation timed out. Terminating. Nov 28 23:14:45 debian systemd[1]: elasticsearch.service: Main process exited, code=killed, status=15/TERM Nov 28 23:14:45 debian systemd[1]: elasticsearch.service: Failed with result 'timeout'. Nov 28 23:14:45 debian systemd[1]: Failed to start Elasticsearch.

Create a service drop-in configuration directory.

$ sudo mkdir /etc/systemd/system/elasticsearch.service.d

Define TimeoutStartSec option to increase startup timeout.

$ echo -e "[Service]\nTimeoutStartSec=180" | sudo tee /etc/systemd/system/elasticsearch.service.d/startup-timeout.conf [Service] TimeoutStartSec=180

Reload systemd manager configuration.

$ sudo systemctl daemon-reload

Inspect altered timeout for start operation.

$ sudo systemctl show elasticsearch | grep ^Timeout TimeoutStartUSec=3min TimeoutStopUSec=infinity

Start the Elasticsearch service.

$ sudo systemctl start elasticsearch

Inspect Elasticsearch service status.

$ sudo systemctl status elasticsearch ● elasticsearch.service - Elasticsearch Loaded: loaded (/etc/systemd/system/elasticsearch.service; enabled; vendor preset: enabled) Drop-In: /etc/systemd/system/elasticsearch.service.d └─startup-timeout.conf Active: active (running) since Thu 2019-11-28 23:43:47 GMT; 1min 49s ago Docs: http://www.elastic.co Main PID: 3990 (java) Tasks: 57 (limit: 238) Memory: 142.5M CGroup: /system.slice/elasticsearch.service ├─3990 /usr/share/elasticsearch/jdk/bin/java -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -Des.networkaddress.cache.ttl=60 -Des.networkaddress.cache.negative.ttl=10 -XX:+AlwaysPreTouch -Xss1m -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Djna.nosys=true -XX:-OmitStackTraceInFastThrow -Dio.netty.noUnsafe=true -Dio.netty.noKeySetOptimization=true -Dio.netty.recycler.maxCapacityPerThread=0 -Dio.netty.allocator.numDirectArenas=0 -Dlog4j.shutdownHookEnabled=false -Dlog4j2.disable.jmx=true -Djava.io.tmpdir=/tmp/elasticsearch-10017090371480218455 -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/var/lib/elasticsearch -XX:ErrorFile=/var/log/elasticsearch/hs_err_pid%p.log -Xlog:gc*,gc+age=trace,safepoint:file=/var/log/elasticsearch/gc.log:utctime,pid,tags:filecount=32,filesize=64m -Djava.locale.providers=COMPAT -Dio.netty.allocator.type=unpooled -XX:MaxDirectMemorySize=60817408 -Des.path.home=/usr/share/elasticsearch -Des.path.conf=/etc/elasticsearch -Des.distribution.flavor=default -Des.distribution.type=deb -Des.bundled_jdk=true -cp /usr/share/elasticsearch/lib/* org.elasticsearch.bootstrap.Elasticsearch -p /var/run/elasticsearch/elasticsearch.pid --quiet └─4082 /usr/share/elasticsearch/modules/x-pack-ml/platform/linux-x86_64/bin/controller Nov 28 23:42:59 debian systemd[1]: Starting Elasticsearch. Nov 28 23:43:01 debian elasticsearch[3990]: OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release. Nov 28 23:43:47 debian systemd[1]: Started Elasticsearch.

The service startup timeout was not triggered this time as it was increased from 1min 30s to 3min.

Follow me on Mastodon , check out source code ad GitHub

Источник

How to change systemd service timeout value?

My systemd service kept timing out because of how long it would take to boot up also, so this fixed it for me:

1. Edit your systemd file:
   • For modern versions of systemd : Run systemctl edit —full node.service (replace «node» with your service name).
     • This will create a system file at /etc/systemd/system/node.service.d/ that will override the system file at /usr/lib/systemd/system/node.service . This is the proper way to configure your system files. More information about how to use systemctl edit is here.
   • Directly editing system file: The system file for me is at /usr/lib/systemd/system/node.service . Replace «node» with your application name. However, it is not safe to directly edit files in /usr/lib/systemd/ (See comments)
2. Use TimeoutStartSec , TimeoutStopSec or TimeoutSec (more info here) to specify how long the timeout should be for starting & stopping the process. Afterwards, this is how my systemd file looked:

[Unit] Description=MyProject Documentation=man:node(1) After=rc-local.service [Service] WorkingDirectory=/home/myproject/GUIServer/Server/ Environment="NODE_PATH=/usr/lib/node_modules" ExecStart=-/usr/bin/node Index.js Type=simple Restart=always KillMode=process TimeoutSec=900 [Install] WantedBy=multi-user.target 

• You can also view the current Timeout status by running any of these (but you’ll need to edit your service to make changes! See step 1). Confusingly, the associated properties have a «U» in their name for microseconds. See this Github issue for more information:
  • systemctl show node.service -p TimeoutStartUSec
  • systemctl show node.service -p TimeoutStopUSec
  • systemctl show node.service -p TimeoutUSec
  • There is also the option to use systemctl mask instead of systemctl start . For more info see here.

  Updates from Comments:

  • TimeoutSec=infinity : Instead of using «infinity» here, put a large amount of time instead, like TimeoutSec=900 (15 min). If the application takes «forever» to exit, then it’s possible that it will block a reboot indefinitely. Credit @Alexis Wilke and @JCCyC
  • Instead of editing /usr/lib/systemd/system , try systemctl edit instead or edit /etc/systemd/system to override them instead. You should never edit service files in /usr/lib/ . Credit @ryeager and @0xC0000022L

  Running systemctl show SERVICE_NAME.service -p TimeoutStopUSec I could at least see the timeout set by systemd to my service.

  I changed the script to a regular unit file one in order for it work properly.

  Источник

  Elasticsearch: How to stop systemd service start operation from timing out

  

  The Elastic Search Platform is a data store, search engine, and analytics solution. This article was necessitated out of the frustrations systemd threw at the writer when he tried to work on Elasticsearch. Hopefully, you will find the steps below useful if you are having a comparable situation as shown in the code box below. Also, please note that If the Elasticsearch service cannot be started within 90 seconds, it will be terminated (by default). This article will show you how to get up and running with your Elasticsearch service. Other helpful tutorials can be found here: Amazon Relational Database Service: How to install Amazon RDS using Terraform, how to install PostgreSQL on Ubuntu, and how to install and configure Prometheus for Monitoring on a Linux Server,

  root@ubuntu:/home/rdgmh# sudo systemctl start elasticsearch Job for elasticsearch.service failed because a timeout was exceeded. See "systemctl status elasticsearch.service" and "journalctl -xe" for details.

  Below are the steps on how to get your systemd service from failing while installing Elasticsearch on Linux, particularly Ubuntu 20.04. Here are some more guides you may want to read: How to deploy a React Application to AWS S3, and How to deploy a .NET application to AWS Elastic Beanstalk using AWS Tool Kit.

   sudo systemctl show elasticsearch | grep ^Timeout

  

  sudo systemctl status elasticsearch

  

  sudo mkdir /etc/systemd/system/elasticsearch.service.d

  echo -e "[Service]\nTimeoutStartSec=180" | sudo tee

  

  sudo systemctl daemon-reload

  sudo systemctl show elasticsearch | grep ^Timeout

  

  sudo systemctl start elasticsearch

   sudo systemctl status elasticsearch

  

  The service launch timeout was not prompted this time since it had been extended from 1 minute 30 seconds to 3 minutes.

  Summary

  We were able to resolve Elasticsearch service timing out because we extended the time from 1 minute 30 seconds to 3 minutes following the series of steps shown above and because of this, our Elasticsearch service was finally able to run without timing out.

  Источник