Thread priorities in linux

How to increase thread priority in pthreads?

I am using pthread in Linux. I would like to increase the thread priority by setting the parameters sched_param.priority . However, I could not find much info from the net regarding the range of the thread priority I could set, or about the description of the thread priority. Also, I would like to know about the relative thread priority as I would not want to set the thread priority to be too high and resulting the OS to halt. Could someone help me with this?

3 Answers 3

The default Linux scheduling policy is SCHED_OTHER , which have no priority choice but a nice level to tweak inside the policy.

You’ll have to change to another scheduling policy using function pthread_setschedparam (see also man sched_setscheduler )

‘Normal’ scheduling policies: (from sched_setscheduler(2) )

 SCHED_OTHER the standard round-robin time-sharing policy; SCHED_BATCH for "batch" style execution of processes; and SCHED_IDLE for running very low priority background jobs. 

Real-time scheduling policies:

 SCHED_FIFO a first-in, first-out policy; and SCHED_RR a round-robin policy. 

In your case maybe you can use SCHED_BATCH as this does not require root privileges.

Warning: wrong usage of real-time scheduling policies may hang your system. That’s why you need root privileges to do this kind of operation.

Just to be sure of what your machine is capable of, you can use chrt tool from util-linux package.
As an example:

$ chrt -m SCHED_OTHER min/max priority : 0/0 SCHED_FIFO min/max priority : 1/99 SCHED_RR min/max priority : 1/99 SCHED_BATCH min/max priority : 0/0 SCHED_IDLE min/max priority : 0/0 

A way to waste less time (which I often use):

alias batchmake='time chrt --batch 0 make --silent' 

While staying with user privileges, this propels the make by 15% (in my case).

Edit: introducing nice , SCHED_BATCH , SCHED_IDLE and chrt tool. For accuracy ! 🙂

@Hasturkun: you’re right there is the nice tweak, as an advice for the scheduler (not a scheduler priority!). Thanks for accuracy !

you don’t actually need root privileges, just need to have the rlimits set so that the max prio you’re allowed to set is > 0

@Spudd86: of course, one can tweak the system file limits.conf (in /etc/security/ on FC) to increase Hard values and then use setrlimit() as a standard user. Among other cases, this is indispensable for real-time audio mixing. This point was missing, I’ll edit to fix.

that’s what I was referring to, yes. You can also raise your nice limit, which would seem to be what would be the appropriate answer to the question, since they don’t want to be able lock the machine.

The current answer from levif (recommending SCHED_BATCH) is not correct for the current NPTL thread implementation on Linux (you can check which implementation your kernel has by running ‘getconf GNU_LIBPTHREAD_VERSION’).

In today’s kernel only the Real Time Scheduling policies allow setting of sched_priority — it is always 0 for the non-RT policies (SCHED_OTHER, SCHED_BATCH, and SCHED_IDLE). Your only choice for the non-RT policies is to set ‘nice’ values e.g. by setpriority(). There are not good specifications of the exact behavior to expect by setting ‘nice’ however, and at least in theory it could vary from kernel version to kernel version. For current Linux kernels ‘nice’ has a very strong effect similar to priority, so you can use it pretty much interchangeably. In order to increase how often your thread is scheduled, you want to lower your ‘nice’ value. This requires CAP_SYS_NICE capability (typically root, although not necessarily, see http://man7.org/linux/man-pages/man7/capabilities.7.html and http://man7.org/linux/man-pages/man3/cap_set_proc.3.html).

In fact SCHED_BATCH is designed for the opposite case to what the questioner requested: it is designed for CPU-intensive, long running jobs, that can live with lower priority. It tells the scheduler to slightly penalize wake-up priority for the threads.

Also to answer one of the earlier comments (I don’t yet have sufficient reputation to comment in response — some upvotes for this answer would help 🙂 ). Yes the bad news is that the POSIX.1 specification says that ‘nice’ affects processes, not individual threads. The good news is that the Linux thread implementations (both NPTL and the original Linux Threads) break the specification and allow it to affect individual threads. I find it amusing that this is often called out in the «BUGS» section of the man pages. I’d say the bug was in the POSIX.1 specification, which should have allowed this behavior, not in the implementations which were compelled to provide it in spite of the specification and did so knowingly and deliberately. In other words — not a bug.

Most of this is detailed on the sched(7) man page (which for some reason is not delivered on my Fedora 20 system): http://man7.org/linux/man-pages/man7/sched.7.html

If you really wanted to affect sched_priority you could look at the Real Time policies such as SCHED_RR).

Источник

How To Use POSIX Thread Priority Scheduling under Linux

The Linux scheduler supports the SCHED_FIFO scheduling policy defined by POSIX.1-2001. Threads scheduled with this “real-time” policy can be assigned a priority (under Linux) in the range 1..99 with 99 representing the highest priority. Since ordinary, “non-real time” processes execute at priority 0 (nice(1) modifies a “dynamic priority” which only affects processes with real-time priority 0 and implements fair timesharing among such processes) SCHED_FIFO threads are always capable to preempt “ordinary processes” (these use the policy SCHED_OTHER). Note, however, that unless the Linux kernel is built with the RT_PREEMPT patch and measures are taken to lock the EPICS process in memory etc., no strictly deterministic latencies can be expected even under the SCHED_FIFO policy which thus is to be considered soft-real time.

Configuring How can I let epicsThreads use the SCHED_FIFO Policy?

In order to let EPICS use the SCHED_FIFO real-time policy you first need to check that the following option is set to “YES” in epics-base/configure/CONFIG_SITE

# Use POSIX thread priority scheduling (YES or NO) USE_POSIX_THREAD_PRIORITY_SCHEDULING = YES

If you find that the current setting is “NO” then you need to rebuild EPICS base ( make clean uninstall; make ) after changing to “YES”. Since engaging the SCHED_FIFO policy gives any thread created under that policy a higher priority than any normal process’, using SCHED_FIFO requires special privileges. Under a reasonably recent Linux equipped with the pam_security module or systemd it is not necessary to execute EPICS IOCs as root. Depending on your Linux system granting an IOC process running as a non-root user the required permissions requires some configuration, though.

SysV-style Init System, or user-mode systemd

On Linux systems using the traditional init system, the system administrator can define the maximum priority that may be used by specific users and/or groups in a file under /etc/security/limits.d/ , see the manpage for limits.conf(5) . E.g., if a file /etc/security.limits.d/epics.conf is created with the following contents (it may be necessary for a user to log out and log back on in order to obtain the new privileges):

someuser hard rtprio 99 someuser soft rtprio 0 someuser hard memlock unlimited someuser soft memlock 0

then processes created by “someuser” still have priority and memlock limts of 0 by default (this is the “soft” value) but “someuser” may increase the rtprio limit up to 99 either from a running program (using the system call setrlimit(2)) or within a shell e.g., with Bash’s ulimit -r utility. Note that a user process can set and/or lower its hard limit but never increase it. RTM.

If you are starting IOCs using user-mode systemd (i.e. with systemctl —user commands) you will also need the above settings or an equivalent for the user account(s) that you run your IOC under.

IOCs as systemd Services

On Linux systems using systemd you probably already have a service file in /etc/systemd/system/myioc.service or elsewhere that starts your IOC as a service. The following example uses procServ and includes the required configuration for granting the IOC process the necessary permissions.

[Unit] Description=EPICS Soft IOC myioc Requires=network.target After=network.target [Service] ExecStart=/usr/bin/procServ --foreground --quiet --port 4051 /path_to_myioc/iocBoot/myiocinstance/st.cmd Restart=always User=iocuser RuntimeDirectory=myioc CPUAffinity=1-3 LimitMEMLOCK=infinity LimitRTPRIO=99 [Install] WantedBy=multi-user.target

Note the last three lines of the [Service] section which are granting the process permissions to run in real-time. Refer to the systemd.exec documentation for details.

Tip: In case your service file is generated automatically by a deployment tool which doesn’t allow you to configure these real-time parameters you can augment an existing service file by dropping the real-time related configuration into /etc/systemd/system/myioc.service.d/10-realtime.conf :

[Service] CPUAffinity=1-3 LimitMEMLOCK=infinity LimitRTPRIO=99

Once your service file is in place, instruct systemd to reload its configuration and to start your IOC:

$ sudo systemctl daemon-reload $ sudo systemctl start myioc.service

Summary

You need to do three things:

1. Build EPICS base with USE_POSIX_THREAD_PRIORITY_SCHEDULING=YES
2. Make sure your EPICS application process has sufficient privileges to use SCHED_FIFO and the desired priority range.
3. SysV Init only: Set the soft and hard RTPRIO limits for your EPICS application process to enable it to actually make use of the privilege. This step is required because it is usually a bad idea to run all processes created by a given user or group at real-time priorities.
   Systemd only: Set the permissions for RTPRIO and MEMLOCK in the [Service] section of your IOC service configuration.

Verify that Real-Time Scheduling has been enabled correctly

Verify that Memory has been locked

$ grep Vm /proc/`pgrep st.cmd`/status

on the command line. If the size reported for “VmLck” is close to “VmSize” your IOC has been locked into memory successfully, if “VmLck” is “0 kB” the IOC process didn’t lock its memory. If an EPICS IOC detects that it has permissions to set real-time thread priorities it tries to lock memory – double check that the IOC process has both permission to adjust real-time thread priorities and to lock memory!

Verify scheduling Policy and Thread Priorities

$ ps -To pid,tid,policy,rtprio,comm -p `pgrep st.cmd`

If your system is configured correctly you should see “FF” (short for SCHED_FIFO) in the “POL” column and different priorities in the “RTPRIO” column. If you see “TS” (short for SCHED_OTHER) and “-” in the “POL” column for the IOC’s threads, your IOC process probably doesn’t have the necessary permissions to run with real-time priority.

If you have linked your IOC against MCore Utils you can also verify this on the IOC shell by running

You should see different priorities for the various threads in the “OSSPRI” column, the “POLICY” column should report “FIFO”.

Verify the CPU Affinity

You can get the CPU affinity of init/systemd as well as the one of your IOC process by running

$ taskset -c -p 1 $ taskset -c -p `pgrep st.cmd`

on the command line. On the IOC again

comes in handy – check the “CPUSET” column.

End of the Story?

Unfortunately, this is not the end of the story. Linux’ sched_get_priority_max(2) system call which is used by EPICS to map EPICS priorities from/to Linux/pthread priorities always reports the maximal supported priority (99) without considering any limits that the system enforces (as discussed above). This has the consequence (see bug #835138) that threads which are created with a desired priority that is greater than the resource limit eventually end up with the lowest priority 0.

Work-around

• Either apply the bugfix (AFAIK the problem is still present in 3.14.12.1, may be fixed thereafter)
• or make sure the RTPRIO limits are set to the maximally possible value (99)

Источник