How do I get the most accurate realtime periodic interrupts in Linux?
I want to be interrupted at frequencies that are powers of ten, so enabling interrupts from /dev/rtc isn’t ideal. I’d like to sleep 1 millisecond or 250 microseconds between interrupts. Enabling periodic interrupts from /dev/hpet works pretty well, but it doesn’t seem to work on some machines. Obviously I can’t use it on machines that don’t actually have a HPET. But I can’t get it working on some machines that have hpet available as a clocksource either. For example, on a Core 2 Quad, the example program included in the kernel documentation fails at HPET_IE_ON when set to poll. It would be nicer to use the itimer interface provided by Linux instead of interfacing with the hardware device driver directly. And on some systems, the itimer provides periodic interrupts that are more stable over time. That is, since the hpet can’t interrupt at exactly the frequency I want, the interrupts start to drift from wall time. But I’m seeing some systems sleep way longer (10+ milliseconds) than they should using an itimer. Here’s a test program using itimer for interrupts. On some systems it will only print out one warning that’s it slept about 100 microseconds or so over the target time. On others, it will print out batches of warning that it slept 10+ milliseconds over the target time. Compile with -lrt and run with sudo chrt -f 50 [name]
#include #include #include #include #include #include #include #include #include #include #include #define NS_PER_SECOND 1000000000LL #define TIMESPEC_TO_NS( aTime ) ( ( NS_PER_SECOND * ( ( long long int ) aTime.tv_sec ) ) \ + aTime.tv_nsec ) int main() < // Block alarm signal, will be waited on explicitly sigset_t lAlarm; sigemptyset( &lAlarm ); sigaddset( &lAlarm, SIGALRM ); sigprocmask( SIG_BLOCK, &lAlarm, NULL ); // Set up periodic interrupt timer struct itimerval lTimer; int lReceivedSignal = 0; lTimer.it_value.tv_sec = 0; lTimer.it_value.tv_usec = 250; lTimer.it_interval = lTimer.it_value; // Start timer if ( setitimer( ITIMER_REAL, &lTimer, NULL ) != 0 ) < error( EXIT_FAILURE, errno, "Could not start interval timer" ); >struct timespec lLastTime; struct timespec lCurrentTime; clock_gettime( CLOCK_REALTIME, &lLastTime ); while ( 1 ) < //Periodic wait if ( sigwait( &lAlarm, &lReceivedSignal ) != 0 ) < error( EXIT_FAILURE, errno, "Failed to wait for next clock tick" ); >clock_gettime( CLOCK_REALTIME, &lCurrentTime ); long long int lDifference = ( TIMESPEC_TO_NS( lCurrentTime ) - TIMESPEC_TO_NS( lLastTime ) ); if ( lDifference > 300000 ) < fprintf( stderr, "Waited too long: %lld\n", lDifference ); >lLastTime = lCurrentTime; > return 0; > This may be a kernel bug. My itimer example seems to work fine on all machines using 2.6.32 but not on 2.6.35 or 2.6.38.
What are Linux Local timer interrupts?
The local timer interrupt is a timer implemented on the APIC that interrupts only a particular CPU instead of raising an interrupt that can be handled by any CPU. It’s discussed in Bovet & Cesati’s «Understanding the Linux Kernel». A snippet:
- The APIC’s timer counter is 32 bits long, while the PIT’s timer counter is 16 bits long; therefore, the local timer can be programmed to issue interrupts at very low frequencies (the counter stores the number of ticks that must elapse before the interrupt is issued).
- The local APIC timer sends an interrupt only to its processor, while the PIT raises a global interrupt, which may be handled by any CPU in the system.
- The APIC’s timer is based on the bus clock signal (or the APIC bus signal, in older machines). It can be programmed in such a way to decrease the timer counter every 1, 2, 4, 8, 16, 32, 64, or 128 bus clock signals. Conversely, the PIT, which makes use of its own clock signals, can be programmed in a more flexible way.
Thanks for the answer. I have this book, but the version of the kernel 2.2 🙁 One more question, how does the kernel use these interrupts ? Which purpose, is it related to scheduling ? to other things ?
@Manuel: I really don’t know too much detail about the kernel’s use of these times. The 3rd edition of the book covers the 2.6 kernel and seems to have pretty good coverage of the topic along with specific function names that you could use to zero in on the relevant source code. I think it would be worth borrowing a copy of the 3rd edition to see if it has the level of detail information you’re looking for.
In fact, you may be able to get what you need by browsing the contents O’Reilly makes available on their page selling the book — click the «Browse Contents» item above the image of the book: shop.oreilly.com/product/9780596005658.do and search for «timekeeping architecture»
Thanks for the answer, I read the chapter online but didn’t find so much answers .. Regarding to scheduling there is also a line called Rescheduling interrupts in /proc/interrupts with a lot of interrupts, aren’t the interrupts for scheduling ?
Create a timer on Linux
The timing of certain events is a common task for a developer. Common scenarios for timers are watchdogs, cyclic execution of tasks, or scheduling events for a specific time. In this article, I show how to create a POSIX-compliant interval timer using timer_create(. ).
You can download the source code for the following examples from GitHub.
Prepare Qt Creator
I used Qt Creator as the IDE for this example. To run and debug the example code in Qt Creator, clone the GitHub repository, open Qt Creator, and go to File -> Open File or Project. and choose the CMakeLists.txt:
Open a project in Qt Creator (CC-BY-SA 4.0)
After selecting the toolchain, click on Configure Project. The project contains three independent examples (we will only cover two of them in this article). With the green-marked menu, switch between the configurations for each example and activate Run in terminal for each of them (see the yellow mark below). The currently active example for building and debugging can be selected over the Debug button on the bottom left corner (see the orange mark below):
Project configuration (CC-BY-SA 4.0)
Threading timer
Let’s take a look at the simple_threading_timer.c example. This is the simplest one: It shows how an interval timer is created, which calls the function expired on expiration. On each expiration, a new thread is created in which the function expiration is called.
#include #include #include #include #include #include #include void expired(union sigval timer_data); pid_t gettid(void); struct t_eventData< int myData; >; int main() < int res = 0; timer_t timerId = 0; struct t_eventData eventData = < .myData = 0 >; /* sigevent specifies behaviour on expiration */ struct sigevent sev = < 0 >; /* specify start delay and interval * it_value and it_interval must not be zero */ struct itimerspec its = < .it_value.tv_sec = 1, .it_value.tv_nsec = 0, .it_interval.tv_sec = 1, .it_interval.tv_nsec = 0 >; printf("Simple Threading Timer - thread-id: %d\n", gettid()); sev.sigev_notify = SIGEV_THREAD; sev.sigev_notify_function = &expired; sev.sigev_value.sival_ptr = &eventData; /* create timer */ res = timer_create(CLOCK_REALTIME, &sev, &timerId); if (res != 0) < fprintf(stderr, "Error timer_create: %s\n", strerror(errno)); exit(-1); >/* start timer */ res = timer_settime(timerId, 0, &its, NULL); if (res != 0) < fprintf(stderr, "Error timer_settime: %s\n", strerror(errno)); exit(-1); >printf("Press ETNER Key to Exit\n"); while(getchar()!='\n')<> return 0; > void expired(union sigval timer_data)< struct t_eventData *data = timer_data.sival_ptr; printf("Timer fired %d - thread-id: %d\n", ++data->myData, gettid()); >The advantage of this approach is its small footprint, in terms of code and simple debugging. The disadvantage is the additional overhead due to the creation of a new thread on expiration and, consequently, the less deterministic behavior.
Interrupt Signal Timer
Another possibility to be notified by an expired timer is based on a kernel signal. Instead of creating a new thread each time the timer expires, the kernel sends a signal to the process, the process is interrupted, and the corresponding signal handler is called.
As the default action when receiving a signal is to terminate the process (see signal man page), we have to prepare Qt Creator in advance so that properly debugging is possible.
The default behavior of Qt Creator when the debuggee receives a signal is:
- Interrupt execution and switch to the debugger context.
- Display a pop-up window that notifies the user about the reception of a signal.
Both actions are not wanted as the reception of a signal is part of our application.
Qt Creator uses GDB in the background. In order to prevent GDB from stopping the execution when the process receives a signal, go to Tools -> Options, select Debugger, and navigate to Locals & Expressions. Add the following expression to Debugging Helper Customization:
Sig 34 no stop with error (CC-BY-SA 4.0)
You can find more information about GDB signal handling in the GDB documentation.
Next, we want to suppress the pop-up window that notifies us every time a signal is received when we stop in the signal handler:
Signal 34 pop-up box (CC-BY-SA 4.0)
To do so, navigate to the tab GDB and uncheck the marked checkbox:
Timer signal windows (CC-BY-SA 4.0)
Now you can properly debug the signal_interrupt_timer. The actual implementation of the signal timer is a bit more complex:
#include #include #include #include #include #include #include #include #include #define UNUSED(x) (void)(x) static void handler(int sig, siginfo_t *si, void *uc); pid_t gettid(void); struct t_eventData< int myData; >; int main() < int res = 0; timer_t timerId = 0; struct sigevent sev = < 0 >; struct t_eventData eventData = < .myData = 0 >; /* specifies the action when receiving a signal */ struct sigaction sa = < 0 >; /* specify start delay and interval */ struct itimerspec its = < .it_value.tv_sec = 1, .it_value.tv_nsec = 0, .it_interval.tv_sec = 1, .it_interval.tv_nsec = 0 >; printf("Signal Interrupt Timer - thread-id: %d\n", gettid()); sev.sigev_notify = SIGEV_SIGNAL; // Linux-specific sev.sigev_signo = SIGRTMIN; sev.sigev_value.sival_ptr = &eventData; /* create timer */ res = timer_create(CLOCK_REALTIME, &sev, &timerId); if ( res != 0) < fprintf(stderr, "Error timer_create: %s\n", strerror(errno)); exit(-1); >/* specifz signal and handler */ sa.sa_flags = SA_SIGINFO; sa.sa_sigaction = handler; /* Initialize signal */ sigemptyset(&sa.sa_mask); printf("Establishing handler for signal %d\n", SIGRTMIN); /* Register signal handler */ if (sigaction(SIGRTMIN, &sa, NULL) == -1) < fprintf(stderr, "Error sigaction: %s\n", strerror(errno)); exit(-1); >/* start timer */ res = timer_settime(timerId, 0, &its, NULL); if ( res != 0) < fprintf(stderr, "Error timer_settime: %s\n", strerror(errno)); exit(-1); >printf("Press ENTER to Exit\n"); while(getchar()!='\n')<> return 0; > static void handler(int sig, siginfo_t *si, void *uc) < UNUSED(sig); UNUSED(uc); struct t_eventData *data = (struct t_eventData *) si->_sifields._rt.si_sigval.sival_ptr; printf("Timer fired %d - thread-id: %d\n", ++data->myData, gettid()); >In contrast to the threading timer, we have to initialize the signal and register a signal handler. This approach is more performant as it won’t cause the creation of additional threads. For this reason, the execution of the signal handler is also more deterministic. The drawback is clearly the extra configuration effort to debug this properly.
Summary
Both methods described in this article are close-to-the-kernel implementations of timers. Even if the timer_create(. ) function is part of the POSIX specification, it is not possible to compile the sample code on a FreeBSD system due to small differences in data structures. Besides this drawback, such an implementation gives you fine-grained control for general-purpose timing applications.
Linux timer interrupt
I have two questions about the Linux kernel. Specifically, does anybody know exactly, what Linux does in the timer interrupt? Is there some documentation about this? And what is affected when changing the CONFIG_HZ setting, when building the kernel? Thanks in advance!
It would be helpful if you could indicate your background knowledge, and which other questions on this and related topics you already looked at (e.g. unix.stackexchange.com/questions/466722/… or unix.stackexchange.com/questions/549616/…)
2 Answers 2
The Linux timer interrupt handler doesn’t do all that much directly. For x86, you’ll find the default PIT/HPET timer interrupt handler in arch/x86/kernel/time.c :
static irqreturn_t timer_interrupt(int irq, void *dev_id) < global_clock_event->event_handler(global_clock_event); return IRQ_HANDLED; > This calls the event handler for global clock events, tick_handler_periodic by default, which updates the jiffies counter, calculates the global load, and updates a few other places where time is tracked.
As a side-effect of an interrupt occurring, __schedule might end up being called, so a timer interrupt can also lead to a task switch (like any other interrupt).
Changing CONFIG_HZ changes the timer interrupt’s periodicity. Increasing HZ means that it fires more often, so there’s more timer-related overhead, but less opportunity for task scheduling to wait for a while (so interactivity is improved); decreasing HZ means that it fires less often, so there’s less timer-related overhead, but a higher risk that tasks will wait to be scheduled (so throughput is improved at the expense of interactive responsiveness). As always, the best compromise depends on your specific workload. Nowadays CONFIG_HZ is less relevant for scheduling aspects anyway; see How to change the length of time-slices used by the Linux CPU scheduler?