On Tuesday 16 May 2006 00:22, Sébastien Dugué wrote: > On Mon, 2006-05-15 at 18:30 -0700, Darren Hart wrote: > > Following Ingo's example I have included the modified test case (please > > see the original mail for librt.h) that starts the trace before each > > sleep and disables it after we wake up. If we have missed a period, we > > print the trace. > I found several flaws in my first two iterations of the test case. Particularly false negatives. It's printing would cause large latencies waiting for the tty, or even the disk if output was being redirected. The new version uses buffered output, and I cleaned up the periodic loop using a MARK() system. You can run the test like this: ./sched_latency_lkml -v4 It will refrain from spewing output until after the test completes, or a missed period is detected. If a missed period is detected it will stop and print the latency trace as well. Note that I restart the trace at each iteration by just calling gettimeofday(0,1) - I don't stop it first. This was because I found that stopping it with gettimeofday(0,0) could take as long as 30ms! The output looks something like this: 000000: ----- ITERATION 1 ----- 000001: MARK 97: 197 us (112606039526 us) delta 0 000002: MARK 102: 214 us (16 us) delta 16 000003: MARK 110: 5024 us (4826 us) delta 4810 000004: MARK 121: 5029 us (4831 us) delta 5 000005: 000006: ----- ITERATION 2 ----- 000007: MARK 97: 5957 us (5759 us) delta 928 000008: MARK 102: 5977 us (19 us) delta 19 000009: MARK 110: 10018 us (4060 us) delta 4041 000010: MARK 121: 10023 us (4065 us) delta 4 Format: MARK LINE TIME_SINCE_START TIME_SINCE_TRACE_START TIME_SINCE_LAST_MARK TIME_SINCE_TRACE_START is handy for indexing the latency_traces if you happen to get one. Without all the added i/o and calls to stop the trace, I have yet to see a missed period with the version of the program. While getting to this point, I did see some things that concerned me: sched_la-4856 3D... 4083us!: math_state_restore (device_not_available) sched_la-4856 3D... 16033us : smp_apic_timer_interrupt (4b3b98e8 0 0) Am I reading that right? 12ms to complete math_state_restore()? What does "device_not_available" mean here? Here are some other similar traces: sched_la-5008 2D... 4104us!: math_state_restore (device_not_available) sched_la-5008 2.... 4992us > sys_clock_gettime (00000001 b7fc8378 0000007b) -- sched_la-4187 3D... 4786us : math_state_restore (device_not_available) sched_la-4187 3D... 4786us!: init_fpu (math_state_restore) sched_la-4187 3D... 5524us : smp_apic_timer_interrupt (4b3b98e8 0 0) -- sched_la-4729 1D... 4078us!: math_state_restore (device_not_available) sched_la-4729 1.... 4978us > sys_clock_gettime (00000001 b7f5a348 0000007b) I'll run the test in a loop overnight and see if I can trigger a missed period. In the meantime, can anyone explain with the math_state_restore() might be causing so much latency? Thanks, -- Darren Hart IBM Linux Technology Center Realtime Linux Team Phone: 503 578 3185 T/L: 775 3185