mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: Julia Lawall <julia.lawall@inria.fr>
To: Julia Lawall <julia.lawall@inria.fr>
Cc: Vincent Guittot <vincent.guittot@linaro.org>,
	 Peter Zijlstra <peterz@infradead.org>,
	Ingo Molnar <mingo@redhat.com>,
	 Dietmar Eggemann <dietmar.eggemann@arm.com>,
	Mel Gorman <mgorman@suse.de>,
	 linux-kernel@vger.kernel.org
Subject: Re: EEVDF and NUMA balancing
Date: Fri, 5 Jan 2024 18:27:19 +0100 (CET)	[thread overview]
Message-ID: <36f2cc93-db10-5977-78ab-d9d07c3f445@inria.fr> (raw)
In-Reply-To: <7a845b43-bd8e-6c7d-6bca-2e6f174f671@inria.fr>



On Fri, 5 Jan 2024, Julia Lawall wrote:

>
>
> On Fri, 5 Jan 2024, Vincent Guittot wrote:
>
> > On Fri, 5 Jan 2024 at 15:51, Julia Lawall <julia.lawall@inria.fr> wrote:
> > >
> > > > Your system is calling the polling mode and not the default
> > > > cpuidle_idle_call() ? This could explain why I don't see such problem
> > > > on my system which doesn't have polling
> > > >
> > > > Are you forcing the use of polling mode ?
> > > > If yes, could you check that this problem disappears without forcing
> > > > polling mode ?
> > >
> > > I expanded the code in do_idle to:
> > >
> > >                 if (cpu_idle_force_poll) { c1++;
> > >                         tick_nohz_idle_restart_tick();
> > >                         cpu_idle_poll();
> > >                 } else if (tick_check_broadcast_expired()) { c2++;
> > >                         tick_nohz_idle_restart_tick();
> > >                         cpu_idle_poll();
> > >                 } else { c3++;
> > >                         cpuidle_idle_call();
> > >                 }
> > >
> > > Later, I have:
> > >
> > >         trace_printk("force poll: %d: c1: %d, c2: %d, c3: %d\n",cpu_idle_force_poll, c1, c2, c3);
> > >         flush_smp_call_function_queue();
> > >         schedule_idle();
> > >
> > > force poll, c1 and c2 are always 0, and c3 is always some non-zero value.
> > > Sometimes small (often 1), and sometimes large (304 or 305).
> > >
> > > So I don't think it's calling cpu_idle_poll().
> >
> > I agree that something else
> >
> > >
> > > x86 has TIF_POLLING_NRFLAG defined to be a non zero value, which I think
> > > is sufficient to cause the issue.
> >
> > Could you trace trace_sched_wake_idle_without_ipi() ans csd traces as well ?
> > I don't understand what set need_resched() in your case; having in
> > mind that I don't see the problem on my Arm systems and IIRC Peter
> > said that he didn't face the problem on his x86 system.
>
> TIF_POLLING_NRFLAG doesn't seem to be defined on Arm.
>
> Peter said that he didn't see the problem, but perhaps that was just
> random.  It requires a NUMA move to occur.  I make 20 runs to be sure to
> see the problem at least once.  But another machine might behave
> differently.
>
> I believe the call chain is:
>
> scheduler_tick
>   trigger_load_balance
>     nohz_balancer_kick
>       kick_ilb
>         smp_call_function_single_async
>           generic_exec_single
>             __smp_call_single_queue
>               send_call_function_single_ipi
>                 call_function_single_prep_ipi
>                   set_nr_if_polling <====== sets need_resched
>
> I'll make a trace to reverify that.

This is what I see at a tick, which corresponds to the call chain shown
above:

          bt.B.x-4184  [046]   466.410605: bputs:                scheduler_tick: calling trigger_load_balance
          bt.B.x-4184  [046]   466.410605: bputs:                trigger_load_balance: calling nohz_balancer_kick
          bt.B.x-4184  [046]   466.410605: bputs:                trigger_load_balance: calling kick_ilb
          bt.B.x-4184  [046]   466.410607: bprint:               trigger_load_balance: calling smp_call_function_single_async 22
          bt.B.x-4184  [046]   466.410607: bputs:                smp_call_function_single_async: calling generic_exec_single
          bt.B.x-4184  [046]   466.410607: bputs:                generic_exec_single: calling __smp_call_single_queue
          bt.B.x-4184  [046]   466.410608: bputs:                __smp_call_single_queue: calling send_call_function_single_ipi
          bt.B.x-4184  [046]   466.410608: bputs:                __smp_call_single_queue: calling call_function_single_prep_ipi
          bt.B.x-4184  [046]   466.410608: bputs:                call_function_single_prep_ipi: calling set_nr_if_polling
          bt.B.x-4184  [046]   466.410609: sched_wake_idle_without_ipi: cpu=22

julia

  reply	other threads:[~2024-01-05 17:27 UTC|newest]

Thread overview: 46+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2023-10-03 20:25 Julia Lawall
2023-10-03 21:51 ` Peter Zijlstra
2023-10-04 12:01   ` Julia Lawall
2023-10-04 12:05     ` Peter Zijlstra
2023-10-04 16:24       ` Julia Lawall
2023-10-04 17:48         ` Peter Zijlstra
2023-10-04 18:04           ` Julia Lawall
2023-10-09 10:29             ` Peter Zijlstra
2023-10-09 14:07               ` Julia Lawall
2023-11-11 12:56               ` Julia Lawall
2023-12-18 13:58               ` Julia Lawall
2023-12-18 17:18                 ` Vincent Guittot
2023-12-18 22:31                   ` Julia Lawall
2023-12-19 17:38                     ` Vincent Guittot
2023-12-19 17:51                       ` Julia Lawall
2023-12-20 17:09                         ` Vincent Guittot
2023-12-21 18:20                           ` Julia Lawall
2023-12-22 14:55                             ` Vincent Guittot
2023-12-22 15:00                               ` Julia Lawall
2023-12-22 15:59                                 ` Vincent Guittot
2023-12-22 16:18                                   ` Julia Lawall
2023-12-22 16:29                                     ` Julia Lawall
2023-12-22 16:42                                       ` Vincent Guittot
2023-12-28 18:34                                         ` Julia Lawall
2023-12-29 15:18                                           ` Julia Lawall
2024-01-04 16:26                                             ` Vincent Guittot
2024-01-04 16:45                                               ` Julia Lawall
2024-01-05 14:51                                               ` Julia Lawall
2024-01-05 16:00                                                 ` Vincent Guittot
2024-01-05 16:39                                                   ` Julia Lawall
2024-01-05 17:27                                                     ` Julia Lawall [this message]
2024-01-18 16:35                                                       ` Vincent Guittot
2024-01-18 16:50                                                         ` Julia Lawall
2024-01-18 17:10                                                           ` Vincent Guittot
2024-01-18 17:43                                                             ` Julia Lawall
2024-01-18 22:13                                                         ` Julia Lawall
2024-01-19 11:26                                                           ` Vincent Guittot
2024-01-19 11:33                                                             ` Julia Lawall
2024-01-26 21:20                                                             ` Julia Lawall
2024-03-10  9:39                                                             ` Julia Lawall
2024-01-05 20:45                                                   ` Julia Lawall
2023-12-20 16:39                       ` Julia Lawall
2023-12-20 17:11                         ` Vincent Guittot
2023-10-04 18:15         ` Ingo Molnar
2023-10-04 18:20           ` Julia Lawall
2023-10-04 19:48           ` Julia Lawall

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=36f2cc93-db10-5977-78ab-d9d07c3f445@inria.fr \
    --to=julia.lawall@inria.fr \
    --cc=dietmar.eggemann@arm.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=mgorman@suse.de \
    --cc=mingo@redhat.com \
    --cc=peterz@infradead.org \
    --cc=vincent.guittot@linaro.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®