* Re: [PATCH 1/2] Always probe the NMI watchdog
@ 2007-06-26 8:04 Mikael Pettersson
2007-06-26 9:57 ` Stephane Eranian
0 siblings, 1 reply; 10+ messages in thread
From: Mikael Pettersson @ 2007-06-26 8:04 UTC (permalink / raw)
To: B.Steinbrink, eranian
Cc: ak, ingo, levon, linux-kernel, oprofile-list, perfmon, wcohen
On Mon, 25 Jun 2007 23:04:25 +0200, B.Steinbrink@gmx.de wrote:
> > I think the tricky part is that we do want to reserve perfctr1 even
> > though the NMI watchdog is not active. This comes from the fact that
> > the NMI watchdog knows about only one counter and if it can't get that
> > one, it probably fails. By reserving it from the start, we ensure NMI
> > watchdog will work when eventually activated.
>
> Can you enable it later on at all? It failed for me when I tried,
> because it didn't know which hardware to use. Had to pass the kernel
> parameter to make the proc files do anything. Seems like it has to be
> enable at boot to work at all.
>
> And AFAICT we never unconditionally reserved a perfctr for the watchdog.
Yes you can dynamically enable/disable the NMI watchdog,
at least if you booted with it enabled.
> In 2.6.21 the nmi watchdog, if enabled, just reserved its perfctrs and
> everything else had to deal with it. Since the cleanup, the watchdog
> will release its perfctr when disabled, so another subsystem can grab
> it. But that also means that that other subsystem must release it again
> before you can reenable the watchdog.
Which is the obvious and correct way to handle a shared resource.
Keeping parts of the PMU HW permanently reserved whether or not
the watchdog is enabled would be a BUG.
/Mikael
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH 1/2] Always probe the NMI watchdog
2007-06-26 8:04 [PATCH 1/2] Always probe the NMI watchdog Mikael Pettersson
@ 2007-06-26 9:57 ` Stephane Eranian
0 siblings, 0 replies; 10+ messages in thread
From: Stephane Eranian @ 2007-06-26 9:57 UTC (permalink / raw)
To: Mikael Pettersson
Cc: B.Steinbrink, ak, ingo, levon, linux-kernel, oprofile-list,
perfmon, wcohen
Mikael,
On Tue, Jun 26, 2007 at 10:04:15AM +0200, Mikael Pettersson wrote:
> On Mon, 25 Jun 2007 23:04:25 +0200, B.Steinbrink@gmx.de wrote:
> > > I think the tricky part is that we do want to reserve perfctr1 even
> > > though the NMI watchdog is not active. This comes from the fact that
> > > the NMI watchdog knows about only one counter and if it can't get that
> > > one, it probably fails. By reserving it from the start, we ensure NMI
> > > watchdog will work when eventually activated.
> >
> > Can you enable it later on at all? It failed for me when I tried,
> > because it didn't know which hardware to use. Had to pass the kernel
> > parameter to make the proc files do anything. Seems like it has to be
> > enable at boot to work at all.
> >
> > And AFAICT we never unconditionally reserved a perfctr for the watchdog.
>
> Yes you can dynamically enable/disable the NMI watchdog,
> at least if you booted with it enabled.
>
> > In 2.6.21 the nmi watchdog, if enabled, just reserved its perfctrs and
> > everything else had to deal with it. Since the cleanup, the watchdog
> > will release its perfctr when disabled, so another subsystem can grab
> > it. But that also means that that other subsystem must release it again
> > before you can reenable the watchdog.
>
> Which is the obvious and correct way to handle a shared resource.
>
Agreed.
> Keeping parts of the PMU HW permanently reserved whether or not
> the watchdog is enabled would be a BUG.
>
True. But the upside is that you guarantee the activation of the NMI
watchdog will always succeed which may be a valuable property given the
goal of the NMI watchdog. Otherwise, if Oprofile or perfmon are active,
the NMI will fail to grab a single counter.
--
-Stephane
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH 1/2] Always probe the NMI watchdog
@ 2007-06-26 10:35 Mikael Pettersson
0 siblings, 0 replies; 10+ messages in thread
From: Mikael Pettersson @ 2007-06-26 10:35 UTC (permalink / raw)
To: eranian, mikpe
Cc: B.Steinbrink, ak, ingo, levon, linux-kernel, oprofile-list,
perfmon, wcohen
On Tue, 26 Jun 2007 02:57:55 -0700, Mikael Pettersson wrote:
> > Keeping parts of the PMU HW permanently reserved whether or not
> > the watchdog is enabled would be a BUG.
> >
> True. But the upside is that you guarantee the activation of the NMI
> watchdog will always succeed which may be a valuable property given the
> goal of the NMI watchdog. Otherwise, if Oprofile or perfmon are active,
> the NMI will fail to grab a single counter.
That relates more to policy than mechanism. I for one don't
consider the watchdog "sacred".
I think it's entirely reasonable that if the watchdog is disabled
(echo 0 > /proc/sys/kernel/nmi_watchdog) in order to give the
_full_ PMU HW to something else, then that something else needs
to be disabled (or otherwise instructed to give up the HW) before
the watchdog can be reenabled. If you don't want to risk not
being able to restart the watchdog, don't disable it.
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH 1/2] Always probe the NMI watchdog
2007-06-25 19:36 ` Andi Kleen
2007-06-25 20:01 ` Stephane Eranian
@ 2007-06-25 21:06 ` Björn Steinbrink
1 sibling, 0 replies; 10+ messages in thread
From: Björn Steinbrink @ 2007-06-25 21:06 UTC (permalink / raw)
To: Andi Kleen
Cc: Andrew Morton, eranian, mingo, linux-kernel, levon, perfmon,
oprofile-list, wcohen
On 2007.06.25 21:36:17 +0200, Andi Kleen wrote:
> On Monday 25 June 2007 21:09, Andrew Morton wrote:
> > On Wed, 20 Jun 2007 20:34:48 +0200
> >
> > Bj__rn Steinbrink <B.Steinbrink@gmx.de> wrote:
> > > The performance counter allocator relies on the nmi watchdog being
> > > probed, so we have to do that even if the watchdog is not enabled.
> >
> > So... what's the status of this lot?
> >
> > I've just merged this patch and the second one:
> >
> > Subject: [PATCH 2/2] Reserve the right performance counter for the Intel
> > PerfMon NMI watchdog Message-ID: <20070620183551.GC3251@atjola.homenet>
> >
> > but there was no followup discussion afaict.
> >
> > Andi, Stephane: acks?
>
> Yes, although I'm still a little uneasy about the always probe one.
>
> > If acked, do we agree that this is 2.6.22 material?
>
> The first (always probe) is probably .22 material, but needs more testing
> first.
Hm, without the second, I expect OProfile to break when the watchdog is
enabled. Alternatively to the patch I sent, we could revert the change
that makes it use perfctr1 instead of perfctr0. Would you prefer that?
Björn
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH 1/2] Always probe the NMI watchdog
2007-06-25 20:01 ` Stephane Eranian
2007-06-25 20:36 ` Andi Kleen
@ 2007-06-25 21:04 ` Björn Steinbrink
1 sibling, 0 replies; 10+ messages in thread
From: Björn Steinbrink @ 2007-06-25 21:04 UTC (permalink / raw)
To: Stephane Eranian
Cc: Andi Kleen, ingo, linux-kernel, levon, perfmon, oprofile-list, wcohen
On 2007.06.25 13:01:58 -0700, Stephane Eranian wrote:
> Hi,
> On Mon, Jun 25, 2007 at 09:36:17PM +0200, Andi Kleen wrote:
> > On Monday 25 June 2007 21:09, Andrew Morton wrote:
> > > On Wed, 20 Jun 2007 20:34:48 +0200
> > >
> > > Bj__rn Steinbrink <B.Steinbrink@gmx.de> wrote:
> > > > The performance counter allocator relies on the nmi watchdog being
> > > > probed, so we have to do that even if the watchdog is not enabled.
> > >
> > > So... what's the status of this lot?
> > >
> > > I've just merged this patch and the second one:
> > >
> > > Subject: [PATCH 2/2] Reserve the right performance counter for the Intel
> > > PerfMon NMI watchdog Message-ID: <20070620183551.GC3251@atjola.homenet>
> > >
> > > but there was no followup discussion afaict.
> > >
> > > Andi, Stephane: acks?
> >
> > Yes, although I'm still a little uneasy about the always probe one.
> >
>
> I looked at the code I have in my tree coming from Bjon's patches and
> I am a bit confused by the flow for probing as well.
>
> The register allocator works globally, i.e., you reserve a register
> for all CPUs at once.
>
> The probe_nmi_watchdog() routine simply probes the CPU type to initialize
> the watchdog data structure (wd_ops). This needs to be done once and for all.
> Why put it in a route that is called with on_each_cpu()?
Ehrm, that's a good question actually... I moved the probing call up
into setup_local_APIC in that other big patch and put a check at the
start of the probing function, so that it is executed only once. No idea
why I did it so weird in this one.
> I think the tricky part is that we do want to reserve perfctr1 even
> though the NMI watchdog is not active. This comes from the fact that
> the NMI watchdog knows about only one counter and if it can't get that
> one, it probably fails. By reserving it from the start, we ensure NMI
> watchdog will work when eventually activated.
Can you enable it later on at all? It failed for me when I tried,
because it didn't know which hardware to use. Had to pass the kernel
parameter to make the proc files do anything. Seems like it has to be
enable at boot to work at all.
And AFAICT we never unconditionally reserved a perfctr for the watchdog.
> Unlike sharing between Oprofile and perfmon which works by enforcing
> mutual exclusion between the two subsystems, the NMI watchdog must
> work concurrently with either Oprofile or Perfmon.
In 2.6.21 the nmi watchdog, if enabled, just reserved its perfctrs and
everything else had to deal with it. Since the cleanup, the watchdog
will release its perfctr when disabled, so another subsystem can grab
it. But that also means that that other subsystem must release it again
before you can reenable the watchdog.
> Bjorn, did I understand the constraints correctly?
I'll tell you, once I'm sure that I understood them correctly ;-)
Björn
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH 1/2] Always probe the NMI watchdog
2007-06-25 20:01 ` Stephane Eranian
@ 2007-06-25 20:36 ` Andi Kleen
2007-06-25 21:04 ` Björn Steinbrink
1 sibling, 0 replies; 10+ messages in thread
From: Andi Kleen @ 2007-06-25 20:36 UTC (permalink / raw)
To: eranian
Cc: b.steinbrink, ingo, linux-kernel, levon, perfmon, oprofile-list, wcohen
> I looked at the code I have in my tree coming from Bjon's patches and
> I am a bit confused by the flow for probing as well.
Yes, it's a little risky. Perhaps it's better to readd the separate CPU switch
from .21 there again for 2.6.22. Ugly, but should be safe
-Andi
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH 1/2] Always probe the NMI watchdog
2007-06-25 19:36 ` Andi Kleen
@ 2007-06-25 20:01 ` Stephane Eranian
2007-06-25 20:36 ` Andi Kleen
2007-06-25 21:04 ` Björn Steinbrink
2007-06-25 21:06 ` Björn Steinbrink
1 sibling, 2 replies; 10+ messages in thread
From: Stephane Eranian @ 2007-06-25 20:01 UTC (permalink / raw)
To: Andi Kleen
Cc: b.steinbrink, ingo, linux-kernel, levon, perfmon, oprofile-list,
wcohen, Stephane Eranian
Hi,
On Mon, Jun 25, 2007 at 09:36:17PM +0200, Andi Kleen wrote:
> On Monday 25 June 2007 21:09, Andrew Morton wrote:
> > On Wed, 20 Jun 2007 20:34:48 +0200
> >
> > Bj__rn Steinbrink <B.Steinbrink@gmx.de> wrote:
> > > The performance counter allocator relies on the nmi watchdog being
> > > probed, so we have to do that even if the watchdog is not enabled.
> >
> > So... what's the status of this lot?
> >
> > I've just merged this patch and the second one:
> >
> > Subject: [PATCH 2/2] Reserve the right performance counter for the Intel
> > PerfMon NMI watchdog Message-ID: <20070620183551.GC3251@atjola.homenet>
> >
> > but there was no followup discussion afaict.
> >
> > Andi, Stephane: acks?
>
> Yes, although I'm still a little uneasy about the always probe one.
>
I looked at the code I have in my tree coming from Bjon's patches and
I am a bit confused by the flow for probing as well.
The register allocator works globally, i.e., you reserve a register
for all CPUs at once.
The probe_nmi_watchdog() routine simply probes the CPU type to initialize
the watchdog data structure (wd_ops). This needs to be done once and for all.
Why put it in a route that is called with on_each_cpu()?
I think the tricky part is that we do want to reserve perfctr1 even though
the NMI watchdog is not active. This comes from the fact that the NMI watchdog
knows about only one counter and if it can't get that one, it probably fails.
By reserving it from the start, we ensure NMI watchdog will work when eventually
activated.
Unlike sharing between Oprofile and perfmon which works by enforcing
mutual exclusion between the two subsystems, the NMI watchdog must work
concurrently with either Oprofile or Perfmon.
Bjorn, did I understand the constraints correctly?
--
-Stephane
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH 1/2] Always probe the NMI watchdog
2007-06-25 19:09 ` Andrew Morton
@ 2007-06-25 19:36 ` Andi Kleen
2007-06-25 20:01 ` Stephane Eranian
2007-06-25 21:06 ` Björn Steinbrink
0 siblings, 2 replies; 10+ messages in thread
From: Andi Kleen @ 2007-06-25 19:36 UTC (permalink / raw)
To: Andrew Morton
Cc: Björn Steinbrink, eranian, mingo, linux-kernel, levon,
perfmon, oprofile-list, wcohen
On Monday 25 June 2007 21:09, Andrew Morton wrote:
> On Wed, 20 Jun 2007 20:34:48 +0200
>
> Bj__rn Steinbrink <B.Steinbrink@gmx.de> wrote:
> > The performance counter allocator relies on the nmi watchdog being
> > probed, so we have to do that even if the watchdog is not enabled.
>
> So... what's the status of this lot?
>
> I've just merged this patch and the second one:
>
> Subject: [PATCH 2/2] Reserve the right performance counter for the Intel
> PerfMon NMI watchdog Message-ID: <20070620183551.GC3251@atjola.homenet>
>
> but there was no followup discussion afaict.
>
> Andi, Stephane: acks?
Yes, although I'm still a little uneasy about the always probe one.
> If acked, do we agree that this is 2.6.22 material?
The first (always probe) is probably .22 material, but needs more testing
first.
-Andi
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH 1/2] Always probe the NMI watchdog
2007-06-20 18:34 ` [PATCH 1/2] Always probe the " Björn Steinbrink
@ 2007-06-25 19:09 ` Andrew Morton
2007-06-25 19:36 ` Andi Kleen
0 siblings, 1 reply; 10+ messages in thread
From: Andrew Morton @ 2007-06-25 19:09 UTC (permalink / raw)
To: Björn Steinbrink
Cc: Andi Kleen, eranian, mingo, linux-kernel, levon, perfmon,
oprofile-list, wcohen
On Wed, 20 Jun 2007 20:34:48 +0200
Bj__rn Steinbrink <B.Steinbrink@gmx.de> wrote:
> The performance counter allocator relies on the nmi watchdog being
> probed, so we have to do that even if the watchdog is not enabled.
>
So... what's the status of this lot?
I've just merged this patch and the second one:
Subject: [PATCH 2/2] Reserve the right performance counter for the Intel PerfMon NMI watchdog
Message-ID: <20070620183551.GC3251@atjola.homenet>
but there was no followup discussion afaict.
Andi, Stephane: acks?
If acked, do we agree that this is 2.6.22 material?
Thanks.
^ permalink raw reply [flat|nested] 10+ messages in thread
* [PATCH 1/2] Always probe the NMI watchdog
2007-06-20 18:33 ` Björn Steinbrink
@ 2007-06-20 18:34 ` Björn Steinbrink
2007-06-25 19:09 ` Andrew Morton
0 siblings, 1 reply; 10+ messages in thread
From: Björn Steinbrink @ 2007-06-20 18:34 UTC (permalink / raw)
To: Andi Kleen, eranian, mingo, linux-kernel, levon, perfmon,
oprofile-list, wcohen, akpm
The performance counter allocator relies on the nmi watchdog being
probed, so we have to do that even if the watchdog is not enabled.
Signed-off-by: Björn Steinbrink <B.Steinbrink@gmx.de>
---
arch/i386/kernel/cpu/perfctr-watchdog.c | 11 +++++------
arch/i386/kernel/nmi.c | 3 +++
arch/x86_64/kernel/nmi.c | 3 +++
include/asm-i386/nmi.h | 1 +
include/asm-x86_64/nmi.h | 1 +
5 files changed, 13 insertions(+), 6 deletions(-)
diff --git a/arch/i386/kernel/cpu/perfctr-watchdog.c b/arch/i386/kernel/cpu/perfctr-watchdog.c
index f0b6763..a12dbcf 100644
--- a/arch/i386/kernel/cpu/perfctr-watchdog.c
+++ b/arch/i386/kernel/cpu/perfctr-watchdog.c
@@ -572,7 +572,7 @@ static struct wd_ops intel_arch_wd_ops = {
.evntsel = MSR_ARCH_PERFMON_EVENTSEL0,
};
-static void probe_nmi_watchdog(void)
+void probe_nmi_watchdog(void)
{
switch (boot_cpu_data.x86_vendor) {
case X86_VENDOR_AMD:
@@ -610,17 +610,16 @@ static void probe_nmi_watchdog(void)
int lapic_watchdog_init(unsigned nmi_hz)
{
- if (!wd_ops) {
- probe_nmi_watchdog();
- if (!wd_ops)
- return -1;
+ if (!wd_ops)
+ return -1;
+ /* hack to make sure that we only try to reserver the perfctrs once */
+ if (smp_processor_id() == 0)
if (!wd_ops->reserve()) {
printk(KERN_ERR
"NMI watchdog: cannot reserve perfctrs\n");
return -1;
}
- }
if (!(wd_ops->setup(nmi_hz))) {
printk(KERN_ERR "Cannot setup NMI watchdog on CPU %d\n",
diff --git a/arch/i386/kernel/nmi.c b/arch/i386/kernel/nmi.c
index fba121f..8788f91 100644
--- a/arch/i386/kernel/nmi.c
+++ b/arch/i386/kernel/nmi.c
@@ -248,6 +248,9 @@ void setup_apic_nmi_watchdog (void *unused)
if ((smp_processor_id() != 0) && (atomic_read(&nmi_active) <= 0))
return;
+ /* always probe the watchdog, the perfctr allocator requires that */
+ probe_nmi_watchdog();
+
switch (nmi_watchdog) {
case NMI_LOCAL_APIC:
__get_cpu_var(wd_enabled) = 1; /* enable it before to avoid race with handler */
diff --git a/arch/x86_64/kernel/nmi.c b/arch/x86_64/kernel/nmi.c
index 931c64b..3229a26 100644
--- a/arch/x86_64/kernel/nmi.c
+++ b/arch/x86_64/kernel/nmi.c
@@ -255,6 +255,9 @@ void setup_apic_nmi_watchdog(void *unused)
if ((smp_processor_id() != 0) && (atomic_read(&nmi_active) <= 0))
return;
+ /* always probe the watchdog, the perfctr allocator requires that */
+ probe_nmi_watchdog();
+
switch (nmi_watchdog) {
case NMI_LOCAL_APIC:
__get_cpu_var(wd_enabled) = 1;
diff --git a/include/asm-i386/nmi.h b/include/asm-i386/nmi.h
index fb1e133..cf87cd0 100644
--- a/include/asm-i386/nmi.h
+++ b/include/asm-i386/nmi.h
@@ -18,6 +18,7 @@
int do_nmi_callback(struct pt_regs *regs, int cpu);
extern int nmi_watchdog_enabled;
+extern void probe_nmi_watchdog(void);
extern int avail_to_resrv_perfctr_nmi_bit(unsigned int);
extern int avail_to_resrv_perfctr_nmi(unsigned int);
extern int reserve_perfctr_nmi(unsigned int);
diff --git a/include/asm-x86_64/nmi.h b/include/asm-x86_64/nmi.h
index d0a7f53..f61653b 100644
--- a/include/asm-x86_64/nmi.h
+++ b/include/asm-x86_64/nmi.h
@@ -45,6 +45,7 @@ extern int panic_on_timeout;
extern int unknown_nmi_panic;
extern int nmi_watchdog_enabled;
+extern void probe_nmi_watchdog(void);
extern int check_nmi_watchdog(void);
extern int avail_to_resrv_perfctr_nmi_bit(unsigned int);
extern int avail_to_resrv_perfctr_nmi(unsigned int);
--
1.5.2.2
^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2007-06-26 10:58 UTC | newest]
Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2007-06-26 8:04 [PATCH 1/2] Always probe the NMI watchdog Mikael Pettersson
2007-06-26 9:57 ` Stephane Eranian
-- strict thread matches above, loose matches on Subject: below --
2007-06-26 10:35 Mikael Pettersson
2007-06-18 10:32 [PATCH] Separate performance counter reservation from nmi watchdog Björn Steinbrink
2007-06-20 12:31 ` [PATCH 1/2] Separate the performance counter allocation from the LAPIC NMI watchdog Andi Kleen
2007-06-20 12:49 ` [perfmon] " Stephane Eranian
2007-06-20 13:01 ` Andi Kleen
2007-06-20 18:33 ` Björn Steinbrink
2007-06-20 18:34 ` [PATCH 1/2] Always probe the " Björn Steinbrink
2007-06-25 19:09 ` Andrew Morton
2007-06-25 19:36 ` Andi Kleen
2007-06-25 20:01 ` Stephane Eranian
2007-06-25 20:36 ` Andi Kleen
2007-06-25 21:04 ` Björn Steinbrink
2007-06-25 21:06 ` Björn Steinbrink
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®