Re: [RFC] [PATCH v2 0/5] Intel Virtual PMU Optimization

[Like Xu <like.xu@linux.intel.com>]

On 2019/3/24 1:28, Peter Zijlstra wrote:
> On Sat, Mar 23, 2019 at 10:18:03PM +0800, Like Xu wrote:
>> === Brief description ===
>>
>> This proposal for Intel vPMU is still committed to optimize the basic
>> functionality by reducing the PMU virtualization overhead and not a blind
>> pass-through of the PMU. The proposal applies to existing models, in short,
>> is "host perf would hand over control to kvm after counter allocation".
>>
>> The pmc_reprogram_counter is a heavyweight and high frequency operation
>> which goes through the host perf software stack to create a perf event for
>> counter assignment, this could take millions of nanoseconds. The current
>> vPMU always does reprogram_counter when the guest changes the eventsel,
>> fixctrl, and global_ctrl msrs. This brings too much overhead to the usage
>> of perf inside the guest, especially the guest PMI handling and context
>> switching of guest threads with perf in use.
> 
> I think I asked for starting with making pmc_reprogram_counter() less
> retarded. I'm not seeing that here.

Do you mean pass perf_event_attr to refactor pmc_reprogram_counter
via paravirt ? Please share more details.

> 
>> We optimize the current vPMU to work in this manner:
>>
>> (1) rely on the existing host perf (perf_event_create_kernel_counter)
>>      to allocate counters for in-use vPMC and always try to reuse events;
>> (2) vPMU captures guest accesses to the eventsel and fixctrl msr directly
>>      to the hardware msr that the corresponding host event is scheduled on
>>      and avoid pollution from host is also needed in its partial runtime;
> 
> If you do pass-through; how do you deal with event constraints >
>> (3) save and restore the counter state during vCPU scheduling in hooks;
>> (4) apply a lazy approach to release the vPMC's perf event. That is, if
>>      the vPMC isn't used in a fixed sched slice, its event will be released.
>>
>> In the use of vPMC, the vPMU always focus on the assigned resources and
>> guest perf would significantly benefit from direct access to hardware and
>> may not care about runtime state of perf_event created by host and always
>> try not to pay for their maintenance. However to avoid events entering into
>> any unexpected state, calling pmc_read_counter in appropriate is necessary.
> 
> what?!

The patch will reuse the created events as much as possible for same 
guest vPMC which may has different config_base in its partial runtime.

The pmc_read_counter is designed to be called in kvm_pmu_rdpmc and 
pmc_stop_counter as legacy does and it's not for vPMU functionality but 
for host perf maintenance (seems to be gone in code,Oops).

> 
> I can't follow that, and the quick look I had at the patches doesn't
> seem to help. I did note it is intel only and that is really sad.

The basic idea of optimization is x86 generic, and the implementation is 
not intentional cause I could not access non-Intel machines and verified it.

> 
> It also makes a mess of who programs what msr when.
> 

who programs: vPMU does as usual in pmc_reprogram_counter

what msr: host perf scheduler make decisions and I'm not sure the hosy 
perf would do cross-mapping scheduling which means to assign a host 
fixed counter to guest gp counter and vice versa.

when programs: every time to call reprogram_gp/fixed_counter && 
pmc_is_assigned(pmc) is false; check the fifth pacth for details.