Re: [PATCH 05/16] perf tools: Add facility to export data in database-friendly way

[Adrian Hunter <adrian.hunter@intel.com>]

On 24/10/14 11:11, Adrian Hunter wrote:
> On 24/10/14 09:02, Namhyung Kim wrote:
>> On Thu, 23 Oct 2014 13:45:13 +0300, Adrian Hunter wrote:
>>> This patch introduces an abstraction for exporting sample
>>> data in a database-friendly way.  The abstraction does not
>>> implement the actual output.  A subsequent patch takes this
>>> facility into use for extending the script interface.
>>>
>>> The abstraction is needed because static data like symbols,
>>> dsos, comms etc need to be exported only once.  That means
>>> allocating them a unique identifier and recording it on each
>>> structure.  The member 'db_id' is used for that.  'db_id'
>>> is just a 64-bit sequence number.
>>
>> Can we do it somewhere in a script not in the core code?  I don't feel
>> comfortable to add those bits into the core code.  What if we export
> 
> Please explain what you mean by "comfortable".

Or rather: What about it is wrong for core code?

> 
>> meta events like task, mmap or some user events to the script optionally
>> and let it process the data?
> 
> Intel PT decoding can generate a lot of data.  Many millions of samples at
> least.
> 
> Each sample contains a lot of duplicated information like symbol names, dso
> names, comms, etc.  So it is not optimal to export that for every sample.
> Even then, each piece of supporting information must be looked up - was it
> the same as the last sample, no then look it up, was it found, no then add a
> record.  That is very very inefficient.
> 
> Exporting each piece of information once, instead of over and over again, is
> a reasonable thing to do.
> 
>>
>> Thanks,
>> Namhyung
>>
>>
>>>
>>> Exporting centres around the db_export__sample() function
>>> which exports the associated data structures if they have
>>> not yet been allocated a db_id.
>>
>>
> 
> 
>