Re: Is it a workqueue related issue in 2.6.37 (Was: Re: [libvirt] blkio cgroup [solved])

[Tejun Heo <tj@kernel.org>]

On Fri, Feb 25, 2011 at 08:24:15AM +0100, Dominik Klein wrote:
> > Maybe watching what the workqueue is doing using the following trace
> > events could be helpful.
> > 
> > # grep workqueue /sys/kernel/debug/tracing/available_events
> > workqueue:workqueue_insertion
> > workqueue:workqueue_execution
> > workqueue:workqueue_creation
> > workqueue:workqueue_destruction
> 
> Since I've never done this before, I will tell you how I captured the
> trace so you know what I did and can see whether I did something wrong
> and can correct me if necessary.
> 
> echo blk > /sys/kernel/debug/tracing/current_tracer
> echo 1 > /sys/block/sdb/trace/enable
> echo workqueue_queue_work >> /sys/kernel/debug/tracing/set_event
> echo workqueue_activate_work >> /sys/kernel/debug/tracing/set_event
> echo workqueue_execute_start >> /sys/kernel/debug/tracing/set_event
> echo workqueue_execute_end >> /sys/kernel/debug/tracing/set_event
> cat /sys/kernel/debug/tracing/trace_pipe
> 
> And that output i gzip'd.
> 
> This was taken with 2.6.37 plus the patch you mentioned on a Dell R815
> with 2 12 core AMD Opteron 6174 CPUs. If you need any more information,
> please let me know.

Hmmm... well, I have no idea what you were trying to do but here are
some info which might be helpful.

* queue_work happens when the work item is queued.

* activate_work happens when the work item becomes eligible for
  execution.  e.g. If the workqueue's @max_active is limited and
  maximum number of work items are already in flight, a new item will
  only get activated after one of the in flight ones retires.

* execute_start marks the actual starting of execution.

* execute_end marks the end of execution.

So, I would look for the matching work function and then try to follow
what happens to it after being scheduled and if it doesn't get
executed what's going on with the target workqueue.

Thanks.

-- 
tejun