How can I migrate a currently running task?

How can I migrate a currently running task?

[夏亮]

Hi,

       I am working on Intel Duo Core with Linux OS 2.6.21, and I'd  
like to migrate task from one cpu to another cpu.
In SMP systems, load_balance() function uses move_tasks() to move  
processes from source runqueue to local runqueue, but it does not move  
a currently running task. If I want to migrate a currently running  
task from source runqueue to local runqueue, how can I do? Any  
suggestion is preferred. Thanks!

     Liang

Re: How can I migrate a currently running task?

[Bart Van Assche]

On Fri, Jun 27, 2008 at 3:17 PM, 夏亮 <xiaiaxaxi@sjtu.edu.cn> wrote:
> I am working on Intel Duo Core with Linux OS 2.6.21, and I'd like to
> migrate task from one cpu to another cpu.
> In SMP systems, load_balance() function uses move_tasks() to move processes
> from source runqueue to local runqueue, but it does not move a currently
> running task. If I want to migrate a currently running task from source
> runqueue to local runqueue, how can I do? Any suggestion is preferred.

Are you familiar with the glibc pthread_setaffinity_np() function
and/or the sched_setaffinity() system call ?

Bart.

Re: How can I migrate a currently running task?

[xialiang]

Quoting Bart Van Assche <bart.vanassche@gmail.com>:

> On Fri, Jun 27, 2008 at 3:17 PM, 夏亮 <xiaiaxaxi@sjtu.edu.cn> wrote:
>> I am working on Intel Duo Core with Linux OS 2.6.21, and I'd like to
>> migrate task from one cpu to another cpu.
>> In SMP systems, load_balance() function uses move_tasks() to move processes
>> from source runqueue to local runqueue, but it does not move a currently
>> running task. If I want to migrate a currently running task from source
>> runqueue to local runqueue, how can I do? Any suggestion is preferred.
>
> Are you familiar with the glibc pthread_setaffinity_np() function
> and/or the sched_setaffinity() system call ?
>
> Bart.
>

Yes. I know sched_setaffinity(), it sets cpu_mask of a task. I want to  
use it in a timer interrupt( scheduler_tick() ), can I?

I use sched_getaffinity() in a timer interrupt, but it doesn't work.As  
following,

scheduler_tick()
{
     every 100ms, balance();
}

static int balance(void)
{
     Get temperature of two cpus.
     int ret;
     cpumask_t mask;

     If(one cpu is much hotter than another one){
         ret = sched_getaffinity(rq->curr->pid, &mask);
     }
}

If I comment sched_getaffinity, it works well.

Re: How can I migrate a currently running task?

[Dmitry Adamushko]

2008/6/27 夏亮 <xiaiaxaxi@sjtu.edu.cn>:
> Hi,
>
>      I am working on Intel Duo Core with Linux OS 2.6.21, and I'd like to
> migrate task from one cpu to another cpu.
> In SMP systems, load_balance() function uses move_tasks() to move processes
> from source runqueue to local runqueue, but it does not move a currently
> running task. If I want to migrate a currently running task from source
> runqueue to local runqueue, how can I do? Any suggestion is preferred.
> Thanks!

Some high-prio task takes away CPU from a 'target' task and then
pushes it onto another CPU. That's what 'migration_thread' does.

Take a look at use-cases of migrate_task() in sched.c.


>
>    Liang
>

-- 
Best regards,
Dmitry Adamushko

Re: How can I migrate a currently running task?

[Robin Holt]

On Fri, Jun 27, 2008 at 09:38:05PM +0800, xialiang wrote:
> Quoting Bart Van Assche <bart.vanassche@gmail.com>:
>
>> On Fri, Jun 27, 2008 at 3:17 PM, 夏亮 <xiaiaxaxi@sjtu.edu.cn> wrote:
>>> I am working on Intel Duo Core with Linux OS 2.6.21, and I'd like to
>>> migrate task from one cpu to another cpu.
>>> In SMP systems, load_balance() function uses move_tasks() to move processes
>>> from source runqueue to local runqueue, but it does not move a currently
>>> running task. If I want to migrate a currently running task from source
>>> runqueue to local runqueue, how can I do? Any suggestion is preferred.
>>
>> Are you familiar with the glibc pthread_setaffinity_np() function
>> and/or the sched_setaffinity() system call ?
>>
>> Bart.
>>
>
> Yes. I know sched_setaffinity(), it sets cpu_mask of a task. I want to  
> use it in a timer interrupt( scheduler_tick() ), can I?

Don't use it from a timer handler.  You can use migrate_task() to move the
task, but your email from a couple days ago said you wanted to move the
task at the head of the runqueue due to cpu heat or something like that.
That is a very imprecise way to move the task as any unfortunate task
that happens to be running when the timer tick occurs could get migrated,
not the one generating the work.

I think you really want to look at the other areas of the kernel which do
stuff like throttling the cpu.  Just moving a task off the cpu does not
prevent it from being used for another compute intesive load.  You could
take the cpu offline.  I guess to do that, I would use schedule_work()
or kthread_create() to get out of the timer context and into a regular
thread context then take the cpu offline from there.

Good Luck,
Robin

Re: How can I migrate a currently running task?

[xialiang]

Quoting Robin Holt <holt@sgi.com>:

> On Fri, Jun 27, 2008 at 09:38:05PM +0800, xialiang wrote:
>> Quoting Bart Van Assche <bart.vanassche@gmail.com>:
>>
>>> On Fri, Jun 27, 2008 at 3:17 PM, 夏亮 <xiaiaxaxi@sjtu.edu.cn> wrote:
>>>> I am working on Intel Duo Core with Linux OS 2.6.21, and I'd like to
>>>> migrate task from one cpu to another cpu.
>>>> In SMP systems, load_balance() function uses move_tasks() to move  
>>>>  processes
>>>> from source runqueue to local runqueue, but it does not move a currently
>>>> running task. If I want to migrate a currently running task from source
>>>> runqueue to local runqueue, how can I do? Any suggestion is preferred.
>>>
>>> Are you familiar with the glibc pthread_setaffinity_np() function
>>> and/or the sched_setaffinity() system call ?
>>>
>>> Bart.
>>>
>>
>> Yes. I know sched_setaffinity(), it sets cpu_mask of a task. I want to
>> use it in a timer interrupt( scheduler_tick() ), can I?
>
> Don't use it from a timer handler.  You can use migrate_task() to move the
> task, but your email from a couple days ago said you wanted to move the
> task at the head of the runqueue due to cpu heat or something like that.
> That is a very imprecise way to move the task as any unfortunate task
> that happens to be running when the timer tick occurs could get migrated,
> not the one generating the work.
>
> I think you really want to look at the other areas of the kernel which do
> stuff like throttling the cpu.  Just moving a task off the cpu does not
> prevent it from being used for another compute intesive load.  You could
> take the cpu offline.  I guess to do that, I would use schedule_work()
> or kthread_create() to get out of the timer context and into a regular
> thread context then take the cpu offline from there.
>
> Good Luck,
> Robin
>

       I would move a task at the head of the runqueue, but I already  
know which task creates the heat. I get the profile of tasks  
beforehand. So if I find the temperature of a cpu is much higher than  
that of the other, and coincidentally the currently running task of  
the cpu is what I want to migrate. Can I migrate it in the  
scheduler_tick()? Or where should I put the migration code? Thanks!

Best regards,
Liang

Re: How can I migrate a currently running task?

[Peter Zijlstra]

On Sat, 2008-06-28 at 12:05 +0800, xialiang wrote:
> Quoting Robin Holt <holt@sgi.com>:
> 
> > On Fri, Jun 27, 2008 at 09:38:05PM +0800, xialiang wrote:
> >> Quoting Bart Van Assche <bart.vanassche@gmail.com>:
> >>
> >>> On Fri, Jun 27, 2008 at 3:17 PM, 夏亮 <xiaiaxaxi@sjtu.edu.cn> wrote:
> >>>> I am working on Intel Duo Core with Linux OS 2.6.21, and I'd like to
> >>>> migrate task from one cpu to another cpu.
> >>>> In SMP systems, load_balance() function uses move_tasks() to move  
> >>>>  processes
> >>>> from source runqueue to local runqueue, but it does not move a currently
> >>>> running task. If I want to migrate a currently running task from source
> >>>> runqueue to local runqueue, how can I do? Any suggestion is preferred.
> >>>
> >>> Are you familiar with the glibc pthread_setaffinity_np() function
> >>> and/or the sched_setaffinity() system call ?
> >>>
> >>> Bart.
> >>>
> >>
> >> Yes. I know sched_setaffinity(), it sets cpu_mask of a task. I want to
> >> use it in a timer interrupt( scheduler_tick() ), can I?
> >
> > Don't use it from a timer handler.  You can use migrate_task() to move the
> > task, but your email from a couple days ago said you wanted to move the
> > task at the head of the runqueue due to cpu heat or something like that.
> > That is a very imprecise way to move the task as any unfortunate task
> > that happens to be running when the timer tick occurs could get migrated,
> > not the one generating the work.
> >
> > I think you really want to look at the other areas of the kernel which do
> > stuff like throttling the cpu.  Just moving a task off the cpu does not
> > prevent it from being used for another compute intesive load.  You could
> > take the cpu offline.  I guess to do that, I would use schedule_work()
> > or kthread_create() to get out of the timer context and into a regular
> > thread context then take the cpu offline from there.
> >
> > Good Luck,
> > Robin
> >
> 
>        I would move a task at the head of the runqueue, but I already  
> know which task creates the heat. I get the profile of tasks  
> beforehand. So if I find the temperature of a cpu is much higher than  
> that of the other, and coincidentally the currently running task of  
> the cpu is what I want to migrate. Can I migrate it in the  
> scheduler_tick()? 

No, you cannot migrate current from hardirq context. 

> Or where should I put the migration code? Thanks!

If you want to balance temperature, the best place would be the load
balancer.

Just don't wreck all the other balance conditions.