Re: SCHED_DEADLINE tasks missing their deadline with SCHED_FLAG_RECLAIM jobs in the mix (using GRUB)

[Juri Lelli <juri.lelli@redhat.com>]

[-- Attachment #1: Type: text/plain, Size: 2080 bytes --]

Hi Marcel,

On 25/05/25 21:29, Marcel Ziswiler wrote:
> Hi Luca
> 
> On Fri, 2025-05-23 at 21:46 +0200, luca abeni wrote:
> > Hi Marcel,
> > 
> > sorry, but I have some additional questions to fully understand your
> > setup...
> 
> No Problem, I am happy to answer any questions :)
> 
> > On Mon, 19 May 2025 15:32:27 +0200
> > Marcel Ziswiler <marcel.ziswiler@codethink.co.uk> wrote:
> > [...]
> > > > just a quick question to better understand your setup (and check
> > > > where the issue comes from):
> > > > in the email below, you say that tasks are statically assigned to
> > > > cores; how did you do this? Did you use isolated cpusets,  
> > > 
> > > Yes, we use the cpuset controller from the cgroup-v2 APIs in the
> > > linux kernel in order to partition CPUs and memory nodes. In detail,
> > > we use the AllowedCPUs and AllowedMemoryNodes in systemd's slice
> > > configurations.
> > 
> > How do you configure systemd? I am having troubles in reproducing your
> > AllowedCPUs configuration... This is an example of what I am trying:
> > 	sudo systemctl set-property --runtime custom-workload.slice AllowedCPUs=1
> > 	sudo systemctl set-property --runtime init.scope AllowedCPUs=0,2,3
> > 	sudo systemctl set-property --runtime system.slice AllowedCPUs=0,2,3
> > 	sudo systemctl set-property --runtime user.slice AllowedCPUs=0,2,3
> > and then I try to run a SCHED_DEADLINE application with
> > 	sudo systemd-run --scope -p Slice=custom-workload.slice <application>
> 
> We just use a bunch of systemd configuration files as follows:
> 

...

> > How are you configuring the cpusets?
> 
> See above.
> 

Could you please add 'debug sched_debug sched_verbose' to your kernel
cmdline and share the complete dmesg before starting your tests?

Also, I am attaching a script that should be able to retrieve cpuset
information if you run it with

# python3 get_cpuset_info.py > cpuset.out

Could you please also do that and share the collected information?

It should help us to better understand your setup and possibly reproduce
the problem you are seeing.

Thanks!
Juri

[-- Attachment #2: get_cpuset_info.py --]
[-- Type: text/plain, Size: 1766 bytes --]

import os

def get_cpuset_info(cgroup_path):
    """Retrieves cpuset information for a given cgroup path."""
    info = {}
    files_to_check = [
        'cpuset.cpus',
        'cpuset.mems',
        'cpuset.cpus.effective',
        'cpuset.mems.effective',
        'cpuset.cpus.exclusive',
        'cpuset.cpus.exclusive.effective',
        'cpuset.cpus.partition'
    ]

    for filename in files_to_check:
        filepath = os.path.join(cgroup_path, filename)
        if os.path.exists(filepath) and os.access(filepath, os.R_OK):
            try:
                with open(filepath, 'r') as f:
                    info[filename] = f.read().strip()
            except Exception as e:
                info[filename] = f"Error reading: {e}"
        # else:
        #     info[filename] = "Not found or not readable" # Uncomment if you want to explicitly show missing files
    return info

def main():
    cgroup_root = '/sys/fs/cgroup'

    print(f"Recursively retrieving cpuset information from {cgroup_root} (cgroup v2):\n")

    for dirpath, dirnames, filenames in os.walk(cgroup_root):
        # Skip the root cgroup directory itself if it's not a delegate
        # and only process subdirectories that might have cpuset info.
        # This is a heuristic; if you want to see info for the root too, remove this if.
        # if dirpath == cgroup_root:
        #     continue

        cpuset_info = get_cpuset_info(dirpath)

        if cpuset_info: # Only print if we found some cpuset information
            print(f"Cgroup: {dirpath.replace(cgroup_root, '') or '/'}")
            for key, value in cpuset_info.items():
                print(f"  {key}: {value}")
            print("-" * 30) # Separator for readability

if __name__ == "__main__":
    main()