binderfs interferes with syzkaller?

binderfs interferes with syzkaller?

[Dmitry Vyukov]

Hi binder maintainers,

It seems that something has happened and now syzbot has 0 coverage in
drivers/android/binder.c:
https://storage.googleapis.com/syzkaller/cover/ci-upstream-kasan-gce-root.html
It covered at least something there before as it found some bugs in binder code.
I _suspect_ it may be related to introduction binderfs, but it's
purely based on the fact that binderfs changed lots of things there.
And I see it claims to be backward compatible.

syzkaller strategy to reach binder devices is to use
CONFIG_ANDROID_BINDER_DEVICES to create a bunch of binderN devices (to
give each test process a private one):
https://github.com/google/syzkaller/blob/master/dashboard/config/upstream-kasan.config#L5671

Then it knows how to open these /dev/binderN devices:
https://github.com/google/syzkaller/blob/master/sys/linux/dev_binder.txt#L22
and do stuff with them.

Did these devices disappear or something?

Re: binderfs interferes with syzkaller?

[Dmitry Vyukov]

On Sat, Jan 25, 2020 at 6:49 PM Dmitry Vyukov <dvyukov@google.com> wrote:
>
> Hi binder maintainers,
>
> It seems that something has happened and now syzbot has 0 coverage in
> drivers/android/binder.c:
> https://storage.googleapis.com/syzkaller/cover/ci-upstream-kasan-gce-root.html
> It covered at least something there before as it found some bugs in binder code.
> I _suspect_ it may be related to introduction binderfs, but it's
> purely based on the fact that binderfs changed lots of things there.
> And I see it claims to be backward compatible.
>
> syzkaller strategy to reach binder devices is to use
> CONFIG_ANDROID_BINDER_DEVICES to create a bunch of binderN devices (to
> give each test process a private one):
> https://github.com/google/syzkaller/blob/master/dashboard/config/upstream-kasan.config#L5671
>
> Then it knows how to open these /dev/binderN devices:
> https://github.com/google/syzkaller/blob/master/sys/linux/dev_binder.txt#L22
> and do stuff with them.
>
> Did these devices disappear or something?

Oh, I see, it's backwards compatible if it's not enabled, right?

if (!IS_ENABLED(CONFIG_ANDROID_BINDERFS) &&
    strcmp(binder_devices_param, "") != 0) {
/*
* Copy the module_parameter string, because we don't want to
* tokenize it in-place.
*/
device_names = kstrdup(binder_devices_param, GFP_KERNEL);
if (!device_names) {
ret = -ENOMEM;
goto err_alloc_device_names_failed;
}

device_tmp = device_names;
while ((device_name = strsep(&device_tmp, ","))) {
ret = init_binder_device(device_name);
if (ret)
goto err_init_binder_device_failed;
}
}

And we enabled it because, well, enabling things generally gives more
coverage. I guess I will disable CONFIG_ANDROID_BINDERFS for now to
restore coverage in the binder itself.

Re: binderfs interferes with syzkaller?

[Christian Brauner]

On Sat, Jan 25, 2020 at 07:13:03PM +0100, Dmitry Vyukov wrote:
> On Sat, Jan 25, 2020 at 6:49 PM Dmitry Vyukov <dvyukov@google.com> wrote:
> >
> > Hi binder maintainers,
> >
> > It seems that something has happened and now syzbot has 0 coverage in
> > drivers/android/binder.c:
> > https://storage.googleapis.com/syzkaller/cover/ci-upstream-kasan-gce-root.html
> > It covered at least something there before as it found some bugs in binder code.
> > I _suspect_ it may be related to introduction binderfs, but it's
> > purely based on the fact that binderfs changed lots of things there.
> > And I see it claims to be backward compatible.
> >
> > syzkaller strategy to reach binder devices is to use
> > CONFIG_ANDROID_BINDER_DEVICES to create a bunch of binderN devices (to
> > give each test process a private one):
> > https://github.com/google/syzkaller/blob/master/dashboard/config/upstream-kasan.config#L5671
> >
> > Then it knows how to open these /dev/binderN devices:
> > https://github.com/google/syzkaller/blob/master/sys/linux/dev_binder.txt#L22
> > and do stuff with them.
> >
> > Did these devices disappear or something?
> 
> Oh, I see, it's backwards compatible if it's not enabled, right?

It's backwards compatible.
Let me give a little more detail. The legacy binder interface would
create all devices listed in the module parameter
CONFIG_ANDROID_BINDER_DEVICES. These devices were created using 
misc_register(&binder_device->miscdev);
in the host's devtmpfs mount.
If binderfs is enabled these devices are now created in the binderfs
instance instead.
For full backwards compatibility with old Android versions symlinks (or
bind mounts) can be provided in the host's devtmpfs. This is what I
recommended a few months ago.

> And we enabled it because, well, enabling things generally gives more
> coverage. I guess I will disable CONFIG_ANDROID_BINDERFS for now to

I would at least try to test both somehow. It's likely that binderfs
will replace legacy binder devices so if we could keep testing it
somehow that would be great.

> restore coverage in the binder itself.

I'm not completely sure what you mean here. The binder IPC codepaths are
nearly the same. The difference is mostly in how the device information
is cast out before actual binder-ipc operations take place.
In any case, testing both would obviously be preferred but binderfs
strikes me as more worthy of testing.

Christian

Re: binderfs interferes with syzkaller?

[Dmitry Vyukov]

On Sun, Jan 26, 2020 at 10:51 AM Christian Brauner
<christian.brauner@ubuntu.com> wrote:
>
> On Sat, Jan 25, 2020 at 07:13:03PM +0100, Dmitry Vyukov wrote:
> > On Sat, Jan 25, 2020 at 6:49 PM Dmitry Vyukov <dvyukov@google.com> wrote:
> > >
> > > Hi binder maintainers,
> > >
> > > It seems that something has happened and now syzbot has 0 coverage in
> > > drivers/android/binder.c:
> > > https://storage.googleapis.com/syzkaller/cover/ci-upstream-kasan-gce-root.html
> > > It covered at least something there before as it found some bugs in binder code.
> > > I _suspect_ it may be related to introduction binderfs, but it's
> > > purely based on the fact that binderfs changed lots of things there.
> > > And I see it claims to be backward compatible.
> > >
> > > syzkaller strategy to reach binder devices is to use
> > > CONFIG_ANDROID_BINDER_DEVICES to create a bunch of binderN devices (to
> > > give each test process a private one):
> > > https://github.com/google/syzkaller/blob/master/dashboard/config/upstream-kasan.config#L5671
> > >
> > > Then it knows how to open these /dev/binderN devices:
> > > https://github.com/google/syzkaller/blob/master/sys/linux/dev_binder.txt#L22
> > > and do stuff with them.
> > >
> > > Did these devices disappear or something?
> >
> > Oh, I see, it's backwards compatible if it's not enabled, right?
>
> It's backwards compatible.
> Let me give a little more detail. The legacy binder interface would
> create all devices listed in the module parameter
> CONFIG_ANDROID_BINDER_DEVICES. These devices were created using
> misc_register(&binder_device->miscdev);
> in the host's devtmpfs mount.
> If binderfs is enabled these devices are now created in the binderfs
> instance instead.
> For full backwards compatibility with old Android versions symlinks (or
> bind mounts) can be provided in the host's devtmpfs. This is what I
> recommended a few months ago.
>
> > And we enabled it because, well, enabling things generally gives more
> > coverage. I guess I will disable CONFIG_ANDROID_BINDERFS for now to
>
> I would at least try to test both somehow. It's likely that binderfs
> will replace legacy binder devices so if we could keep testing it
> somehow that would be great.
>
> > restore coverage in the binder itself.
>
> I'm not completely sure what you mean here. The binder IPC codepaths are
> nearly the same. The difference is mostly in how the device information
> is cast out before actual binder-ipc operations take place.
> In any case, testing both would obviously be preferred but binderfs
> strikes me as more worthy of testing.

I've reverted CONFIG_ANDROID_BINDERFS to "is not set" for now as the
easiest option. We've got 35% coverage of binder.c again.

By "restore coverage" I meant that even if the IPC codepaths are
nearly the same tests knew to open /dev/binderN and these disappeared,
so no test could open any binder device and get any coverage in binder
code.

If anybody wants to try to test both, you are very welcome ;)
If you are hinting that I should do it, a single person can't test all
2000 subsystems in kernel, and testing each of them requires quite
some domain expertise. This really requires involvement of developers
of the code. I am sure there is lots to improve in the IPC codepaths
as well.

Re: binderfs interferes with syzkaller?

[Greg Kroah-Hartman]

On Sat, Jan 25, 2020 at 06:49:49PM +0100, Dmitry Vyukov wrote:
> Hi binder maintainers,
> 
> It seems that something has happened and now syzbot has 0 coverage in
> drivers/android/binder.c:
> https://storage.googleapis.com/syzkaller/cover/ci-upstream-kasan-gce-root.html
> It covered at least something there before as it found some bugs in binder code.
> I _suspect_ it may be related to introduction binderfs, but it's
> purely based on the fact that binderfs changed lots of things there.
> And I see it claims to be backward compatible.

It is backwards compatible if you mount binderfs, right?

> syzkaller strategy to reach binder devices is to use
> CONFIG_ANDROID_BINDER_DEVICES to create a bunch of binderN devices (to
> give each test process a private one):
> https://github.com/google/syzkaller/blob/master/dashboard/config/upstream-kasan.config#L5671
> 
> Then it knows how to open these /dev/binderN devices:
> https://github.com/google/syzkaller/blob/master/sys/linux/dev_binder.txt#L22
> and do stuff with them.
> 
> Did these devices disappear or something?

Try mounting binderfs and then you should be able to see them all.

thanks,

greg k-h

Re: binderfs interferes with syzkaller?

[Christian Brauner]

On Sun, Jan 26, 2020 at 09:55:35AM +0100, Greg Kroah-Hartman wrote:
> On Sat, Jan 25, 2020 at 06:49:49PM +0100, Dmitry Vyukov wrote:
> > Hi binder maintainers,
> > 
> > It seems that something has happened and now syzbot has 0 coverage in
> > drivers/android/binder.c:
> > https://storage.googleapis.com/syzkaller/cover/ci-upstream-kasan-gce-root.html
> > It covered at least something there before as it found some bugs in binder code.
> > I _suspect_ it may be related to introduction binderfs, but it's
> > purely based on the fact that binderfs changed lots of things there.
> > And I see it claims to be backward compatible.
> 
> It is backwards compatible if you mount binderfs, right?

Yes, it is backwards compatible. The devices that would usually be
created in devtmpfs are now created in binderfs. The core
binder-codepaths are the same.

Christian

Re: binderfs interferes with syzkaller?

[Eric W. Biederman]

Christian Brauner <christian.brauner@ubuntu.com> writes:

> On Sun, Jan 26, 2020 at 09:55:35AM +0100, Greg Kroah-Hartman wrote:
>> On Sat, Jan 25, 2020 at 06:49:49PM +0100, Dmitry Vyukov wrote:
>> > Hi binder maintainers,
>> > 
>> > It seems that something has happened and now syzbot has 0 coverage in
>> > drivers/android/binder.c:
>> > https://storage.googleapis.com/syzkaller/cover/ci-upstream-kasan-gce-root.html
>> > It covered at least something there before as it found some bugs in binder code.
>> > I _suspect_ it may be related to introduction binderfs, but it's
>> > purely based on the fact that binderfs changed lots of things there.
>> > And I see it claims to be backward compatible.
>> 
>> It is backwards compatible if you mount binderfs, right?
>
> Yes, it is backwards compatible. The devices that would usually be
> created in devtmpfs are now created in binderfs. The core
> binder-codepaths are the same.

Any chance you can add code to the binderfs case to automatically
create the symlinks to the standard mount location in devtmpfs?

That way existing userspace might not need to care how the kernel is
configured.

Eric

Re: binderfs interferes with syzkaller?

[Christian Brauner]

On Tue, Jan 28, 2020 at 07:46:08AM -0600, Eric W. Biederman wrote:
> Christian Brauner <christian.brauner@ubuntu.com> writes:
> 
> > On Sun, Jan 26, 2020 at 09:55:35AM +0100, Greg Kroah-Hartman wrote:
> >> On Sat, Jan 25, 2020 at 06:49:49PM +0100, Dmitry Vyukov wrote:
> >> > Hi binder maintainers,
> >> > 
> >> > It seems that something has happened and now syzbot has 0 coverage in
> >> > drivers/android/binder.c:
> >> > https://storage.googleapis.com/syzkaller/cover/ci-upstream-kasan-gce-root.html
> >> > It covered at least something there before as it found some bugs in binder code.
> >> > I _suspect_ it may be related to introduction binderfs, but it's
> >> > purely based on the fact that binderfs changed lots of things there.
> >> > And I see it claims to be backward compatible.
> >> 
> >> It is backwards compatible if you mount binderfs, right?
> >
> > Yes, it is backwards compatible. The devices that would usually be
> > created in devtmpfs are now created in binderfs. The core
> > binder-codepaths are the same.
> 
> Any chance you can add code to the binderfs case to automatically
> create the symlinks to the standard mount location in devtmpfs?

Yeah, that's certainly doable and should be fairly easy. My reasoning
for not doing it was that it would be trivial for userspace to add in
the symlinks with an init script or service file.
We can also place this CONFIG_BINDERFS_DEVTMFPS_SYMLINK (random name)
which defaults to Y. Then - if userspace decides to completely move
from /dev/binder* to /dev/binderfs/binder* nodes and doesn't need the
symlinks - they can opt out of this by setting it to N. If Todd agrees
that something like this makes for Android too then we can do this.

Christian

Re: binderfs interferes with syzkaller?

[Todd Kjos]

On Sun, Jan 26, 2020 at 12:55 AM Greg Kroah-Hartman
<gregkh@linuxfoundation.org> wrote:
>
> On Sat, Jan 25, 2020 at 06:49:49PM +0100, Dmitry Vyukov wrote:
> > Hi binder maintainers,
> >
> > It seems that something has happened and now syzbot has 0 coverage in
> > drivers/android/binder.c:
> > https://storage.googleapis.com/syzkaller/cover/ci-upstream-kasan-gce-root.html
> > It covered at least something there before as it found some bugs in binder code.
> > I _suspect_ it may be related to introduction binderfs, but it's
> > purely based on the fact that binderfs changed lots of things there.
> > And I see it claims to be backward compatible.
>
> It is backwards compatible if you mount binderfs, right?

If binderfs is enabled and binderfs is mounted, then it's compatible
except that the path to be opened is under the binderfs mount point
instead of /dev/binderX.

>
> > syzkaller strategy to reach binder devices is to use
> > CONFIG_ANDROID_BINDER_DEVICES to create a bunch of binderN devices (to
> > give each test process a private one):
> > https://github.com/google/syzkaller/blob/master/dashboard/config/upstream-kasan.config#L5671
> >
> > Then it knows how to open these /dev/binderN devices:
> > https://github.com/google/syzkaller/blob/master/sys/linux/dev_binder.txt#L22
> > and do stuff with them.
> >
> > Did these devices disappear or something?
>
> Try mounting binderfs and then you should be able to see them all.
>
> thanks,
>
> greg k-h