* eBPF verifier does not load libxdp dispatcher eBPF program @ 2023-01-23 11:06 Chris Ward 2023-01-23 12:35 ` Chris Ward 0 siblings, 1 reply; 8+ messages in thread From: Chris Ward @ 2023-01-23 11:06 UTC (permalink / raw) To: linux-kernel; +Cc: Chris Ward I am trying to use the 'bleeding edge' kernel to determine whether a problem I see has already been fixed, but with this kernel the eBPF verifier will not load the dispatcher program that is contained within libxdp. I am testing kernel commit hash 2475bf0 which fails, and the kernel in Ubuntu 22.04 (5.15.0-58-generic) works properly. I am running the test case from https://github.com/tjcw/bpf-examples/tree/tjcw-explore-sameeth ; to build it go to the AF_XDP-filter directory and type 'make', and to run it go to the AF_XDP-filter/runscripts/iperf3-namespace directory and type 'sudo FILTER=af_xdp_kern PORT=50000 ./run.sh' . The lines from the run output indicating the failure are libbpf: prog 'xdp_dispatcher': BPF program load failed: Invalid argument libbpf: prog 'xdp_dispatcher': -- BEGIN PROG LOAD LOG -- Func#11 is safe for any args that match its prototype btf_vmlinux is malformed reg type unsupported for arg#0 function xdp_dispatcher#29 0: R1=ctx(off=0,imm=0) R10=fp0 ; int xdp_dispatcher(struct xdp_md *ctx) 0: (bf) r6 = r1 ; R1=ctx(off=0,imm=0) R6_w=ctx(off=0,imm=0) 1: (b7) r0 = 2 ; R0_w=2 ; __u8 num_progs_enabled = conf.num_progs_enabled; 2: (18) r8 = 0xffffb2f6c06d8000 ; R8_w=map_value(off=0,ks=4,vs=84,imm=0) 4: (71) r7 = *(u8 *)(r8 +0) ; R7=1 R8=map_value(off=0,ks=4,vs=84,imm=0) ; if (num_progs_enabled < 1) 5: (15) if r7 == 0x0 goto pc+141 ; R7=1 ; ret = prog0(ctx); 6: (bf) r1 = r6 ; R1_w=ctx(off=0,imm=0) R6=ctx(off=0,imm=0) 7: (85) call pc+140 btf_vmlinux is malformed R1 type=ctx expected=fp Caller passes invalid args into func#1 processed 84 insns (limit 1000000) max_states_per_insn 0 total_states 9 peak_states 9 mark_read 1 -- END PROG LOAD LOG -- libbpf: prog 'xdp_dispatcher': failed to load: -22 libbpf: failed to load object 'xdp-dispatcher.o' libxdp: Failed to load dispatcher: Invalid argument libxdp: Falling back to loading single prog without dispatcher Can this regression be fixed before kernel 6.2 ships ? ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: eBPF verifier does not load libxdp dispatcher eBPF program 2023-01-23 11:06 eBPF verifier does not load libxdp dispatcher eBPF program Chris Ward @ 2023-01-23 12:35 ` Chris Ward 2025-02-25 15:55 ` Marc Kleine-Budde 0 siblings, 1 reply; 8+ messages in thread From: Chris Ward @ 2023-01-23 12:35 UTC (permalink / raw) To: linux-kernel; +Cc: Chris Ward The 5.15.0 kernel (built by 'git checkout v5.15' from the kernel.org torvalds tree) fails in the same way that the 6.2.0-rc5+ kernel fails. So it seems that something Canonical did for the Ubuntu 20.04 kernel causes eBPF to work correctly. On Mon, 23 Jan 2023 at 11:06, Chris Ward <tjcw01@gmail.com> wrote: > > I am trying to use the 'bleeding edge' kernel to determine whether a > problem I see has already been fixed, but with this kernel the eBPF > verifier will not load the dispatcher program that is contained within > libxdp. I am testing kernel commit hash 2475bf0 which fails, and the > kernel in Ubuntu 22.04 (5.15.0-58-generic) works properly. I am > running the test case from > https://github.com/tjcw/bpf-examples/tree/tjcw-explore-sameeth ; to > build it go to the AF_XDP-filter directory and type 'make', and to run > it go to the AF_XDP-filter/runscripts/iperf3-namespace directory and > type 'sudo FILTER=af_xdp_kern PORT=50000 ./run.sh' . > The lines from the run output indicating the failure are > libbpf: prog 'xdp_dispatcher': BPF program load failed: Invalid argument > libbpf: prog 'xdp_dispatcher': -- BEGIN PROG LOAD LOG -- > Func#11 is safe for any args that match its prototype > btf_vmlinux is malformed > reg type unsupported for arg#0 function xdp_dispatcher#29 > 0: R1=ctx(off=0,imm=0) R10=fp0 > ; int xdp_dispatcher(struct xdp_md *ctx) > 0: (bf) r6 = r1 ; R1=ctx(off=0,imm=0) > R6_w=ctx(off=0,imm=0) > 1: (b7) r0 = 2 ; R0_w=2 > ; __u8 num_progs_enabled = conf.num_progs_enabled; > 2: (18) r8 = 0xffffb2f6c06d8000 ; R8_w=map_value(off=0,ks=4,vs=84,imm=0) > 4: (71) r7 = *(u8 *)(r8 +0) ; R7=1 > R8=map_value(off=0,ks=4,vs=84,imm=0) > ; if (num_progs_enabled < 1) > 5: (15) if r7 == 0x0 goto pc+141 ; R7=1 > ; ret = prog0(ctx); > 6: (bf) r1 = r6 ; R1_w=ctx(off=0,imm=0) > R6=ctx(off=0,imm=0) > 7: (85) call pc+140 > btf_vmlinux is malformed > R1 type=ctx expected=fp > Caller passes invalid args into func#1 > processed 84 insns (limit 1000000) max_states_per_insn 0 total_states > 9 peak_states 9 mark_read 1 > -- END PROG LOAD LOG -- > libbpf: prog 'xdp_dispatcher': failed to load: -22 > libbpf: failed to load object 'xdp-dispatcher.o' > libxdp: Failed to load dispatcher: Invalid argument > libxdp: Falling back to loading single prog without dispatcher > > Can this regression be fixed before kernel 6.2 ships ? ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: eBPF verifier does not load libxdp dispatcher eBPF program 2023-01-23 12:35 ` Chris Ward @ 2025-02-25 15:55 ` Marc Kleine-Budde 2025-02-25 18:18 ` Eduard Zingerman 2025-02-25 18:21 ` Eduard Zingerman 0 siblings, 2 replies; 8+ messages in thread From: Marc Kleine-Budde @ 2025-02-25 15:55 UTC (permalink / raw) To: Chris Ward, Alexei Starovoitov, Daniel Borkmann, John Fastabend Cc: linux-kernel, Chris Ward, bpf [-- Attachment #1: Type: text/plain, Size: 3002 bytes --] On 23.01.2023 12:35:41, Chris Ward wrote: > The 5.15.0 kernel (built by 'git checkout v5.15' from the kernel.org > torvalds tree) fails in the same way that the 6.2.0-rc5+ kernel fails. > So it seems that something Canonical did for the Ubuntu 20.04 kernel > causes eBPF to work correctly. > > On Mon, 23 Jan 2023 at 11:06, Chris Ward <tjcw01@gmail.com> wrote: > > > > I am trying to use the 'bleeding edge' kernel to determine whether a > > problem I see has already been fixed, but with this kernel the eBPF > > verifier will not load the dispatcher program that is contained within > > libxdp. I am testing kernel commit hash 2475bf0 which fails, and the > > kernel in Ubuntu 22.04 (5.15.0-58-generic) works properly. I am > > running the test case from > > https://github.com/tjcw/bpf-examples/tree/tjcw-explore-sameeth ; to > > build it go to the AF_XDP-filter directory and type 'make', and to run > > it go to the AF_XDP-filter/runscripts/iperf3-namespace directory and > > type 'sudo FILTER=af_xdp_kern PORT=50000 ./run.sh' . > > The lines from the run output indicating the failure are > > libbpf: prog 'xdp_dispatcher': BPF program load failed: Invalid argument > > libbpf: prog 'xdp_dispatcher': -- BEGIN PROG LOAD LOG -- > > Func#11 is safe for any args that match its prototype > > btf_vmlinux is malformed > > reg type unsupported for arg#0 function xdp_dispatcher#29 > > 0: R1=ctx(off=0,imm=0) R10=fp0 > > ; int xdp_dispatcher(struct xdp_md *ctx) > > 0: (bf) r6 = r1 ; R1=ctx(off=0,imm=0) > > R6_w=ctx(off=0,imm=0) > > 1: (b7) r0 = 2 ; R0_w=2 > > ; __u8 num_progs_enabled = conf.num_progs_enabled; > > 2: (18) r8 = 0xffffb2f6c06d8000 ; R8_w=map_value(off=0,ks=4,vs=84,imm=0) > > 4: (71) r7 = *(u8 *)(r8 +0) ; R7=1 > > R8=map_value(off=0,ks=4,vs=84,imm=0) > > ; if (num_progs_enabled < 1) > > 5: (15) if r7 == 0x0 goto pc+141 ; R7=1 > > ; ret = prog0(ctx); > > 6: (bf) r1 = r6 ; R1_w=ctx(off=0,imm=0) > > R6=ctx(off=0,imm=0) > > 7: (85) call pc+140 > > btf_vmlinux is malformed > > R1 type=ctx expected=fp > > Caller passes invalid args into func#1 > > processed 84 insns (limit 1000000) max_states_per_insn 0 total_states > > 9 peak_states 9 mark_read 1 > > -- END PROG LOAD LOG -- > > libbpf: prog 'xdp_dispatcher': failed to load: -22 > > libbpf: failed to load object 'xdp-dispatcher.o' > > libxdp: Failed to load dispatcher: Invalid argument > > libxdp: Falling back to loading single prog without dispatcher > > > > Can this regression be fixed before kernel 6.2 ships ? I'm seeing the same failure on 32 bit ARM on v6.13. Have you found a solution? regards, Marc -- Pengutronix e.K. | Marc Kleine-Budde | Embedded Linux | https://www.pengutronix.de | Vertretung Nürnberg | Phone: +49-5121-206917-129 | Amtsgericht Hildesheim, HRA 2686 | Fax: +49-5121-206917-9 | [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 488 bytes --] ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: eBPF verifier does not load libxdp dispatcher eBPF program 2025-02-25 15:55 ` Marc Kleine-Budde @ 2025-02-25 18:18 ` Eduard Zingerman 2025-02-25 18:21 ` Eduard Zingerman 1 sibling, 0 replies; 8+ messages in thread From: Eduard Zingerman @ 2025-02-25 18:18 UTC (permalink / raw) To: Marc Kleine-Budde, Chris Ward, Alexei Starovoitov, Daniel Borkmann, John Fastabend Cc: linux-kernel, Chris Ward, bpf Skip to content [79] Navigation Menu Toggle navigation [80] Sign in [81] * Product GitHub Copilot Write better code with AI [37] Security Find and fix vulnerabilities [71] Actions Automate any workflow [72] Codespaces Instant dev environments [73] Issues Plan and track work [74] Code Review Manage code changes [75] Discussions Collaborate outside of code [76] Code Search Find more, search less [77] Explore All features [35] Documentation [22] GitHub Skills [26] Blog [15] * Solutions By company size Enterprises [36] Small and medium teams [40] Startups [69] Nonprofits [70] By use case DevSecOps [65] DevOps [66] CI/CD [67] View all use cases [68] By industry Healthcare [60] Financial services [61] Manufacturing [62] Government [63] View all industries [64] View all solutions [78] * Resources Topics AI [55] DevOps [56] Security [57] Software Development [58] View all [59] Explore Learning Pathways [51] Events & Webinars [41] Ebooks & Whitepapers [52] Customer Stories [53] Partners [30] Executive Insights [54] * Open Source GitHub Sponsors Fund open source developers [50] The ReadME Project GitHub community articles [16] Repositories Topics [47] Trending [48] Collections [49] * Enterprise Enterprise platform AI-powered developer platform [36] Available add-ons Advanced Security Enterprise-grade security features [44] GitHub Copilot Enterprise-grade AI features [45] Premium Support Enterprise-grade 24/7 support [46] * Pricing [39] undefined Search or jump to... undefinedundefined Search code, repositories, users, issues, pull requests... Provide feedback undefined We read every piece of feedback, and take your input very seriously. Include my email address so I can be contacted Cancel Submit feedback Saved searches Use saved searches to filter your results more quickly undefined Cancel Create saved search Sign in [81] Sign in to GitHub Username or email address Password Forgot password? [82] Sign up [83] Reseting focus 404 “This is not the web page you are looking for” Find code, projects, and people on GitHub: Search Contact Support [84] — GitHub Status [85] — @githubstatus [86] Site-wide Links [80] Subscribe to our developer newsletter Get tips, technical guides, and best practices. Twice a month. Right in your inbox. Subscribe [87] Product * Features [35] * Enterprise [36] * Copilot [37] * Security [38] * Pricing [39] * Team [40] * Resources [41] * Roadmap [42] * Compare GitHub [43] Platform * Developer API [29] * Partners [30] * Education [31] * GitHub CLI [32] * GitHub Desktop [33] * GitHub Mobile [34] Support * Docs [22] * Community Forum [23] * Professional Services [24] * Premium Support [25] * Skills [26] * Status [27] * Contact GitHub [28] Company * About [13] * Customer stories [14] * Blog [15] * The ReadME Project [16] * Careers [17] * Newsroom [18] * Inclusion [19] * Social Impact [20] * Shop [21] * © 2025 GitHub, Inc. * Terms [8] * Privacy [10] (Updated 02/2024 [9]) * Sitemap [11] * What is Git? [12] * Manage cookies * Do not share my personal information * GitHub on LinkedIn [1] * Instagram GitHub on Instagram [2] * GitHub on YouTube [3] * GitHub on X [4] * TikTok GitHub on TikTok [5] * Twitch GitHub on Twitch [6] * GitHub’s organization on GitHub [7] [1] GitHub on LinkedIn https://www.linkedin.com/company/github [2] Instagram GitHub on Instagram https://www.instagram.com/github [3] GitHub on YouTube https://www.youtube.com/github [4] GitHub on X https://x.com/github [5] TikTok GitHub on TikTok https://www.tiktok.com/@github [6] Twitch GitHub on Twitch https://www.twitch.tv/github [7] GitHub’s organization on GitHub https://github.com/github [8] Terms https://docs.github.com/site-policy/github-terms/github-terms-of-service [9] Updated 02/2024 https://github.com/github/site-policy/pull/582 [10] Privacy https://docs.github.com/site-policy/privacy-policies/github-privacy-statement [11] Sitemap https://github.com/sitemap [12] What is Git? https://github.com/git-guides [13] About https://github.com/about [14] Customer stories https://github.com/customer-stories?type=enterprise [15] Blog https://github.blog/ [16] The ReadME Project https://github.com/readme [17] Careers https://github.careers/ [18] Newsroom https://github.com/newsroom [19] Inclusion https://github.com/about/diversity [20] Social Impact https://socialimpact.github.com/ [21] Shop https://shop.github.com/ [22] Docs https://docs.github.com/ [23] Community Forum https://github.community/ [24] Professional Services https://services.github.com/ [25] Premium Support https://github.com/enterprise/premium-support [26] Skills https://skills.github.com/ [27] Status https://www.githubstatus.com/ [28] Contact GitHub https://support.github.com/?tags=dotcom-footer [29] Developer API https://docs.github.com/get-started/exploring-integrations/about-building-integrations [30] Partners https://partner.github.com/ [31] Education https://github.com/edu [32] GitHub CLI https://cli.github.com/ [33] GitHub Desktop https://desktop.github.com/ [34] GitHub Mobile https://github.com/mobile [35] Features https://github.com/features [36] Enterprise https://github.com/enterprise [37] Copilot https://github.com/features/copilot [38] Security https://github.com/security [39] Pricing https://github.com/pricing [40] Team https://github.com/team [41] Resources https://resources.github.com/ [42] Roadmap https://github.com/github/roadmap [43] Compare GitHub https://resources.github.com/devops/tools/compare [44] Advanced Security Enterprise-grade security features https://github.com/enterprise/advanced-security [45] GitHub Copilot Enterprise-grade AI features https://github.com/features/copilot#enterprise [46] Premium Support Enterprise-grade 24/7 support https://github.com/premium-support [47] Topics https://github.com/topics [48] Trending https://github.com/trending [49] Collections https://github.com/collections [50] GitHub Sponsors Fund open source developers https://github.com/sponsors [51] Learning Pathways https://resources.github.com/learn/pathways [52] Ebooks & Whitepapers https://github.com/resources/whitepapers [53] Customer Stories https://github.com/customer-stories [54] Executive Insights https://github.com/solutions/executive-insights [55] AI https://github.com/resources/articles/ai [56] DevOps https://github.com/resources/articles/devops [57] Security https://github.com/resources/articles/security [58] Software Development https://github.com/resources/articles/software-development [59] View all https://github.com/resources/articles [60] Healthcare https://github.com/solutions/industry/healthcare [61] Financial services https://github.com/solutions/industry/financial-services [62] Manufacturing https://github.com/solutions/industry/manufacturing [63] Government https://github.com/solutions/industry/government [64] View all industries https://github.com/solutions/industry [65] DevSecOps https://github.com/solutions/use-case/devsecops [66] DevOps https://github.com/solutions/use-case/devops [67] CI/CD https://github.com/solutions/use-case/ci-cd [68] View all use cases https://github.com/solutions/use-case [69] Startups https://github.com/enterprise/startups [70] Nonprofits https://github.com/solutions/industry/nonprofits [71] Security Find and fix vulnerabilities https://github.com/features/security [72] Actions Automate any workflow https://github.com/features/actions [73] Codespaces Instant dev environments https://github.com/features/codespaces [74] Issues Plan and track work https://github.com/features/issues [75] Code Review Manage code changes https://github.com/features/code-review [76] Discussions Collaborate outside of code https://github.com/features/discussions [77] Code Search Find more, search less https://github.com/features/code-search [78] View all solutions https://github.com/solutions [79] Skip to content https://github.com/tjcw/bpf-examples/tree/tjcw-explore-sameeth#start-of-content [80] https://github.com/ [81] Sign in https://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Ftjcw%2Fbpf-examples%2Ftree%2Ftjcw-explore-sameeth [82] Forgot password? https://github.com/password_reset [83] Sign up https://github.com/signup?ref_cta=Sign+up&ref_loc=header+logged+out&ref_page=%2Ftjcw%2Fbpf-examples%2Ftree%2Ftjcw-explore-sameeth&source=header [84] Contact Support https://support.github.com/?tags=dotcom-404 [85] GitHub Status https://githubstatus.com/ [86] @githubstatus https://twitter.com/githubstatus [87] Subscribe https://resources.github.com/newsletter/ ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: eBPF verifier does not load libxdp dispatcher eBPF program 2025-02-25 15:55 ` Marc Kleine-Budde 2025-02-25 18:18 ` Eduard Zingerman @ 2025-02-25 18:21 ` Eduard Zingerman 2025-02-25 22:57 ` Marc Kleine-Budde 1 sibling, 1 reply; 8+ messages in thread From: Eduard Zingerman @ 2025-02-25 18:21 UTC (permalink / raw) To: Marc Kleine-Budde, Chris Ward, Alexei Starovoitov, Daniel Borkmann, John Fastabend Cc: linux-kernel, Chris Ward, bpf On Tue, 2025-02-25 at 16:55 +0100, Marc Kleine-Budde wrote: > On 23.01.2023 12:35:41, Chris Ward wrote: > > The 5.15.0 kernel (built by 'git checkout v5.15' from the kernel.org > > torvalds tree) fails in the same way that the 6.2.0-rc5+ kernel fails. > > So it seems that something Canonical did for the Ubuntu 20.04 kernel > > causes eBPF to work correctly. > > > > On Mon, 23 Jan 2023 at 11:06, Chris Ward <tjcw01@gmail.com> wrote: > > > > > > I am trying to use the 'bleeding edge' kernel to determine whether a > > > problem I see has already been fixed, but with this kernel the eBPF > > > verifier will not load the dispatcher program that is contained within > > > libxdp. I am testing kernel commit hash 2475bf0 which fails, and the > > > kernel in Ubuntu 22.04 (5.15.0-58-generic) works properly. I am > > > running the test case from > > > https://github.com/tjcw/bpf-examples/tree/tjcw-explore-sameeth ; to > > > build it go to the AF_XDP-filter directory and type 'make', and to run > > > it go to the AF_XDP-filter/runscripts/iperf3-namespace directory and > > > type 'sudo FILTER=af_xdp_kern PORT=50000 ./run.sh' . > > > The lines from the run output indicating the failure are > > > libbpf: prog 'xdp_dispatcher': BPF program load failed: Invalid argument > > > libbpf: prog 'xdp_dispatcher': -- BEGIN PROG LOAD LOG -- > > > Func#11 is safe for any args that match its prototype > > > btf_vmlinux is malformed > > > reg type unsupported for arg#0 function xdp_dispatcher#29 > > > 0: R1=ctx(off=0,imm=0) R10=fp0 > > > ; int xdp_dispatcher(struct xdp_md *ctx) > > > 0: (bf) r6 = r1 ; R1=ctx(off=0,imm=0) > > > R6_w=ctx(off=0,imm=0) > > > 1: (b7) r0 = 2 ; R0_w=2 > > > ; __u8 num_progs_enabled = conf.num_progs_enabled; > > > 2: (18) r8 = 0xffffb2f6c06d8000 ; R8_w=map_value(off=0,ks=4,vs=84,imm=0) > > > 4: (71) r7 = *(u8 *)(r8 +0) ; R7=1 > > > R8=map_value(off=0,ks=4,vs=84,imm=0) > > > ; if (num_progs_enabled < 1) > > > 5: (15) if r7 == 0x0 goto pc+141 ; R7=1 > > > ; ret = prog0(ctx); > > > 6: (bf) r1 = r6 ; R1_w=ctx(off=0,imm=0) > > > R6=ctx(off=0,imm=0) > > > 7: (85) call pc+140 > > > btf_vmlinux is malformed > > > R1 type=ctx expected=fp > > > Caller passes invalid args into func#1 > > > processed 84 insns (limit 1000000) max_states_per_insn 0 total_states > > > 9 peak_states 9 mark_read 1 > > > -- END PROG LOAD LOG -- > > > libbpf: prog 'xdp_dispatcher': failed to load: -22 > > > libbpf: failed to load object 'xdp-dispatcher.o' > > > libxdp: Failed to load dispatcher: Invalid argument > > > libxdp: Falling back to loading single prog without dispatcher > > > > > > Can this regression be fixed before kernel 6.2 ships ? > > I'm seeing the same failure on 32 bit ARM on v6.13. > > Have you found a solution? > > regards, > Marc > Hello, When I try the link from the discussion: https://github.com/tjcw/bpf-examples/tree/tjcw-explore-sameeth I get a 404 error from github. (Sorry for previous email, it was sent unintentionally). ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: eBPF verifier does not load libxdp dispatcher eBPF program 2025-02-25 18:21 ` Eduard Zingerman @ 2025-02-25 22:57 ` Marc Kleine-Budde 2025-02-26 16:01 ` Marc Kleine-Budde 0 siblings, 1 reply; 8+ messages in thread From: Marc Kleine-Budde @ 2025-02-25 22:57 UTC (permalink / raw) To: Eduard Zingerman Cc: Chris Ward, Alexei Starovoitov, Daniel Borkmann, John Fastabend, linux-kernel, Chris Ward, bpf [-- Attachment #1: Type: text/plain, Size: 8080 bytes --] On 25.02.2025 10:21:11, Eduard Zingerman wrote: > On Tue, 2025-02-25 at 16:55 +0100, Marc Kleine-Budde wrote: > > On 23.01.2023 12:35:41, Chris Ward wrote: > > > The 5.15.0 kernel (built by 'git checkout v5.15' from the kernel.org > > > torvalds tree) fails in the same way that the 6.2.0-rc5+ kernel fails. > > > So it seems that something Canonical did for the Ubuntu 20.04 kernel > > > causes eBPF to work correctly. > > > > > > On Mon, 23 Jan 2023 at 11:06, Chris Ward <tjcw01@gmail.com> wrote: > > > > > > > > I am trying to use the 'bleeding edge' kernel to determine whether a > > > > problem I see has already been fixed, but with this kernel the eBPF > > > > verifier will not load the dispatcher program that is contained within > > > > libxdp. I am testing kernel commit hash 2475bf0 which fails, and the > > > > kernel in Ubuntu 22.04 (5.15.0-58-generic) works properly. I am > > > > running the test case from > > > > https://github.com/tjcw/bpf-examples/tree/tjcw-explore-sameeth ; to > > > > build it go to the AF_XDP-filter directory and type 'make', and to run > > > > it go to the AF_XDP-filter/runscripts/iperf3-namespace directory and > > > > type 'sudo FILTER=af_xdp_kern PORT=50000 ./run.sh' . > > > > The lines from the run output indicating the failure are > > > > libbpf: prog 'xdp_dispatcher': BPF program load failed: Invalid argument > > > > libbpf: prog 'xdp_dispatcher': -- BEGIN PROG LOAD LOG -- > > > > Func#11 is safe for any args that match its prototype > > > > btf_vmlinux is malformed > > > > reg type unsupported for arg#0 function xdp_dispatcher#29 > > > > 0: R1=ctx(off=0,imm=0) R10=fp0 > > > > ; int xdp_dispatcher(struct xdp_md *ctx) > > > > 0: (bf) r6 = r1 ; R1=ctx(off=0,imm=0) > > > > R6_w=ctx(off=0,imm=0) > > > > 1: (b7) r0 = 2 ; R0_w=2 > > > > ; __u8 num_progs_enabled = conf.num_progs_enabled; > > > > 2: (18) r8 = 0xffffb2f6c06d8000 ; R8_w=map_value(off=0,ks=4,vs=84,imm=0) > > > > 4: (71) r7 = *(u8 *)(r8 +0) ; R7=1 > > > > R8=map_value(off=0,ks=4,vs=84,imm=0) > > > > ; if (num_progs_enabled < 1) > > > > 5: (15) if r7 == 0x0 goto pc+141 ; R7=1 > > > > ; ret = prog0(ctx); > > > > 6: (bf) r1 = r6 ; R1_w=ctx(off=0,imm=0) > > > > R6=ctx(off=0,imm=0) > > > > 7: (85) call pc+140 > > > > btf_vmlinux is malformed > > > > R1 type=ctx expected=fp > > > > Caller passes invalid args into func#1 > > > > processed 84 insns (limit 1000000) max_states_per_insn 0 total_states > > > > 9 peak_states 9 mark_read 1 > > > > -- END PROG LOAD LOG -- > > > > libbpf: prog 'xdp_dispatcher': failed to load: -22 > > > > libbpf: failed to load object 'xdp-dispatcher.o' > > > > libxdp: Failed to load dispatcher: Invalid argument > > > > libxdp: Falling back to loading single prog without dispatcher > > > > > > > > Can this regression be fixed before kernel 6.2 ships ? > > > > I'm seeing the same failure on 32 bit ARM on v6.13. > > > > Have you found a solution? > When I try the link from the discussion: > https://github.com/tjcw/bpf-examples/tree/tjcw-explore-sameeth > I get a 404 error from github. I'm have the same error as Chris Ward wrote in their original mail. But I'm using the xdp-tutorial's [1] basic01-xdp-pass/xdp_pass_user example. [1] https://github.com/xdp-project/xdp-tutorial.git This is my error message. | sudo ./xdp_pass_user -d lan0 | libbpf: elf: skipping unrecognized data section(7) xdp_metadata | libbpf: elf: skipping unrecognized data section(7) xdp_metadata | libbpf: prog 'xdp_dispatcher': BPF program load failed: Invalid argument | libbpf: prog 'xdp_dispatcher': -- BEGIN PROG LOAD LOG -- | btf_vmlinux is malformed ^^^^^^^^^^^^^^^^^^^^^^^^ Now I understand, what this error message wants to tell me. I should recompile my kernel with CONFIG_DEBUG_INFO_BTF=y. | 0: R1=ctx() R10=fp0 | ; int xdp_dispatcher(struct xdp_md *ctx) @ xdp-dispatcher.c:118 | 0: (bf) r6 = r1 ; R1=ctx() R6_w=ctx() | ; __u8 num_progs_enabled = conf.num_progs_enabled; @ xdp-dispatcher.c:120 | 1: (18) r8 = 0xc3b45cc8 ; R8_w=map_value(map=xdp_disp.rodata,ks=4,vs=124) | 3: (71) r7 = *(u8 *)(r8 +2) ; R7_w=1 R8_w=map_value(map=xdp_disp.rodata,ks=4,vs=124) | 4: (b7) r0 = 2 ; R0_w=2 | ; if (num_progs_enabled < 1) @ xdp-dispatcher.c:123 | 5: (15) if r7 == 0x0 goto pc+136 ; R7_w=1 | ; ret = prog0(ctx); @ xdp-dispatcher.c:125 | 6: (bf) r1 = r6 ; R1_w=ctx() R6_w=ctx() | 7: (85) call pc+135 | btf_vmlinux is malformed | R1 type=ctx expected=fp | Caller passes invalid args into func#1 ('prog0') | processed 7 insns (limit 1000000) max_states_per_insn 0 total_states 0 peak_states 0 mark_read 0 | -- END PROG LOAD LOG -- | libbpf: prog 'xdp_dispatcher': failed to load: -22 | libbpf: failed to load object 'xdp-dispatcher.o' | libxdp: Failed to load dispatcher: Invalid argument | libxdp: Falling back to loading single prog without dispatcher | libbpf: elf: skipping unrecognized data section(7) xdp_metadata | Success: Loading XDP prog name:xdp_prog_simple(id:118) on device:lan0(ifindex:4) With the CONFIG_DEBUG_INFO_BTF=y kernel the verifier seems to be more happy. Now it fails with "-22": | sudo ./xdp_pass_user -d lan0 | libbpf: elf: skipping unrecognized data section(7) xdp_metadata | libbpf: elf: skipping unrecognized data section(7) xdp_metadata | libbpf: elf: skipping unrecognized data section(7) xdp_metadata | libbpf: elf: skipping unrecognized data section(7) xdp_metadata | libbpf: prog 'xdp_pass': BPF program load failed: Invalid argument | libbpf: prog 'xdp_pass': -- BEGIN PROG LOAD LOG -- | Extension programs should be JITed | processed 0 insns (limit 1000000) max_states_per_insn 0 total_states 0 peak_states 0 mark_read 0 | -- END PROG LOAD LOG -- | libbpf: prog 'xdp_pass': failed to load: -22 | libbpf: failed to load object 'xdp-dispatcher.o' | libxdp: Compatibility check for dispatcher program failed: Invalid argument | libxdp: Falling back to loading single prog without dispatcher | libbpf: elf: skipping unrecognized data section(7) xdp_metadata | Success: Loading XDP prog name:xdp_prog_simple(id:20) on device:lan0(ifindex:4) After unloading and enabling the JIT... | ➜ (pts/0) frogger@riot:xdp-tutorial/basic01-xdp-pass (main✗) echo 1 |sudo tee /proc/sys/net/core/bpf_jit_enable ... the dispatcher fails to load with "524". Yes, the number is positive. | ➜ (pts/0) frogger@riot:xdp-tutorial/basic01-xdp-pass (main✗) sudo ./xdp_pass_user -d lan0 --unload-all | Success: Unloading XDP prog name: xdp_prog_simple | ➜ (pts/0) frogger@riot:xdp-tutorial/basic01-xdp-pass (main✗) sudo ./xdp_pass_user -d lan0 | libbpf: elf: skipping unrecognized data section(7) xdp_metadata | libbpf: elf: skipping unrecognized data section(7) xdp_metadata | libbpf: elf: skipping unrecognized data section(7) xdp_metadata | libbpf: elf: skipping unrecognized data section(7) xdp_metadata | libxdp: Compatibility check for dispatcher program failed: Unknown error 524 | libxdp: Falling back to loading single prog without dispatcher | libbpf: elf: skipping unrecognized data section(7) xdp_metadata | Success: Loading XDP prog name:xdp_prog_simple(id:48) on device:lan0(ifindex:4) strace indicates this syscalls fails: | bpf(BPF_RAW_TRACEPOINT_OPEN, {raw_tracepoint={name=NULL, prog_fd=17}}, 16) = -1 ENOTSUPP (Unknown error 524) I'm on a armv7l, i.e. a 32 bit ARM system. Maybe I'm missing some kernel option or BPF_RAW_TRACEPOINT_OPEN is not supported on armv7l. Will look deeper into the kernel config options tomorrow. regards, Marc -- Pengutronix e.K. | Marc Kleine-Budde | Embedded Linux | https://www.pengutronix.de | Vertretung Nürnberg | Phone: +49-5121-206917-129 | Amtsgericht Hildesheim, HRA 2686 | Fax: +49-5121-206917-9 | [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 488 bytes --] ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: eBPF verifier does not load libxdp dispatcher eBPF program 2025-02-25 22:57 ` Marc Kleine-Budde @ 2025-02-26 16:01 ` Marc Kleine-Budde 0 siblings, 0 replies; 8+ messages in thread From: Marc Kleine-Budde @ 2025-02-26 16:01 UTC (permalink / raw) To: Eduard Zingerman Cc: Chris Ward, Alexei Starovoitov, Daniel Borkmann, John Fastabend, linux-kernel, Chris Ward, bpf [-- Attachment #1: Type: text/plain, Size: 3158 bytes --] On 25.02.2025 23:57:47, Marc Kleine-Budde wrote: [...] > With the CONFIG_DEBUG_INFO_BTF=y kernel the verifier seems to be more > happy. Now it fails with "-22": > > | sudo ./xdp_pass_user -d lan0 > | libbpf: elf: skipping unrecognized data section(7) xdp_metadata > | libbpf: elf: skipping unrecognized data section(7) xdp_metadata > | libbpf: elf: skipping unrecognized data section(7) xdp_metadata > | libbpf: elf: skipping unrecognized data section(7) xdp_metadata > | libbpf: prog 'xdp_pass': BPF program load failed: Invalid argument > | libbpf: prog 'xdp_pass': -- BEGIN PROG LOAD LOG -- > | Extension programs should be JITed > | processed 0 insns (limit 1000000) max_states_per_insn 0 total_states 0 peak_states 0 mark_read 0 > | -- END PROG LOAD LOG -- > | libbpf: prog 'xdp_pass': failed to load: -22 > | libbpf: failed to load object 'xdp-dispatcher.o' > | libxdp: Compatibility check for dispatcher program failed: Invalid argument > | libxdp: Falling back to loading single prog without dispatcher > | libbpf: elf: skipping unrecognized data section(7) xdp_metadata > | Success: Loading XDP prog name:xdp_prog_simple(id:20) on device:lan0(ifindex:4) > > > After unloading and enabling the JIT... > > | ➜ (pts/0) frogger@riot:xdp-tutorial/basic01-xdp-pass (main✗) echo 1 |sudo tee /proc/sys/net/core/bpf_jit_enable > > ... the dispatcher fails to load with "524". Yes, the number is > positive. > > | ➜ (pts/0) frogger@riot:xdp-tutorial/basic01-xdp-pass (main✗) sudo ./xdp_pass_user -d lan0 --unload-all > | Success: Unloading XDP prog name: xdp_prog_simple > | ➜ (pts/0) frogger@riot:xdp-tutorial/basic01-xdp-pass (main✗) sudo ./xdp_pass_user -d lan0 > | libbpf: elf: skipping unrecognized data section(7) xdp_metadata > | libbpf: elf: skipping unrecognized data section(7) xdp_metadata > | libbpf: elf: skipping unrecognized data section(7) xdp_metadata > | libbpf: elf: skipping unrecognized data section(7) xdp_metadata > | libxdp: Compatibility check for dispatcher program failed: Unknown error 524 > | libxdp: Falling back to loading single prog without dispatcher > | libbpf: elf: skipping unrecognized data section(7) xdp_metadata > | Success: Loading XDP prog name:xdp_prog_simple(id:48) on device:lan0(ifindex:4) > > strace indicates this syscalls fails: > > | bpf(BPF_RAW_TRACEPOINT_OPEN, {raw_tracepoint={name=NULL, prog_fd=17}}, 16) = -1 ENOTSUPP (Unknown error 524) > > I'm on a armv7l, i.e. a 32 bit ARM system. Maybe I'm missing some kernel > option or BPF_RAW_TRACEPOINT_OPEN is not supported on armv7l. Will look > deeper into the kernel config options tomorrow. FTR: I figured out that the dispatcher needs eBPF trampoline support, which doesn't exist on ARM. | https://docs.ebpf.io/linux/concepts/trampolines/ regards, Marc -- Pengutronix e.K. | Marc Kleine-Budde | Embedded Linux | https://www.pengutronix.de | Vertretung Nürnberg | Phone: +49-5121-206917-129 | Amtsgericht Hildesheim, HRA 2686 | Fax: +49-5121-206917-9 | [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 488 bytes --] ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: eBPF verifier does not load libxdp dispatcher eBPF program @ 2023-01-23 13:23 Chris Ward 0 siblings, 0 replies; 8+ messages in thread From: Chris Ward @ 2023-01-23 13:23 UTC (permalink / raw) To: linux-kernel, Chris Ward The 5.15.0 kernel (built by 'git checkout v5.15' from the kernel.org torvalds tree) fails in the same way that the 6.2.0-rc5+ kernel fails. So it seems that something Canonical did for the Ubuntu 20.04 kernel causes eBPF to work correctly. Chris Ward ^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2025-02-26 16:01 UTC | newest] Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2023-01-23 11:06 eBPF verifier does not load libxdp dispatcher eBPF program Chris Ward 2023-01-23 12:35 ` Chris Ward 2025-02-25 15:55 ` Marc Kleine-Budde 2025-02-25 18:18 ` Eduard Zingerman 2025-02-25 18:21 ` Eduard Zingerman 2025-02-25 22:57 ` Marc Kleine-Budde 2025-02-26 16:01 ` Marc Kleine-Budde 2023-01-23 13:23 Chris Ward
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox
Powered by JetHome