mirror of https://lore.kernel.org/lkml/
 help / color / mirror / Atom feed
From: Julian Anastasov <ja@ssi.bg>
To: Zihan Xi <zihanx@nebusec.ai>
Cc: Simon Horman <horms@verge.net.au>,
	Pablo Neira Ayuso <pablo@netfilter.org>,
	Florian Westphal <fw@strlen.de>, Phil Sutter <phil@nwl.cc>,
	"David S. Miller" <davem@davemloft.net>,
	Eric Dumazet <edumazet@google.com>,
	Jakub Kicinski <kuba@kernel.org>, Paolo Abeni <pabeni@redhat.com>,
	netdev@vger.kernel.org, lvs-devel@vger.kernel.org,
	netfilter-devel@vger.kernel.org, coreteam@netfilter.org,
	linux-kernel@vger.kernel.org
Subject: Re: [PATCH net v2 1/2] ipvs: avoid stack overflow from recursive connection expiration
Date: Sat, 19 Sep 2026 13:06:23 +0300 (EEST)	[thread overview]
Message-ID: <1233faca-355c-b1b6-5c8d-9b36b4cc427a@ssi.bg> (raw)
In-Reply-To: <c104a8ee-eebc-1e80-c685-3e22f23ffbc8@ssi.bg>


	Hello,

On Fri, 18 Sep 2026, Julian Anastasov wrote:

> On Thu, 17 Sep 2026, Zihan Xi wrote:
> 
> > When a controlled IPVS connection expires, its controller may be expired
> > synchronously if it has no remaining controlled connections. A chain of
> > controlled connections can then cause recursive calls to
> > ip_vs_conn_expire() and exhaust the kernel stack during namespace cleanup.
> > 
> > Make ip_vs_conn_del_put() report whether it deleted the controller timer.
> > When it succeeds, continue expiration with the controller instead of
> > calling ip_vs_conn_expire() recursively. This keeps chain cleanup
> > synchronous while using one stack frame for the whole chain.
> > 
> > Fixes: f9200a52eedf ("ipvs: avoid expiring many connections from timer")
> > Cc: stable@vger.kernel.org
> > Reported-by: Vega <vega@nebusec.ai>
> > Assisted-by: LLM
> > Co-developed-by: Luxing Yin <root@tr0jan.top>
> > Signed-off-by: Luxing Yin <root@tr0jan.top>
> > Signed-off-by: Zihan Xi <zihanx@nebusec.ai>
> 
> 	Patch looks good to me for the nf tree, thanks!
> 
> Acked-by: Julian Anastasov <ja@ssi.bg>

	In fact, Sashiko detects problem with connections
that are deleted and traffic that can restart the timer and
its callback deleteing the connection:

https://sashiko.dev/#/patchset/cover.1789435989.git.zihanx%40nebusec.ai

	Events are in this order:

CPU 1                   CPU 2
timer_delete,
refcnt is 1
                        find conn, get refcnt
                        mod_timer, put refcnt => 1

                        run timer callback
			and expire the conn,
			refcnt=0

touching cp->control is
safe under RCU,
but we mod_timer
with refcnt=0

	The problem is that the timer callback runs without
conn reference and not under RCU lock. OTOH, we delete the
connection only under RCU read lock and can take measures
if the callback removed the connection before us.

	Dropping conns only via timer callback is something we try
to avoid, we have to rethink this change.

pw-bot: changes-requested

> 	Next time use "nf"/"nf-next" tag for the IPVS patches.
> 
> > ---
> > changes in v2:
> >   - Use a repeat path for controller cleanup so expiration stays
> >     synchronous without recursive calls or extra timer ticks.
> >   - v1 Link:
> >     https://lore.kernel.org/all/cover.1789110326.git.zihanx@nebusec.ai/
> > 
> >  net/netfilter/ipvs/ip_vs_conn.c | 17 ++++++++++++-----
> >  1 file changed, 12 insertions(+), 5 deletions(-)
> > 
> > diff --git a/net/netfilter/ipvs/ip_vs_conn.c b/net/netfilter/ipvs/ip_vs_conn.c
> > index 6fa3e1dc534c3..c7b88ce1765dc 100644
> > --- a/net/netfilter/ipvs/ip_vs_conn.c
> > +++ b/net/netfilter/ipvs/ip_vs_conn.c
> > @@ -1331,17 +1331,18 @@ static void ip_vs_conn_del(struct ip_vs_conn *cp)
> >  }
> >  
> >  /* Try to delete connection while holding reference */
> > -static void ip_vs_conn_del_put(struct ip_vs_conn *cp)
> > +static bool ip_vs_conn_del_put(struct ip_vs_conn *cp)
> >  {
> >  	if (timer_delete(&cp->timer)) {
> >  		/* Drop cp->control chain too */
> >  		if (cp->control)
> >  			cp->timeout = 0;
> >  		__ip_vs_conn_put(cp);
> > -		ip_vs_conn_expire(&cp->timer);
> > -	} else {
> > -		__ip_vs_conn_put(cp);
> > +		return true;
> >  	}
> > +
> > +	__ip_vs_conn_put(cp);
> > +	return false;
> >  }
> >  
> >  static void ip_vs_conn_expire(struct timer_list *t)
> > @@ -1349,6 +1350,7 @@ static void ip_vs_conn_expire(struct timer_list *t)
> >  	struct ip_vs_conn *cp = timer_container_of(cp, t, timer);
> >  	struct netns_ipvs *ipvs = cp->ipvs;
> >  
> > +repeat:
> >  	/*
> >  	 *	do I control anybody?
> >  	 */
> > @@ -1358,6 +1360,7 @@ static void ip_vs_conn_expire(struct timer_list *t)
> >  	/* Unlink conn if not referenced anymore */
> >  	if (likely(ip_vs_conn_unlink(cp))) {
> >  		struct ip_vs_conn *ct = cp->control;
> > +		bool next = false;
> >  
> >  		/* delete the timer if it is activated by other users */
> >  		timer_delete(&cp->timer);
> > @@ -1372,7 +1375,7 @@ static void ip_vs_conn_expire(struct timer_list *t)
> >  			    (!(ct->flags & IP_VS_CONN_F_TEMPLATE) ||
> >  			     !(ct->state & IP_VS_CTPL_S_ASSURED))) {
> >  				IP_VS_DBG(4, "drop controlling connection\n");
> > -				ip_vs_conn_del_put(ct);
> > +				next = ip_vs_conn_del_put(ct);
> >  			} else if (has_ref) {
> >  				__ip_vs_conn_put(ct);
> >  			}
> > @@ -1402,6 +1405,10 @@ static void ip_vs_conn_expire(struct timer_list *t)
> >  		else
> >  			call_rcu(&cp->rcu_head, ip_vs_conn_rcu_free);
> >  		atomic_dec(&ipvs->conn_count);
> > +		if (next) {
> > +			cp = ct;
> > +			goto repeat;
> > +		}
> >  		return;
> >  	}
> >  
> > -- 
> > 2.43.0

Regards

--
Julian Anastasov <ja@ssi.bg>


  reply	other threads:[~2026-09-19 10:06 UTC|newest]

Thread overview: 10+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-09-17  3:02 [PATCH net v2 0/2] " Zihan Xi
2026-09-17  3:02 ` [PATCH net v2 1/2] " Zihan Xi
2026-09-18 16:49   ` Julian Anastasov
2026-09-19 10:06     ` Julian Anastasov [this message]
2026-09-21  3:19   ` netdev-bot+sashiko
2026-09-21  6:50     ` Julian Anastasov
2026-09-21  7:10       ` zihan xi
2026-09-17  3:02 ` [PATCH net v2 2/2] ipvs: reject FTP control ports as data ports Zihan Xi
2026-09-18 16:54   ` Julian Anastasov
2026-09-19 10:08     ` Julian Anastasov

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1233faca-355c-b1b6-5c8d-9b36b4cc427a@ssi.bg \
    --to=ja@ssi.bg \
    --cc=coreteam@netfilter.org \
    --cc=davem@davemloft.net \
    --cc=edumazet@google.com \
    --cc=fw@strlen.de \
    --cc=horms@verge.net.au \
    --cc=kuba@kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=lvs-devel@vger.kernel.org \
    --cc=netdev@vger.kernel.org \
    --cc=netfilter-devel@vger.kernel.org \
    --cc=pabeni@redhat.com \
    --cc=pablo@netfilter.org \
    --cc=phil@nwl.cc \
    --cc=zihanx@nebusec.ai \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox

all inboxes | Powered by JetHome®