syslog spam: TCP segment has incorrect auth options set

syslog spam: TCP segment has incorrect auth options set

[Christian Kujau]

Ever since commit 2717b5adea9e ("net/tcp: Add tcp_hash_fail() ratelimited 
logs") the following is printed, in waves of small floods, to syslog:

 kernel: TCP: TCP segment has incorrect auth options set for XX.20.239.12.54681->XX.XX.90.103.80 [S]

This host is connected to the open internet and serves as a small HTTP and 
SSH login server, not much traffic is happening here. So I'd assume these 
messages to be the result of random internet scans and/or fingerprinting 
attempts or the like. While not really a concern, these messages are 
flooding the dmesg buffer over time :-(

Is there a way to adjust the severity of these messages?

* In include/net/tcp.h this gets logged with tcp_hash_fail(), which is
* defined in include/net/tcp_ao.h and calls net_info_ratelimited(), which
* is in turn defined in include/linux/net.h and calls pr_info().

Can e.g. net_dbg_ratelimited be used instead?

Thanks,
Christian.
-- 
BOFH excuse #78:

Yes, yes, its called a design limitation

Re: syslog spam: TCP segment has incorrect auth options set

[Dmitry Safonov]

Hi Christian,

Thanks for the report,

On 1/4/24 10:55, Christian Kujau wrote:
> Ever since commit 2717b5adea9e ("net/tcp: Add tcp_hash_fail() ratelimited 
> logs") the following is printed, in waves of small floods, to syslog:
> 
>  kernel: TCP: TCP segment has incorrect auth options set for XX.20.239.12.54681->XX.XX.90.103.80 [S]
> 
> This host is connected to the open internet and serves as a small HTTP and 
> SSH login server, not much traffic is happening here. So I'd assume these 
> messages to be the result of random internet scans and/or fingerprinting 
> attempts or the like. While not really a concern, these messages are 
> flooding the dmesg buffer over time :-(
> 
> Is there a way to adjust the severity of these messages?
> 
> * In include/net/tcp.h this gets logged with tcp_hash_fail(), which is
> * defined in include/net/tcp_ao.h and calls net_info_ratelimited(), which
> * is in turn defined in include/linux/net.h and calls pr_info().
> 
> Can e.g. net_dbg_ratelimited be used instead?

Yeah, I guess it's possible to down the severity of these logs, but may
be unexpected by admins: TCP-MD5 messages existed for long time and
there may be userspace that expects them (i.e. in arista there are tests
that look for these specific messages - those would be easy to fix, but
there may be others outside this company).

While thinking on the origin of your issue, it seems that the logs
produced by either TCP-MD5 or TCP-AO are desired by a user when they
add/use the authentication. Could you try this and see if that solves
the issue for you?

https://lore.kernel.org/all/20240104-tcp_hash_fail-logs-v1-1-ff3e1f6f9e72@arista.com/

Thanks,
             Dmitry

Re: syslog spam: TCP segment has incorrect auth options set

[Christian Kujau]

On Thu, 4 Jan 2024, Dmitry Safonov wrote:
> Yeah, I guess it's possible to down the severity of these logs, but may
> be unexpected by admins: TCP-MD5 messages existed for long time and
> there may be userspace that expects them (i.e. in arista there are tests
> that look for these specific messages - those would be easy to fix, but
> there may be others outside this company).

Understood, thanks for explaining that.

> While thinking on the origin of your issue, it seems that the logs
> produced by either TCP-MD5 or TCP-AO are desired by a user when they
> add/use the authentication. Could you try this and see if that solves
> the issue for you?

Thanks for preparing that patch so quickly, did not expect that :-)

I've applied this on top of 6.7.0-rc8 and will report back if I see those 
messages again in the next days.

Thank you!
Christian.
-- 
BOFH excuse #127:

Sticky bits on disk.

Re: syslog spam: TCP segment has incorrect auth options set

[Christian Kujau]

On Thu, 4 Jan 2024, Christian Kujau wrote:
> On Thu, 4 Jan 2024, Dmitry Safonov wrote:
> > Yeah, I guess it's possible to down the severity of these logs, but may
> > be unexpected by admins: TCP-MD5 messages existed for long time and
> > there may be userspace that expects them (i.e. in arista there are tests
> > that look for these specific messages - those would be easy to fix, but
> > there may be others outside this company).
> 
> Understood, thanks for explaining that.
> 
> > While thinking on the origin of your issue, it seems that the logs
> > produced by either TCP-MD5 or TCP-AO are desired by a user when they
> > add/use the authentication. Could you try this and see if that solves
> > the issue for you?
> 
> Thanks for preparing that patch so quickly, did not expect that :-)
> 
> I've applied this on top of 6.7.0-rc8 and will report back if I see those 
> messages again in the next days.

No messages so far, great!

Tested-by: Christian Kujau <lists@nerdbynature.de>

Thanks again for fixing this so quickly,
Christian.
-- 
BOFH excuse #323:

Your processor has processed too many instructions.  Turn it off immediately, do not type any commands!!

Re: syslog spam: TCP segment has incorrect auth options set

[Dmitry Safonov]

On 1/7/24 19:27, Christian Kujau wrote:
> On Thu, 4 Jan 2024, Christian Kujau wrote:
>> On Thu, 4 Jan 2024, Dmitry Safonov wrote:
>>> Yeah, I guess it's possible to down the severity of these logs, but may
>>> be unexpected by admins: TCP-MD5 messages existed for long time and
>>> there may be userspace that expects them (i.e. in arista there are tests
>>> that look for these specific messages - those would be easy to fix, but
>>> there may be others outside this company).
>>
>> Understood, thanks for explaining that.
>>
>>> While thinking on the origin of your issue, it seems that the logs
>>> produced by either TCP-MD5 or TCP-AO are desired by a user when they
>>> add/use the authentication. Could you try this and see if that solves
>>> the issue for you?
>>
>> Thanks for preparing that patch so quickly, did not expect that :-)
>>
>> I've applied this on top of 6.7.0-rc8 and will report back if I see those 
>> messages again in the next days.
> 
> No messages so far, great!
> 
> Tested-by: Christian Kujau <lists@nerdbynature.de>
> 
> Thanks again for fixing this so quickly,

Thank you for the report and the testing.
It's a bit late to add your tag, but on a good side the patch went in
v6.7 release, so you will not have to have any out-of-tree fix for
this :-)

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=4c8530dc7d7d

Thanks,
            Dmitry