Hi all I have discovered a bug in the advanced routing / masquerading code 2 weeks ago and have tried to make a patch since (patch attached after the bug description). The bug is reproduceable: 1. you need 2 PCs (at minimum), each one with a network card 2. PC 1 acts as a source-based, masquerading router 3. PC 2 acts as the client, using PC 1 as default gateway You should have something like this (both addresses of PC1 can be on one interface without problems): ----- 10.0.3.2 10.0.3.1 ----- 10.0.1.2 10.0.1.1 ------ |PC2|---------------------|PC1|---------------------| GW |-... ----- ----- ------ 4. setup PC 1 as follows: ip addr add 10.0.1.2/24 dev eth0 ip addr add 10.0.3.1/24 dev eth0 ip rule add prio 1 table main ip route add table 2 default via 10.0.1.1 src 10.0.1.2 dev eth0 ip rule add prio 102 iif eth0 from 10.0.3.0/24 nat 10.0.1.2 table 2 ip rule add prio 32000 iif lo table 2 ip route flush cache echo 1 > /proc/sys/net/ipv4/ip_forward echo 3 > /proc/sys/net/ipv4/ip_masq_debug for i in /proc/sys/net/ipv4/conf/*/log_martians; do echo 1 > $i done for i in /proc/sys/net/ipv4/conf/*/rp_filter; do echo 1 > $i done ipchains -I input -l ipchains -I output -l ipchains -I forward -l 5. setup PC 2 as follows: ip addr add 10.0.3.2/24 dev eth0 ip route add default via 10.0.3.1 6. on PC 2, do ping 10.0.1.1 (or ping if PC 1 is connected to others) You can see that the ping works and that PC 1 does masquerading. 7. on PC 1, do ip rule del prio 32000 Now PC 1 has no default route (no default route for packets coming from iface lo), pings do not work from PC 1 - that's ok. 8. on PC 2, do ping 10.0.1.1 (or ping ) Now the ping does not work. If you look at the syslog on PC 1, you can see that it masquerades the packets coming from 10.0.3.2 as 10.0.1.2, which is perfectly ok. Then the packets come back from 10.0.1.1 to 10.0.1.2 (they come through the input chain) and get thrown away. They do not appear in the output chain, just silently dropped (if log_martians is switched on, there is a message). I was able to track the bug down to the following: In net/ipv4/route.c, function ip_route_input_slow, line 1183, fib_validate_source is called. There, the reverse fib_lookup in line 207 fails because there is no routing entry for packets coming -from- iface lo. After that, fib_validate_source returns an error in line 234 when rp_filter is activated. This causes ip_route_input_slow to drop the packet and log a "martian source". I think this behaviour should be changed, because the kernel masqueraded the packets in the forward direction, but does not allow them to be masqueraded. This patch solves the problem for me, but I am completely unsure about the performance impact of calling ip_fw_demasquerade and I am also unsure if it changes the packet in a bad way during the check. Please see this patch as an idea on what should be changed. I do not understand the masquerading code fully and therefore I do not claim that there aren't side-effects. Also this is my first kernel patch, so please be patient with me :) Please could somebody comment on the solution ? Would it be better to check this in fib_validate_source ? I think so, but then I would need to change the interface of fib_validate_source. The problem is still there in 2.3.99-pre5 (the codes are equal), but my patch can't work on 2.3.99, since there is no ip_fw_demasquerade and I don't know anything about how the netfilter framework does masquerading now. However, the same procedure (with the ipchains module loaded of course) can be used to reproduce the bug with 2.3.99-pre5. Please reply directly to me, since I am not subscribed to the mailing list. best greets, Rene Mayrhofer