summaryrefslogtreecommitdiff
path: root/net/ipv6
AgeCommit message (Collapse)Author
2019-04-11ipv6: Refactor __ip6_route_redirectDavid Ahern
Move the nexthop evaluation of a fib entry to a helper that can be leveraged for each fib6_nh in a multipath nexthop object. In the move, 'continue' statements means the helper returns false (loop should continue) and 'break' means return true (found the entry of interest). Signed-off-by: David Ahern <dsahern@gmail.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2019-04-11ipv6: Refactor rt6_device_matchDavid Ahern
Move the device and gateway checks in the fib6_next loop to a helper that can be called per fib6_nh entry. Signed-off-by: David Ahern <dsahern@gmail.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2019-04-11ipv6: Move fib6_multipath_select down in ip6_pol_routeDavid Ahern
Move the siblings and fib6_multipath_select after the null entry check since a null entry can not have siblings. Signed-off-by: David Ahern <dsahern@gmail.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2019-04-11ipv6: Be smarter with null_entry handling in ip6_pol_route_lookupDavid Ahern
Clean up the fib6_null_entry handling in ip6_pol_route_lookup. rt6_device_match can return fib6_null_entry, but fib6_multipath_select can not. Consolidate the fib6_null_entry handling and on the final null_entry check set rt and goto out - no need to defer to a second check after rt6_find_cached_rt. Signed-off-by: David Ahern <dsahern@gmail.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2019-04-11ipv6: Refactor find_rr_leafDavid Ahern
find_rr_leaf has 3 loops over fib_entries calling find_match. The loops are very similar with differences in start point and whether the metric is evaluated: 1. start at rr_head, no extra loop compare, check fib metric 2. start at leaf, compare rt against rr_head, check metric 3. start at cont (potential saved point from earlier loops), no extra loop compare, no metric check Create 1 loop that is called 3 different times. This will make a later change with multipath nexthop objects much simpler. Signed-off-by: David Ahern <dsahern@gmail.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2019-04-11ipv6: Refactor find_matchDavid Ahern
find_match primarily needs a fib6_nh (and fib6_flags which it passes through to rt6_score_route). Move fib6_check_expired up to the call sites so find_match is only called for relevant entries. Remove the match argument which is mostly a pass through and use the return boolean to decide if match gets set in the call sites. The end result is a helper that can be called per fib6_nh struct which is needed once fib entries reference nexthop objects that have more than one fib6_nh. Signed-off-by: David Ahern <dsahern@gmail.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2019-04-11ipv6: Pass fib6_nh and flags to rt6_score_routeDavid Ahern
rt6_score_route only needs the fib6_flags and nexthop data. Change it accordingly. Allows re-use later for nexthop based fib6_nh. Signed-off-by: David Ahern <dsahern@gmail.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2019-04-11ipv6: Change rt6_probe to take a fib6_nhDavid Ahern
rt6_probe sends probes for gateways in a nexthop. As such it really depends on a fib6_nh, not a fib entry. Move last_probe to fib6_nh and update rt6_probe to a fib6_nh struct. Signed-off-by: David Ahern <dsahern@gmail.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2019-04-11ipv6: Remove rt6_check_devDavid Ahern
rt6_check_dev is a simpler helper with only 1 caller. Fold the code into rt6_score_route. Signed-off-by: David Ahern <dsahern@gmail.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2019-04-11ipv6: Only call rt6_check_neigh for nexthop with gatewayDavid Ahern
Change rt6_check_neigh to take a fib6_nh instead of a fib entry. Move the check on fib_flags and whether the nexthop has a gateway up to the one caller. Remove the inline from the definition as well. Not necessary. Signed-off-by: David Ahern <dsahern@gmail.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2019-04-08Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/netDavid S. Miller
2019-04-08net: ip6_gre: fix possible use-after-free in ip6erspan_rcvLorenzo Bianconi
erspan_v6 tunnels run __iptunnel_pull_header on received skbs to remove erspan header. This can determine a possible use-after-free accessing pkt_md pointer in ip6erspan_rcv since the packet will be 'uncloned' running pskb_expand_head if it is a cloned gso skb (e.g if the packet has been sent though a veth device). Fix it resetting pkt_md pointer after __iptunnel_pull_header Fixes: 1d7e2ed22f8d ("net: erspan: refactor existing erspan code") Signed-off-by: Lorenzo Bianconi <lorenzo.bianconi@redhat.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2019-04-08neighbor: Add skip_cache argument to neigh_outputDavid Ahern
A later patch allows an IPv6 gateway with an IPv4 route. The neighbor entry will exist in the v6 ndisc table and the cached header will contain the ipv6 protocol which is wrong for an IPv4 packet. For an IPv4 packet to use the v6 neighbor entry, neigh_output needs to skip the cached header and just use the output callback for the neigh entry. A future patchset can look at expanding the hh_cache to handle 2 protocols. For now, IPv6 gateways with an IPv4 route will take the extra overhead of generating the header. Signed-off-by: David Ahern <dsahern@gmail.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2019-04-08net: Replace nhc_has_gw with nhc_gw_familyDavid Ahern
Allow the gateway in a fib_nh_common to be from a different address family than the outer fib{6}_nh. To that end, replace nhc_has_gw with nhc_gw_family and update users of nhc_has_gw to check nhc_gw_family. Now nhc_family is used to know if the nh_common is part of a fib_nh or fib6_nh (used for container_of to get to route family specific data), and nhc_gw_family represents the address family for the gateway. Signed-off-by: David Ahern <dsahern@gmail.com> Reviewed-by: Ido Schimmel <idosch@mellanox.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2019-04-08ipv6: Add fib6_nh_init and release to stubsDavid Ahern
Add fib6_nh_init and fib6_nh_release to ipv6_stubs. If fib6_nh_init fails, callers should not invoke fib6_nh_release, so there is no reason to have a dummy stub for the IPv6 is not enabled case. Signed-off-by: David Ahern <dsahern@gmail.com> Reviewed-by: Ido Schimmel <idosch@mellanox.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2019-04-08datagram: remove rendundant 'peeked' argumentPaolo Abeni
After commit a297569fe00a ("net/udp: do not touch skb->peeked unless really needed") the 'peeked' argument of __skb_try_recv_datagram() and friends is always equal to !!'flags & MSG_PEEK'. Since such argument is really a boolean info, and the callers have already 'flags & MSG_PEEK' handy, we can remove it and clean-up the code a bit. Signed-off-by: Paolo Abeni <pabeni@redhat.com> Acked-by: Willem de Bruijn <willemb@google.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2019-04-07rhashtable: use bit_spin_locks to protect hash bucket.NeilBrown
This patch changes rhashtables to use a bit_spin_lock on BIT(1) of the bucket pointer to lock the hash chain for that bucket. The benefits of a bit spin_lock are: - no need to allocate a separate array of locks. - no need to have a configuration option to guide the choice of the size of this array - locking cost is often a single test-and-set in a cache line that will have to be loaded anyway. When inserting at, or removing from, the head of the chain, the unlock is free - writing the new address in the bucket head implicitly clears the lock bit. For __rhashtable_insert_fast() we ensure this always happens when adding a new key. - even when lockings costs 2 updates (lock and unlock), they are in a cacheline that needs to be read anyway. The cost of using a bit spin_lock is a little bit of code complexity, which I think is quite manageable. Bit spin_locks are sometimes inappropriate because they are not fair - if multiple CPUs repeatedly contend of the same lock, one CPU can easily be starved. This is not a credible situation with rhashtable. Multiple CPUs may want to repeatedly add or remove objects, but they will typically do so at different buckets, so they will attempt to acquire different locks. As we have more bit-locks than we previously had spinlocks (by at least a factor of two) we can expect slightly less contention to go with the slightly better cache behavior and reduced memory consumption. To enhance type checking, a new struct is introduced to represent the pointer plus lock-bit that is stored in the bucket-table. This is "struct rhash_lock_head" and is empty. A pointer to this needs to be cast to either an unsigned lock, or a "struct rhash_head *" to be useful. Variables of this type are most often called "bkt". Previously "pprev" would sometimes point to a bucket, and sometimes a ->next pointer in an rhash_head. As these are now different types, pprev is NULL when it would have pointed to the bucket. In that case, 'blk' is used, together with correct locking protocol. Signed-off-by: NeilBrown <neilb@suse.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2019-04-05Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/netDavid S. Miller
Minor comment merge conflict in mlx5. Staging driver has a fixup due to the skb->xmit_more changes in 'net-next', but was removed in 'net'. Signed-off-by: David S. Miller <davem@davemloft.net>
2019-04-04ipv6: sit: reset ip header pointer in ipip6_rcvLorenzo Bianconi
ipip6 tunnels run iptunnel_pull_header on received skbs. This can determine the following use-after-free accessing iph pointer since the packet will be 'uncloned' running pskb_expand_head if it is a cloned gso skb (e.g if the packet has been sent though a veth device) [ 706.369655] BUG: KASAN: use-after-free in ipip6_rcv+0x1678/0x16e0 [sit] [ 706.449056] Read of size 1 at addr ffffe01b6bd855f5 by task ksoftirqd/1/= [ 706.669494] Hardware name: HPE ProLiant m400 Server/ProLiant m400 Server, BIOS U02 08/19/2016 [ 706.771839] Call trace: [ 706.801159] dump_backtrace+0x0/0x2f8 [ 706.845079] show_stack+0x24/0x30 [ 706.884833] dump_stack+0xe0/0x11c [ 706.925629] print_address_description+0x68/0x260 [ 706.982070] kasan_report+0x178/0x340 [ 707.025995] __asan_report_load1_noabort+0x30/0x40 [ 707.083481] ipip6_rcv+0x1678/0x16e0 [sit] [ 707.132623] tunnel64_rcv+0xd4/0x200 [tunnel4] [ 707.185940] ip_local_deliver_finish+0x3b8/0x988 [ 707.241338] ip_local_deliver+0x144/0x470 [ 707.289436] ip_rcv_finish+0x43c/0x14b0 [ 707.335447] ip_rcv+0x628/0x1138 [ 707.374151] __netif_receive_skb_core+0x1670/0x2600 [ 707.432680] __netif_receive_skb+0x28/0x190 [ 707.482859] process_backlog+0x1d0/0x610 [ 707.529913] net_rx_action+0x37c/0xf68 [ 707.574882] __do_softirq+0x288/0x1018 [ 707.619852] run_ksoftirqd+0x70/0xa8 [ 707.662734] smpboot_thread_fn+0x3a4/0x9e8 [ 707.711875] kthread+0x2c8/0x350 [ 707.750583] ret_from_fork+0x10/0x18 [ 707.811302] Allocated by task 16982: [ 707.854182] kasan_kmalloc.part.1+0x40/0x108 [ 707.905405] kasan_kmalloc+0xb4/0xc8 [ 707.948291] kasan_slab_alloc+0x14/0x20 [ 707.994309] __kmalloc_node_track_caller+0x158/0x5e0 [ 708.053902] __kmalloc_reserve.isra.8+0x54/0xe0 [ 708.108280] __alloc_skb+0xd8/0x400 [ 708.150139] sk_stream_alloc_skb+0xa4/0x638 [ 708.200346] tcp_sendmsg_locked+0x818/0x2b90 [ 708.251581] tcp_sendmsg+0x40/0x60 [ 708.292376] inet_sendmsg+0xf0/0x520 [ 708.335259] sock_sendmsg+0xac/0xf8 [ 708.377096] sock_write_iter+0x1c0/0x2c0 [ 708.424154] new_sync_write+0x358/0x4a8 [ 708.470162] __vfs_write+0xc4/0xf8 [ 708.510950] vfs_write+0x12c/0x3d0 [ 708.551739] ksys_write+0xcc/0x178 [ 708.592533] __arm64_sys_write+0x70/0xa0 [ 708.639593] el0_svc_handler+0x13c/0x298 [ 708.686646] el0_svc+0x8/0xc [ 708.739019] Freed by task 17: [ 708.774597] __kasan_slab_free+0x114/0x228 [ 708.823736] kasan_slab_free+0x10/0x18 [ 708.868703] kfree+0x100/0x3d8 [ 708.905320] skb_free_head+0x7c/0x98 [ 708.948204] skb_release_data+0x320/0x490 [ 708.996301] pskb_expand_head+0x60c/0x970 [ 709.044399] __iptunnel_pull_header+0x3b8/0x5d0 [ 709.098770] ipip6_rcv+0x41c/0x16e0 [sit] [ 709.146873] tunnel64_rcv+0xd4/0x200 [tunnel4] [ 709.200195] ip_local_deliver_finish+0x3b8/0x988 [ 709.255596] ip_local_deliver+0x144/0x470 [ 709.303692] ip_rcv_finish+0x43c/0x14b0 [ 709.349705] ip_rcv+0x628/0x1138 [ 709.388413] __netif_receive_skb_core+0x1670/0x2600 [ 709.446943] __netif_receive_skb+0x28/0x190 [ 709.497120] process_backlog+0x1d0/0x610 [ 709.544169] net_rx_action+0x37c/0xf68 [ 709.589131] __do_softirq+0x288/0x1018 [ 709.651938] The buggy address belongs to the object at ffffe01b6bd85580 which belongs to the cache kmalloc-1024 of size 1024 [ 709.804356] The buggy address is located 117 bytes inside of 1024-byte region [ffffe01b6bd85580, ffffe01b6bd85980) [ 709.946340] The buggy address belongs to the page: [ 710.003824] page:ffff7ff806daf600 count:1 mapcount:0 mapping:ffffe01c4001f600 index:0x0 [ 710.099914] flags: 0xfffff8000000100(slab) [ 710.149059] raw: 0fffff8000000100 dead000000000100 dead000000000200 ffffe01c4001f600 [ 710.242011] raw: 0000000000000000 0000000000380038 00000001ffffffff 0000000000000000 [ 710.334966] page dumped because: kasan: bad access detected Fix it resetting iph pointer after iptunnel_pull_header Fixes: a09a4c8dd1ec ("tunnels: Remove encapsulation offloads on decap") Tested-by: Jianlin Shi <jishi@redhat.com> Signed-off-by: Lorenzo Bianconi <lorenzo.bianconi@redhat.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2019-04-03ipv6: Flip to fib_nexthop_infoDavid Ahern
Export fib_nexthop_info and fib_add_nexthop for use by IPv6 code. Remove rt6_nexthop_info and rt6_add_nexthop in favor of the IPv4 versions. Update fib_nexthop_info for IPv6 linkdown check and RTA_GATEWAY for AF_INET6. Signed-off-by: David Ahern <dsahern@gmail.com> Acked-by: Martin KaFai Lau <kafai@fb.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2019-04-03ipv6: Fix dangling pointer when ipv6 fragmentJunwei Hu
At the beginning of ip6_fragment func, the prevhdr pointer is obtained in the ip6_find_1stfragopt func. However, all the pointers pointing into skb header may change when calling skb_checksum_help func with skb->ip_summed = CHECKSUM_PARTIAL condition. The prevhdr pointe will be dangling if it is not reloaded after calling __skb_linearize func in skb_checksum_help func. Here, I add a variable, nexthdr_offset, to evaluate the offset, which does not changes even after calling __skb_linearize func. Fixes: 405c92f7a541 ("ipv6: add defensive check for CHECKSUM_PARTIAL skbs in ip_fragment") Signed-off-by: Junwei Hu <hujunwei4@huawei.com> Reported-by: Wenhao Zhang <zhangwenhao8@huawei.com> Reported-by: syzbot+e8ce541d095e486074fc@syzkaller.appspotmail.com Reviewed-by: Zhiqiang Liu <liuzhiqiang26@huawei.com> Acked-by: Martin KaFai Lau <kafai@fb.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2019-04-02ip6_tunnel: Match to ARPHRD_TUNNEL6 for dev typeSheena Mira-ato
The device type for ip6 tunnels is set to ARPHRD_TUNNEL6. However, the ip4ip6_err function is expecting the device type of the tunnel to be ARPHRD_TUNNEL. Since the device types do not match, the function exits and the ICMP error packet is not sent to the originating host. Note that the device type for IPv4 tunnels is set to ARPHRD_TUNNEL. Fix is to expect a tunnel device type of ARPHRD_TUNNEL6 instead. Now the tunnel device type matches and the ICMP error packet is sent to the originating host. Signed-off-by: Sheena Mira-ato <sheena.mira-ato@alliedtelesis.co.nz> Signed-off-by: David S. Miller <davem@davemloft.net>
2019-04-01tcp: fix tcp_inet6_sk() for 32bit kernelsEric Dumazet
It turns out that struct ipv6_pinfo is not located as we think. inet6_sk_generic() and tcp_inet6_sk() disagree on 32bit kernels by 4-bytes, because struct tcp_sock has 8-bytes alignment, but ipv6_pinfo size is not a multiple of 8. sizeof(struct ipv6_pinfo): 116 (not padded to 8) I actually first coded tcp_inet6_sk() as this patch does, but thought that "container_of(tcp_sk(sk), struct tcp6_sock, tcp)" was cleaner. As Julian told me : Nobody should use tcp6_sock.inet6 directly, it should be accessed via tcp_inet6_sk() or inet6_sk(). This happened when we added the first u64 field in struct tcp_sock. Fixes: 93a77c11ae79 ("tcp: add tcp_inet6_sk() helper") Signed-off-by: Eric Dumazet <edumazet@google.com> Bisected-by: Julian Anastasov <ja@ssi.bg> Signed-off-by: David S. Miller <davem@davemloft.net>
2019-03-29ipv6: Move ipv6 stubs to a separate header fileDavid Ahern
The number of stubs is growing and has nothing to do with addrconf. Move the definition of the stubs to a separate header file and update users. In the move, drop the vxlan specific comment before ipv6_stub. Code move only; no functional change intended. Signed-off-by: David Ahern <dsahern@gmail.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2019-03-29net: Use common nexthop init and release helpersDavid Ahern
With fib_nh_common in place, move common initialization and release code into helpers used by both ipv4 and ipv6. For the moment, the init is just the lwt encap and the release is both the netdev reference and the the lwt state reference. More will be added later. Signed-off-by: David Ahern <dsahern@gmail.com> Reviewed-by: Ido Schimmel <idosch@mellanox.com> Acked-by: Alexei Starovoitov <ast@kernel.org> Signed-off-by: David S. Miller <davem@davemloft.net>
2019-03-29net: Add fib_nh_common and update fib_nh and fib6_nhDavid Ahern
Add fib_nh_common struct with common nexthop attributes. Convert fib_nh and fib6_nh to use it. Use macros to move existing fib_nh_* references to the new nh_common.nhc_*. Signed-off-by: David Ahern <dsahern@gmail.com> Reviewed-by: Ido Schimmel <idosch@mellanox.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2019-03-29ipv6: Rename fib6_nh entriesDavid Ahern
Rename fib6_nh entries that will be moved to a fib_nh_common struct. Specifically, the device, gateway, flags, and lwtstate are common with all nexthop definitions. In some places new temporary variables are declared or local variables renamed to maintain line lengths. Rename only; no functional change intended. Signed-off-by: David Ahern <dsahern@gmail.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2019-03-29ipv6: Change rt6_add_nexthop and rt6_nexthop_info to take fib6_nhDavid Ahern
rt6_add_nexthop and rt6_nexthop_info only need the fib6_info for the gateway flag and the nexthop weight, and the presence of a gateway is now per-nexthop. Update the signatures to take a fib6_nh and nexthop weight and better align with the ipv4 versions. Signed-off-by: David Ahern <dsahern@gmail.com> Reviewed-by: Ido Schimmel <idosch@mellanox.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2019-03-29ipv6: Refactor fib6_ignore_linkdownDavid Ahern
fib6_ignore_linkdown takes a fib6_info but only looks at the net_device and its IPv6 config. Change it to take a net_device over a fib6_info as its input argument. In addition, move it to a header file to make the check inline and usable later with IPv4 code without going through the ipv6 stub, and rename to ip6_ignore_linkdown since it is only checking the setting based on the ipv6 struct on a device. Signed-off-by: David Ahern <dsahern@gmail.com> Reviewed-by: Ido Schimmel <idosch@mellanox.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2019-03-29ipv6: Move gateway checks to a fib6_nh settingDavid Ahern
The gateway setting is not per fib6_info entry but per-fib6_nh. Add a new fib_nh_has_gw flag to fib6_nh and convert references to RTF_GATEWAY to the new flag. For IPv6 address the flag is cheaper than checking that nh_gw is non-0 like IPv4 does. While this increases fib6_nh by 8-bytes, the effective allocation size of a fib6_info is unchanged. The 8 bytes is recovered later with a fib_nh_common change. Signed-off-by: David Ahern <dsahern@gmail.com> Reviewed-by: Ido Schimmel <idosch@mellanox.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2019-03-29ipv6: Create cleanup helper for fib6_nhDavid Ahern
Move the fib6_nh cleanup code to a new helper, fib6_nh_release. Signed-off-by: David Ahern <dsahern@gmail.com> Reviewed-by: Ido Schimmel <idosch@mellanox.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2019-03-29ipv6: Create init helper for fib6_nhDavid Ahern
Similar to IPv4, consolidate the fib6_nh initialization into a helper. As a new standalone function, add a cleanup path to put lwtstate on error. To avoid modifying fib6_config flags, move the reject check to a helper that is invoked once by fib6_nh_init to reset the device and then again in ip6_route_info_create to set the fib6_flags. Signed-off-by: David Ahern <dsahern@gmail.com> Reviewed-by: Ido Schimmel <idosch@mellanox.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2019-03-27ila: Fix rhashtable walker list corruptionHerbert Xu
ila_xlat_nl_cmd_flush uses rhashtable walkers allocated from the stack but it never frees them. This corrupts the walker list of the hash table. This patch fixes it. Reported-by: syzbot+dae72a112334aa65a159@syzkaller.appspotmail.com Fixes: b6e71bdebb12 ("ila: Flush netlink command to clear xlat...") Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au> Signed-off-by: David S. Miller <davem@davemloft.net>
2019-03-27Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/netDavid S. Miller
2019-03-27inet: switch IP ID generator to siphashEric Dumazet
According to Amit Klein and Benny Pinkas, IP ID generation is too weak and might be used by attackers. Even with recent net_hash_mix() fix (netns: provide pure entropy for net_hash_mix()) having 64bit key and Jenkins hash is risky. It is time to switch to siphash and its 128bit keys. Signed-off-by: Eric Dumazet <edumazet@google.com> Reported-by: Amit Klein <aksecurity@gmail.com> Reported-by: Benny Pinkas <benny@pinkas.net> Signed-off-by: David S. Miller <davem@davemloft.net>
2019-03-23tcp: add one skb cache for rxEric Dumazet
Often times, recvmsg() system calls and BH handling for a particular TCP socket are done on different cpus. This means the incoming skb had to be allocated on a cpu, but freed on another. This incurs a high spinlock contention in slab layer for small rpc, but also a high number of cache line ping pongs for larger packets. A full size GRO packet might use 45 page fragments, meaning that up to 45 put_page() can be involved. More over performing the __kfree_skb() in the recvmsg() context adds a latency for user applications, and increase probability of trapping them in backlog processing, since the BH handler might found the socket owned by the user. This patch, combined with the prior one increases the rpc performance by about 10 % on servers with large number of cores. (tcp_rr workload with 10,000 flows and 112 threads reach 9 Mpps instead of 8 Mpps) This also increases single bulk flow performance on 40Gbit+ links, since in this case there are often two cpus working in tandem : - CPU handling the NIC rx interrupts, feeding the receive queue, and (after this patch) freeing the skbs that were consumed. - CPU in recvmsg() system call, essentially 100 % busy copying out data to user space. Having at most one skb in a per-socket cache has very little risk of memory exhaustion, and since it is protected by socket lock, its management is essentially free. Note that if rps/rfs is used, we do not enable this feature, because there is high chance that the same cpu is handling both the recvmsg() system call and the TCP rx path, but that another cpu did the skb allocations in the device driver right before the RPS/RFS logic. To properly handle this case, it seems we would need to record on which cpu skb was allocated, and use a different channel to give skbs back to this cpu. Signed-off-by: Eric Dumazet <edumazet@google.com> Acked-by: Soheil Hassas Yeganeh <soheil@google.com> Acked-by: Willem de Bruijn <willemb@google.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2019-03-22genetlink: make policy common to familyJohannes Berg
Since maxattr is common, the policy can't really differ sanely, so make it common as well. The only user that did in fact manage to make a non-common policy is taskstats, which has to be really careful about it (since it's still using a common maxattr!). This is no longer supported, but we can fake it using pre_doit. This reduces the size of e.g. nl80211.o (which has lots of commands): text data bss dec hex filename 398745 14323 2240 415308 6564c net/wireless/nl80211.o (before) 397913 14331 2240 414484 65314 net/wireless/nl80211.o (after) -------------------------------- -832 +8 0 -824 Which is obviously just 8 bytes for each command, and an added 8 bytes for the new policy pointer. I'm not sure why the ops list is counted as .text though. Most of the code transformations were done using the following spatch: @ops@ identifier OPS; expression POLICY; @@ struct genl_ops OPS[] = { ..., { - .policy = POLICY, }, ... }; @@ identifier ops.OPS; expression ops.POLICY; identifier fam; expression M; @@ struct genl_family fam = { .ops = OPS, .maxattr = M, + .policy = POLICY, ... }; This also gets rid of devlink_nl_cmd_region_read_dumpit() accessing the cb->data as ops, which we want to change in a later genl patch. Signed-off-by: Johannes Berg <johannes.berg@intel.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2019-03-21ipv6: Remove fallback argument from ip6_hold_safeDavid Ahern
net and null_fallback are redundant. Remove null_fallback in favor of !net check. Signed-off-by: David Ahern <dsahern@gmail.com> Acked-by: Wei Wang <weiwan@google.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2019-03-21ipv6: Change addrconf_f6i_alloc to use ip6_route_info_createDavid Ahern
Change addrconf_f6i_alloc to generate a fib6_config and call ip6_route_info_create. addrconf_f6i_alloc is the last caller to fib6_info_alloc besides ip6_route_info_create, and there is no reason for it to do its own initialization on a fib6_info. Host routes need to be created even if the device is down, so add a new flag, fc_ignore_dev_down, to fib6_config and update fib6_nh_init to not error out if device is not up. Notes on the conversion: - ip_fib_metrics_init is the same as fib6_config has fc_mx set to NULL and fc_mx_len set to 0 - dst_nocount is handled by the RTF_ADDRCONF flag - dst_host is handled by fc_dst_len = 128 nh_gw does not get set after the conversion to ip6_route_info_create but it should not be set in addrconf_f6i_alloc since this is a host route not a gateway route. Everything else is a straight forward map between fib6_info and fib6_config. Signed-off-by: David Ahern <dsahern@gmail.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2019-03-21ipv6: Move setting default metric for routesDavid Ahern
ip6_route_info_create is a low level function for ensuring fc_metric is set. Move the check and default setting to the 2 locations that do not already set fc_metric before calling ip6_route_info_create. This is required for the next patch which moves addrconf allocations to ip6_route_info_create and want the metric for host routes to be 0. Signed-off-by: David Ahern <dsahern@gmail.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2019-03-21Merge git://git.kernel.org/pub/scm/linux/kernel/git/pablo/nfDavid S. Miller
Pablo Neira Ayuso says: ==================== Netfilter fixes for net The following patchset contains Netfilter fixes for your net tree: 1) Remove a direct dependency with IPv6 introduced by the sip_external_media feature, from Alin Nastac. 2) Fix bogus ENOENT when removing interval elements from set. 3) Set transport_header from br_netfilter to mimic the stack behaviour, this partially fixes a checksum validation bug from the SCTP connection tracking, from Xin Long. 4) Fix undefined reference to symbol in xt_TEE, due to missing Kconfig dependencies, from Arnd Bergmann. 5) Check for NULL in skb_header_pointer() calls in ip6t_shr, from Kangjie Lu. 6) Fix bogus EBUSY when removing an existing conntrack helper from a transaction. 7) Fix module autoload of the redirect extension. 8) Remove duplicated transition in flowtable diagram in the existing documentation. 9) Missing .release_ops call from error path in newrule() which results module refcount leak, from Taehee Yoo. ==================== Signed-off-by: David S. Miller <davem@davemloft.net>
2019-03-20ipv6: Add icmp_echo_ignore_anycast for ICMPv6Stephen Suryaputra
In addition to icmp_echo_ignore_multicast, there is a need to also prevent responding to pings to anycast addresses for security. Signed-off-by: Stephen Suryaputra <ssuryaextr@gmail.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2019-03-20ipv6: make ip6_create_rt_rcu return ip6_null_entry instead of NULLXin Long
Jianlin reported a crash: [ 381.484332] BUG: unable to handle kernel NULL pointer dereference at 0000000000000068 [ 381.619802] RIP: 0010:fib6_rule_lookup+0xa3/0x160 [ 382.009615] Call Trace: [ 382.020762] <IRQ> [ 382.030174] ip6_route_redirect.isra.52+0xc9/0xf0 [ 382.050984] ip6_redirect+0xb6/0xf0 [ 382.066731] icmpv6_notify+0xca/0x190 [ 382.083185] ndisc_redirect_rcv+0x10f/0x160 [ 382.102569] ndisc_rcv+0xfb/0x100 [ 382.117725] icmpv6_rcv+0x3f2/0x520 [ 382.133637] ip6_input_finish+0xbf/0x460 [ 382.151634] ip6_input+0x3b/0xb0 [ 382.166097] ipv6_rcv+0x378/0x4e0 It was caused by the lookup function __ip6_route_redirect() returns NULL in fib6_rule_lookup() when ip6_create_rt_rcu() returns NULL. So we fix it by simply making ip6_create_rt_rcu() return ip6_null_entry instead of NULL. v1->v2: - move down 'fallback:' to make it more readable. Fixes: e873e4b9cc7e ("ipv6: use fib6_info_hold_safe() when necessary") Reported-by: Jianlin Shi <jishi@redhat.com> Suggested-by: Paolo Abeni <pabeni@redhat.com> Signed-off-by: Xin Long <lucien.xin@gmail.com> Reviewed-by: David Ahern <dsahern@gmail.com> Acked-by: Wei Wang <weiwan@google.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2019-03-19ipv6: Add icmp_echo_ignore_multicast support for ICMPv6Stephen Suryaputra
IPv4 has icmp_echo_ignore_broadcast to prevent responding to broadcast pings. IPv6 needs a similar mechanism. v1->v2: - Remove NET_IPV6_ICMP_ECHO_IGNORE_MULTICAST. Signed-off-by: Stephen Suryaputra <ssuryaextr@gmail.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2019-03-19tcp: add tcp_inet6_sk() helperEric Dumazet
TCP ipv6 fast path dereferences a pointer to get to the inet6 part of a tcp socket, but given the fixed memory placement, we can do better and avoid a possible cache line miss. This also reduces register pressure, since we let the compiler know about this memory placement. Signed-off-by: Eric Dumazet <edumazet@google.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2019-03-19tcp: do not use ipv6 header for ipv4 flowEric Dumazet
When a dual stack tcp listener accepts an ipv4 flow, it should not attempt to use an ipv6 header or tcp_v6_iif() helper. Fixes: 1397ed35f22d ("ipv6: add flowinfo for tcp6 pkt_options for all cases") Fixes: df3687ffc665 ("ipv6: add the IPV6_FL_F_REFLECT flag to IPV6_FL_A_GET") Fixes: 1da177e4c3f4 ("Linux-2.6.12-rc2") Signed-off-by: Eric Dumazet <edumazet@google.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2019-03-18netfilter: ip6t_srh: fix NULL pointer dereferencesKangjie Lu
skb_header_pointer may return NULL. The current code dereference its return values without a NULL check. The fix inserts the checks to avoid NULL pointer dereferences. Fixes: 202a8ff545cc ("netfilter: add IPv6 segment routing header 'srh' match") Signed-off-by: Kangjie Lu <kjlu@umn.edu> Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
2019-03-11net: sit: fix UBSAN Undefined behaviour in check_6rdMiaohe Lin
In func check_6rd,tunnel->ip6rd.relay_prefixlen may equal to 32,so UBSAN complain about it. UBSAN: Undefined behaviour in net/ipv6/sit.c:781:47 shift exponent 32 is too large for 32-bit type 'unsigned int' CPU: 6 PID: 20036 Comm: syz-executor.0 Not tainted 4.19.27 #2 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.10.2-1ubuntu1 04/01/2014 Call Trace: __dump_stack lib/dump_stack.c:77 [inline] dump_stack+0xca/0x13e lib/dump_stack.c:113 ubsan_epilogue+0xe/0x81 lib/ubsan.c:159 __ubsan_handle_shift_out_of_bounds+0x293/0x2e8 lib/ubsan.c:425 check_6rd.constprop.9+0x433/0x4e0 net/ipv6/sit.c:781 try_6rd net/ipv6/sit.c:806 [inline] ipip6_tunnel_xmit net/ipv6/sit.c:866 [inline] sit_tunnel_xmit+0x141c/0x2720 net/ipv6/sit.c:1033 __netdev_start_xmit include/linux/netdevice.h:4300 [inline] netdev_start_xmit include/linux/netdevice.h:4309 [inline] xmit_one net/core/dev.c:3243 [inline] dev_hard_start_xmit+0x17c/0x780 net/core/dev.c:3259 __dev_queue_xmit+0x1656/0x2500 net/core/dev.c:3829 neigh_output include/net/neighbour.h:501 [inline] ip6_finish_output2+0xa36/0x2290 net/ipv6/ip6_output.c:120 ip6_finish_output+0x3e7/0xa20 net/ipv6/ip6_output.c:154 NF_HOOK_COND include/linux/netfilter.h:278 [inline] ip6_output+0x1e2/0x720 net/ipv6/ip6_output.c:171 dst_output include/net/dst.h:444 [inline] ip6_local_out+0x99/0x170 net/ipv6/output_core.c:176 ip6_send_skb+0x9d/0x2f0 net/ipv6/ip6_output.c:1697 ip6_push_pending_frames+0xc0/0x100 net/ipv6/ip6_output.c:1717 rawv6_push_pending_frames net/ipv6/raw.c:616 [inline] rawv6_sendmsg+0x2435/0x3530 net/ipv6/raw.c:946 inet_sendmsg+0xf8/0x5c0 net/ipv4/af_inet.c:798 sock_sendmsg_nosec net/socket.c:621 [inline] sock_sendmsg+0xc8/0x110 net/socket.c:631 ___sys_sendmsg+0x6cf/0x890 net/socket.c:2114 __sys_sendmsg+0xf0/0x1b0 net/socket.c:2152 do_syscall_64+0xc8/0x580 arch/x86/entry/common.c:290 entry_SYSCALL_64_after_hwframe+0x49/0xbe Signed-off-by: linmiaohe <linmiaohe@huawei.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2019-03-08fou, fou6: avoid uninit-value in gue_err() and gue6_err()Eric Dumazet
My prior commit missed the fact that these functions were using udp_hdr() (aka skb_transport_header()) to get access to GUE header. Since pskb_transport_may_pull() does not exist yet, we have to add transport_offset to our pskb_may_pull() calls. BUG: KMSAN: uninit-value in gue_err+0x514/0xfa0 net/ipv4/fou.c:1032 CPU: 1 PID: 10648 Comm: syz-executor.1 Not tainted 5.0.0+ #11 Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011 Call Trace: <IRQ> __dump_stack lib/dump_stack.c:77 [inline] dump_stack+0x173/0x1d0 lib/dump_stack.c:113 kmsan_report+0x12e/0x2a0 mm/kmsan/kmsan.c:600 __msan_warning+0x82/0xf0 mm/kmsan/kmsan_instr.c:313 gue_err+0x514/0xfa0 net/ipv4/fou.c:1032 __udp4_lib_err_encap_no_sk net/ipv4/udp.c:571 [inline] __udp4_lib_err_encap net/ipv4/udp.c:626 [inline] __udp4_lib_err+0x12e6/0x1d40 net/ipv4/udp.c:665 udp_err+0x74/0x90 net/ipv4/udp.c:737 icmp_socket_deliver net/ipv4/icmp.c:767 [inline] icmp_unreach+0xb65/0x1070 net/ipv4/icmp.c:884 icmp_rcv+0x11a1/0x1950 net/ipv4/icmp.c:1066 ip_protocol_deliver_rcu+0x584/0xbb0 net/ipv4/ip_input.c:208 ip_local_deliver_finish net/ipv4/ip_input.c:234 [inline] NF_HOOK include/linux/netfilter.h:289 [inline] ip_local_deliver+0x624/0x7b0 net/ipv4/ip_input.c:255 dst_input include/net/dst.h:450 [inline] ip_rcv_finish net/ipv4/ip_input.c:414 [inline] NF_HOOK include/linux/netfilter.h:289 [inline] ip_rcv+0x6bd/0x740 net/ipv4/ip_input.c:524 __netif_receive_skb_one_core net/core/dev.c:4973 [inline] __netif_receive_skb net/core/dev.c:5083 [inline] process_backlog+0x756/0x10e0 net/core/dev.c:5923 napi_poll net/core/dev.c:6346 [inline] net_rx_action+0x78b/0x1a60 net/core/dev.c:6412 __do_softirq+0x53f/0x93a kernel/softirq.c:293 invoke_softirq kernel/softirq.c:375 [inline] irq_exit+0x214/0x250 kernel/softirq.c:416 exiting_irq+0xe/0x10 arch/x86/include/asm/apic.h:536 smp_apic_timer_interrupt+0x48/0x70 arch/x86/kernel/apic/apic.c:1064 apic_timer_interrupt+0x2e/0x40 arch/x86/entry/entry_64.S:814 </IRQ> RIP: 0010:finish_lock_switch+0x2b/0x40 kernel/sched/core.c:2597 Code: 48 89 e5 53 48 89 fb e8 63 e7 95 00 8b b8 88 0c 00 00 48 8b 00 48 85 c0 75 12 48 89 df e8 dd db 95 00 c6 00 00 c6 03 00 fb 5b <5d> c3 e8 4e e6 95 00 eb e7 66 90 66 2e 0f 1f 84 00 00 00 00 00 55 RSP: 0018:ffff888081a0fc80 EFLAGS: 00000296 ORIG_RAX: ffffffffffffff13 RAX: ffff88821fd6bd80 RBX: ffff888027898000 RCX: ccccccccccccd000 RDX: ffff88821fca8d80 RSI: ffff888000000000 RDI: 00000000000004a0 RBP: ffff888081a0fc80 R08: 0000000000000002 R09: ffff888081a0fb08 R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000001 R13: ffff88811130e388 R14: ffff88811130da00 R15: ffff88812fdb7d80 finish_task_switch+0xfc/0x2d0 kernel/sched/core.c:2698 context_switch kernel/sched/core.c:2851 [inline] __schedule+0x6cc/0x800 kernel/sched/core.c:3491 schedule+0x15b/0x240 kernel/sched/core.c:3535 freezable_schedule include/linux/freezer.h:172 [inline] do_nanosleep+0x2ba/0x980 kernel/time/hrtimer.c:1679 hrtimer_nanosleep kernel/time/hrtimer.c:1733 [inline] __do_sys_nanosleep kernel/time/hrtimer.c:1767 [inline] __se_sys_nanosleep+0x746/0x960 kernel/time/hrtimer.c:1754 __x64_sys_nanosleep+0x3e/0x60 kernel/time/hrtimer.c:1754 do_syscall_64+0xbc/0xf0 arch/x86/entry/common.c:291 entry_SYSCALL_64_after_hwframe+0x63/0xe7 RIP: 0033:0x4855a0 Code: 00 00 48 c7 c0 d4 ff ff ff 64 c7 00 16 00 00 00 31 c0 eb be 66 0f 1f 44 00 00 83 3d b1 11 5d 00 00 75 14 b8 23 00 00 00 0f 05 <48> 3d 01 f0 ff ff 0f 83 04 e2 f8 ff c3 48 83 ec 08 e8 3a 55 fd ff RSP: 002b:0000000000a4fd58 EFLAGS: 00000246 ORIG_RAX: 0000000000000023 RAX: ffffffffffffffda RBX: 0000000000085780 RCX: 00000000004855a0 RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000a4fd60 RBP: 00000000000007ec R08: 0000000000000001 R09: 0000000000ceb940 R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000008 R13: 0000000000a4fdb0 R14: 0000000000085711 R15: 0000000000a4fdc0 Uninit was created at: kmsan_save_stack_with_flags mm/kmsan/kmsan.c:205 [inline] kmsan_internal_poison_shadow+0x92/0x150 mm/kmsan/kmsan.c:159 kmsan_kmalloc+0xa6/0x130 mm/kmsan/kmsan_hooks.c:176 kmsan_slab_alloc+0xe/0x10 mm/kmsan/kmsan_hooks.c:185 slab_post_alloc_hook mm/slab.h:445 [inline] slab_alloc_node mm/slub.c:2773 [inline] __kmalloc_node_track_caller+0xe9e/0xff0 mm/slub.c:4398 __kmalloc_reserve net/core/skbuff.c:140 [inline] __alloc_skb+0x309/0xa20 net/core/skbuff.c:208 alloc_skb include/linux/skbuff.h:1012 [inline] alloc_skb_with_frags+0x186/0xa60 net/core/skbuff.c:5287 sock_alloc_send_pskb+0xafd/0x10a0 net/core/sock.c:2091 sock_alloc_send_skb+0xca/0xe0 net/core/sock.c:2108 __ip_append_data+0x34cd/0x5000 net/ipv4/ip_output.c:998 ip_append_data+0x324/0x480 net/ipv4/ip_output.c:1220 icmp_push_reply+0x23d/0x7e0 net/ipv4/icmp.c:375 __icmp_send+0x2ea3/0x30f0 net/ipv4/icmp.c:737 icmp_send include/net/icmp.h:47 [inline] ipv4_link_failure+0x6d/0x230 net/ipv4/route.c:1190 dst_link_failure include/net/dst.h:427 [inline] arp_error_report+0x106/0x1a0 net/ipv4/arp.c:297 neigh_invalidate+0x359/0x8e0 net/core/neighbour.c:992 neigh_timer_handler+0xdf2/0x1280 net/core/neighbour.c:1078 call_timer_fn+0x285/0x600 kernel/time/timer.c:1325 expire_timers kernel/time/timer.c:1362 [inline] __run_timers+0xdb4/0x11d0 kernel/time/timer.c:1681 run_timer_softirq+0x2e/0x50 kernel/time/timer.c:1694 __do_softirq+0x53f/0x93a kernel/softirq.c:293 Fixes: 26fc181e6cac ("fou, fou6: do not assume linear skbs") Signed-off-by: Eric Dumazet <edumazet@google.com> Reported-by: syzbot <syzkaller@googlegroups.com> Cc: Stefano Brivio <sbrivio@redhat.com> Cc: Sabrina Dubroca <sd@queasysnail.net> Acked-by: Stefano Brivio <sbrivio@redhat.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2019-03-04net: ignore sysctl_devconf_inherit_init_net without SYSCTLArnd Bergmann
When CONFIG_SYSCTL is turned off, we get a link failure for the newly introduced tuning knob. net/ipv6/addrconf.o: In function `addrconf_init_net': addrconf.c:(.text+0x31dc): undefined reference to `sysctl_devconf_inherit_init_net' Add an IS_ENABLED() check to fall back to the default behavior (sysctl_devconf_inherit_init_net=0) here. Fixes: 856c395cfa63 ("net: introduce a knob to control whether to inherit devconf config") Signed-off-by: Arnd Bergmann <arnd@arndb.de> Acked-by: Christian Brauner <christian@brauner.io> Signed-off-by: David S. Miller <davem@davemloft.net>