summaryrefslogtreecommitdiff
path: root/drivers/net/ethernet/google/gve/gve.h
AgeCommit message (Collapse)Author
2021-10-25gve: Implement packet continuation for RX.David Awogbemila
This enables the driver to receive RX packets spread across multiple buffers: For a given multi-fragment packet the "packet continuation" bit is set on all descriptors except the last one. These descriptors' payloads are combined into a single SKB before the SKB is handed to the networking stack. This change adds a "packet buffer size" notion for RX queues. The CreateRxQueue AdminQueue command sent to the device now includes the packet_buffer_size. We opt for a packet_buffer_size of PAGE_SIZE / 2 to give the driver the opportunity to flip pages where we can instead of copying. Signed-off-by: David Awogbemila <awogbemila@google.com> Signed-off-by: Jeroen de Borst <jeroendb@google.com> Reviewed-by: Catherine Sullivan <csully@google.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2021-10-25gve: Add RX context.David Awogbemila
This refactor moves the skb_head and skb_tail fields into a new gve_rx_ctx struct. This new struct will contain information about the current packet being processed. This is in preparation for multi-descriptor RX packets. Signed-off-by: David Awogbemila <awogbemila@google.com> Signed-off-by: Jeroen de Borst <jeroendb@google.com> Reviewed-by: Catherine Sullivan <csully@google.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2021-10-11gve: Recover from queue stall due to missed IRQJohn Fraker
Don't always reset the driver on a TX timeout. Attempt to recover by kicking the queue in case an IRQ was missed. Fixes: 9e5f7d26a4c08 ("gve: Add workqueue and reset support") Signed-off-by: John Fraker <jfraker@google.com> Signed-off-by: David Awogbemila <awogbemila@google.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2021-10-11gve: Do lazy cleanup in TX pathTao Liu
When TX queue is full, attemt to process enough TX completions to avoid stalling the queue. Fixes: f5cedc84a30d2 ("gve: Add transmit and receive support") Signed-off-by: Tao Liu <xliutaox@google.com> Signed-off-by: Catherine Sullivan <csully@google.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2021-10-11gve: Switch to use napi_complete_doneYangchun Fu
Use napi_complete_done to allow for the use of gro_flush_timeout. Fixes: f5cedc84a30d2 ("gve: Add transmit and receive support") Signed-off-by: Yangchun Fu <yangchun@google.com> Signed-off-by: Catherine Sullivan <csully@google.com> Signed-off-by: David Awogbemila <awogbemila@google.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2021-10-07Merge git://git.kernel.org/pub/scm/linux/kernel/git/netdev/netJakub Kicinski
No conflicts. Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2021-10-06gve: Correct available tx qpl checkCatherine Sullivan
The qpl_map_size is rounded up to a multiple of sizeof(long), but the number of qpls doesn't have to be. Fixes: f5cedc84a30d2 ("gve: Add transmit and receive support") Signed-off-by: Catherine Sullivan <csully@google.com> Signed-off-by: Jeroen de Borst <jeroendb@google.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2021-09-28gve: DQO: avoid unused variable warningsArnd Bergmann
The use of dma_unmap_addr()/dma_unmap_len() in the driver causes multiple warnings when these macros are defined as empty, e.g. in an ARCH=i386 allmodconfig build: drivers/net/ethernet/google/gve/gve_tx_dqo.c: In function 'gve_tx_add_skb_no_copy_dqo': drivers/net/ethernet/google/gve/gve_tx_dqo.c:494:40: error: unused variable 'buf' [-Werror=unused-variable] 494 | struct gve_tx_dma_buf *buf = This is not how the NEED_DMA_MAP_STATE macros are meant to work, as they rely on never using local variables or a temporary structure like gve_tx_dma_buf. Remote the gve_tx_dma_buf definition and open-code the contents in all places to avoid the warning. This causes some rather long lines but otherwise ends up making the driver slightly smaller. Fixes: a57e5de476be ("gve: DQO: Add TX path") Link: https://lore.kernel.org/netdev/20210723231957.1113800-1-bcf@google.com/ Link: https://lore.kernel.org/netdev/20210721151100.2042139-1-arnd@kernel.org/ Signed-off-by: Arnd Bergmann <arnd@arndb.de> Signed-off-by: David S. Miller <davem@davemloft.net>
2021-06-24gve: DQO: Add RX pathBailey Forrest
The RX queue has an array of `gve_rx_buf_state_dqo` objects. All allocated pages have an associated buf_state object. When a buffer is posted on the RX buffer queue, the buffer ID will be the buf_state's index into the RX queue's array. On packet reception, the RX queue will have one descriptor for each buffer associated with a received packet. Each RX descriptor will have a buffer_id that was posted on the buffer queue. Notable mentions: - We use a default buffer size of 2048 bytes. Based on page size, we may post separate sections of a single page as separate buffers. - The driver holds an extra reference on pages passed up the receive path with an skb and keeps these pages on a list. When posting new buffers to the NIC, we check if any of these pages has only our reference, or another buffer sized segment of the page has no references. If so, it is free to reuse. This page recycling approach is a common netdev optimization that reduces page alloc/free calls. - Pages in the free list have a page_count bias in order to avoid an atomic increment of pagecount every time we attempt to reuse a page. # references = page_count() - bias - In order to track when a page is safe to reuse, we keep track of the last offset which had a single SKB reference. When this occurs, it implies that every single other offset is reusable. Otherwise, we don't know if offsets can be safely reused. - We maintain two free lists of pages. List #1 (recycled_buf_states) contains pages we know can be reused right away. List #2 (used_buf_states) contains pages which cannot be used right away. We only attempt to get pages from list #2 when list #1 is empty. We only attempt to use a small fixed number pages from list #2 before giving up and allocating a new page. Both lists are FIFOs in hope that by the time we attempt to reuse a page, the references were dropped. Signed-off-by: Bailey Forrest <bcf@google.com> Reviewed-by: Willem de Bruijn <willemb@google.com> Reviewed-by: Catherine Sullivan <csully@google.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2021-06-24gve: DQO: Add ring allocation and initializationBailey Forrest
Allocate the buffer and completion ring structures. Do not populate the rings yet. That will happen in the respective rx and tx datapath follow-on patches Signed-off-by: Bailey Forrest <bcf@google.com> Reviewed-by: Willem de Bruijn <willemb@google.com> Reviewed-by: Catherine Sullivan <csully@google.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2021-06-24gve: DQO: Add core netdev featuresBailey Forrest
Add napi netdev device registration, interrupt handling and initial tx and rx polling stubs. The stubs will be filled in follow-on patches. Also: - LRO feature advertisement and handling - Also update ethtool logic Signed-off-by: Bailey Forrest <bcf@google.com> Reviewed-by: Willem de Bruijn <willemb@google.com> Reviewed-by: Catherine Sullivan <csully@google.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2021-06-24gve: Update adminq commands to support DQO queuesBailey Forrest
DQO queue creation requires additional parameters: - TX completion/RX buffer queue size - TX completion/RX buffer queue address - TX/RX queue size - RX buffer size Signed-off-by: Bailey Forrest <bcf@google.com> Reviewed-by: Willem de Bruijn <willemb@google.com> Reviewed-by: Catherine Sullivan <csully@google.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2021-06-24gve: Add DQO fields for core data structuresBailey Forrest
- Add new DQO datapath structures: - `gve_rx_buf_queue_dqo` - `gve_rx_compl_queue_dqo` - `gve_rx_buf_state_dqo` - `gve_tx_desc_dqo` - `gve_tx_pending_packet_dqo` - Incorporate these into the existing ring data structures: - `gve_rx_ring` - `gve_tx_ring` Noteworthy mentions: - `gve_rx_buf_state` represents an RX buffer which was posted to HW. Each RX queue has an array of these objects and the index into the array is used as the buffer_id when posted to HW. - `gve_tx_pending_packet_dqo` is treated similarly for TX queues. The completion_tag is the index into the array. - These two structures have links for linked lists which are represented by 16b indexes into a contiguous array of these structures. This reduces memory footprint compared to 64b pointers. - We use unions for the writeable datapath structures to reduce cache footprint. GQI specific members will renamed like DQO members in a future patch. Signed-off-by: Bailey Forrest <bcf@google.com> Reviewed-by: Willem de Bruijn <willemb@google.com> Reviewed-by: Catherine Sullivan <csully@google.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2021-06-24gve: Add support for DQO RX PTYPE mapBailey Forrest
Unlike GQI, DQO RX descriptors do not contain the L3 and L4 type of the packet. L3 and L4 types are necessary in order to set the hash and csum on RX SKBs correctly. DQO RX descriptors instead contain a 10 bit PTYPE index. The PTYPE map enables the device to tell the driver how to map from PTYPE index to L3/L4 type. The device doesn't provide any guarantees about the range of possible PTYPEs, so we just use a 1024 entry array to implement a fast mapping structure. Signed-off-by: Bailey Forrest <bcf@google.com> Reviewed-by: Willem de Bruijn <willemb@google.com> Reviewed-by: Catherine Sullivan <csully@google.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2021-06-24gve: adminq: DQO specific device descriptor logicBailey Forrest
- In addition to TX and RX queues, DQO has TX completion and RX buffer queues. - TX completions are received when the device has completed sending a packet on the wire. - RX buffers are posted on a separate queue form the RX completions. - DQO descriptor rings are allowed to be smaller than PAGE_SIZE. Signed-off-by: Bailey Forrest <bcf@google.com> Reviewed-by: Willem de Bruijn <willemb@google.com> Reviewed-by: Catherine Sullivan <csully@google.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2021-06-24gve: Introduce per netdev `enum gve_queue_format`Bailey Forrest
The currently supported queue formats are: - GQI_RDA - GQI with raw DMA addressing - GQI_QPL - GQI with queue page list - DQO_RDA - DQO with raw DMA addressing The old `gve_priv.raw_addressing` value is only used for GQI_RDA, so we remove it in favor of just checking against GQI_RDA Signed-off-by: Bailey Forrest <bcf@google.com> Reviewed-by: Willem de Bruijn <willemb@google.com> Reviewed-by: Catherine Sullivan <csully@google.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2021-06-24gve: Make gve_rx_slot_page_info.page_offset an absolute offsetBailey Forrest
Using `page_offset` like a boolean means a page may only be split into two sections. With page sizes larger than 4k, this can be very wasteful. Future commits in this patchset use `struct gve_rx_slot_page_info` in a way which supports a fixed buffer size and a variable page size. Signed-off-by: Bailey Forrest <bcf@google.com> Reviewed-by: Willem de Bruijn <willemb@google.com> Reviewed-by: Catherine Sullivan <csully@google.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2020-12-08gve: Add support for raw addressing in the tx pathCatherine Sullivan
During TX, skbs' data addresses are dma_map'ed and passed to the NIC. This means that the device can perform DMA directly from these addresses and the driver does not have to copy the buffer content into pre-allocated buffers/qpls (as in qpl mode). Reviewed-by: Yangchun Fu <yangchun@google.com> Signed-off-by: Catherine Sullivan <csully@google.com> Signed-off-by: David Awogbemila <awogbemila@google.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2020-12-08gve: Rx Buffer RecyclingDavid Awogbemila
This patch lets the driver reuse buffers that have been freed by the networking stack. In the raw addressing case, this allows the driver avoid allocating new buffers. In the qpl case, the driver can avoid copies. This patch separates the page refcount tracking mechanism into a function gve_rx_can_recycle_buffer which uses get_page - this will be changed in a future patch to entirely eliminate the use of get_page in tracking page refcounts. Signed-off-by: David Awogbemila <awogbemila@google.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2020-12-08gve: Add support for raw addressing to the rx pathCatherine Sullivan
Add support to use raw dma addresses in the rx path. Due to this new support we can alloc a new buffer instead of making a copy. RX buffers are handed to the networking stack and are re-allocated as needed, avoiding the need to use skb_copy_to_linear_data() as in "qpl" mode. Reviewed-by: Yangchun Fu <yangchun@google.com> Signed-off-by: Catherine Sullivan <csully@google.com> Signed-off-by: David Awogbemila <awogbemila@google.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2020-12-08gve: Add support for raw addressing device optionCatherine Sullivan
Add support to describe device for parsing device options. As the first device option, add raw addressing. "Raw Addressing" mode (as opposed to the current "qpl" mode) is an operational mode which allows the driver avoid bounce buffer copies which it currently performs using pre-allocated qpls (queue_page_lists) when sending and receiving packets. For egress packets, the provided skb data addresses will be dma_map'ed and passed to the device, allowing the NIC can perform DMA directly - the driver will not have to copy the buffer content into pre-allocated buffers/qpls (as in qpl mode). For ingress packets, copies are also eliminated as buffers are handed to the networking stack and then recycled or re-allocated as necessary, avoiding the use of skb_copy_to_linear_data(). This patch only introduces the option to the driver. Subsequent patches will add the ingress and egress functionality. Reviewed-by: Yangchun Fu <yangchun@google.com> Signed-off-by: Catherine Sullivan <csully@google.com> Signed-off-by: David Awogbemila <awogbemila@google.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2020-09-11gve: Enable Link Speed Reporting in the driver.David Awogbemila
This change allows the driver to report the device link speed when the ethtool command: ethtool <nic name> is run. Getting the link speed is done via a new admin queue command: ReportLinkSpeed. Reviewed-by: Yangchun Fu <yangchun@google.com> Signed-off-by: David Awogbemila <awogbemila@google.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2020-09-11gve: NIC stats for report-stats and for ethtoolDavid Awogbemila
This adds per queue NIC stats to ethtool stats and to report-stats. These stats are always exposed to guest whether or not the report-stats flag is turned on. Signed-off-by: David Awogbemila <awogbemila@google.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2020-09-11gve: Add Gvnic stats AQ command and ethtool show/set-priv-flags.Kuo Zhao
This adds functionality to report driver stats to Hypervisor. (Users may want to turn this feature off as a matter of privacy so a "report-stats" flag is added as an ethtool priv option. It is also disabled by default.) The hypervisor would trigger a stats report in case "too many" packets dropped; the stats would be useful in debugging stuck queues. A "stats_report_trigger_cnt" stat is added to count the number of times the hypervisor attempts to trigger stats report. A timer is also added so that when report-stats is enabled, stat are updated once every 20 seconds. Reviewed-by: Yangchun Fu <yangchun@google.com> Signed-off-by: Kuo Zhao <kuozhao@google.com> Signed-off-by: David Awogbemila <awogbemila@google.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2020-09-11gve: Add stats for gve.Kuo Zhao
Sample output of "ethtool -S <interface-name>" with 1 RX queue and 1 TX queue: NIC statistics: rx_packets: 1039 tx_packets: 37 rx_bytes: 822071 tx_bytes: 4100 rx_dropped: 0 tx_dropped: 0 tx_timeouts: 0 rx_skb_alloc_fail: 0 rx_buf_alloc_fail: 0 rx_desc_err_dropped_pkt: 0 interface_up_cnt: 1 interface_down_cnt: 0 reset_cnt: 0 page_alloc_fail: 0 dma_mapping_error: 0 rx_posted_desc[0]: 1365 rx_completed_desc[0]: 341 rx_bytes[0]: 215094 rx_dropped_pkt[0]: 0 rx_copybreak_pkt[0]: 3 rx_copied_pkt[0]: 3 tx_posted_desc[0]: 6 tx_completed_desc[0]: 6 tx_bytes[0]: 420 tx_wake[0]: 0 tx_stop[0]: 0 tx_event_counter[0]: 6 adminq_prod_cnt: 34 adminq_cmd_fail: 0 adminq_timeouts: 0 adminq_describe_device_cnt: 1 adminq_cfg_device_resources_cnt: 1 adminq_register_page_list_cnt: 16 adminq_unregister_page_list_cnt: 0 adminq_create_tx_queue_cnt: 8 adminq_create_rx_queue_cnt: 8 adminq_destroy_tx_queue_cnt: 0 adminq_destroy_rx_queue_cnt: 0 adminq_dcfg_device_resources_cnt: 0 adminq_set_driver_parameter_cnt: 0 Reviewed-by: Yangchun Fu <yangchun@google.com> Signed-off-by: Kuo Zhao <kuozhao@google.com> Signed-off-by: David Awogbemila <awogbemila@google.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2019-08-05gve: Fix case where desc_cnt and data_cnt can get out of syncCatherine Sullivan
desc_cnt and data_cnt should always be equal. In the case of a dropped packet desc_cnt was still getting updated (correctly), data_cnt was not. To eliminate this bug and prevent it from recurring this patch combines them into one ring level cnt. Signed-off-by: Catherine Sullivan <csully@google.com> Reviewed-by: Sagi Shahar <sagis@google.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2019-07-01gve: Add ethtool supportCatherine Sullivan
Add support for the following ethtool commands: ethtool -s|--change devname [msglvl N] [msglevel type on|off] ethtool -S|--statistics devname ethtool -i|--driver devname ethtool -l|--show-channels devname ethtool -L|--set-channels devname ethtool -g|--show-ring devname ethtool --reset devname Signed-off-by: Catherine Sullivan <csully@google.com> Signed-off-by: Sagi Shahar <sagis@google.com> Signed-off-by: Jon Olson <jonolson@google.com> Acked-by: Willem de Bruijn <willemb@google.com> Reviewed-by: Luigi Rizzo <lrizzo@google.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2019-07-01gve: Add workqueue and reset supportCatherine Sullivan
Add support for the workqueue to handle management interrupts and support for resets. Signed-off-by: Catherine Sullivan <csully@google.com> Signed-off-by: Sagi Shahar <sagis@google.com> Signed-off-by: Jon Olson <jonolson@google.com> Acked-by: Willem de Bruijn <willemb@google.com> Reviewed-by: Luigi Rizzo <lrizzo@google.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2019-07-01gve: Add transmit and receive supportCatherine Sullivan
Add support for passing traffic. Signed-off-by: Catherine Sullivan <csully@google.com> Signed-off-by: Sagi Shahar <sagis@google.com> Signed-off-by: Jon Olson <jonolson@google.com> Acked-by: Willem de Bruijn <willemb@google.com> Reviewed-by: Luigi Rizzo <lrizzo@google.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2019-07-01gve: Add basic driver framework for Compute Engine Virtual NICCatherine Sullivan
Add a driver framework for the Compute Engine Virtual NIC that will be available in the future. At this point the only functionality is loading the driver. Signed-off-by: Catherine Sullivan <csully@google.com> Signed-off-by: Sagi Shahar <sagis@google.com> Signed-off-by: Jon Olson <jonolson@google.com> Acked-by: Willem de Bruijn <willemb@google.com> Reviewed-by: Luigi Rizzo <lrizzo@google.com> Signed-off-by: David S. Miller <davem@davemloft.net>