From: Vineeth Karumanchi <vineeth.karumanchi@amd.com>
To: "Théo Lebrun" <theo.lebrun@bootlin.com>,
"Conor Dooley" <conor.dooley@microchip.com>,
"Andrew Lunn" <andrew+netdev@lunn.ch>,
"David S. Miller" <davem@davemloft.net>,
"Eric Dumazet" <edumazet@kernel.org>,
"Jakub Kicinski" <kuba@kernel.org>,
"Paolo Abeni" <pabeni@redhat.com>,
"Russell King" <linux@armlinux.org.uk>
Cc: netdev@vger.kernel.org, linux-kernel@vger.kernel.org,
"Nicolai Buchwitz" <nb@tipi-net.de>,
"Paolo Valerio" <pvalerio@redhat.com>,
"Vladimir Kondratiev" <vladimir.kondratiev@mobileye.com>,
"Gregory CLEMENT" <gregory.clement@bootlin.com>,
"Benoît Monin" <benoit.monin@bootlin.com>,
"Tawfik Bayouk" <tawfik.bayouk@mobileye.com>,
"Thomas Petazzoni" <thomas.petazzoni@bootlin.com>
Subject: Re: [PATCH net-next v7 1/3] net: macb: cache ENST config across link down/up
Date: Thu, 19 Nov 2026 18:06:33 +0530 [thread overview]
Message-ID: <fe08db37-4415-4b4b-b226-cc1a0ec4f029@amd.com> (raw)
In-Reply-To: <20261118-macb-set-channels-v7-1-2dbc00f686af@bootlin.com>
Hi Théo,
On 11/19/2026 2:23 AM, Théo Lebrun wrote:
> Currently, ENST is not explicitly restored across interface close/open.
> If HW keeps its register state then ENST is kept, otherwise it is lost;
> but we do not write the ENST schedule again on open.
>
> This causes trouble if the config is not compatible with the new open
> and link-up. The two issues that can occur are a link speed change (as
> all timings are configured in a unit relative to the link speed) or a
> change in the # of queues.
>
Thanks for the patch.
We are working on the same problem for CBS, covering the four scenarios
listed in:
https://lore.kernel.org/netdev/a04aa4d5-335b-47c1-b969-e89b18c9b142@amd.com/
Since this patch reworks how ENST state is kept and restored, it would
be good to introduce a generic top-level tc structure here, which ENST,
CBS and any later TSN offloads can share, instead of an ENST-specific one.
> The latter is the main reason we do this change before implementing
> ethtool -L support.
>
> If on link-up, the cached ENST config is invalid (link speed changed),
> then we drop the cached config, log a warning and explicitly override
> all ENST timings previously configured in the registers. If it is
> valid, we explicitly re-apply it.
>
> Take this opportunity to replace a kmalloc in
> macb_taprio_setup_replace() with a simple and tiny (104b) stack value.
>
> Drop the old hack of checking for pm_runtime_suspended(), which has
> little value because we could race with the runtime_suspend() call.
>
> Synchronise our config using the existing bp->mac_cfg_lock mutex, taken
> by macb_taprio_setup_replace() and macb_taprio_destroy(). To avoid an
> ABBA deadlock between bp->mac_cfg_lock and phydev->lock taken by
> phylink_ethtool_ksettings_get(), cache the link speed as bp->speed_mbps.
>
> Signed-off-by: Théo Lebrun <theo.lebrun@bootlin.com>
> ---
> drivers/net/ethernet/cadence/macb.h | 47 +++++---
> drivers/net/ethernet/cadence/macb_main.c | 187 +++++++++++++++----------------
> 2 files changed, 124 insertions(+), 110 deletions(-)
>
> diff --git a/drivers/net/ethernet/cadence/macb.h b/drivers/net/ethernet/cadence/macb.h
> index b7b4b6a38298..00258866623f 100644
> --- a/drivers/net/ethernet/cadence/macb.h
> +++ b/drivers/net/ethernet/cadence/macb.h
> @@ -1327,6 +1327,35 @@ struct macb_context {
> struct macb_txq txq[MACB_MAX_QUEUES];
> };
>
> +/**
> + * struct macb_queue_enst_config - Configuration for Enhanced Scheduled Traffic
> + * @start_time_mask: Bitmask representing the start time for the queue
> + * @on_time_bytes: "on" time nsec expressed in bytes
> + * @off_time_bytes: "off" time nsec expressed in bytes
> + *
> + * This structure holds the configuration parameters for an ENST queue,
> + * used to control time-based transmission scheduling in the MACB driver.
> + */
> +struct macb_queue_enst_config {
> + u32 start_time_mask;
> + u32 on_time_bytes;
> + u32 off_time_bytes;
> +};
> +
> +/**
> + * struct macb_enst - Cached Enhanced Scheduled Traffic configuration
> + * @queues: per-queue timing (zeroed for queues without schedule)
> + * @speed_mbps: link speed the timings were computed for, 0 if no schedule
> + * @queue_mask: queues enabled by the current schedule, 0 if no schedule
> + *
> + * The timings are expressed in bytes, so they depend on the link speed.
> + */
> +struct macb_enst {
> + struct macb_queue_enst_config queues[MACB_MAX_QUEUES];
> + u32 speed_mbps;
> + u32 queue_mask;
> +};
> +
Could this be a per-queue structure shared by all tc offloads, so that
new features only add their own part?
For example:
struct macb_tc_queue {
struct {
u32 idleslope; /* kbps */
} cbs;
struct {
u32 start_time; /* ENST_START_TIME register value */
u32 on_time_ns; /* gate open interval */
u64 off_time_ns; /* gate closed interval */
} enst;
};
> struct macb {
> void __iomem *regs;
> bool native_io;
> @@ -1427,11 +1456,14 @@ struct macb {
>
> /* Redundant to netif_carrier_ok(), but set under bp->mac_cfg_lock. */
> bool link_up;
> + u32 speed_mbps;
>
> u32 rx_intr_mask;
>
> struct macb_pm_data pm_data;
> const struct macb_usrio_config *usrio;
> +
> + struct macb_enst enst;
> };
>
> #ifdef CONFIG_MACB_USE_HWSTAMP
> @@ -1532,19 +1564,4 @@ struct macb_platform_data {
> struct clk *hclk;
> };
>
> -/**
> - * struct macb_queue_enst_config - Configuration for Enhanced Scheduled Traffic
> - * @start_time_mask: Bitmask representing the start time for the queue
> - * @on_time_bytes: "on" time nsec expressed in bytes
> - * @off_time_bytes: "off" time nsec expressed in bytes
> - *
> - * This structure holds the configuration parameters for an ENST queue,
> - * used to control time-based transmission scheduling in the MACB driver.
> - */
> -struct macb_queue_enst_config {
> - u32 start_time_mask;
> - u32 on_time_bytes;
> - u32 off_time_bytes;
> -};
> -
> #endif /* _MACB_H */
> diff --git a/drivers/net/ethernet/cadence/macb_main.c b/drivers/net/ethernet/cadence/macb_main.c
> index 9f6111661b40..1e0666692116 100644
> --- a/drivers/net/ethernet/cadence/macb_main.c
> +++ b/drivers/net/ethernet/cadence/macb_main.c
> @@ -847,6 +847,7 @@ static void macb_mac_link_down(struct phylink_config *config, unsigned int mode,
> mutex_lock(&bp->mac_cfg_lock);
>
> bp->link_up = false;
> + bp->speed_mbps = 0;
>
> if (!(bp->caps & MACB_CAPS_MACB_IS_EMAC))
> for (q = 0, queue = bp->queues; q < bp->num_queues; ++q, ++queue)
> @@ -955,6 +956,43 @@ static void gem_shuffle_tx_rings(struct macb *bp)
> gem_shuffle_tx_one_ring(queue);
> }
>
> +static void macb_apply_enst_config(struct macb *bp)
> +{
> + struct macb_queue_enst_config *config;
> + struct macb_queue *queue;
> + u32 queue_mask;
> + size_t q;
> +
> + /* Short-circuit to avoid writing to invalid register offsets */
> + if (!(bp->caps & MACB_CAPS_QBV))
> + return;
> +
> + guard(spinlock_irqsave)(&bp->lock);
> +
> + /* Disable ENST for all queues.
> + * Reconfiguring timings must be done with ENST disabled.
> + */
> + queue_mask = BIT_U32(bp->num_queues) - 1;
> + gem_writel(bp, ENST_CONTROL,
> + queue_mask << GEM_ENST_DISABLE_QUEUE_OFFSET);
> +
> + for (q = 0, queue = bp->queues; q < bp->num_queues; ++q, ++queue) {
> + /* Configure queue timing registers.
> + *
> + * Those fields must be zero if the queue has ENST disabled:
> + * notice we don't bother reading bp->enst.queue_mask & BIT(q).
> + */
> + config = &bp->enst.queues[q];
> + queue_writel(queue, ENST_START_TIME, config->start_time_mask);
> + queue_writel(queue, ENST_ON_TIME, config->on_time_bytes);
> + queue_writel(queue, ENST_OFF_TIME, config->off_time_bytes);
> + }
> +
> + /* Enable ENST for all configured queues in one write */
> + if (bp->enst.queue_mask)
> + gem_writel(bp, ENST_CONTROL, bp->enst.queue_mask);
> +}
> +
> static void macb_mac_link_up(struct phylink_config *config,
> struct phy_device *phydev,
> unsigned int mode, phy_interface_t interface,
> @@ -972,6 +1010,7 @@ static void macb_mac_link_up(struct phylink_config *config,
> spin_lock_irqsave(&bp->lock, flags);
>
> bp->link_up = true;
> + bp->speed_mbps = speed > 0 ? speed : 0;
>
> ctrl = macb_or_gem_readl(bp, NCFGR);
>
> @@ -1009,6 +1048,15 @@ static void macb_mac_link_up(struct phylink_config *config,
>
> spin_unlock_irqrestore(&bp->lock, flags);
>
> + if (bp->enst.speed_mbps && speed != bp->enst.speed_mbps) {
Why drop the schedule on a speed change? The taprio qdisc stays
installed, so the user's configuration is still valid;
Could we instead recompute the ON/OFF times for the new speed on
link-up, and only disable (with a warning) the queues whose intervals no
longer fit the hardware range at that speed?
> + netdev_warn(bp->netdev,
> + "TAPRIO dropped because link speed changed\n");
> +
> + memset(&bp->enst, 0, sizeof(bp->enst));
> + }
> +
> + macb_apply_enst_config(bp);
> +
Could this be a single macb_apply_tc_config() (or similar) that
reprograms every enabled tc offload? CBS needs exactly the same
handling, since idleslope is also relative to the link rate.
Thanks,
> if (!(bp->caps & MACB_CAPS_MACB_IS_EMAC)) {
> macb_set_tx_clk(bp, speed);
> gem_shuffle_tx_rings(bp);
> @@ -4754,19 +4802,23 @@ static int macb_taprio_setup_replace(struct net_device *netdev,
> struct tc_taprio_qopt_offload *conf)
> {
> u64 total_on_time = 0, start_time_sec = 0, start_time = conf->base_time;
> - u32 configured_queues = 0, speed = 0, start_time_nsec;
> - struct macb_queue_enst_config *enst_queue;
> - struct ethtool_link_ksettings kset = {};
> + u32 configured_queues = 0, start_time_nsec;
> struct macb *bp = netdev_priv(netdev);
> struct tc_taprio_sched_entry *entry;
> - struct macb_queue *queue;
> - u32 queue_mask;
> + struct macb_enst config = {};
> u8 queue_id;
> - size_t i, q;
> - int err;
> + u64 off;
> + size_t i;
> +
> + guard(mutex)(&bp->mac_cfg_lock);
> +
> + if (unlikely(!bp->speed_mbps)) {
> + netdev_err(netdev, "Invalid speed, link-down?\n");
> + return -EINVAL;
> + }
>
> if (conf->num_entries > bp->num_queues) {
> - netdev_err(netdev, "Too many TAPRIO entries: %zu > %d queues\n",
> + netdev_err(netdev, "Too many TAPRIO entries: %zu > %u active queues\n",
> conf->num_entries, bp->num_queues);
> return -EINVAL;
> }
> @@ -4777,23 +4829,6 @@ static int macb_taprio_setup_replace(struct net_device *netdev,
> return -ERANGE;
> }
>
> - /* Get the current link speed */
> - err = phylink_ethtool_ksettings_get(bp->phylink, &kset);
> - if (unlikely(err)) {
> - netdev_err(netdev, "Failed to get link settings: %d\n", err);
> - return err;
> - }
> -
> - speed = kset.base.speed;
> - if (unlikely(speed == SPEED_UNKNOWN || !speed)) {
> - netdev_err(netdev, "Invalid speed %d, link-down?\n", speed);
> - return -EINVAL;
> - }
> -
> - enst_queue = kzalloc_objs(*enst_queue, bp->num_queues);
> - if (unlikely(!enst_queue))
> - return -ENOMEM;
> -
> /* Pre-validate all entries before making any hardware changes */
> for (i = 0; i < conf->num_entries; i++) {
> entry = &conf->entries[i];
> @@ -4801,25 +4836,23 @@ static int macb_taprio_setup_replace(struct net_device *netdev,
> if (entry->command != TC_TAPRIO_CMD_SET_GATES) {
> netdev_err(netdev, "Entry %zu: unsupported command %d\n",
> i, entry->command);
> - err = -EOPNOTSUPP;
> - goto cleanup;
> + return -EOPNOTSUPP;
> }
>
> /* Validate gate_mask: must be nonzero, single queue, and within range */
> if (!is_power_of_2(entry->gate_mask)) {
> netdev_err(netdev, "Entry %zu: gate_mask 0x%x is not a power of 2 (only one queue per entry allowed)\n",
> i, entry->gate_mask);
> - err = -EINVAL;
> - goto cleanup;
> + return -EINVAL;
> }
>
> /* gate_mask must not select queues outside the valid queues */
> queue_id = order_base_2(entry->gate_mask);
> if (queue_id >= bp->num_queues) {
> - netdev_err(netdev, "Entry %zu: gate_mask 0x%x exceeds queue range (max_queues=%d)\n",
> - i, entry->gate_mask, bp->num_queues);
> - err = -EINVAL;
> - goto cleanup;
> + netdev_err(netdev, "Entry %zu: gate_mask 0x%x exceeds active queue range (num_queues=%u)\n",
> + i, entry->gate_mask,
> + bp->num_queues);
> + return -EINVAL;
> }
>
> /* Check for start time limits */
> @@ -4828,34 +4861,33 @@ static int macb_taprio_setup_replace(struct net_device *netdev,
> if (start_time_sec > GENMASK(GEM_START_TIME_SEC_SIZE - 1, 0)) {
> netdev_err(netdev, "Entry %zu: Start time %llu s exceeds hardware limit\n",
> i, start_time_sec);
> - err = -ERANGE;
> - goto cleanup;
> + return -ERANGE;
> }
>
> /* Check for on time limit */
> - if (entry->interval > enst_max_hw_interval(speed)) {
> + if (entry->interval > enst_max_hw_interval(bp->speed_mbps)) {
> netdev_err(netdev, "Entry %zu: interval %u ns exceeds hardware limit %llu ns\n",
> - i, entry->interval, enst_max_hw_interval(speed));
> - err = -ERANGE;
> - goto cleanup;
> + i, entry->interval,
> + enst_max_hw_interval(bp->speed_mbps));
> + return -ERANGE;
> }
>
> /* Check for off time limit*/
> - if ((conf->cycle_time - entry->interval) > enst_max_hw_interval(speed)) {
> + off = conf->cycle_time - entry->interval;
> + if (off > enst_max_hw_interval(bp->speed_mbps)) {
> netdev_err(netdev, "Entry %zu: off_time %llu ns exceeds hardware limit %llu ns\n",
> - i, conf->cycle_time - entry->interval,
> - enst_max_hw_interval(speed));
> - err = -ERANGE;
> - goto cleanup;
> + i, off,
> + enst_max_hw_interval(bp->speed_mbps));
> + return -ERANGE;
> }
>
> - enst_queue[queue_id].start_time_mask =
> + config.queues[queue_id].start_time_mask =
> (start_time_sec << GEM_START_TIME_SEC_OFFSET) |
> start_time_nsec;
> - enst_queue[queue_id].on_time_bytes =
> - enst_ns_to_hw_units(entry->interval, speed);
> - enst_queue[queue_id].off_time_bytes =
> - enst_ns_to_hw_units(conf->cycle_time - entry->interval, speed);
> + config.queues[queue_id].on_time_bytes =
> + enst_ns_to_hw_units(entry->interval, bp->speed_mbps);
> + config.queues[queue_id].off_time_bytes =
> + enst_ns_to_hw_units(off, bp->speed_mbps);
>
> configured_queues |= entry->gate_mask;
> total_on_time += entry->interval;
> @@ -4866,82 +4898,47 @@ static int macb_taprio_setup_replace(struct net_device *netdev,
> if (total_on_time > conf->cycle_time) {
> netdev_err(netdev, "Total ON %llu ns exceeds cycle time %llu ns\n",
> total_on_time, conf->cycle_time);
> - err = -EINVAL;
> - goto cleanup;
> + return -EINVAL;
> }
>
> netdev_dbg(netdev, "TAPRIO setup: %zu entries, base_time=%lld ns, cycle_time=%llu ns\n",
> conf->num_entries, conf->base_time, conf->cycle_time);
>
> - /* All validations passed - proceed with hardware configuration */
> - scoped_guard(spinlock_irqsave, &bp->lock) {
> - /* Disable ENST queues if running before configuring */
> - queue_mask = BIT_U32(bp->num_queues) - 1;
> - gem_writel(bp, ENST_CONTROL,
> - queue_mask << GEM_ENST_DISABLE_QUEUE_OFFSET);
> + config.queue_mask = configured_queues;
> + config.speed_mbps = bp->speed_mbps;
> + bp->enst = config;
>
> - for (q = 0, queue = bp->queues; q < bp->num_queues; ++q, ++queue) {
> - /* Configure queue timing registers */
> - queue_writel(queue, ENST_START_TIME,
> - enst_queue[q].start_time_mask);
> - queue_writel(queue, ENST_ON_TIME,
> - enst_queue[q].on_time_bytes);
> - queue_writel(queue, ENST_OFF_TIME,
> - enst_queue[q].off_time_bytes);
> - }
> -
> - /* Enable ENST for all configured queues in one write */
> - gem_writel(bp, ENST_CONTROL, configured_queues);
> - }
> + macb_apply_enst_config(bp);
>
> netdev_info(netdev, "TAPRIO configuration completed successfully: %zu entries, %d queues configured\n",
> conf->num_entries, hweight32(configured_queues));
>
> -cleanup:
> - kfree(enst_queue);
> - return err;
> + return 0;
> }
>
> static void macb_taprio_destroy(struct net_device *netdev)
> {
> struct macb *bp = netdev_priv(netdev);
> - struct macb_queue *queue;
> - u32 queue_mask;
> - unsigned int q;
> +
> + guard(mutex)(&bp->mac_cfg_lock);
>
> netdev_reset_tc(netdev);
> - queue_mask = BIT_U32(bp->num_queues) - 1;
> + memset(&bp->enst, 0, sizeof(bp->enst));
>
> - scoped_guard(spinlock_irqsave, &bp->lock) {
> - /* Single disable command for all queues */
> - gem_writel(bp, ENST_CONTROL,
> - queue_mask << GEM_ENST_DISABLE_QUEUE_OFFSET);
> + if (bp->link_up)
> + macb_apply_enst_config(bp);
>
> - /* Clear all queue ENST registers in batch */
> - for (q = 0, queue = bp->queues; q < bp->num_queues; ++q, ++queue) {
> - queue_writel(queue, ENST_START_TIME, 0);
> - queue_writel(queue, ENST_ON_TIME, 0);
> - queue_writel(queue, ENST_OFF_TIME, 0);
> - }
> - }
> netdev_info(netdev, "TAPRIO destroy: All gates disabled\n");
> }
>
> static int macb_setup_taprio(struct net_device *netdev,
> struct tc_taprio_qopt_offload *taprio)
> {
> - struct macb *bp = netdev_priv(netdev);
> int err = 0;
>
> if (unlikely(!(netdev->hw_features & NETIF_F_HW_TC)))
> return -EOPNOTSUPP;
>
> - /* Check if Device is in runtime suspend */
> - if (unlikely(pm_runtime_suspended(&bp->pdev->dev))) {
> - netdev_err(netdev, "Device is in runtime suspend\n");
> - return -EOPNOTSUPP;
> - }
> -
> switch (taprio->cmd) {
> case TAPRIO_CMD_REPLACE:
> err = macb_taprio_setup_replace(netdev, taprio);
>
--
🙏 Vineeth
next prev parent reply other threads:[~2026-11-19 12:36 UTC|newest]
Thread overview: 8+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-18 20:53 [PATCH net-next v7 0/3] net: macb: implement ethtool set channels count operation Théo Lebrun
2026-11-18 20:53 ` [PATCH net-next v7 1/3] net: macb: cache ENST config across link down/up Théo Lebrun
2026-11-19 11:49 ` Nicolai Buchwitz
2026-11-19 12:36 ` Vineeth Karumanchi [this message]
2026-11-18 20:53 ` [PATCH net-next v7 2/3] net: macb: implement ethtool_ops.get|set_channels() Théo Lebrun
2026-11-19 11:54 ` Nicolai Buchwitz
2026-11-18 20:53 ` [PATCH net-next v7 3/3] net: macb: distribute evenly Tx SRAM segments Théo Lebrun
2026-11-19 11:45 ` Nicolai Buchwitz
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=fe08db37-4415-4b4b-b226-cc1a0ec4f029@amd.com \
--to=vineeth.karumanchi@amd.com \
--cc=andrew+netdev@lunn.ch \
--cc=benoit.monin@bootlin.com \
--cc=conor.dooley@microchip.com \
--cc=davem@davemloft.net \
--cc=edumazet@kernel.org \
--cc=gregory.clement@bootlin.com \
--cc=kuba@kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux@armlinux.org.uk \
--cc=nb@tipi-net.de \
--cc=netdev@vger.kernel.org \
--cc=pabeni@redhat.com \
--cc=pvalerio@redhat.com \
--cc=tawfik.bayouk@mobileye.com \
--cc=theo.lebrun@bootlin.com \
--cc=thomas.petazzoni@bootlin.com \
--cc=vladimir.kondratiev@mobileye.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®