[PATCH] serial: 8250_of: set UART_CAP_NOTEMT for rts-gpios RS485 direction

[Nicolas Thibert <nithibert@gmail.com>]

__stop_tx() (8250_port.c) only calls the RS485 rs485_stop_tx() hook
(which de-asserts the direction GPIO/RTS line) once it has observed
both UART_LSR_THRE and UART_LSR_TEMT for the last byte. If TEMT is
never seen and the driver hasn't set UART_CAP_NOTEMT, the function
returns without scheduling any retry -- the direction line is left
asserted (driver enabled) forever, with nothing to un-stick it short
of another kernel-visible LSR event.

of_platform_serial_setup()/of_platform_serial_probe() unconditionally
wire up the generic em485 GPIO-RTS RS485 support
(rs485_config/rs485_start_tx/rs485_stop_tx) for every port they
register, but never set UART_CAP_NOTEMT, so any board using this
driver whose 16550-compatible core doesn't reliably surface TEMT for
its shift register hits the stuck-direction-GPIO case above.

Confirmed live on an ath79 QCA9531 board (SoC-internal ns16550a-
compatible UART, RS485 transceiver DE/RE tied together on a GPIO via
rts-gpios, linux,rs485-enabled-at-boot-time): the direction GPIO
correctly asserts for the duration of a transmit, but never
de-asserts afterwards -- confirmed by sampling the GPIO's debugfs
state through and after a multi-hundred-byte write, on both the first
transmit and repeated back-to-back transmits. Setting
UART_CAP_NOTEMT, which makes __stop_tx() fall back to a frame-time-
based timer instead of waiting indefinitely on TEMT, makes the
direction GPIO reliably return low right after each transmit
completes.

Scope the fix to ports that declare a GPIO-controlled direction line
(rts-gpios), rather than setting it unconditionally for every port
this driver registers: this is the class of hardware actually
affected (RTS state has to be explicitly un-stuck by software, unlike
a UART's native RTS pin), and it avoids adding the extra frame-time
margin to ports relying on the native RTS pin, which this has not
been observed to need.

Set it in of_platform_serial_probe(), after the existing
"if (port8250.port.fifosize) port8250.capabilities = UART_CAP_FIFO;"
assignment, rather than in of_platform_serial_setup(): that later
plain assignment (not "|=") unconditionally overwrites
port8250.capabilities on any port whose "fifo-size" DT property is
set, silently discarding a capability bit set earlier in setup(). Not
observed on the reporter's own board (it has no "fifo-size" property,
so port.fifosize stays 0 and that assignment is skipped), but a real
regression waiting to happen on any other of_platform_serial user
that does set "fifo-size" -- which is a common, documented property
for this binding.

Signed-off-by: Nicolas Thibert <nithibert@gmail.com>
Assisted-by: LLM (Claude Sonnet 5, Anthropic)
---
 drivers/tty/serial/8250/8250_of.c | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)

--- a/drivers/tty/serial/8250/8250_of.c
+++ b/drivers/tty/serial/8250/8250_of.c
@@ -233,6 +233,24 @@ static int of_platform_serial_probe(struct platform_device *ofdev)
 			&port8250.overrun_backoff_time_ms) != 0)
 		port8250.overrun_backoff_time_ms = 0;

+	/*
+	 * This generic driver never enables a dedicated line-status
+	 * interrupt on TEMT, so for ports whose RS485 direction is
+	 * controlled via a GPIO (rts-gpios) rather than the native RTS
+	 * pin, __stop_tx() (8250_port.c) can see THRE without TEMT on the
+	 * last byte and bail out without ever retrying -- leaving the
+	 * direction GPIO stuck asserted after the last byte sent, on
+	 * hardware whose shift register doesn't reliably surface TEMT.
+	 * UART_CAP_NOTEMT makes it fall back to a frame-time-based timer
+	 * instead of waiting on that interrupt. Scoped to rts-gpios users
+	 * only, to avoid changing timing for ports relying on the native
+	 * RTS pin, which this has not been observed to affect. Set here,
+	 * after the fifosize-based capabilities assignment above, so it
+	 * isn't clobbered by it.
+	 */
+	if (of_property_present(ofdev->dev.of_node, "rts-gpios"))
+		port8250.capabilities |= UART_CAP_NOTEMT;
+
 	ret = serial8250_register_8250_port(&port8250);
 	if (ret < 0)
 		goto err_dispose;