* [PATCH v3 0/3] fbcon: fix rotated font buffer handling on font
@ 2026-10-16 7:30 Yao Yiqi
2026-10-16 7:30 ` [PATCH v3 1/3] fbcon: Include font geometry in the rotated font cache Yao Yiqi
` (2 more replies)
0 siblings, 3 replies; 8+ messages in thread
From: Yao Yiqi @ 2026-10-16 7:30 UTC (permalink / raw)
To: deller, tzimmermann, simona
Cc: dri-devel, linux-fbdev, linux-kernel, yaoyiqi3, Yao Yiqi
This series fixes out-of-bounds reads of the rotated font buffer on
consoles that use fbcon's software rotation (fbcon=rotate:N).
Changing the console font through KDFONTOP installs the new font
geometry and then calls vc_resize(), whose redraw indexes
par->rotated.buf with the new cell size while the buffer is still
sized and laid out for the old font. KASAN reports, for example:
BUG: KASAN: slab-out-of-bounds in soft_cursor
Read of size 64 at addr ...
memcpy
soft_cursor
cw_cursor
fbcon_cursor
hide_cursor
redraw_screen
vc_do_resize
fbcon_do_set_font
fbcon_set_font
con_font_op
vt_ioctl
Patch 2 adds the missing rebuild for the font-change path, and
compensates in the error path of fbcon_do_set_font(): if vc_resize()
fails, the font state is rolled back, and the rendering state has to
be selected and the rotated cache rebuilt for the restored font as
well. The rebuild itself cannot be undone, because font_data_rotate()
has already reused or freed the old buffer.
The cache key in fbcon_rotate_font() compared only the fontdata
pointer and the rotation. Raw font data does not store geometry, and
fbcon_set_font() deduplicates uploads by byte equality, so re-uploading
the same bytes with different dimensions keeps the pointer unchanged
while the geometry changes; the early-out then skips the rebuild and
the stale buffer is overrun during the resize redraw. Patch 1 adds the
font geometry to the cache key. Patch 3 makes a failed buffer
allocation retried on the next call, instead of blanking the console
until a font or rotation with a different key is installed.
The unrotated-rendering fallback that fbcon_init(), fbcon_switch() and
fbcon_modechanged() attempt when rotate_font() fails is ineffective:
set_blitting_type() immediately re-derives par->rotate from
par->p->con_rotate and reinstalls the rotated bitops. This is a
pre-existing issue that is independent of the font-change path, so it
is deliberately not addressed here and will be sent separately.
Changes since v2:
- patch 2: also select the blitting type again in the error path of
fbcon_do_set_font(). On tile-blitting hardware, set_blitting_type()
uploads the font map to the text engine; without the second call, a
failed vc_resize() left the engine with the rejected font (found by
the Sashiko review)
Changes since v1 [1]:
- new patch 1 ("Include font geometry in the rotated font cache") in
response to the review, which noted that the early-out relies on the
fontdata pointer alone
- patch 2 (was patch 1): also rebuild in the error path of
fbcon_do_set_font(), so a failed vc_resize() no longer leaves the
cache describing the rejected font while the console renders with
the restored one; and select the console's blitting state with
set_blitting_type() before rebuilding, like the other rotate_font()
callers, instead of writing par->p directly
- patch 3 (was patch 2): unchanged in essence; the retry now covers
the geometry-aware key of patch 1
[1] https://lore.kernel.org/all/20261014085102.497789-1-yaoyiqi3@huawei.com/
Yao Yiqi (3):
fbcon: Include font geometry in the rotated font cache
fbcon: Rebuild rotated font buffer on font change
fbcon: Retry rotated font allocation after failure
drivers/video/fbdev/core/fbcon.c | 23 +++++++++++++++++++++++
drivers/video/fbdev/core/fbcon.h | 9 ++++++---
drivers/video/fbdev/core/fbcon_rotate.c | 10 +++++++++-
3 files changed, 38 insertions(+), 4 deletions(-)
--
2.56.0
^ permalink raw reply [flat|nested] 8+ messages in thread
* [PATCH v3 1/3] fbcon: Include font geometry in the rotated font cache
2026-10-16 7:30 [PATCH v3 0/3] fbcon: fix rotated font buffer handling on font Yao Yiqi
@ 2026-10-16 7:30 ` Yao Yiqi
2026-10-18 14:05 ` Helge Deller
2026-10-16 7:30 ` [PATCH v3 2/3] fbcon: Rebuild rotated font buffer on font change Yao Yiqi
2026-10-16 7:30 ` [PATCH v3 3/3] fbcon: Retry rotated font allocation after failure Yao Yiqi
2 siblings, 1 reply; 8+ messages in thread
From: Yao Yiqi @ 2026-10-16 7:30 UTC (permalink / raw)
To: deller, tzimmermann, simona
Cc: dri-devel, linux-fbdev, linux-kernel, yaoyiqi3, Yao Yiqi, stable
From: Yao Yiqi <yaoyiqi3@huawei.com>
fbcon_rotate_font() caches a rotated copy of the console font in
par->rotated.buf and skips the rebuild when the cache appears to be
up to date. The cache is keyed on the source font's data pointer and
the rotation alone, on the assumption that an unchanged fontdata
pointer implies an unchanged font.
That assumption does not hold. Raw font data (font_data_t) stores no
glyph geometry; width, height and charcount are tracked separately in
vc->vc_font. When a font is uploaded, fbcon_set_font() deduplicates
it against the fonts of all consoles with font_data_is_equal(), which
compares only the byte count, the checksum and the raw bytes. Two
uploads of the same data that declare different dimensions therefore
end up sharing one font_data_t, and a console can install a new
geometry while its fontdata pointer stays the same.
The early-out in fbcon_rotate_font() then skips the rebuild, and the
rotated blitting routines index the stale buffer with cell sizes
computed from the new geometry:
src = par->rotated.buf + (scr_readw(s) & charmask) * cellsize;
Rotation swaps a glyph's width and height, so the size of the rotated
cell depends on pitch rounding and differs between geometries. A 4x16
font whose 4096 bytes are re-uploaded as 16x8 on a CW-rotated console,
for example, grows the rotated cell size from 8 to 16 bytes, and the
largest character then reads past the end of the 2048-byte buffer that
was built for the 4x16 geometry.
The key has never included the geometry. Its exposure was limited,
but not eliminated, while the deduplication in fbcon_set_font() also
compared the consoles' font widths and total byte counts: with the
charcount restricted to 256 or 512, differing uploads could then only
trade height against charcount (such as 4x18x256 against 4x9x512),
and even those can change the rotated cell size. Commit 1e3c49aa03fb
("lib/fonts: Compare font data for equality with font_data_is_equal()")
dropped the width comparison, so identical bytes now share font data
across arbitrary dimensions.
Record the geometry that the buffer was built from and compare it in
the early-out, so a geometry change always triggers a rebuild.
Fixes: 6cc50e1c5b57 ("[PATCH] fbcon: Console Rotation - Add support to rotate font bitmap")
Cc: stable@vger.kernel.org
Signed-off-by: Yao Yiqi <yaoyiqi3@huawei.com>
---
drivers/video/fbdev/core/fbcon.h | 9 ++++++---
drivers/video/fbdev/core/fbcon_rotate.c | 9 ++++++++-
2 files changed, 14 insertions(+), 4 deletions(-)
diff --git a/drivers/video/fbdev/core/fbcon.h b/drivers/video/fbdev/core/fbcon.h
index 407d207b14f1..bfbd70f8b1cd 100644
--- a/drivers/video/fbdev/core/fbcon.h
+++ b/drivers/video/fbdev/core/fbcon.h
@@ -89,10 +89,13 @@ struct fbcon_par {
char *cursor_data;
#ifdef CONFIG_FRAMEBUFFER_CONSOLE_ROTATION
struct {
- font_data_t *fontdata; /* source font */
- u8 *buf; /* rotated glyphs */
+ font_data_t *fontdata; /* source font */
+ u8 *buf; /* rotated glyphs */
size_t bufsize;
- int buf_rotate; /* rotation of buf */
+ int buf_rotate; /* rotation of buf */
+ unsigned int buf_width; /* geometry of source font */
+ unsigned int buf_height;
+ unsigned int buf_charcount;
} rotated;
#endif
u8 *cursor_src;
diff --git a/drivers/video/fbdev/core/fbcon_rotate.c b/drivers/video/fbdev/core/fbcon_rotate.c
index 6cdbc96eeca6..660a3fa81bae 100644
--- a/drivers/video/fbdev/core/fbcon_rotate.c
+++ b/drivers/video/fbdev/core/fbcon_rotate.c
@@ -21,11 +21,18 @@ int fbcon_rotate_font(struct fb_info *info, struct vc_data *vc)
unsigned char *buf;
int ret;
- if (par->p->fontdata == par->rotated.fontdata && par->rotate == par->rotated.buf_rotate)
+ if (par->p->fontdata == par->rotated.fontdata &&
+ par->rotate == par->rotated.buf_rotate &&
+ vc->vc_font.width == par->rotated.buf_width &&
+ vc->vc_font.height == par->rotated.buf_height &&
+ vc->vc_font.charcount == par->rotated.buf_charcount)
return 0;
par->rotated.fontdata = par->p->fontdata;
par->rotated.buf_rotate = par->rotate;
+ par->rotated.buf_width = vc->vc_font.width;
+ par->rotated.buf_height = vc->vc_font.height;
+ par->rotated.buf_charcount = vc->vc_font.charcount;
if (info->fbops->fb_sync)
info->fbops->fb_sync(info);
--
2.56.0
^ permalink raw reply [flat|nested] 8+ messages in thread
* [PATCH v3 2/3] fbcon: Rebuild rotated font buffer on font change
2026-10-16 7:30 [PATCH v3 0/3] fbcon: fix rotated font buffer handling on font Yao Yiqi
2026-10-16 7:30 ` [PATCH v3 1/3] fbcon: Include font geometry in the rotated font cache Yao Yiqi
@ 2026-10-16 7:30 ` Yao Yiqi
2026-10-16 7:59 ` Thomas Zimmermann
2026-10-16 7:30 ` [PATCH v3 3/3] fbcon: Retry rotated font allocation after failure Yao Yiqi
2 siblings, 1 reply; 8+ messages in thread
From: Yao Yiqi @ 2026-10-16 7:30 UTC (permalink / raw)
To: deller, tzimmermann, simona
Cc: dri-devel, linux-fbdev, linux-kernel, yaoyiqi3, Yao Yiqi, stable
From: Yao Yiqi <yaoyiqi3@huawei.com>
fbcon_rotate_font() caches a rotated copy of the console font in
par->rotated.buf. The buffer is sized from the font's width, height
and charcount, and is rebuilt in fbcon_init(), fbcon_switch() and
fbcon_modechanged(), but not in fbcon_do_set_font().
When the console font is changed through KDFONTOP/KD_FONT_OP_SET, the
new font data and geometry are installed and vc_resize() is called.
The redraw that follows reaches the rotated bitops while the buffer
still holds the old font: the rotated putcs/cursor ops index
par->rotated.buf with the new cell size and character codes, so a
stale, too-small buffer is read out of bounds.
KASAN reports, for example:
BUG: KASAN: slab-out-of-bounds in soft_cursor
Read of size 64 at addr ...
memcpy
soft_cursor
cw_cursor
fbcon_cursor
hide_cursor
redraw_screen
vc_do_resize
fbcon_do_set_font
fbcon_set_font
con_font_op
vt_ioctl
Rebuild the buffer for the visible console right after the new font
geometry has been installed and before any redraw. Select the
console's blitting state with set_blitting_type() first, as
fbcon_init(), fbcon_switch() and fbcon_modechanged() do before their
rotate_font() calls: fbcon_rotate_font() reads the source font
through par->p, which may otherwise still refer to another console of
this framebuffer. On tile-blitting hardware, the same call uploads
the new font's map to the text engine, which previously happened only
when the console was switched. Background consoles keep their rebuild
at fbcon_switch() time, as the buffer is shared per framebuffer and
must not be rebuilt for a console that is not being displayed.
If vc_resize() fails, err_out restores the previous font. The rotated
buffer that was rebuilt for the new font earlier in the function
cannot be restored with it: font_data_rotate() has already reused or
freed the buffer that belonged to the old font. Select the blitting
type and rebuild the buffer again after the restore: the upload to
tile-blitting hardware and the rotated cache would otherwise keep
describing the rejected font, and par->rotated.fontdata would keep
referencing the font data whose last reference the error path drops.
A failing rebuild leaves par->rotated.buf NULL, so the rotated ops
skip drawing instead of reading a mismatched buffer; the next patch
retries the allocation on the following call.
Unlike the other callers, this one does not fall back to unrotated
rendering on failure. The "par->rotate = FB_ROTATE_UR;
set_blitting_type(vc, info);" fallback they use does not actually
switch to unrotated rendering: set_blitting_type() calls
fbcon_set_rotation(), which immediately re-derives par->rotate from
par->p->con_rotate. Replicating that pattern here would not change
the outcome; repairing the fallback is left to separate work.
Fixes: 6cc50e1c5b57 ("[PATCH] fbcon: Console Rotation - Add support to rotate font bitmap")
Cc: stable@vger.kernel.org
Signed-off-by: Yao Yiqi <yaoyiqi3@huawei.com>
---
Changes in v3:
- Also select the blitting type again in the error path of
fbcon_do_set_font(). On tile-blitting hardware, set_blitting_type()
uploads the font map to the text engine; without the second call, a
failed vc_resize() left the engine with the rejected font (found by
the Sashiko review).
Changes in v2:
- Rebuild the buffer again in the error path of fbcon_do_set_font().
Without this, a failed vc_resize() left the rotated cache describing
the rejected font while the console rendered with the restored one,
and par->rotated.fontdata referenced freed font data (found by the
Sashiko review).
- Select the console's blitting state with set_blitting_type() before
rebuilding, like the other rotate_font() callers, instead of writing
par->p directly.
- Reworded the note about the unrotated-rendering fallback, which is
now left to separate work.
---
drivers/video/fbdev/core/fbcon.c | 23 +++++++++++++++++++++++
1 file changed, 23 insertions(+)
diff --git a/drivers/video/fbdev/core/fbcon.c b/drivers/video/fbdev/core/fbcon.c
index 01715873ea49..d90a2ad45053 100644
--- a/drivers/video/fbdev/core/fbcon.c
+++ b/drivers/video/fbdev/core/fbcon.c
@@ -2432,6 +2432,17 @@ static int fbcon_do_set_font(struct vc_data *vc, int w, int h, int charcount,
else if (!vc->vc_hi_font_mask && charcount == 512)
set_vc_hi_font(vc, true);
+ /*
+ * Select the console's blitting state and update par->rotated to
+ * match the new font, so that the rendering below does not use a
+ * cache that was built for the old font.
+ */
+ if (con_is_visible(vc)) {
+ set_blitting_type(vc, info);
+ if (par->bitops->rotate_font)
+ par->bitops->rotate_font(info, vc);
+ }
+
if (resize) {
int cols, rows;
@@ -2466,6 +2477,18 @@ static int fbcon_do_set_font(struct vc_data *vc, int w, int h, int charcount,
else if (!old_hi_font_mask && vc->vc_hi_font_mask)
set_vc_hi_font(vc, false);
+ /*
+ * Restore the rendering state for the old font: re-select the
+ * blitting type, which re-uploads the font map on tile-blitting
+ * hardware, and overwrite par->rotated with the restored font's
+ * state.
+ */
+ if (con_is_visible(vc)) {
+ set_blitting_type(vc, info);
+ if (par->bitops->rotate_font)
+ par->bitops->rotate_font(info, vc);
+ }
+
font_data_put(data);
return ret;
--
2.56.0
^ permalink raw reply [flat|nested] 8+ messages in thread
* [PATCH v3 3/3] fbcon: Retry rotated font allocation after failure
2026-10-16 7:30 [PATCH v3 0/3] fbcon: fix rotated font buffer handling on font Yao Yiqi
2026-10-16 7:30 ` [PATCH v3 1/3] fbcon: Include font geometry in the rotated font cache Yao Yiqi
2026-10-16 7:30 ` [PATCH v3 2/3] fbcon: Rebuild rotated font buffer on font change Yao Yiqi
@ 2026-10-16 7:30 ` Yao Yiqi
2 siblings, 0 replies; 8+ messages in thread
From: Yao Yiqi @ 2026-10-16 7:30 UTC (permalink / raw)
To: deller, tzimmermann, simona
Cc: dri-devel, linux-fbdev, linux-kernel, yaoyiqi3, Yao Yiqi, stable
From: Yao Yiqi <yaoyiqi3@huawei.com>
fbcon_rotate_font() records the font, its geometry and the rotation it
is about to handle in par->rotated before calling font_data_rotate().
If the allocation fails, the error path clears par->rotated.buf, but
the recorded font, geometry and rotation already look up to date, so
the next call takes the early-out and never retries. The rotated
putcs/cursor ops keep returning early on the NULL buffer and the
console stays blank until a font or rotation with a different key is
installed, even once memory is available again.
Also test par->rotated.buf in the early-out, so a failed allocation is
retried the next time fbcon_rotate_font() runs: at the next
fbcon_switch(), fbcon_modechanged() or font change.
Fixes: e4ef723d8975 ("fbcon: Avoid OOB font access if console rotation fails")
Cc: stable@vger.kernel.org
Signed-off-by: Yao Yiqi <yaoyiqi3@huawei.com>
---
Changes in v2:
- The early-out now also compares the font geometry (previous patch in
this series); the retry covers the combined key.
---
drivers/video/fbdev/core/fbcon_rotate.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/drivers/video/fbdev/core/fbcon_rotate.c b/drivers/video/fbdev/core/fbcon_rotate.c
index 660a3fa81bae..7afd63dcee7b 100644
--- a/drivers/video/fbdev/core/fbcon_rotate.c
+++ b/drivers/video/fbdev/core/fbcon_rotate.c
@@ -25,7 +25,8 @@ int fbcon_rotate_font(struct fb_info *info, struct vc_data *vc)
par->rotate == par->rotated.buf_rotate &&
vc->vc_font.width == par->rotated.buf_width &&
vc->vc_font.height == par->rotated.buf_height &&
- vc->vc_font.charcount == par->rotated.buf_charcount)
+ vc->vc_font.charcount == par->rotated.buf_charcount &&
+ par->rotated.buf)
return 0;
par->rotated.fontdata = par->p->fontdata;
--
2.56.0
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH v3 2/3] fbcon: Rebuild rotated font buffer on font change
2026-10-16 7:30 ` [PATCH v3 2/3] fbcon: Rebuild rotated font buffer on font change Yao Yiqi
@ 2026-10-16 7:59 ` Thomas Zimmermann
2026-10-19 8:46 ` yyaqia
0 siblings, 1 reply; 8+ messages in thread
From: Thomas Zimmermann @ 2026-10-16 7:59 UTC (permalink / raw)
To: Yao Yiqi, deller, simona
Cc: dri-devel, linux-fbdev, linux-kernel, yaoyiqi3, stable
Hi
Am 16.10.26 um 09:30 schrieb Yao Yiqi:
> From: Yao Yiqi <yaoyiqi3@huawei.com>
>
> fbcon_rotate_font() caches a rotated copy of the console font in
> par->rotated.buf. The buffer is sized from the font's width, height
> and charcount, and is rebuilt in fbcon_init(), fbcon_switch() and
> fbcon_modechanged(), but not in fbcon_do_set_font().
>
> When the console font is changed through KDFONTOP/KD_FONT_OP_SET, the
> new font data and geometry are installed and vc_resize() is called.
> The redraw that follows reaches the rotated bitops while the buffer
> still holds the old font: the rotated putcs/cursor ops index
> par->rotated.buf with the new cell size and character codes, so a
> stale, too-small buffer is read out of bounds.
>
> KASAN reports, for example:
>
> BUG: KASAN: slab-out-of-bounds in soft_cursor
> Read of size 64 at addr ...
> memcpy
> soft_cursor
> cw_cursor
> fbcon_cursor
> hide_cursor
> redraw_screen
> vc_do_resize
> fbcon_do_set_font
> fbcon_set_font
> con_font_op
> vt_ioctl
>
> Rebuild the buffer for the visible console right after the new font
> geometry has been installed and before any redraw. Select the
> console's blitting state with set_blitting_type() first, as
> fbcon_init(), fbcon_switch() and fbcon_modechanged() do before their
> rotate_font() calls: fbcon_rotate_font() reads the source font
> through par->p, which may otherwise still refer to another console of
> this framebuffer. On tile-blitting hardware, the same call uploads
> the new font's map to the text engine, which previously happened only
> when the console was switched. Background consoles keep their rebuild
> at fbcon_switch() time, as the buffer is shared per framebuffer and
> must not be rebuilt for a console that is not being displayed.
>
> If vc_resize() fails, err_out restores the previous font. The rotated
> buffer that was rebuilt for the new font earlier in the function
> cannot be restored with it: font_data_rotate() has already reused or
> freed the buffer that belonged to the old font. Select the blitting
> type and rebuild the buffer again after the restore: the upload to
> tile-blitting hardware and the rotated cache would otherwise keep
> describing the rejected font, and par->rotated.fontdata would keep
> referencing the font data whose last reference the error path drops.
> A failing rebuild leaves par->rotated.buf NULL, so the rotated ops
> skip drawing instead of reading a mismatched buffer; the next patch
> retries the allocation on the following call.
>
> Unlike the other callers, this one does not fall back to unrotated
> rendering on failure. The "par->rotate = FB_ROTATE_UR;
> set_blitting_type(vc, info);" fallback they use does not actually
> switch to unrotated rendering: set_blitting_type() calls
> fbcon_set_rotation(), which immediately re-derives par->rotate from
> par->p->con_rotate. Replicating that pattern here would not change
> the outcome; repairing the fallback is left to separate work.
It's way too long and boring to read. Can you please try to shorten the
AI-generated commit description?
>
> Fixes: 6cc50e1c5b57 ("[PATCH] fbcon: Console Rotation - Add support to rotate font bitmap")
> Cc: stable@vger.kernel.org
> Signed-off-by: Yao Yiqi <yaoyiqi3@huawei.com>
> ---
> Changes in v3:
>
> - Also select the blitting type again in the error path of
> fbcon_do_set_font(). On tile-blitting hardware, set_blitting_type()
> uploads the font map to the text engine; without the second call, a
> failed vc_resize() left the engine with the rejected font (found by
> the Sashiko review).
The problem is that rotate_font can fail as well. Handling that in the
error path is not possible; hence results might be worse than keeping
the rejected font. Can you try to keep the original rotated font and
reinstall that directly?
Best regards
Thomas
>
> Changes in v2:
>
> - Rebuild the buffer again in the error path of fbcon_do_set_font().
> Without this, a failed vc_resize() left the rotated cache describing
> the rejected font while the console rendered with the restored one,
> and par->rotated.fontdata referenced freed font data (found by the
> Sashiko review).
> - Select the console's blitting state with set_blitting_type() before
> rebuilding, like the other rotate_font() callers, instead of writing
> par->p directly.
> - Reworded the note about the unrotated-rendering fallback, which is
> now left to separate work.
> ---
> drivers/video/fbdev/core/fbcon.c | 23 +++++++++++++++++++++++
> 1 file changed, 23 insertions(+)
>
> diff --git a/drivers/video/fbdev/core/fbcon.c b/drivers/video/fbdev/core/fbcon.c
> index 01715873ea49..d90a2ad45053 100644
> --- a/drivers/video/fbdev/core/fbcon.c
> +++ b/drivers/video/fbdev/core/fbcon.c
> @@ -2432,6 +2432,17 @@ static int fbcon_do_set_font(struct vc_data *vc, int w, int h, int charcount,
> else if (!vc->vc_hi_font_mask && charcount == 512)
> set_vc_hi_font(vc, true);
>
> + /*
> + * Select the console's blitting state and update par->rotated to
> + * match the new font, so that the rendering below does not use a
> + * cache that was built for the old font.
> + */
> + if (con_is_visible(vc)) {
> + set_blitting_type(vc, info);
> + if (par->bitops->rotate_font)
> + par->bitops->rotate_font(info, vc);
> + }
> +
> if (resize) {
> int cols, rows;
>
> @@ -2466,6 +2477,18 @@ static int fbcon_do_set_font(struct vc_data *vc, int w, int h, int charcount,
> else if (!old_hi_font_mask && vc->vc_hi_font_mask)
> set_vc_hi_font(vc, false);
>
> + /*
> + * Restore the rendering state for the old font: re-select the
> + * blitting type, which re-uploads the font map on tile-blitting
> + * hardware, and overwrite par->rotated with the restored font's
> + * state.
> + */
> + if (con_is_visible(vc)) {
> + set_blitting_type(vc, info);
> + if (par->bitops->rotate_font)
> + par->bitops->rotate_font(info, vc);
> + }
> +
> font_data_put(data);
>
> return ret;
--
--
Thomas Zimmermann
Graphics Driver Developer
SUSE Software Solutions Germany GmbH
Frankenstr. 146, 90461 Nürnberg, Germany, www.suse.com
GF: Stefan Gaiser, Jochen Jaser, Abhinav Puri, (HRB 36809, AG Nürnberg)
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH v3 1/3] fbcon: Include font geometry in the rotated font cache
2026-10-16 7:30 ` [PATCH v3 1/3] fbcon: Include font geometry in the rotated font cache Yao Yiqi
@ 2026-10-18 14:05 ` Helge Deller
2026-10-19 9:12 ` yyaqia
0 siblings, 1 reply; 8+ messages in thread
From: Helge Deller @ 2026-10-18 14:05 UTC (permalink / raw)
To: Yao Yiqi, tzimmermann, simona
Cc: dri-devel, linux-fbdev, linux-kernel, yaoyiqi3, stable
On 10/16/26 09:30, Yao Yiqi wrote:
> From: Yao Yiqi <yaoyiqi3@huawei.com>
>
> fbcon_rotate_font() caches a rotated copy of the console font in
> par->rotated.buf and skips the rebuild when the cache appears to be
> up to date. The cache is keyed on the source font's data pointer and
> the rotation alone, on the assumption that an unchanged fontdata
> pointer implies an unchanged font.
>
> That assumption does not hold. Raw font data (font_data_t) stores no
> glyph geometry; width, height and charcount are tracked separately in
> vc->vc_font. When a font is uploaded, fbcon_set_font() deduplicates
> it against the fonts of all consoles with font_data_is_equal(), which
> compares only the byte count, the checksum and the raw bytes. Two
> uploads of the same data that declare different dimensions therefore
> end up sharing one font_data_t, and a console can install a new
> geometry while its fontdata pointer stays the same.
>
> The early-out in fbcon_rotate_font() then skips the rebuild, and the
> rotated blitting routines index the stale buffer with cell sizes
> computed from the new geometry:
>
> src = par->rotated.buf + (scr_readw(s) & charmask) * cellsize;
>
> Rotation swaps a glyph's width and height, so the size of the rotated
> cell depends on pitch rounding and differs between geometries. A 4x16
> font whose 4096 bytes are re-uploaded as 16x8 on a CW-rotated console,
> for example, grows the rotated cell size from 8 to 16 bytes, and the
> largest character then reads past the end of the 2048-byte buffer that
> was built for the 4x16 geometry.
>
> The key has never included the geometry. Its exposure was limited,
> but not eliminated, while the deduplication in fbcon_set_font() also
> compared the consoles' font widths and total byte counts: with the
> charcount restricted to 256 or 512, differing uploads could then only
> trade height against charcount (such as 4x18x256 against 4x9x512),
> and even those can change the rotated cell size. Commit 1e3c49aa03fb
> ("lib/fonts: Compare font data for equality with font_data_is_equal()")
> dropped the width comparison, so identical bytes now share font data
> across arbitrary dimensions.
>
> Record the geometry that the buffer was built from and compare it in
> the early-out, so a geometry change always triggers a rebuild.
>
> Fixes: 6cc50e1c5b57 ("[PATCH] fbcon: Console Rotation - Add support to rotate font bitmap")
> Cc: stable@vger.kernel.org
> Signed-off-by: Yao Yiqi <yaoyiqi3@huawei.com>
> ---
> drivers/video/fbdev/core/fbcon.h | 9 ++++++---
> drivers/video/fbdev/core/fbcon_rotate.c | 9 ++++++++-
> 2 files changed, 14 insertions(+), 4 deletions(-)
I've replaced the v1 version of this series with this v3 version
in fbdev/for-next git tree for further testing.
Thanks!
Helge
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH v3 2/3] fbcon: Rebuild rotated font buffer on font change
2026-10-16 7:59 ` Thomas Zimmermann
@ 2026-10-19 8:46 ` yyaqia
0 siblings, 0 replies; 8+ messages in thread
From: yyaqia @ 2026-10-19 8:46 UTC (permalink / raw)
To: Thomas Zimmermann, deller, simona
Cc: dri-devel, linux-fbdev, linux-kernel, yaoyiqi3, stable
On 2026-10-16 09:59, Thomas Zimmermann wrote:
>
> It's way too long and boring to read. Can you please try to shorten the
> AI-generated commit description?
>
>
Sorry for that, I was a bit lazy to think commit desc by myself. I am
new to kernel and a little afraid that my message does not cover all
things.
> > - Also select the blitting type again in the error path of
> > fbcon_do_set_font(). On tile-blitting hardware, set_blitting_type()
> > uploads the font map to the text engine; without the second call, a
> > failed vc_resize() left the engine with the rejected font (found by
> > the Sashiko review).
>
> The problem is that rotate_font can fail as well. Handling that in the error
> path is not possible; hence results might be worse than keeping the rejected
> font. Can you try to keep the original rotated font and reinstall that
> directly?
>
Good advice, I will fix this in V4.
Thanks for your review,
Yao Yiqi
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH v3 1/3] fbcon: Include font geometry in the rotated font cache
2026-10-18 14:05 ` Helge Deller
@ 2026-10-19 9:12 ` yyaqia
0 siblings, 0 replies; 8+ messages in thread
From: yyaqia @ 2026-10-19 9:12 UTC (permalink / raw)
To: Helge Deller, tzimmermann, simona
Cc: dri-devel, linux-fbdev, linux-kernel, yaoyiqi3, stable
On 2026-10-18 16:05, Helge Deller wrote:
> I've replaced the v1 version of this series with this v3 version
> in fbdev/for-next git tree for further testing.
Sorry, the patch was not good enough, Zimmermann reviewed the code and I
did some modification after that. So we might need to switch to v4
version, sadly. Anyway, appreciate your continuous work for testing.
Thanks,
Yao Yiqi
^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2026-10-19 9:12 UTC | newest]
Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2026-10-16 7:30 [PATCH v3 0/3] fbcon: fix rotated font buffer handling on font Yao Yiqi
2026-10-16 7:30 ` [PATCH v3 1/3] fbcon: Include font geometry in the rotated font cache Yao Yiqi
2026-10-18 14:05 ` Helge Deller
2026-10-19 9:12 ` yyaqia
2026-10-16 7:30 ` [PATCH v3 2/3] fbcon: Rebuild rotated font buffer on font change Yao Yiqi
2026-10-16 7:59 ` Thomas Zimmermann
2026-10-19 8:46 ` yyaqia
2026-10-16 7:30 ` [PATCH v3 3/3] fbcon: Retry rotated font allocation after failure Yao Yiqi
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
all inboxes | Powered by JetHome®