Re: [PATCH 00/11] Fix PDF doc builds on major distros

[Mauro Carvalho Chehab <mchehab+huawei@kernel.org>]

Em Thu, 21 Aug 2025 09:52:21 +0200
Mauro Carvalho Chehab <mchehab+huawei@kernel.org> escreveu:

> Em Thu, 21 Aug 2025 09:09:41 +0900
> Akira Yokosawa <akiyks@gmail.com> escreveu:
> 
> > Hi,
> > 
> > Let me do a quick follow up.
> > I messed up the table.
> > 
> > On Thu, 21 Aug 2025 08:53:04 +0900, Akira Yokosawa wrote:
> > > Hi,
> > > 
> > > Commenting on your observation quoted below.
> > > 
> > > On Wed, 20 Aug 2025 18:48:10 +0200, Mauro Carvalho Chehab wrote:
> > > [...]
> > >   
> > >> If you want a more comprehensive answer:
> > >>
> > >> LaTeX is highly dependent lots of packages, including fonts. The
> > >> reason why you can't reproduce the font issues with Docker
> > >> (I wasn't able to reproduce with Docker here as well) is
> > >> probably due to either packaging differences between the
> > >> two containers, due to different environment technologies
> > >> or even due to the way Docker and LXC handles OS virtualization.
> > >>  
> > > 
> > > I'm not saying there is no difference between Docker and LXC.
> > > 
> > > Can you fill in ???? cells in the table below ?  
> > I mean                                          with this series applied
> > 
> > > Docker column is my observation of "FROM ubuntu:plucky" podman runs.
> > > 
> > >  "make SPHINXDIRS=gpu pdfdocs" under Ubuntu Plucky
> > >   
> > 
> > I meant:
> > 
> >      --------------- --------- ----------
> >      SVG --> PDF     Docker    LXC
> >      --------------- --------- ----------
> >      imagemagick     FAIL      ????
> >      inkscape        SUCCESS   ????
> >      imagemagick [*] FAIL      ????
> >      --------------- --------- ----------
> 
> This is after my series, with doesn't deal with imagemagick/inkscape,
> it only fixes broken texlive dependencies and fix font handling:
> 
> # (dpkg -l |grep -i magick; dpkg -l |grep inkscape)|grep ii
> ii  imagemagick                   8:7.1.1.43+dfsg1-1                    amd64        image manipulation programs -- binaries
> ii  imagemagick-7-common          8:7.1.1.43+dfsg1-1                    all          image manipulation programs -- infrastructure
> ii  imagemagick-7.q16             8:7.1.1.43+dfsg1-1                    amd64        image manipulation programs -- quantum depth Q16
> ii  libmagickcore-7.q16-10:amd64  8:7.1.1.43+dfsg1-1                    amd64        low-level image manipulation library -- quantum depth Q16
> ii  libmagickwand-7.q16-10:amd64  8:7.1.1.43+dfsg1-1                    amd64        image manipulation library -- quantum depth Q16
> 
> # make SPHINXDIRS=gpu pdfdocs
> 
> Summary
> =======
> gpu: gpu/pdf/gpu.pdf
> 
> All PDF files were built.
> 
> # rm -rf Documentation/output/gpu/*
> # (dpkg -l |grep -i magick; dpkg -l |grep inkscape)|grep ii
> ii  imagemagick                     8:7.1.1.43+dfsg1-1                    amd64        image manipulation programs -- binaries
> ii  imagemagick-7-common            8:7.1.1.43+dfsg1-1                    all          image manipulation programs -- infrastructure
> ii  imagemagick-7.q16               8:7.1.1.43+dfsg1-1                    amd64        image manipulation programs -- quantum depth Q16
> ii  libimage-magick-perl            8:7.1.1.43+dfsg1-1                    all          Perl interface to the ImageMagick graphics routines
> ii  libimage-magick-q16-perl        8:7.1.1.43+dfsg1-1                    amd64        Perl interface to the ImageMagick graphics routines -- Q16 version
> ii  libmagickcore-7.q16-10:amd64    8:7.1.1.43+dfsg1-1                    amd64        low-level image manipulation library -- quantum depth Q16
> ii  libmagickwand-7.q16-10:amd64    8:7.1.1.43+dfsg1-1                    amd64        image manipulation library -- quantum depth Q16
> ii  inkscape                   
> 
> # rm -rf Documentation/output/gpu/*
> # (dpkg -l |grep -i magick; dpkg -l |grep inkscape)|grep ii
> ii  inkscape                        1.2.2-8build1                         amd64        vector-based drawing program
> 
> # make SPHINXDIRS=gpu pdfdocs
> 
> Summary
> =======
> gpu: gpu/pdf/gpu.pdf
> 
> > >     --------------- --------- ----------
> > >     SVG --> PDF     Docker    LXC
> > >     --------------- --------- ----------
> > >     imagemagick     FAIL      FAIL
> > >     inkscape        SUCCESS   ????
> > >     imagemagick [*] FAIL      ????
> > >     --------------- --------- ----------
> > > 
> > > [*] after installing both inkscape and imagemagick, remove inkscape
> > >     with all its dependencies kept.
> > > 
> > > Do you see any difference between Docker and LXC columns in the table?
> > > I'm all ears.
> 
> Yes. After having texlive dependencies fixed, and addressing the broken
> conf.py file that is not aligned with modern Sphinx practices, it passed
> on all three scenarios.
> 
> Please notice that addressing image was not the intent of this series.
> 
> The goal was *just* to fix texlive dependencies and fix text font
> mapping that were causing troubles on Ubuntu and on other distros.
> 
> 
> Thanks,
> Mauro

Thanks,
Mauro

Hmm.. I ended doing the above tests with this patch on my pile.
It could be affecting the results, as it prevents kfigure.py to
crash when writing PDF output.


commit df1602df0da3a6254d58a782654e7f2e60512dc8
Author: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>
Date:   Wed Aug 20 09:17:50 2025 +0200

    docs: kfigure.py: don't crash during read/write
    
    By default, Python does a very bad job when reading/writing
    from files, as it tries to enforce that the character is < 128.
    Nothing prevents a SVG file to contain, for instance, a comment
    with an utf-8 accented copyright notice - or even an utf-8
    invalid char.
    
    While testing PDF and html builds, I recently faced one build
    that got an error at kfigure.py saying that a char was > 128,
    crashing PDF output.
    
    To avoid such issues, let's use PEP 383 subrogate escape encoding
    to prevent read/write errors on such cases.
    
    Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>

diff --git a/Documentation/sphinx/kfigure.py b/Documentation/sphinx/kfigure.py
index ad495c0da270..8ba07344a1c8 100644
--- a/Documentation/sphinx/kfigure.py
+++ b/Documentation/sphinx/kfigure.py
@@ -88,7 +88,7 @@ def mkdir(folder, mode=0o775):
         os.makedirs(folder, mode)
 
 def file2literal(fname):
-    with open(fname, "r") as src:
+    with open(fname, "r", encoding='utf8', errors='surrogateescape') as src:
         data = src.read()
         node = nodes.literal_block(data, data)
     return node
@@ -355,7 +355,7 @@ def dot2format(app, dot_fname, out_fname):
     cmd = [dot_cmd, '-T%s' % out_format, dot_fname]
     exit_code = 42
 
-    with open(out_fname, "w") as out:
+    with open(out_fname, "w", encoding='utf8', errors='surrogateescape') as out:
         exit_code = subprocess.call(cmd, stdout = out)
         if exit_code != 0:
             logger.warning(
@@ -533,7 +533,7 @@ def visit_kernel_render(self, node):
     literal_block = node[0]
 
     code      = literal_block.astext()
-    hashobj   = code.encode('utf-8') #  str(node.attributes)
+    hashobj   = code.encode('utf-8', errors='surrogateescape')) #  str(node.attributes)
     fname     = path.join('%s-%s' % (srclang, sha1(hashobj).hexdigest()))
 
     tmp_fname = path.join(
@@ -541,7 +541,7 @@ def visit_kernel_render(self, node):
 
     if not path.isfile(tmp_fname):
         mkdir(path.dirname(tmp_fname))
-        with open(tmp_fname, "w") as out:
+        with open(tmp_fname, "w", encoding='utf8', errors='surrogateescape') as out:
             out.write(code)
 
     img_node = nodes.image(node.rawsource, **node.attributes)