Re: [PATCH v3 15/15] Introduce v18 dtb version

[David Gibson <david@gibson.dropbear.id.au>]

[-- Attachment #1: Type: text/plain, Size: 9855 bytes --]

On Mon, Oct 19, 2026 at 03:11:21PM +0200, Herve Codina wrote:
> Hi David,
> 
> On Fri, 16 Oct 2026 16:18:03 +1100
> David Gibson <david@gibson.dropbear.id.au> wrote:
> 
> > On Tue, Oct 13, 2026 at 07:03:47PM +0200, Herve Codina wrote:
> > > Hi David, Rob,
> > > 
> > > On Tue, 13 Oct 2026 18:10:39 +1100
> > > David Gibson <david@gibson.dropbear.id.au> wrote:
> > >   
> > > > On Fri, Sep 25, 2026 at 03:21:57PM +0200, Herve Codina wrote:  
> > > > > Hi David,
> > > > > 
> > > > > On Mon, 21 Sep 2026 16:20:02 +1000
> > > > > David Gibson <david@gibson.dropbear.id.au> wrote:
> > > > >     
> > > > > > On Wed, Aug 26, 2026 at 10:31:46AM +0200, Herve Codina wrote:    
> > > > > > > This v18 version will add support for
> > > > > > >   - Structured tags.
> > > > > > >     Those tags value definition will allow "old" libfdt, dtc and other
> > > > > > >     tools to skip unknown tags if encountered in future dtb version.
> > > > > > > 
> > > > > > >     libfdt, dtc and other tools implementing version v18 will be able to
> > > > > > >     skip unknown tags available in dtbd generated with later version of
> > > > > > >     dtc.
> > > > > > > 
> > > > > > >   - dt_flags header field.
> > > > > > >     For now this flag field is set to 0. It is a placeholder for future
> > > > > > >     dtb version and could be used to store some dtb related information
> > > > > > >     such as the kind of dtb. For instance, the future addons format will
> > > > > > >     use this field to clearly identify that the dtb is an addon dtb.
> > > > > > > 
> > > > > > >   - last_comp_version_w header field.
> > > > > > >     This field is similar to last_comp_version but for writing.
> > > > > > >     It contains the lowest version of the devicetree data structure with
> > > > > > >     which the version used can safely perform modifications (taking into
> > > > > > >     account following rules related to unknown tags).
> > > > > > >     If this lowest version is greater than the last known supported
> > > > > > >     version, modification are simply forbidden and lead to a
> > > > > > >     FDT_ERR_BADVERSION error.
> > > > > > > 
> > > > > > > For modification, when an unknown tag that can be skipped is involved
> > > > > > > and last_comp_version_w allows modifications, the following rules
> > > > > > > apply:      
> > > > > > 
> > > > > > "last_comp_version_w allows modifications" is a bit hard to
> > > > > > understand.  What exactly is the version number against which we're
> > > > > > comparing that value?    
> > > > > 
> > > > > We had one last_comp_version.
> > > > > 
> > > > > Now we have two last_comp_version, one for reading and one for writing.
> > > > > We compare again the dtb version supported by our libfdt version.
> > > > > 
> > > > > This comes from the discussion [1]
> > > > > 
> > > > > [1] https://lore.kernel.org/all/CAL_JsqLRrbZje_gGZPBDni6StFa+6rdiECtk49on8VfkP7CDvw@mail.gmail.com/    
> > > > 
> > > > Ok, I now see that the logic is you simply fail to proceed with writes
> > > > if the library's latest understood version is < last_comp_version_w.
> > > > 
> > > > As I said elsewhere, this approach doesn't particularly make sense to
> > > > me.  If lib_version >= last_comp_version you *can* write, even if it
> > > > means stripping everything that's not understood at lib_version.
> > > > 
> > > > So the real question here is how do we know, for any given structured
> > > > tag, what write operations require the version to be downgraded (which
> > > > might in turn require this and/or other structured tags to be stripped
> > > > entirely).  
> > > 
> > > The rules is well known.  
> > 
> > I really don't think they are.  You've proposed one set of rules (as
> > code) - although I don't think they quite make sense.  I've proposed
> > another.  Other variants are possible.
> > 
> > > An old libfdt version will strip the tag and
> > > downgrade the version. If, when we define a new tags this rules cannot be
> > > follow by olf libfdt version, last_comp_version_w will be set to protect
> > > against modification.
> > > 
> > > Maybe the problem is just downgrading the version.
> > > 
> > > Instead of downgrading it could be better if we mark the dtb "one or more
> > > 'unknown' tags have been removed due to a write by an old libfdt version" ?
> > > 
> > > We have a dt_flags header field. We can dedicate one bit for this marker.
> > > 
> > > Could it make sense ?  
> > 
> > I really dislike this idea.  It means we effectively have at least two
> > variants of every version which strikes me as pretty confusing.
> > 
> > > > This is related to (though not the same as) the question of whether
> > > > introducing a new structured tags requires a bump to the dtb version
> > > > (though not to last_comp_version, of course).
> > > > 
> > > > Here's the simplest model I can think of.  It's what I'd suggest,
> > > > unless one of it's limitations is a showstopper for what you have in mind:
> > > > 
> > > >  - version and last_comp_version retain their meanings.  No new header
> > > >    version.
> > > >  - New structured tags *do* require a new version.  If a dtb wants to
> > > >    include a specific tag, it must label itself with a version that
> > > >    defines that tag.  To enforce that:
> > > >     - For read operations, if we encounter a (known) tag that
> > > >       doesn't belong in the header version, we throw an error (either
> > > >       BADSTRUCTURE, or a new error for the purpose).
> > > >        - If version <= FDT_LAST_SUPPORTED_VERSION and we encounter an
> > > >          unknown tag that also trips the error (since if it was
> > > > 	 defined in this version, we should know about it).
> > > >     - For write operations (including in-place), we always error if
> > > >       version > FDT_LAST_SUPPORTED_VERSION (done in fdt_rw_probe_()).
> > > >     - fdt_open_into() _is_ permitted if version > FDT_LAST_SUPPORTED_VERSION.
> > > >       It will strip all unknown structured tags from the entire tree
> > > >       and downgrade the version to FDT_LAST_SUPPORTED_VERSION.
> > > > 
> 
> Ok, my proposal seems to be a no-go and so I will follow your proposal given
> above.

To be clear, I'm rejecting the original proposal because I think it's
internally inconsistent (if you follow the consequences of decisions
far enough).  I'm not necessarily opposed to something other than the
above, as long as the rules for when unknown tags must be invalidated
(and what that does to the version) are made very clear in the
structured tag spec.  e.g. see an alternative proposal below.

> Just to clarify, correct me if I am wrong.
> 
> When read if we encountered a skippable unknown tag (this is possible if
> version > FDT_LAST_SUPPORTED_VERSION), we just skip the tag without any
> error.

Correct.  Btw, I know I suggested "skippable tag" as a term, but given
how that gets kind of misleading in the case of writes, I'm coming to
prefer the orignal "structured tag" term.

> For write, "we always error if version > FDT_LAST_SUPPORTED_VERSION".
> If fdt_open_into() has been called before the write operation, this will
> not happen. Indeed, we have downgraded the version to FDT_LAST_SUPPORTED_VERSION
> and removed all unknown skippable tags.

Correct.

> Also, during this downgrade process (downgrade version + remove skippable
> unknown tags), if a unknown tag that cannot be skipped is found, the downgrade
> process fails an return an error.

Yes.  This kind of happens naturally: since non-skippable tags are no
longer structured tags in current thinking, we won't be able to
complete scanning the tree to strip tags if we encounter any
non-skippable unknown tags.

Essentially the theory of the proposal above is to be maximally
conservative: any write (adding, removing, changing, in-place,
whatever) will invalidate all unknown tags and downgrade the version.


But, here's an alternative proposal for invalidation/version semantics
that I think is consistent, while being a bit closer to your original
proposal.

1. version and last_comp_version retain their meanings.  No new header
   version.
2. If version >= 18 the tree may include any structured tags, even if
   they weren't defined as of v18.
3. Some future version might _require_ certain structured tags to be
   present.  But a client must only rely on those tags if the version
   indicates it can.
     - e.g. suppose FDT_FOO tag is introduced in v20.  A client
       reading a v19 tree _might_ see some FDT_FOO tags (because of
       (2)), but cannot rely on them being present in all the places
       they should be.
4. Any write to a property (including in place) invalidates all
   unknown tags on that property (they must be stripped)
5. Any write to a property invalidates all unknown tags on that
   property's node
6. Adding or deleting a property on a node invalidates all unknown tags
   on the node
7. Adding or deleting a subnode invalidates all unknown tags on the
   node
8. Whenever a tag is invalidated and stripped, the overall version
   must be downgraded (to let clients know they can't count on tags
   being present)

Points (4)-(7) are all somewhat negotiable.  In particular, I'm not
sure how feasible (6) is to implement.

Note that because writes don't invalidate tags on ancestor nodes
beyond one level, this proposal doesn't permit tags to contain summary
information about an entire subtree that could be invalidated by
writes further down in that tree.

-- 
David Gibson (he or they)	| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you, not the other way
				| around.
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]