Define a way to provide unique identifiers for glTF data (nodes/etc)

[aaronfranke]

Continuing from the discussion here: #1051 (comment)

The glTF standard does not currently endorse any particular way to define unique identifiers. There are no UIDs, nothing beyond names is provided to identify glTF objects. This can make it tricky for applications to deterministically keep track of objects in glTF files. The problem is not isolated to game engines, it also affects glXF files.

The problem:

• You import a glTF scene into a game engine with node "MyNode".
• In-engine, you alter this node, such as by adding children, changing the materials, etc.
• In your modeling application, you rename to "OtherNode", or reparent to "Parent/MyNode", and re-export a glTF file.
• When the game engine imports this again, it will look for "MyNode" but not find it, so it will not be able to tell where to put the added children or custom materials, so they will be discarded, and they will have to be applied again.

Some options for solutions: (EDIT: Removed number 2)

1. Recommend using node names as unique identifiers, and do not add a UID property. This is in line with what glXF already does, it can reference nodes in a glTF scene by unique name.

   • Minor note: Godot already enforces unique node names, but in addition Godot also needs the path to match to be considered the same node, it doesn't follow a node's name around the tree.

"nodes": [
    {
        "name": "Block", // No other node may have this name, and it is expected to not change.
        "mesh": 0
    }
]

3. Create an extension KHR_unique_id that has one property: "uid" (and for glXF, a way to refer to these).

"nodes": [
    {
        "name": "Block",
        "mesh": 0,
        "extensions": {
            "KHR_unique_id": {
                "uid": "ab2e0958-b67d-4f28-8f6f-22e41c23a4cc"
            }
        }
    }
]

4. Add a "uid" property to the base "core glTF" spec itself. This is probably not the preferred solution given that the glTF spec is pretty much frozen, but if we did add this, it wouldn't break either forward or backward compatibility because it is optional, so it could be done in a hypothetical glTF 2.1 (no need to wait for a hypothetical glTF 3.0).

5. EDIT: Another option, currently my favorite. This is a combination of ideas 1 and 3. Basically, we have an extension that enforces the constraint that glTF files have unique identifiers. By default, use the name as the unique identifier (idea 1), but optionally we can supply a separate UID (idea 3). This allows retroactively adding UIDs to files that didn't start with one (ex: node "A", renamed to "B" with UID "A", then it can track that as the same node).

Regardless of which option is chosen:

• We should define the scope of what glTF intends to support.
  • Is pointing to a node or resource inside of a glTF something we want to support?
  • Is keeping track of unique identifiers something we want to have in glTF? Why isn't the name suitable?
  • Is it acceptable that changing a node name or path can break applications trying to reference that node?
  • Is it sensible for applications to "look for" particular parts of a glTF file which it is "tailored" to have, or does this relationship create a new problem of "My specific game/engine knows what to do with this specific model, when it contains a node that has this specific ID"? (see here)
  • Is modifying a glTF scene after import in a game engine considered an important use case? (I think yes) (see here)
• We must note that using this field is optional. This field is only to be used when the content creation program has its own UID system and therefore exporting UIDs can be done deterministically. We don't want exporters to generate random UIDs, as that would defeat the whole point of UIDs.
• We should note that UIDs are not guaranteed to be unique between different glTF files, and in fact for the UIDs to function as expected they should keep the same UIDs between different versions of the same file.
• We should ensure the glXF format is able to refer to nodes using this UID (in addition to the glXF itself being able to define UIDs for its nodes in case anything wants to reference the contents of a glXF).
• We should note that UIDs are specifically only useful when the loader already knows what to expect when loading, such as in a game engine or in a glXF file. UIDs are completely useless for dynamically loading arbitrary content at runtime in most applications which are not looking for UIDs.
• We must work toward solving all of these questions in good faith.

[apostrophedottilde]

Sound like a great idea

[slumberface]

This would be a huge leap forward for 3D workflows in Godot

[bhouston]

There is already a generic way of specifying metadata everywhere in a glTF. IT is a little cumbersome, but it does technically have this capability. It is via this already approved extension: https://github.com/KhronosGroup/glTF/blob/main/extensions/2.0/Khronos/KHR_xmp_json_ld/README.md

What you would do is create a bunch of xml packages at the top level, each one containing metadata, and then reference them from each node or material.

Would that be possible? The idea was to avoid extensions that just add metadata and unify them into this single extension.

[aaronfranke]

@bhouston This would be a single string per node, XML is vastly overkill for this.

(Also note: I have a Godot implementation of KHR_xmp_json_ld, but I can't say that I'm a fan of the spec...)

[hybridherbst]

I think this may also be relevant:
https://github.com/KhronosGroup/glTF/blob/main/specification/2.0/ObjectModel.adoc
It's more about "runtime properties" but since the goal is "identifying asset properties" there may be overlap.

[bhouston]

@hybridherbst wrote:

I think this may also be relevant:
https://github.com/KhronosGroup/glTF/blob/main/specification/2.0/ObjectModel.adoc

Unfortunately it doesn't add any new identifiers such as uuid.

@aaronfranke wrote:

@bhouston This would be a single string per node, XML is vastly overkill for this.

This doesn't involve XML, rather it uses XMP. This spec defines all metadata at the top level and then you reference it from the resource you want to use it from. This allows you to reference duplicated metadata on multiple nodes if you wanted to, but in this case you want unique metadata per node but it stills support that use case as well. This isn't that hard to use.

At the top level you would define your UUIDs:

"extensions": {
  "KHR_xmp_json_ld": {
    "packets": [
      {
        "@id": "",
      },
     {
        "@id": "",
      },
     {
        "@id": "",
      }
   ]
 }
}

And then in each node, mesh, texture or material or scene you would do:

  "meshes": [
    {
      [...rest of mesh definition...]
      "extensions": {
        "KHR_xmp_json_ld": {
          "packet": 0
        }
      }
   },
  {
     [...rest of mesh definition...]
      "extensions": {
        "KHR_xmp_json_ld": {
          "packet": 1
        }
      }
    }
 ]

[bhouston]

@aaronfranke do you understand the example usage I have outlined above using the existing extension? It isn't that much work to do. I think it would work perfectly for your use case, no? I bet you could write support for it in a hour or less.

EDIT: I think that maybe the "dc:identifier" may be a better top level name than "@id"?

https://www.dublincore.org/specifications/dublin-core/dcmi-terms/#http://purl.org/dc/elements/1.1/identifier

[reduz]

I really think a dedicated extension is probably better for this, to better encourage other 3D DCCs to implement this functionality.

[Wolve-3DTech]

It MUST be integrated into Godot and i fully support this !

[Scoppio]

A unique indentifier should be present in absolutely everything that deals with transfer and storage of data, and it has to be a first-class property, not an aftertought.
Yes, the KHR_xmp_json_ld could work, but its non-intuitive, the "packet" number is the index of where the "packet" is in an ordered list, which holds a json with arbitrary names and values, its an addon, which means that you are adding friction between the "need" and the "solution", since it is not readily available, and lastly

warning: pseudocode/python ahead.

pakcets = gltf["extensions"]["KHR_xmp_json_ld"]["packets"]

for mesh in gltf["meshes"]:
    index = mesh["extensions"]["KHR_xmp_json_ld"]["packet"]
    print(f'mesh id = {pakcets[index]["dc:identifier"]}')

I will be 100% honest here, this is not an acceptable way to access a unique identifier, what does even mean "dc:*"? This adds unnecessary clutter, the way to grab the information is noisy and convoluted. This is not something that is reasonable in any situation.

Compare with this:

for mesh in gltf["meshes"]:
    print(f'mesh id = {mesh.get("uid")}')

[RandomShaper]

I also support the idea of this becoming its own extension. It would be awkward that tool A exported uids via some raw metadata field and then tool B checked precisely those fields that happen to have a very specific meaning. It would be like having a de facto extension, only not specified.

[reduz]

Right, I mean, this is something that once defined every game engine and a lot of DCCs will be happy to implement. It is precisely why it should be standardized.

[FedTheCat]

Yes please!

[mrussogit]

Yes!

[javagl]

The first comment links to an issue that already contains some discussion, and it summarizes some of the discussion points, clarifications, and open questions from there. And judging from the high number of 👍 's (and comments, even though many of them do not go beyond the meaning of a 👍 ), there seems to be a high demand for this feature. But I think that it is really important to be clear about the scope, intended use, and behavior of such an extension. Therefore, I will ask "Devil's advocate" questions (again). I'm asking these with the goal of fleshing out what could eventually become the 'Introduction' section of an extension specification, and make sure that everybody is on the same page (and I hope that there will not be toooo many people accusing me of being stupid and/or dismissive for asking these questions...)

Referring to the high-level use-case description:

• You import a glTF scene into a game engine with node "MyNode".
• In-engine, you alter this node, such as by adding children, changing the materials, etc.
• In your modeling application, you rename to "OtherNode", or reparent to "Parent/MyNode", and re-export a glTF file.
• When the game engine imports this again, it will look for "MyNode" but not find it, so it will not be able to tell where to put the added children or custom materials, so they will be discarded, and they will have to be applied again.

This sounds like something that is mainly (or only?) relevant during the authoring phase of an asset.

(NOTE: I'm aware that glTF - even though it is primarily intended as a 'last mile' delivery format - is used in authoring workflows. And I acknowledge that unique identifiers could be useful for supporting the authoring workflow. The following is really about the scope, "management", and intended usage of these IDs).

The assumption that this is mainly relevant during the authoring seems to be confirmed by this point:

We should note that UIDs are specifically only useful when the loader already knows what to expect when loading

A model is imported into an engine. And this model is expected to contain a node ("MyNode") that is supposed to be identified with an ID. Who is responsible for establishing the connection between the IDs that have been assigned by the modeling application and the use of these IDs in the engine, on a technical level? In how far do the authoring application and the engine have to "know each other" (and the IDs that they are assigning and expecting)? Or to put it that way: The description sounds like this only refers to the case where the engine imports a model, and it should be possible for the engine to assume that ...

• this is "the same model" that it had imported previously (from the same authoring application?)
• the model was not processed in a way that did destroy or modify one, specific ID

Is that correct?

(If it is correct, then some constraints for a possible specification can be derived from that. For example, that no engine may assume the presence of IDs to begin with, and that authoring applications may never modify IDs that are already there. It could be emphasized that the extension/IDs are only useful within that "authoring cycle" of editing/importing models between authoring applications and engines that both support the extension in exactly this way)

Another important question is: What are engines allowed to do based on these IDs?

For example: An engine finds a certain mesh primitive, based on its ID. And it could assign a new material to this mesh primitive. This would mean that the appearance of that glTF asset does no longer depend on the glTF asset itself, but on the presence of a certain ID (and the specific engine that is importing it).

The follow-up question would be: Shouldn't it be possible to eventually "bake" these modifications into the asset itself, after the authoring process is 'finished'? (Meaning that at the end, one could also remove all IDs?) Otherwise, glTF may lose some of its portability. Specifically: I wonder which aspects of the current portability of glTF might be endangered by ~"modifications of the asset that are based on the presence of certain IDs"...

---

Again: I'm not opposed to introducing identifiers (for example, to support common authoring workflows). But it should be made clear which kinds of behaviors and interconnections between authoring applications and engines are expected to (or rather: "allowed to") be established with these IDs.

[theraot]

@javagl

What we want is to be able to transfer modifications made to an scene imported from a glTF to another scene imported from a newer version of the glTF.

Ideally we would not create a new scene from the new version of the glTF, instead we would reuse the existing objects, something like this:

• Fist the glTF scene is imported into the engine, this requires parsing it, and building an representation of it with engine specific types. During this process the uid would be stored associated with said representation (either as part of it, or in an auxiliary data structure).
• Second, the user would do modifications to the imported scene, such as adding engine specific materials (i.e. non-PBR materials made in engine), adding physics, composing multiple scenes (including bone attachments), and other engine specific medications. These modifications are not done to the glTF, but the objects that were created from it. Since these modifications are engine specific it does not make sense to author them in another software or to transmit them from software to software.
• Third, a new version of the glTF is imported. This will also be parsed, but instead of creating new objects we want to reuse the existing ones with their engine specific modifications. For this, the uids allow to identify nodes even if they have been renamed or reparented.

Assume the glTF is authored by a different person to who is manipulating it in engine. While this is not necessarily true, since these are different know-hows it is common that they are done by a different person.

I hope this makes sense.

[reduz]

@javagl I agree this is only useful for authoring. But reality is that GLTF is used hugely for this. It is a very fast format to export (due to its binary nature) and reimport into an engine.

Most game engines do not use GLTF natively though, they only use it for opening the assets, then use their own formats to export. This is the case of Godot and pretty much any other engine I can think of.

My feeling is that, if you are making a game engine and you want to actually ship the very same GLTFs, then its up to you to clean it up.

GLTF already provides a lot of information that may be redundant (as example, names), so its up to the engine do do this clean up process when shipping.

[javagl]

@theraot and @reduz

That description sounds reasonable, and seems to be in line with what someone (maybe even one of you, I'd have to look it up) said in the linked discussion. Roughly: The glTF itself is used only as a basis for building the "engine-specific asset". And this may even be stored in an engine-specific asset (file) format, which only refers to the glTF as its input. In this case, of course, any engine-specific additions have to know which element of the glTF they refer to.

One specific example (just to get an idea of whether I got the intention right): There might be a glTF asset with some avatar/character, and it has identified elements like 'head, torso, armL, armR, legL, legR'. The engine-specific asset uses this as the basis for building some ~"physics computation structures" (say, for some ragdoll effect, or something else that cannot be modeled with glTF animations).

In this case, the base glTF asset would not even be intended to be "portable" any more (not beyond showing that avatar in the T-pose in which it was originally authored). It might therefore be that some of my concerns are negligible in the "real world".

But I'd still wonder how to avoid "obscure" usages of these IDs. I know, this is kind of a "worst case" scenario, but people could just throw a bunch of meshes (that are not attached to nodes) or materials (that are not associated to meshes) or just empty nodes that only contain IDs (!) into an asset, and say: Yeah, my engine knows how to assemble something that makes sense from these fragments, based on their IDs.

Right now, I can drag-and-drop any GLB file into Three.js, Babylon.js, Filament, PlayCanvas, Cesium.js, the Khronos glTF Viewer, ClayGL, Hilo3d, RedCube.js, and any other glTF viewer, and they will all behave the same way, and show the same result ... mostly: Achieving portability is an important goal of glTF, ranging from the lowest level of mathematical details of the specification of PBR, up to the question of what the (visual) "ground truth" actually is (see for example https://modelviewer.dev/fidelity/ ).

We should be careful to keep these goals in mind, and in the context of a possible specification of 'unique identifiers', we should clearly describe their intended use (with the focus on the "authoring cycle") and the limits (what they should not be used for, to ensure that glTF keeps its portability).

[hybridherbst]

I think one aspect here is that unique IDs in glTF would allow for one – and only one – workflow to be better:

• Going from Application A (with its own internal data format)
• to glTF
• to Application B (with its own internal data format)

in a repeatable way while renaming/moving nodes around.

Going from Application B to Application C would have likely different unique IDs since internal data formats do not necessarily match the glTF data model (e.g. some engines have submeshes - aka multiple primitives per mesh - while others don't). In practice, importing and exporting glTF is almost never a perfect roundtrip due to these engine-specific differences. So IDs would be useful for importing and might change when exporting again unless you expect every application to somehow keep the IDs and have perfect roundtrips, ignoring their own data format.

[theraot]

@javagl

It is already possible to store arbitrary data in glTF. Take for example the suggested alternatives to this proposal: extra and KHR_xmp_json_ld. We could have an authoring tool and game engine combo that use glTFs that consist of empty nodes with extra or KHR_xmp_json_ld which would be unusable in any other software.

However, in practice, we do not see this. I believe the incentives are either non-existent or negligible. The general availability of glTF viewers would mitigate such attempts, as users expect glTF to be portable across software an platforms, and not being able to view a glTF in a general purpose glTF viewer would suggest that something is wrong with it.

Beyond that, if there is a solution to prevent misuse of extra or KHR_xmp_json_ld (of which I'm unaware), it probably applies to this proposal too. And I think it would be in everybody's interest to have a look at it.

---

@hybridherbst

The idea of a roundtrip is alluring. However it is not the goal, and would not always be possible.

While we would be interested in tools generating consisten uids for different versions of the same glTF. No tool would be required to do that, as the uids would be optional, and even generating random uids would also result in valid glTFs.

Similarly, no tool should be required to preserve uids from imported glTF when exporting them. While preserving them could be useful for some workflows, it would not always be possible (e.g. in case of importing multiple glTFs that have colliding uids and exporting them as a single glTF).

However, as game engines take advantage of uids, authoring tools that have game developers in their target audience are likely to follow. This is because game developers would rather use tools that generate consistent uids, and would request such feature.

---

In the topic of adoption by tools... As far as I know extra and KHR_xmp_json_ld lack semantics. Software could follow the robustness principle and look for an uid there in case some other software outputs it that way, but we would still want a recommendation of how to output uids. Without this, it would be hard to get the maintainers of existing engines and authoring tools to use uids… And if we managed to get them to use uids over extra or KHR_xmp_json_ld, it would be akin to a secret society handshake.

[javagl]

In practice, importing and exporting glTF is almost never a perfect roundtrip due to these engine-specific differences. So IDs would be useful for importing and might change when exporting again unless you expect every application to somehow keep the IDs and have perfect roundtrips, ignoring their own data format.

The term "roundtrip" also came up in the earlier discussion (and it was said that this was not the primary goal). But in view of the alternatives that have been mentioned here (name, extras, and KHR_xmp_json_ld), one question for a specification of an extension could be:

In how far would these IDs go beyond what an authoring application and an engine could achieve with a bilateral agreement?

The application and the engine could just agree to store the ID, as a string, in the name property. And if the IDs should go beyond that, then I think that "going beyond that" does exactly mean that there are constraints, on the level of the specification, clearly stating the expected behavior for generating and consuming these IDs, in different workflows.

For example, the specification could require a roundtrip capability for the most simple case (that does not involve editing), by stating: "Importing (then not modifiying) and exporting an asset MUST keep the original IDs". Of course, there are many cases to consider: What if multiple assets with conflicting IDs are merged, and the result is exported again? All this could go down into very nitty-gritty details, e.g.: What if an authoring application just swaps two nodes? Do they keep their IDs (because they are still the same nodes), or do they have to receive new IDs (because they have new parents)?

I think that some/many of these constraints could only sensibly refer to authoring applications in particular. For example, one could expect that Blender builds sophisticated (authoring-oriented) structures that allow to keep track of all IDs that have been read from the input. In constrast to that, a glTF loader library in a game engine might have methods to read glTF into the engine-specific 'model' object, or write such a 'model' as a glTF, but it might perform operations (e.g. optimzations, like dropping "unnecessary" data, like the IDs themself, unused materials, or nodes in node chains) that make it impossible to reconstruct the original IDs.

[aaronfranke]

@bhouston I understand perfectly what you mean, but I completely disagree. It's not a helpful layer of abstraction. Using KHR_xmp_json_ld will just increase complexity and file size for absolutely zero benefit. The whole point of putting data in top-level arrays is sharing data, but UIDs are unique, so they can never be shared. What specifically is the problem you are trying to solve here (see this chart)? The only argument in favor of using KHR_xmp_json_ld is a misguided belief that we need to unify all data under it in one format. But we already have a unified data format, it's called JSON, we can store metadata without extensions in "extras".

The follow-up question would be: Shouldn't it be possible to eventually "bake" these modifications into the asset itself, after the authoring process is 'finished'?

This is not always possible. For example, setting a material may use an engine-specific material type that has no way to be saved to glTF. Or, you may attach a script written in that game engine's programming language using that game engine's API, which can never be fully portable to glTF.

Roughly: The glTF itself is used only as a basis for building the "engine-specific asset". And this may even be stored in an engine-specific asset (file) format, which only refers to the glTF as its input. In this case, of course, any engine-specific additions have to know which element of the glTF they refer to.

Yes, this is precisely the idea. Well, ideally, the glTF is a very large part of the asset, like 90% or more. This way when you need to have some data specified in your engine-specific format, you can have 90% or more of your asset be a portable glTF, instead of 0% (with all data being stored in the engine-specific format).

In this case, the base glTF asset would not even be intended to be "portable" any more

Not necessarily. The base asset could be portable, but it would just be missing whatever functionality you added in-engine. For example, when making an avatar for VRChat, you start with the base model with a mesh, skeleton, materials, etc, and may replace materials, add new functionality (like spring bones for dynamic hair/tails/ears/etc), etc. The base model is still fairly portable, it can still be moved to other apps, used with its skeleton etc, but it will just be missing the final last-stage tweaks (like the hair will be static relative to the head).

Of course ideally we should continue building standards to allow specifying more and more of the data in the glTF file itself (for example, the VRM consortium has a glTF extension for spring bones), but there will always be application-specific needs that go beyond what's standardized.

[theraot]

@javagl

The application and the engine could just agree to store the ID, as a string, in the name property.

To quote JonathanDotCel's #1051 (comment):

names are for people and GUIDSs are for machines.

Since the engine builds and shows a very close representation of the glTF structure, we have the expectation that the name carries out. If it is the name what is held unique and unchanging, we want a display name property, so we would be talking about adding a new property any way.

---

What if an authoring application just swaps two nodes? Do they keep their IDs (because they are still the same nodes), or do they have to receive new IDs (because they have new parents)?

If they have uids, I'd expect them to keep their uids. If they were to receive new uids because being reparented that would defeat the purpose.

---

Yes, there would be workflows. The following is what comes to mind:

Ignoring the uids on import. And not outputting uids, would be equivalent to what already exist.

Authoring tools could also create new uids for new nodes, keep track of all of them, and persist them in their own authoring format, so they can be consistent across multiple exports to glTF.

The authoring tools could also preserve uids of imported glTF. And here we run into possible conflicts (see below).

Scrapping the uids on export is also OK. For example, when exporting the final game, as stated.

However, game engines could also keep track of uids to be able to match them among multiple versions of the same glTF.

---

The following are my ideas of what to do with conflicts:

Under the premise that exported uids are not required to match the imported uids, the authoring tool can replace conflicting uids with new ones. Doing this without losing track of the original uids suggest that the new uids should be derived from the original uids plus some reference to the source glTF. Two strategies comes to mind:

• If arbitrary strings are allowed, the uids could be prefixed with an id of the original glTF.
• Otherwise, the new uids could be derived using a hash based algorithm.

In eiher case, it seems that some id for glTFs as a whole would be useful to solve conflicts. I'm inclined to believe that authoring software could use (a hash of) the (relative) file path for this purpose. I'm suggesting a hash here as to not embed potentially confidential paths, and I'm suggesting relative paths to allow moving the files of the project as a whole without breaking these ids.

[javagl]

Shouldn't it be possible to eventually "bake" these modifications into the asset itself, after the authoring process is 'finished'?

This is not always possible. For example, setting a material may use an engine-specific material type that has no way to be saved to glTF.

One could consider to explicitly recommend to not use the IDs for a purpose that can be achieved with pure glTF. For example, they could be used to assign "physical material properties" to meshes, but should probably not be used to model something like parent-child relations between nodes (or anything else that can already be represented as pure glTF).

It could be hard to phrase that precisely. It can hardly be a strict requirement, because the features of glTF will be extended with ... other extensions. But it might be something on the level of a hint about the scope of the extension, like a "best practice", or an "Implementation Note"...

names are for people and GUIDSs are for machines.

Since the engine builds and shows a very close representation of the glTF structure, we have the expectation that the name carries out.

The comment about using the name was to emphasize that an extension specification will raise tricky questions (and some of them have already been mentioned in the meantime). And it will be necessary to have a clear idea about the behavior of the IDs in these cases, considering that this is supposed to not only be a mutual agreement between two parties. It has to stand the test of time, across many applications, and throughout different workflows.

What if an authoring application just swaps two nodes? Do they keep their IDs (because they are still the same nodes), or do they have to receive new IDs (because they have new parents)?

If they have uids, I'd expect them to keep their uids. If they were to receive new uids because being reparented that would defeat the purpose.

Yes, it would defeat the specific purpose of 'identifying a node regardless of its parent'.

I could now ask further (overly specific) questions: Will a node receive a new ID when a child node is attached? Will it receive a new ID when a mesh is assigned to or removed from it? Will a mesh primitive receive a new ID when its material is changed? But the obvious generalization of these questions is:

Which editing operations in an authoring application MAY/MUST cause the IDs to change, and which ones MAY/MUST NOT affect the ID?

(More technically, this could be seen as a question about the concept of 'equality'. Or more philosophically, as an instance of the thought experiment of the 'Ship Of Theseus'...)

For each answer, one could come up with scenarios. For example: If the ID was intended to identify a node that is expected to contain a mesh (let's say a node with a mesh for which physics computations should be performed), then removing the mesh will make the node "invalid" for that purpose. Iff something like this was an intended use case, then I'd be curious about the behavior that is expected from an authoring application and an engine in such a case.

(Note: These questions are not meant to dismiss the idea. And the answers to these questions may very well be "This is not relevant", or start with the usual "That depends...". They are intended to get a clearer idea about what could (reasonably) be specified for such an extension, beyond the JSON schema that says that there is some extension object with a uid: string property).

[aaronfranke]

Will a node receive a new ID when a child node is attached?

No, that would defeat the point of UIDs.

Will it receive a new ID when a mesh is assigned to or removed from it?

No. In editors where this is possible, the UID should not change because it's the same node. But also, note that in many apps like in Blender or Godot, creating a new mesh requires creating a new node.

For example: If the ID was intended to identify a node that is expected to contain a mesh (let's say a node with a mesh for which physics computations should be performed), then removing the mesh will make the node "invalid" for that purpose.

Yes, but if you are looking for a mesh and that mesh is gone, there is no possible configuration that would allow mesh modifications to be re-applied. So this is expected. Also, this scenario won't occur with Blender or Godot, because the only way to add/remove meshes to nodes is to add/remove the nodes and have them be of type mesh.

Will a mesh primitive receive a new ID when its material is changed?

No, that would defeat the point of UIDs.

Which editing operations in an authoring application MAY/MUST cause the IDs to change, and which ones MAY/MUST NOT affect the ID?

Creating a node MAY give it a UID. Or, if there's an existing file without UIDs, one may wish to add them afterwards. Once a node has a UID, it MAY be stripped for the "final product", but it MUST NOT be changed or replaced with any other editing operation. There are no valid editing operations that will result in the UID property automatically changing from one UID to another UID. If a node has a UID, it must never have any other UID.

[hybridherbst]

Just for clarification,

There are no valid editing operations that will result in the UID property automatically changing from one UID to another UID. If a node has a UID, it must never have any other UID.

You do mean: in the same file in the same application, right? "Exporting the GLB and importing it again in the same software" may already result in new UIDs due to internal format differences or collisions (things that aren't nodes are turned into nodes on export and vice versa). "Exporting the GLB and importing it elsewhere and exporting it again" may also result in new UIDs.

[javagl]

Once a node has a UID, it MAY be stripped for the "final product", but it MUST NOT be changed or replaced with any other editing operation.

That's the core of the answer that my questions aimed at. And is the strictest requirement that can be imposed here (and that could go into the specification accordingly).

But I'm s stickler, and will ask a few more questions. I think that this is important for the long(er) term goal of a robust technical specification. Some of these questions might seem to be hypothetical (for example, because they are not applicable for one specific authoring tool - even though they may be applicable to another one). If someone thinks that this is not important, then these questions can be ignored.

---

There seems to be a level where we could talk about the vocabulary:

...the only way to add/remove meshes to nodes is to add/remove the nodes and have them be of type mesh.

A "node of type 'mesh'" is something that doesn't translate well to glTF. When I'm talking about a 'node', then this refers to a glTF node, as something that determines the hierarchical structure of the scene. This might be mapped to the concepts of authoring applications in different ways.

Regarding the question of 'removing a mesh from a node':

... removing the mesh will make the node "invalid"

Yes, but if you are looking for a mesh and that mesh is gone, there is no possible configuration that would allow mesh modifications to be re-applied.

One caveat here is that a 'mesh' in glTF cannot sensibly receive an ID. Of course, the mesh object itself can receive an ID, but meshes in glTF are basically instantiated when a node refers to a mesh. This raises some questions. Imagine the ID is supposed to be used to identify a mesh for which 'physics' computations should be performed (like building some collision detection data structure or whatnot). The mesh could be identified via its ID. But the authoring application could remove this mesh from a node, and assign it to a different node, or even to multiple different nodes. Should the physics data structures now be created for each instance of that mesh?

The question about 'removing a mesh from a node' could therefore be phrased in a more abstract and generic way: Who is responsible for ensuring that structural modifications (that keep existing IDs) do not interfere with the purpose of the identified element?

To emphasize this again: There might be 'simple' answers to that, like "This is not relevant for the specification", or "It's the responsibility of the person who edits the model and who has to know which modifications are allowed". And that's perfectly fine. On this level, one could say that I just want to know whether there is such a 'simple' answer or not...)

(And an aside: If someone is supposed to actually implement the handling of glTF IDs in an authoring application, there will be some questions on a far lower technical level. For example: When you delete a node and press CTRL-Z (undo) - will it be guaranteed to receive the same ID again? It should be, right? What about the squence CTRL-C-X-V-V (copy, cut, paste, paste) - what will be the IDs of the pasted elements?)

[aaronfranke]

A "node of type 'mesh'" is something that doesn't translate well to glTF.

Sure, I'm just mentioning this to argue that the case you mention will not occur in Godot and Blender. But anyway, even in applications where it can occur, it's not a problem, changing the contents of a node should still keep the UID.

One caveat here is that a 'mesh' in glTF cannot sensibly receive an ID. Of course, the mesh object itself can receive an ID, but meshes in glTF are basically instantiated when a node refers to a mesh. ... assign it to a different node, or even to multiple different nodes. ...

In Godot, this is not a problem. Mesh resources are stored in memory. If multiple nodes use the same mesh, then by default they share the same mesh resource. So a single mesh instanced multiple times is still one mesh with one object in memory and one UID (if it has a UID). I suppose this may not be the case in all apps, but most engines have the concept of instancing a mesh. What gives you the impression that glTF meshes "cannot sensibly receive" a UID?

Also, the case you mention about physics - I get what you're saying in a hypothetical sense, but in this particular case it's not a valid example, because glTF physics does not work like that (in both of the competing extensions, a physics shape may use a mesh, but they do not add any data to the mesh resource itself).

[javagl]

The point about the uniqueness and instancing of meshes was probably not stated properly. I'll try to be more specific. This attempt to be more specific bears the risk of being too specific, causing the response: "That's now how it is done in engine X". But the behavior and handling of IDs should be consistent across multiple applications, as far as reasonably possible (!), and within the intended use-cases. For an application-independent format, one should be able to either say 1. what the behavior should be, or 2. explicitly (!) say that a certain aspect of the behavior is not specified. (And again: that's fine. I'm trying to find 'the limits of what can be specified' here...).

A glTF file may contain a certain mesh. And this mesh has an ID. The engine knows that it should create, say, collision detection information for this mesh, based on its ID. For example, if the mesh is attached to a node with a translation of (1,2,3), then the engine may create its collision detection information (some BVH or spatial hash) specifically for the mesh at the position (1,2,3). When the mesh is attached to a different node with a different translation (in the authoring application), then the engine will build the collision detection data structure for a different location. That's fine, And one of the explicitly intended use-cases, as far as I understood.
Now, when the same mesh (or rather the identical mesh, as of the meaning of 'ID') is attached to 10 nodes, then the engine would create this collision data 10 times.
The point is: Is the collision detection info (or whatever should be associated with the ID) specific for the mesh instance (as it is created via the mesh: 123 reference in a node), or is it specific to the mesh itself (regardless of the nodes that it is instantiated in)?
(Corollary: If a mesh should appear 2 times, but only one instance should receive collision detection information, then the ID in the mesh can not be used as a basis for the collision detection data. It has to be based on an ID in the node that refers to (i.e. "instantiates") the mesh)

[reduz]

@javagl I think you are making it more complex than it need to be.

For unique IDs, to me everything should potentially be able to have it. Both instance and mesh, as well as material, texture, animation, etc.

[fire]

The issue we're facing with glTF scenes in game engines is due to the lack of unique node identifiers in the glTF file. Changes made within the engine are tied to the node's name, but if this name or its parentage changes in the modeling application and the glTF file is re-exported, the engine loses its connection with the old-named node, resulting in the loss of any modifications made within the engine.

A proposed gltf extension by donmccordy would be to simply enforce unique names for all resources.

Advantage is that this 1. avoids uids 2. avoids xmp 3. avoids making downstream applications change their design.

Now you have a primary identifier of the unique name and the secondary identifier of the animation pointer path. You can use both identifiers to handle collisions.

[reduz]

@donmccurdy @weegeekps Seriously, this take of wanting to add unique IDs in a generic container extension is really confusing to me. Why are we making a standard otherwise?

IMO these things should be used only when the exporter and importer want to send custom information, not when you want to push for something standard.

Unique IDs is a feature with a lot of demand that we expect will be implemented in game engines and DCCs all over the place. Trying to "push it to generic container" IMO is not the right way to go on this, given the demand.

Is this not what extensions are for, agree on a way we all want things and have a centralized place (this repository) where you can find them?

[vpenades]

As I understand it, a first level KHR extension needs to be carefully reviewed and tested against many use cases beyond what Unity or Godot may require, in order to ensure the extension is consistent and it is free of loop holes and design flaws.

Otherwise, the solution is simple, like other third parties do, it's possible to propose a GODOT_dcc_uid that does exactly what's required by the engine without ambiguities. And I am sure if it's published in the main repo, other third parties will have no problem adopting it. After all, calling it GODOT_dcc_uid or KHR_dcc_uid is just semantics.

[reduz]

@fire

Advantage is that this 1. avoids uids 2. avoids xmp 3. avoids making downstream applications change their design.

Major disadvantage is, if user renames asset int he DCC, then you expect it to be renamed in the GLTF too, hence point of extension is moot. UID is the only real solution to this.

@vpenades

As I understand it, a first level KHR extension needs to be carefully reviewed and tested against many use cases beyond what Unity or Godot may require, in order to ensure the extension is consistent and it is free of loop holes and design flaws.

To be honest discussing or speculating around this further is what should be justified, not the other way around. The use case is 100% clear and obvious here and the extension is extremely simple:

• Use case: Agree on a way where DCCs can identify objects in a persistent way between exports, so game engines that consume them can keep track of added metadata to them.
• Proposed solution: Just add the optional ability to add an UID to any type of data in the GLTF file. The UID is just unique in the scope of the file, that's it.

So:

• I think we agree that the use case here is clear as water, there is no room for misinterpretation.
• We know we need a standard extension to do this, not a GODOT extension.
• There is literally only one possible way to implement this extension. No loopholes possible.

IMO, then as a standard body, the steps that I expect should be taken here should be quite clear too:

• Draft an extension (we can provide a draft on our side if this saves work).
• Request for feedback.
• Implement changes if they arise.
• Seek consensus among those interested in using this extension.
• Make it official.

If there is really something further that needs to be left to the realm of speculation, IMO at this stage could be done in the request for feedback.

But saying that more should we discussed because there could be more potential use cases IMO is not the right way a standards body should work. If there are more use cases, then they should be a separate extension. There is no reason to try to cram and discuss everything that potentially and speculatively happen under the sun in a single extension.

This is a standards body and IMO the purpose of creating extensions should be by focusing only on concrete, explicit use cases with real world demand, not speculation.

[vpenades]

@reduz by "other use cases" I don't mean uses cases other than DCC=>Engine

I mean things like this:

Let's say the UID in the extension is a STRING, so anything goes as long as it's unique.

Then a given DCC exports UIDs that are INTEGERS.
But then, the importing engine requires GUIDs, so it will fail to parse them.

So how do you address that?

I maintain a tool that allows transforming gltfs, merging them, etc. One of the operations the tool can do is to apply an Axis switch, which is typically done by adding a root node with a 90 degree rotation and moving everything else inside that node. Let's say someone processes a model that has UIDs.... the resulting model will have UIDs in the pre-existing nodes, but NOT in the root node. How tools like mine should handle that?

Then there's the question of asset merging and collision UIDs, how do you handle them?

All these questions need to be answered in the specification so all tools and pipelines behave in a consistent way

We know we need a standard extension to do this, not a GODOT extension.

Then you have to be sensitive to other parties requirements and concerns.

[reduz]

@vpenades

Then a given DCC exports UIDs that are INTEGERS.
But then, the importing engine requires GUIDs, so it will fail to parse them.
So how do you address that?

Game engines do not keep track of this currently, so IMO this is what I mean with speculation, but if they did they do and this was a problem, they either add a workaround or they can simply SHA/MD5 the string.

I maintain a tool that allows transforming gltfs, merging them, etc.
...
How tools like mine should handle that?

That's up to you. The use case here is basically for the DCC -> Importer workflow which is a very common use case. Yours is not, you can give options to users on what to do.

Then there's the question of asset merging and collision UIDs, how do you handle them?

Again, all this is speculation to me and I don't think a standards body should act upon speculation. It should act upon concrete use cases. In this case the concrete use case is tracking your data from the DCC. Asset merging has nothing to do with it and can't be considered a concrete use case.

That said, given those are text, you can simply append the filename to it and be it; like "uinqueid-file1.glb" or give the choice to users if they want a re-hashed UID between both. IMO this is dependent on the situation and what you are doing, so it should be on the merging side.

Another solution would be enforcing universal unique IDs (uuids), but IMO this would make the extension far more complex because then you would have to determine in the extension how these IDs need to be generated (precise algorithm). This would make the extension far more difficult to implement and process, and for use cases that are entirely speculative.

Then you have to be sensitive to other parties requirements and concerns

Why? As I said if other parties have a need, they can propose their own extension and justify why they need this, not try to cram their needs on something unrelated to it. Doing speculation is free, anyone can do it, hence these opinions should not be taken into account.

The only reality that matters is when you have an actual, real world need that needs a solution. This should be acted upon. Speculation means solutions in search of a problem, should not be acted upon.

[reduz]

@vpenades Actually upon further thought, I wonder if an extension could suggest the creation of UUIDs to implementers with any library of their choice, but not require it.

[Psychpsyo]

On the question of "What operations should and shouldn't affect UIDs?":
The main point of the UID is that when something re-imports a newer version of a known file, it has a straightforward way of determining which of the things in there correspond to what it already knows about from last time.
I think editors and converters should be able to determine what is 'the same thing' in their newly generated files in any way they like. ('the same thing' shouldn't imply identical or even comparable, just "this thing is the new, potentially different version of that thing from last time")
Also, any program that wants to read the UIDs should make no assumptions about which parts of the file have UIDs or what UIDs there are.

[javagl]

I'll try to limit my nitpicking questions here to a minimum. It might very well be that there are "obvious" answers to some of the questions, and I'm just not aware of them, because I don't know the technical details of the interaction between authoring applications and engines for specific workflows. But...

I think editors and converters should be able to determine what is 'the same thing' in their newly generated files in any way they like.

... this sounds like much of the responsibility is moved to the person who edits the asset, and has to be aware of the concept of IDs, how they are assigned, and how they are used. (The latter may be very specific for the authoring application, the engine, and maybe even the model). I could imagine that someone picks out one node from a hierarchy, attaches it to a different parent node, adds new child nodes, assigns a different mesh to this node, and maybe makes this node part of a skeleton for vertex skinning or adds an animation that modifies the transform of this node... the meaning, context, and purpose of that node will then have changed completely, but the IDentity of that node may remain the same. However, if this is not expected to cause trouble, then that's fine...

[reduz]

@javagl By the time this enters the game engine, it has the file context added, so if things are combined inside the game engine this is not a problem.

To my make position clearer, if you workflow is this:

[DCC] -> [Game Engine]

Which I am confident will cover well over 99% of cases. You are good.

If you do this:

[DCC] -> [Some processing] -> [Game Engine]

Then it is up to whatever the processing is doing. If, as mentioned above, you are somehow mixing GLTF files, then I can see this benefiting from UUID instead of UID (though I am pretty sure this can be worked around by just appending the file name to the UID, but whathever).

If you do this:

[DCC] -> [Some processing] -> [Something else]

Then you will most likely not care about UIDs anyway, since this is a feature geared for game engines.

This is why I think, if you want to cover 100% all bases, the extension may have a recommendation on using UUID for the UIDs, but definitely should not be incorporated to the extension due to the high complexity on generating those.

[Psychpsyo]

... this sounds like much of the responsibility is moved to the person who edits the asset, and has to be aware of the concept of IDs, how they are assigned, and how they are used. (The latter may be very specific for the authoring application, the engine, and maybe even the model). I could imagine that someone picks out one node from a hierarchy, attaches it to a different parent node, adds new child nodes, assigns a different mesh to this node, and maybe makes this node part of a skeleton for vertex skinning or adds an animation that modifies the transform of this node... the meaning, context, and purpose of that node will then have changed completely, but the IDentity of that node may remain the same. However, if this is not expected to cause trouble, then that's fine...

That is a fair point but I think the concept of a node still being the same node even if you do a lot of changes to it is a fairly intuitive one. (Someone editing the asset wouldn't need to know that this is a UID under the hood, just that a thing generally retains its identity even if they modify it)
I also think that nodes getting completely repurposed isn't a very common thing. It seems much more straightforward in most cases to just delete the old thing and to make a new one instead of ship-of-theseus-ing one thing into another thing that's intended to be completely different anyways.
(Though it might be good to get input from more people on this since there are a lot of programs and workflows out there so maybe this is common in some of them.)

[donmccurdy]

I maintain a tool that allows transforming gltfs, merging them, etc.
...
How tools like mine should handle that?

That's up to you. The use case here is basically for the DCC -> Importer workflow which is a very common use case. Yours is not, you can give options to users on what to do.

I see @vpenades's example as a very important workflow, as well. But I'm pretty confident that whatever approach we decide on will be OK in that regard, and that details can be worked out off-thread.

Example / proposal

Processing tools adding or merging nodes SHOULD aim to generate a UID that is deterministic, and reproducible given the same input. This might mean an +Zup→+Yup transform node added to the scene root is always given UID "<toolname>_scene_up". Or it might mean nodes injected to reorient a light or camera (as Blender does) are always given UID "<child_uid>_orientation".

Not a strict requirement, and obviously needs a little more detail, but I view the risk that we cannot solve this as "low". I expect that a similar approach would be fine for glTF Transform, gltfpack, and other optimization pipeline tools.

---

Advantage is that this 1. avoids uids 2. avoids xmp 3. avoids making downstream applications change their design.

Major disadvantage is, if user renames asset int he DCC, then you expect it to be renamed in the GLTF too, hence point of extension is moot. UID is the only real solution to this.

Unfortunately, I'm not aware that Blender has any unique ID associated with an object, other than the name... Do other DCC tools? Do exports to FBX or USD avoid this issue today? If there were unique IDs already sitting around in DCC tools that glTF simply can't use, that would be really useful to know. 🙂

To be clear — I'm broadly in favor of solving the issue raised here, and OK with the possibility that it requires a new extension. But let's not treat questions that aren't central to [DCC] -> [Game Engine] as speculation or unimportant. There is overwhelming representation of one game engine in this thread, and we will need positive signal from >1 implementor before something becomes a Khronos-ratified standard. Particularly if it's an area in which existing DCC tools and file formats haven't paved a clear path.

---

Putting my three.js hat on for a moment ... unique, human-readable names are really nice. I wish glTF names were unique per resource type, as Blender's are. That would solve some issues for three.js users. Unique integers or hash strings are, well, still useful, but I'm going to have to think about how/if we can change our implementation to use a new "type" of ID in addition to names and our existing three.js UUIDs. I'm not saying this in the interest of blocking the proposal here, but to indicate that I can't (yet!) give positive signal as a second implementor.

[aaronfranke]

Putting my three.js hat on for a moment ... unique, human-readable names are really nice. I wish glTF names were unique per resource type, as Blender's are.

We could have an extension that does nothing except enforce the constraint that node names must be unique (see also #2329 which is an extension that only enforces a constraint). Implementations could then look for this and use it as a hint that the names should be treated as unique IDs. This would solve the case of paths changing such as when nodes are moved or parents are renamed, and has the advantage of no new fields.

However, this does not fully solve the problem, because some operations (renaming a modified node, or deleting a node and creating another with the same name) will have different behavior compared to UIDs.

[aaronfranke]

Hmm, another idea: What if an extension does both of these things at once? Meaning:

• If a file has KHR_uid or whatever we want to call it:
  • If a node has "extensions": { "KHR_uid": { "uid": "abc123whatever" } } then that is the UID.
  • If a node does not have KHR_uid, but does have a name, the name should be treated as a unique identifier.
  • This way we can ensure that all resources in a file with KHR_uid can be identified uniquely, without adding unnecessary data, and future files can retroactively add UIDs to handle name changes.
    • For example, if a node is named "A", and is renamed to "B", the "B" node could have a UID of "A" that says what was previously named "A" is now named "B".

[weegeekps]

It would be best to choose one or the other approach. Either/or and specifications doesn't generally mix well and only adds to developer confusion.

Between the two ideas the idea to have an extension where it's inclusion guarantees that name properties will be unique is still a better option than a separate property for specifying another identifier. I would not be surprised if there are others who are already treating name properties as unique.

However, this does not fully solve the problem, because some operations (renaming a modified node, or deleting a node and creating another with the same name) will have different behavior compared to UIDs.

It's worth mentioning that a uid field will likely have many of the same problems. It also creates some additional edge cases such as what happens if an artist duplicates and then accidentally deletes the non-original node. In that case the UID would be different and the in-engine links would be broken, but the artist may believe they deleted the duplicate.

It's likely that many of the DCCs would keep UID as hidden as they do not today have mechanisms for display of these. Even if they could show them, most users can't tell the difference between two UUIDs quickly and speaking anecdotally for a moment, most artists will complain vehemently about having to deal with them. (I've worked in systems before where UUIDs were the main identifier.)

UI constraints could also be an issue from the DCC standpoint, which is why we really need their representation here. I can definitely see cases where DCCs may refuse to implement this extension because it falls outside of the norm of their UX workflows. We've dealt with similar cases in the past for other metadata extensions.

The UX aspect is actually another reason why I'm really a fan of enforcing uniqueness. It leaves it up to the artist to determine the name, as long as it's unique. Typically they will chose one that they can easily identify and that may be a beneficial matter. If it's made clear that the engine is using the name to link custom materials and other in-engine components to a particular node, in my experiences most artists will take note of that. This downside is outweighed by the fact that now they can also easily identify a node X as having custom Y in the engine while they're working on modifications in Blender. Many of the DCCs also essentially already enforce uniqueness themselves so this may be really easily implemented.

[reduz]

@weegeekps @aaronfranke

Between the two ideas the idea to have an extension where it's inclusion guarantees that name properties will be unique is still a better option than a separate property for specifying another identifier. I would not be surprised if there are others who are already treating name properties as unique.

I think that's absolutely not the way to go. As I said before, I think this is a terrible idea and it should not even be further discussed:

• You still need proper names to identify objects in the game engine upon import.
• Unique names in the GLTF file still does not protect you from an artist renaming an object in the DCC, which is the main point of this proposal.
• If the DCC allows two objects to be called the same, then the artists duplicates and has 5 objects called the same, what is to ensure the GLTF exporter will do a consistent name resolution between those five?

IMO unique ID as an extra property is the only way to do this properly.

[weegeekps]

Unique names in the GLTF file still does not protect you from an artist renaming an object in the DCC, which is the main point of this proposal.

This is the biggest uphill battle I see with this proposal then. We need direct input from the DCCs at this point because my gut is telling me that most will not have interest in implementing this. Internally, most work with their own formats and do not keep track of details between glTF import/exports, and the proposal as it currently stands would require them to start doing so. I suspect this will be an onerous ask.

[reduz]

@weegeekps You don't really need the DCCs to implement this. AFAIK all the popular DCCs can store metadata in the edited objects (Blender, Max and Maya can). Additionally, you know as well as I do that Autodesk will never support GLTF2 officially. All you need is the GLTF2 exporter plugin authors to implement this.

[vpenades]

My final thoughts;

The title of this issue is misleading because it proposes a mechanism to tag glTF elements with a general purpose UID.

A general purpose UID extension requires discussing issues like equality, scope, collision resolution and affected use cases, which are well known issues of unique identifiers. I don't see interest discussing these here.

So I agree this issue must be restricted to DCC=>Engine domain, which means any extension branding must reflect that. I dare to suggest: KHR_dcc_export_uid

This proposal asks to resolve a problem, which is to add a mechanism to identify parts of a glTF model when exported from a DCC and imported into an engine, so the engine can apply modifications to said parts.

Now, whatever solution it is agreed here, it must go through the process of developers of DCC export plugins adopting that solution.

That solution, would look like this in the export dialog

Export GLTF
☑ Ensure all names exist and are unique

So if the current scene fails to meet these requirements, it can show an error and tell the artist to fix the problem before trying again.

At this point any model exported with these plugins will be garanteed to have unique names.

Then, as it's been stated here many times, the exported glTF goes straight away into the game engine, this will ensure all names are unique, solving the problem without any extension at all.

And a very important detail here: let's say an Extension is added for this, the modifications to the plugins would need to be done anyway, the only difference is that the plugins would also include the extension in the file.

In other words, such an extension would only serve as a seal of approval

But the artist can change the name!

Yes, and as @weegeekps and others have stated, the artist can do many other operations that would break the model anyway, like cloning what's on scene and deleting the original. Renaming a node is just another way in which the artist can break the model.

Furthermore. if the artist breaks the model by renaming something in the scene, names have an advantage over internal UIDs you can rename the node back to it's original name, which would fix the problem.

With UIDs, if the DCC chooses to recycle/refresh them, or the artist performs a destructive UID operation for whatever reason, you're sold , there's no way to fix the UIDs back.

So I dare say that names are a superior solution over internal unique UIDs

My feelings? an uid system is not going to fix a flawed workflow (I'm looking at you, Unity), no matter how common it is nowadays.

End of line.

[reduz]

@vpenades

Yes, and as @weegeekps and others have stated, the artist can do many other operations that would break the model anyway, like cloning what's on scene and deleting the original. Renaming a node is just another way in which the artist can break the model.

If you are not familiar with this problem or this problem does not affect you, then I would very much appreciate you simply mind your own and don´t try to gaslight other people that their problems are not real.

This is very much a real problem and it needs a real solution. We have plenty of users complaining about this. Suggesting It's not a real problem is extremely unpolite from your part.

[vpenades]

This is very much a real problem and it needs a real solution. We have plenty of users complaining about this. Suggesting It's not a real problem is extremely unpolite from your part.

I never said that, I know It's a VERY important problem. I've been developing games, and even 3dsmax export plugins, since almost 30 years now, and I know the problems of exporting assets from DCCs to Engines maybe too well.

That's why I offered alternatives to the problem that may probe useful.

[reduz]

@vpenades

That's why I offered alternatives to the problem that may probe useful.

Thanks for taking the effort to do this, but I insist the real root of the problem is that the following to me are facts:

• The object name can´t be trusted.
• Enforcing unique names does not solve this problem. I explained why.
• All major 3D DCC has systems to store persistent metadata in objects, hence you don´t necessarily need consensus from the DCC to implement this (and effectively Autodesk does not even support GLTF2 anyway),
• So the only consensus needed should be from GLTF exporter plugin authors, both importers and exporters.

If you have this consensus, it seems reasonable to me that this extension can be proposed. I can ensure we get this consensus, but we must agree this is the way forward.

[javagl]

(reduz) We know we need a standard extension to do this, not a GODOT extension.

(vpenades) Then you have to be sensitive to other parties requirements and concerns.

(redzuz) Why?

That "Why?" could be moved up one indentation level. Everybody can propose an extension. For example, a GODOT_uuid extension. That GODOT prefix does not have a meaning. The prefix is only intended for disambiguation. (This is important - the name serves as some sort of ID here...). Such a proposal has to contain a schema, and a specification text. People may or may not implement this extension. When there are two independent implementations, the specification can go through a ratification process. It does not matter whether the prefix is GODOT or KHR or EXT at this point.

(I just wanted to point that out, in view of the overall extension development workflow. Any further statements that I could make here would not be constructive (on a technical level) and therefore self-contradictory...)

[BastiaanOlij]

@reduz I've been following this discussion along and as a fellow Godot developer I fully understand our needs here, but I would like to ask you to take a step back and look at what we're asking from a point of view of 3D authoring software, it is not as simple as you believe.

Yes you are absolutely right that DCCs can store meta data, but that is only the start of what is needed here. There are many, many features in DCCs that will break the ability to guarantee these unique IDs are actually unique and are properly tracked.

One good example here is Blenders ability to reference multiple blend files, blend files artists often will have purchased from assets stores, assets that often begin life as a single blend file with a number of base objects, that are then duplicated to create variations on the same theme. Blend files that may then be referenced from the actual blend file the artist is using to create a level/asset/etc. At this point in time there will be a large swat of overlapping UIDs that break the system.

Another example are tools like Houdini or other generative systems that often have no way of persisting UIDs as geometry is recreated, users will be even more frustrated here as changes will not propagate properly as UIDs change while to the user it's damn obvious the same object was recreated with new settings.

I can easily list many more.

I'm not saying these issues can't be fixed, but we are putting a requirement up as a consumer of the GLTF format that will stifle the abilities of many authoring software to a point that it is unlikely many will be willing to implement the extension, or if they do, do so with a big asterix saying that the system will likely fail in complex situations.

That all said, the most important thing here is to listen, not to me, but everyone here reacting. The problem isn't that they do not understand the needs of a game engine, the problem here is that the needs of a game engine do not outweigh the larger needs of this industry.

My humble opinion here is that this is an unsolvable problem because DCCs have too many features that make a simple UID solution unmanageable. I'm also leaning more to either enforcing unique names, or at the very least warn users that re-using names will lead to an inability to track changes over versions.

I fully agree that there are weaknesses to this approach and that the user can easily break things by changing names.
But equally using UIDs create different breaking situations that are even harder to fix when the user has no control (duplication, propagation to the wrong object, etc.).

[javagl]

There is an issue #1713 and apparently something that is called ASOBO_unique_id - maybe someone can provide some information about that - e.g. whether there is a specification available for that, or how it was implemented.

(We could consider to explicitly ping @Selmar, who seems to have implemented this extension (based on some comments in that issue), or @ghost, who seems to have implemented it in AsoboStudio/glTF-Blender-IO-MSFS#145 , but I hesitate to create noise. Maybe it's enough to just link here...)

[reduz]

Sorry folks for the heated debate and I apologize for being pushy with this. I appreciate all the feedback given and will try to contact the right stakeholders here to put together a better formed specification proposal to find a way to solve this, where we can better discuss on something concrete.

[Selmar]

There is an issue #1713 and apparently something that is called ASOBO_unique_id - maybe someone can provide some information about that - e.g. whether there is a specification available for that, or how it was implemented.

Yep.

EDIT: Another option, currently my favorite. This is a combination of ideas 1 and 3. Basically, we have an extension that enforces the constraint that glTF files have unique identifiers. By default, use the name as the unique identifier (idea 1), but optionally we can supply a separate UID (idea 3). This allows retroactively adding UIDs to files that didn't start with one (ex: node "A", renamed to "B" with UID "A", then it can track that as the same node).

@aaronfranke this is more or less what we did (though we never used UIDs, we just used the name)

We created this extension for many of the reasons mentioned here. I left Asobo at the end of last year, so I can't speak for experiences past that, but I'll write down some notes. The origin of this extension is here.

I must say that, in practice, this indirection of identification does complexify a number of things. It is an effective solution for many problems, but relying on unique ids requires a certain understanding from everyone involved in the pipeline. For example, an artist can't simply destroy and recreate an object and "give it the same name". Additional tooling (and knowledge) is required for such cases. Some verifications on exports were added too.

Flight simulator uses a somewhat modified version of glTF that imports directly into the game. We used this extension to be able to link up glTF hierarchies at runtime. This allowed us to export animation files and "mesh liveries" separately from the models, and allows us to change things as long as the nodes targeted for merging and the hierarchies in between them don't change (this means the hierarchy above an animation-targeted node can actually be changed without issue). For an SDK perspective, see the MSFS docs about submodel merging.

Inside the simulation, if a unique id extension is marked as used in a file, the unique id of a node is either the id specified by the extension, or its name if the extension is absent. Gameplay code and scripts did not use the unique id extension, it was used exclusively for the model merging, though that is certainly also because it was created very late in the project.

I can't find the json schema for the extension, but it's very simple anyway:

            "extensions": {
                "ASOBO_unique_id": {
                    "id": "HIPS"
                }
            },

We also created (at the same time) an extension called ASOBO_animation_retargeting which is part of the animation[i]/channel/target object. This extension replaces the node reference, so the animation data may be exported without a node hierarchy. Though perhaps this data is completely redundant, since I believe we have never actually exported animation data file without also exporting the nodes (and I suspect the importer wouldn't accept it either).