Authorship Metadata - #106
Conversation
9f38c93 to
cb776b3
Compare
|
Thanks for the feedback, Spiff. Lots of good questions and I think this is a bit of a can of worms to a degree. I'll wait for others to leave some feedback and do a pass accordingly |
|
I think there's a stronger case for layer metadata as the store for authorship information. Many of the fields proposed are symmetrical with document (ie. layer) authorship more than composed scene graph state. Per some of Spiff's points, a multi-apply namespace isn't protection against colliding fields. Users will have to impose a design on their structures on these namespaces that I suspect in practice might resemble a bijection with the underlying layer structure. The solution of applying identical schema applications to all root prims to simulate layer level metadata is awkward when multiple root prims exist. Consumers may also miss root level The primary concern with using layer metadata appears to be that this information may be lost under flattening. There are several examples we can look to for solutions here. In OpenUSD itself, the flattening operation records a breadcrumb in its comment field. One can imagine having similar semantics for flattening authorship fields similar to the breadcrumb in the comment. I'd also offer that in image formats, generally, authorship is encoded as a single set of fields. An EXR image may be "composed" of many different input assets (textures, geometry, graphics, etc.), but ultimately only records a single set of documentation fields. With a carefully formulated layer metadata approach, I think you can more robustly encode user authorship even under flattening. |
The things that are "nice" about switching to a layer-based encoding are that to do robust provenance tracking we are going to need to get into it at the layer-level, and you'd be able to disambiguate "authorship" of a referenced asset vs overrides to it that came in at a stronger layer. It does make for a more fragile mechanism (as you say, "carefully formulated")... we have been reluctant to complicate UsdStage::Flatten() (particularly because it can't reasonably manage special data that comes in from downstream schema domains), and even when we introduced core metadata |
|
I think we should align with C2PA. C2PA is widely adopted, and in particular AOUSD partners are using it in their products already. C2PA applies to the package/archive containing assets, and has a specific Manifest Store file that should be in the archive. The AuthorshipAPI declares the identity of a prim. C2PA signs a manifest store with a claim about the current asset; it applies at asset granularity. It doesn't have a provision for chain-of-custody or changes, nor for granular information about components of a signed asset. C2PA and AuthorshipAPI are highly complementary, and do not preclude future work on chain-of-custody and other provenance work. AuthorshipAPI Compatibility with C2PA can be achieved by adopting C2PA-standardized keys, mapped to XMP/IPTC vocabularies. I suggest we change the AuthorshipAPI proposed names to the C2PA names in the second column, and then we are immediately and obviously aligned with C2PA. This would dovetail nicely with existing C2PA tooling and integration, and would open the door to creating C2PA Manifest Stores in a straightforward manner without any need for remapping. |
|
@nporcino-pixar jsut to clarify, you're not suggesting to adopt the C2PA signing requirements but just align the names? |
|
@nvmkuruc my concerns about using layer metadata is say you have a 3D scene and I use GenAI to modify a single mesh.
|
|
Yes, I'm specifically suggesting to align the names. |
|
|
In the case of relationships to materials etc, I'm not sure I quite follow how that wouldn't play well with the schema route? Could you elaborate with an example scenario? I do worry that requiring people to split up their content just to comply with regulation based use in their pipelines makes it difficult to use this meaningfully. For example, most DCCs can only export a single file at once. They'd have to either adopt the view that any use of GenAI taints your whole scene completely and irrevocably or that we'd need to work with every dcc to make sure they split stuff out for users. If we have to rely on using a particular incantation of flatten to do the right thing , then I think we also start introducing the view that OpenUSD becomes responsible for more than just the metadata but also the persistence of it. I'm not sure I'm comfortable with either of those myself based on what I've seen of the GenAI landscape or out of the box dcc pipelines. |
I think "any use taints" is unreasonable.
I think flattening in general is a separate problem, and not USD's responsibility. i.e. flattening just does what it says. If a DCC wants to e.g. record all provenance in data, then great, collect that data, flatten, and then reintroduce the subtle authoring again. Just like Profiles. Profiles is an author's declaration and a best intention. We provide tools to derive a ground truth and declare it, but is not a whole new magical pseudo-provenance system. I think Authorship is similar in that regard, in not being a magical usd feature, but a please for well intentioned authors to declare their intentions.
This is entirely possible within the proposal as stands, and by the best intention logic, would be something someone could certainly do, one possible pattern amongst any number. The key point is discoverability, which is aided by consistency. With a multi-apply schema you could declare simultaneously human and gen authorship at the top level so a DCC could report "mixed authorship". I don't think it has to go deeper or fancier than that. |
|
@spiffmon @asluk I have taken a pass at all your notes. Please let me know if you have more. @nporcino-pixar I aligned most of the names but left a table with reasoning where they differ. @nvmkuruc I left a section about layer metadata and why I felt it wasn't the right fit as is for this, but left it as an open question. I'm not opposed to finding a solution for it, but I feel like my intuition says per prim metadata is the way to go for the time being. |
|
Separate from the discussion about layers and prims, I have hesitations about using Consider the following layers The I'm sharing this as a strawman alternative. Not suggesting this is an ideal, but it shows an alternative view where authorship lives on the layer. It uses the dictionary key as the |
|
@nvmkuruc , regarding #106 (comment)
|
|
@nvmkuruc , I realized your concern about API schemas is not specific to this proposed use - it applies equally to all use of API schemas, which are becoming more and more vital to describing and combining behaviors. Are you aware of Are you concerned about Sdf-level authoring code? There is one, I believe place where we would produce an explicit list, in UsdStage::Flatten()... the idea of using Flatten() to produce a layer that you apply over another layer seems odd/wrong to me, but obviously we can't prevent it... would it be worth changing the output of Flatten() to be additive, also? Or are there places where documentation/best-practices should be emphasized? |
|
As the Chair of the Technical Working Group for the C2PA, as well as the co-chair of the Technical Pillar of the ISO/IE/ITU's World Standards Collaboration on AI & Multimedia Authenticity (https://aiforgood.itu.int/multimedia-authenticity/), let me weigh in here... First and foremost, let me clarify that the C2PA's Content Credential standard is focused on providing a tamper-evident record of provenance for assets (either singularly or in a collection). It enables that provenance from "glass to glass" (aka creation to consumption). Second, authorship is just one of many types of provenance that creators/producers/publishers/etc. want to incorporate. You could also find "basic" information such as devices used, locations, etc. - but also about ownership, renumeration, etc. The Content Credential is also extensible for various industry extensions, of which there are many (both in existence and in process) Finally, that provenance record is asset type/format agnostic, but we document the specific locations within various formats when embedding the Content Credential. (though it can also live in the cloud, on disks, etc.). And, as noted, many of the folks in your consortium have already standardized on it for images, videos, audio and documents. And related to that, I have active PRs going for support for the USD formats (.usda, .usdc and .usdz), as well as various others in your ecosystem - EXR, MaterialX, glTF/GLB and more. Instead of having you folks create your own solution - it would be great for you to collaborate with the rest of the industry to adopt C2PA's Content Credential as your solution not only for AI labelling, but for the wider use cases of provenance. |
|
Hi Leonard! It's great to see you here. There is a great deal of activity in flight recently around related concerns, see also: #110 (Profiles and Capabilities) You mention:
Is that something we can see today? As you note all of these impact or are integral parts of our ecosystem, it would be great to understand the bigger picture! |
|
I put some stuff into your Slack channels for more conversation. Happy to share them here as well! |
|
Thanks everyone for the thorough notes! I've pushed an update that I think works through all of them, but I'm of course open to more notes. A quick breakdown of what changed: @nvmkuruc on apiSchemas listop shadowing: I've added a section ("When apiSchemas Gets Shadowed") that documents this as a known gotcha. I understand the concerns, but I think on balance it's probably the solution which has the fewest infrastructure lifts, and fewest current edge cases. @spiffmon — on the instanceID / version / model-version confusion: I've tried to clarify this more in the text now. producer + version together identify the @lrosenthol — thanks for the C2PA/XMP detail. I also folded in some of the Slack ASWF thread comments too. I've tried to make more mention of the C2PA package manifest you mentioned, and tried to draw a clearer line on how this proposal and provenance standards like C2PA aren't at odds, and serve subtly but distinctive needs. @nporcino-pixar good pont on the AI-only reading. I've made it explicit that the schema covers human, tool, and algorithmic authorship equally, and that @asluk , On Decentralized Identifiers (raised separately in slack) — there's now an "Identifiers" section that explains our reverse-domain convention for producer and UUID4 for instanceID, I've also added an explicit Goals section just to make sure our thinking is reflected up front, to correspond with the non-goals section right after it. Please take another pass and let me know if anything still feels off! |
88280b0 to
e076f57
Compare
|
The current Authorship proposal advocates for a composed and hierarchically inherited Observations
The claim that it's "hard for a simple file-level tag to accurately describe what happened" is
Few tools and DCCs operate directly on OpenUSD. OpenUSD layers are often the result of an export
Layer metadata doesn't become lost or ambiguous when referenced or sublayered. (The below example Surviving flattening is an interesting question that we'll follow up on later.
The example provided here asserts that tracking different authorship for the document organized by It's worth being mindful that not all tooling in the broader OpenUSD ecosystem support hierarchical
OpenUSD provides layers as its native solution for mixed-authorship workstreams.
It's not correct to say there's no special semantics-- specifically "hierarchial inheritance" is suggested here-- There have been several suggestions in discussions outside of this proposal about how one might Consider the following layer-- There's no native OpenUSD way to identify the connection of RecommendationThis counter proposal recommends working within the constraints of layers is not to suggest that layer The hierchically composed proposal underspecified yielding unpredictable results for authors This critique recommends a single authorship dictionary per layer which may be populated with the same We'd generally expect most usage would be simple annotations. We'd expect in the common case for differences in authorship to align with different workflows If it's a strong requirement that multiple producers (say a primary application A variation of this would be "instance as key" to allow for multiple producers Much of the current proposal's approach comes from a suggested One final point of consideration would be to make the authorship field This proposal showed several examples of how layer metadata can address the |
|
@nvmkuruc Matt, for human readers, could I ask you to summarize what the take away is? My brain is finding it very hard to extract the argument from the long (agentic?) explanation. |
|
I'm super confused, but I think* the argument is for a compromise? i.e. Allow for authoring on prims, and also on layers? |
|
My sincere apologies for jumping to an assumption of agentic authorship, @nvmkuruc - it is terrible that I have gotten to a place where long explanatory texts automatically trigger an assumption that machine reasoning is on display. |
|
Also to clarify my question, your conclusion was clear but left a question in my mind:
Very specifically, what I was confused by whether the proposal you are making is "layer metadata in addition to authoring on prims" or "layer metadata instead of authoring on prims". |
|
Hey Nick. Personally, the idea of one authorship dictionary per layer makes the most sense to me and feels to me like the common use case, but I tried to show in the recommendations section a few different forms which accommodate multiple producer and/or instance ids for the same layer to better support some of the scenarios in the proposal. For either the dictionary or schema form, I can see cases where authorship is best keyed by producer, instance id, producer/instance id combo, producer with an optional instance id, etc.. If you're looking for one proposed form to work with-- I'd put this form out there. It would compactly accommodate the "one layer one author scenario". while allowing for more granular annotations
Possibly! I think the compromise idea allows both with the idea that something like a "profile" might guide towards best practices in the future. At the same time, two different forms could yield their own complexities. |
|
Thank you for the additional examples, this helps form it up for me at least. This really speaks to the larger world of provenance, and properly understanding who originated what (when will also be critical to that discussion). Although it has only just started landing, I am reminded of the Profiles and Capabilities system by analogy; where claims are made by an author, which can be validated by introspection, or simply accepted (the author claims you need to have OpenVdb, a validator may discover no such dependency). The issues you point out about behavior under composition and flattening are questions about truth and provenance. The authorship metadata is about a claim, which cannot be validated by introspection because USD does not embody a chain of evidence of modification. As a consumer of data, the only choice on the table is trust the author's claim. In terms of what a trustworthy or untrustworthy claim achieves, is a place for a content creation program or an individual author, to make a claim, satisfactory to EU and CA AI disclosure laws. Your analysis and criticism holds water, and I think it is part of the larger "separation of IP concerns" umbrella that @asluk has introduced. In the case of this Authorship proposal, I argue that computational truth is not the goal, the goal would be a granular, unverifiable claim of intent in accordance with a legal framework to be satisfied. That much I believe is achieved and has been externally verified. I wonder if we might harvest (as it were) your careful analysis for the larger "separation" discussion? |
Propose a standardized multiple-apply AuthorshipAPI schema for recording authorship and provenance metadata on USD prims.
|
Thanks @nvmkuruc for the feedback. I've incorporated it as best as I could, and tried to put clarifications where possible to remove the ambiguity in the cases you mentioned. In terms of lines of code changed, its pretty hefty but most are clarifications so the proposal hasn't changed substantively too much at least. I do still feel strongly about having the data on a per prim basis, as that's a hard requirement from our perspective, but I do like the dictionary alternatives you suggested and incorporated them as options. I also took the compositional cases you brought up here and on slack, and added them in as thought experiments. I don't think they actually have good tractable answers, but I think it's good to call them out and provide guidance. |
|
At AOUSD TAC this morning, we discussed encodings and techniques for "lofting up" authorship data to "the root" of a scene, whether that be layer metadata or a root prim(s). Afterwards, that started to seem deeply limiting to me, primarily (but not exclusively) due to the presence of VariantSets in an asset/scene. It seems entirely plausibe to me that we might have an asset with multiple variants, only one/some of which are AI generated (or what have you). Any pre-computed encoding of authorship is going to have a really hard time capturing that level of granularity. And it doesn't seem right to reject, based on AI-grounds, an asset that is not currently configured with any AI-produced content. But the larger point is that none of the proposed computations-of-aggregated-authorship for either encoding can capture opinions from uncomposed, but present and potentially selectable, content. I'm therefore preferring a) an applied-schema encoding, which can naturally be specified per-Variant on any prim, and b) no pre-caching of aggregated authorship (plus of course utilities for performing the computation given the current configuration of the stage). We might someday reach a point where we decide we must provide some form of caching for performance reasons, but while this space and workflows are still exploratory, I'd argue it's better not to add one prematurely. |
|
@spiffmon ah yeah I think that's a great point regarding variants. Especially because you might have a pipeline where someone has a variant for the GenAI as standin proxy while the human made one is in progress. Or multiple iterations of something to visualize in scene I also agree that we can add any lofting / indirection in a second pass independent of the initial one. |
|
The remaining question, though, is whether there is value or anti-value (given Apple's per-prim granularity concerns) in including the ability to compute the "most conservative/inclusive" authorship for an asset. None of the proposed techniques afford that, but it would be possible using the UsdUtils asset dependency analysis logic to find all reachable layers/assets, and walk the layers extracting known authorship data. |
Description of Proposal
Link to Rendered Proposal
As global regulations increasingly mandate the identification of AI-generated content, the industry needs a unified standard for tracking provenance within Universal Scene Description (USD). This proposal introduces a standardized metadata schema—ideally implemented as a multiple-apply API—to record authorship across the 3D pipeline.
While prompted by the rise of generative AI, this schema is designed for universal utility. It enables a single prim to carry a persistent, auditable trail from every stage of its creation, whether produced by a human artist, a generative model, or algorithmic tools like photogrammetry. By aligning on these fields, we ensure regulatory compliance, proper artist attribution, and clear studio ownership across the 3D ecosystem.
A quick example of what it might look like in practice:
Disclaimer: This proposal should not be taken as an indication of any upcoming
features in any particular product.
It is being provided to garner community feedback and help guide the ecosystem.
I am also going to emphasize that I am not a lawyer.
While I have consulted with our great legal team to see if we have holes in coverage, that doesn't extend to providing any legal advice.
Contributing