Make TLE optional for Rekor v2 compat

[loosebazooka]

Summary

Changes required for a TLE that is compatible with both V1 and V2 logs.

• Make integrated_time optional
• Add some comments for handling (maybe could use some work on the wording).

Release Note

• TransparencyLogEntry.integrated_time is now optional as future log implementations may not return anything. Clients will be expected to attach another form of verified time to the bundle, such as an rfc3161 timestamp.

Documentation

architecture docs may need to be updated

• https://github.com/sigstore/architecture-docs/blob/main/client-spec.md
• https://github.com/sigstore/docs/blob/main/content/en/about/bundle.md

[Hayden-IO]

nit, SET isn't defined til later, can we say inclusion_promise?

[loosebazooka]

yeah

[Hayden-IO]

"log upload responses" rather than "log entries"?

[loosebazooka]

I've been doing a lot of flipflopping on how to handle this.

While clients can know during rekorv2/putEntries how to parse the returned rekor request, handling it at verification time for bundles is a bit different.

Clients that try to parse the canonicalized entry will break if we try to re-use the canonicalized_body code path.

So the idea that we wont break clients was not necessarily true, there is no "free verification" of rekor v2 bundles. Clients will fail in unexpected ways. We need new verification paths and branching anyway. If we want to reuse the TransparencyLogEntry from V1, clients need to rewrite parsing so they use the kind/version information in the TransparencyLogEntry to determine how to parse the CanonicalizedBody -- and not parse the CanonicalizedBody partially to determine kind/version (most client do this -- java/go/python).

@steiza @kommendorkapten @jku @haydentherapper @codysoyland -- Current clients cannot handle rekorv2 bundles and they may fail in strange ways (probably mostly json unmarshalling error)

Failures:
java

java.lang.IllegalStateException: Cannot build RekorEntryBody, some of required attributes are not set [apiVersion, kind, spec]

go (via examples/sigstore-go-verification)

validation error: invalid character 'h' looking for beginning of object key string

Failure modes:

• reuse old tlog type: clients fail when trying to unmarshall json for canonicalizedEntry/rekorBody.
• TransparencyLogEntryV2: clients fail with "no logentry found"
  • client code is a lot cleaner here too.

Both approaches will still require code changes and failures will occur on all existing clients.

[jku]

I've been doing a lot of flipflopping on how to handle this.

While clients can know during rekorv2/putEntries how to parse the returned rekor request, handling it at verification time for bundles is a bit different.

Clients that try to parse the canonicalized entry will break if we try to re-use the canonicalized_body code path.

So the idea that we wont break clients was not necessarily true, there is no "free verification" of rekor v2 bundles

Thanks for digging into it. I've not had to deal with code that handles canonicalized body so I have trouble following the argument here.

I suppose the core failure here is that the proto is unclear: "contents of this field are the same as the body field in a Rekor response" is not ideal.

In any case, I'd appreciate more details on what has changed. I'm sure the canonicalized body is documented for both rekor versions... but I have trouble finding those docs and I can't see the issue by just looking at python client code.

[loosebazooka]

moved discussion to: #633