This is fairly in-the-weeds, but I've worked over this analysis with some colleagues and I believe our interpretation to be worth mentioning here.
The test claims to produce an invalid chain, but the "root" cert is not actually considered in this situation for path validation purposes (both RFC5280 and X.509 exclude self-issued self-signed certificates from consideration in path validation, even if it were in the path). The "root" in this scenario is a public key certificate that is used as a trust anchor - it is aggressively parsed as an input to path validation (RFC5280 6.1.1(d+), X.509 12.1), but the trust anchor is not subject to basic certificate check requirements that are part of path validation (X.509 12.5.1, 5280 6.1.3), as it's not actually in the path. The use of a public-key certificate as a trust anchor by a relying party is a choice - but not a requirement - and parsing of that certificate is not controlled by path resolution rules. Instead, it would fall back to parsing dictated by the standard for individual certificates (X.509 7.2 and 9.4.2.2 for the extension in particular, which does not require it be critical to be valid).
Putting it another way, we view the handling of the test (as we understand it) by an implementation as follows:
trusted-store = { root }
untrusted-store = { }
cert-to-validate = leaf
Given the above input, an RFC5280-compliant implementation will produce the following (non-default) path processing inputs:
path = [ leaf ]
trust-anchor-public-key = root.public_key
initial-permitted-subtrees-set = root.NameConstraints.permittedSubtrees
initial-excluded-subtrees-set = root.NameConstraints.excludedSubtrees
- assume defaults for all other input variables
- A "pure" X.509 implementation would not be required to read NameConstraints from the trust anchor, but RFC5280 seems to encourage this given the paragraph before 6.1.1(e) (implying, we think, that the variables mention in (e) through (i) may be populated from the trust anchor if available), and certainly most implementations interpret it this way.
Our view is that the "root" in the test case is not in the path generated by the resolver, but even if it were it would be skipped for the vast majority of processing given that X.509 12.5.1 would ignore it as a self-signed certificate (which RFC5280 also follows). An implementation that initializes initial-permitted-subtrees-set from the trust anchor certificate does so before it starts path processing, and thus the criticality of the NameConstraints extension is irrelevant for populating the input variable. As a result, the only cert in the path is the EE cert and it is valid given the path processing input variables in this environment.
A relying party could make the choice by policy that public key certificates supplied as trust anchors will be validated with a non-standard set of rules that would include requiring criticality for some extensions, and that could of course cause a failure, but RFC5280 hasn't defined such a policy (we do not pretend to be more than passingly familiar with CAB/F, so it's possible that the policy is supplied there - or could be in the future - although it would still be difficult to apply to trust anchors that come from directory services).
This is fairly in-the-weeds, but I've worked over this analysis with some colleagues and I believe our interpretation to be worth mentioning here.
The test claims to produce an invalid chain, but the "root" cert is not actually considered in this situation for path validation purposes (both RFC5280 and X.509 exclude self-issued self-signed certificates from consideration in path validation, even if it were in the path). The "root" in this scenario is a public key certificate that is used as a trust anchor - it is aggressively parsed as an input to path validation (RFC5280 6.1.1(d+), X.509 12.1), but the trust anchor is not subject to basic certificate check requirements that are part of path validation (X.509 12.5.1, 5280 6.1.3), as it's not actually in the path. The use of a public-key certificate as a trust anchor by a relying party is a choice - but not a requirement - and parsing of that certificate is not controlled by path resolution rules. Instead, it would fall back to parsing dictated by the standard for individual certificates (X.509 7.2 and 9.4.2.2 for the extension in particular, which does not require it be critical to be valid).
Putting it another way, we view the handling of the test (as we understand it) by an implementation as follows:
Given the above input, an RFC5280-compliant implementation will produce the following (non-default) path processing inputs:
Our view is that the "root" in the test case is not in the path generated by the resolver, but even if it were it would be skipped for the vast majority of processing given that X.509 12.5.1 would ignore it as a self-signed certificate (which RFC5280 also follows). An implementation that initializes initial-permitted-subtrees-set from the trust anchor certificate does so before it starts path processing, and thus the criticality of the NameConstraints extension is irrelevant for populating the input variable. As a result, the only cert in the path is the EE cert and it is valid given the path processing input variables in this environment.
A relying party could make the choice by policy that public key certificates supplied as trust anchors will be validated with a non-standard set of rules that would include requiring criticality for some extensions, and that could of course cause a failure, but RFC5280 hasn't defined such a policy (we do not pretend to be more than passingly familiar with CAB/F, so it's possible that the policy is supplied there - or could be in the future - although it would still be difficult to apply to trust anchors that come from directory services).