TCK: Servlet TCK tests still using web.xml with DTDs

[robertpatrick]

Challenged tests

1. servlet.tck.compat.LeadingSlash.WithLeadingSlash.WithLeadingSlashTests.WithLeadingSlashTest
2. servlet.tck.compat.LeadingSlash.WithoutLeadingSlash.WithoutLeadingSlashTests.WithoutLeadingSlashTest
3. servlet.tck.pluggability.api.jakarta_servlet.registration.RegistrationTests.getRegistrationInitParameterTest
4. servlet.tck.pluggability.api.jakarta_servlet.registration.RegistrationTests.filterServletMappingTest
5. servlet.tck.pluggability.api.jakarta_servlet.registration.RegistrationTests.getRegistrationInitParametersTest
6. servlet.tck.pluggability.api.jakarta_servlet.registration.RegistrationTests.servletRegistrationsTest
7. servlet.tck.pluggability.api.jakarta_servlet.registration.RegistrationTests.servletURLMappingTest
8. servlet.tck.pluggability.api.jakarta_servlet.registration.RegistrationTests.getRegistrationNameTest
9. servlet.tck.pluggability.api.jakarta_servlet.registration.RegistrationTests.getRegistrationClassNameTest
10. servlet.tck.pluggability.api.jakarta_servlet.registration.RegistrationTests.getFilterRegistrationsTest
11. servlet.tck.pluggability.api.jakarta_servlet.registration.RegistrationTests.getServletRegistrationTest
12. servlet.tck.pluggability.api.jakarta_servlet.registration.RegistrationTests.getFilterRegistrationTest
13. servlet.tck.spec.defaultmapping.DefaultMappingTests.defaultservletTest1
14. servlet.tck.spec.defaultmapping.DefaultMappingTests.defaultservletTest
15. servlet.tck.spec.servletresponse.servletResponseTests.testFlushBuffer
16. servlet.tck.spec.servletresponse.servletResponseTests.testFlushBufferHttp

TCK Version
Jakarta Servlet 6.1.0

Description
All 16 of the tests are failing with the error:

java.lang.RuntimeException: Could not lookup value for field public java.net.URL servlet.tck.common.client.BaseTckTest.url
	at org.jboss.arquillian.test.impl.enricher.resource.ArquillianResourceTestEnricher.enrich(ArquillianResourceTestEnricher.java:67)

Our engineer diagnosed the problem with servlet.tck.compat.LeadingSlash.WithLeadingSlash.WithLeadingSlashTests.WithLeadingSlashTest as being caused by the use of a web.xml with a DTD reference. Given that there really is no backward compatibility in Jakarta EE prior to Jakarta EE 9 (due to the javax => jakarta package change) and the web.xml was changed from DTDs to schemas in J2EE 1.4's Servlet 2.4 specification, there is no reason to require support for pre-servlet 2.4 deployment descriptor in the Servlet 6.1 TCK.

Additional context
We are trying to pass the EE11 TCK but this is a blocking issue.

[markt-asf]

This challenge is rejected based on chapter 14 of the Servlet specification

For backwards compatibility of applications written to previous versions of the API, any
deployment descriptors written to comply with earlier versions of the specification, must still be
supported such that applications continue to deploy.

[robertpatrick]

@markt-asf This seems inconsistent with jakartaee/platform-tck#1313

[edbratt]

As @robertpatrick points out, the Platform TCK relaxed this requirement as part of the Servlet compatibility requirements when Servlet TCK was included in the Platform TCK. It does appear that we should relax the legacy DTD requirement to align with the prior decision.

[robertpatrick]

More context and discussion around why this should be revisited at https://eclipsefoundationhq.slack.com/archives/C0131MLD538/p1773318403506959.

[arjantijms]

@markt-asf if possible, it would be great if you could reconsider. DTDs are really ancient artefacts, and there's no real usecase to force implementations to support them,

Of course, any implementation that wants to support them is free to do so. We have support in GlassFish still for them I believe, and a few custom tests outside the TCK are more than enough to make sure it still works. For our newer Servlet implementation, Piranja, it would rather not have to support DTDs too.

A compromise could be perhaps to mark them optional. Any implementation can then test for it, and report they passes those tests, but the full EE or web profiles would not have that requirement.

WDYT?

[robertpatrick]

It seems that only 2 test classes are using these old descriptors:

1. servlet.tck.compat.LeadingSlash.WithLeadingSlash.WithLeadingSlashTests.WithLeadingSlashTest
2. servlet.tck.compat.LeadingSlash.WithoutLeadingSlash.WithoutLeadingSlashTests.WithoutLeadingSlashTest

The other 14 are failing with the same error but for different reasons.

[markt-asf]

It looks like the spec document needs to be updated to align with the previous update to the platform specification. I don't recall any prior discussion in this project about such a change but I could well have forgotten it.

I'm assuming that the text in the platform specification overrides that in the Servlet specification. Given I don't recall an explicit decision by the Servlet project to override the platform specification on this point, that seems reasonable.

On that basis, the challenge is accepted and those two tests can be excluded.

The Servlet project therefore needs to:

• update the text in the Servlet specification document
• update those web.xml files to use a Jakarta EE 8 or later schema (using a Jakarta EE 8 schema would provide a test for support for that schema)
• check for any other web.xml files using pre Jakarta EE 8 descriptors.
• release an updated Servlet 6.1 TCK

[markt-asf]

The fix is even simpler. The tests are checking for a behaviour required only by Servlet 2.2 so then entire servlet.tck.compat can be removed.

[markt-asf]

A 6.1.2-SNAPSHOT TCK build is available from https://download.eclipse.org/ee4j/servlet/jakartaee11/staged/eftl/

If that addresses all the issues, I can release that as 6.1.2. If there are further issues, I'd prefer to fix them and then release 6.1.2 in a form that works for you.

I've run that SNAPSHOT (strictly a local build TCK build from the same source) against Apache Tomcat and all tests pass so there should be no impact on containers that have passed earlier versions of the 6.1.x TCK.

[olamy]

could we have snapshot jars deployed somewhere? central snapshots?

[markt-asf]

could we have snapshot jars deployed somewhere? central snapshots?

Yes. I've been looking into how to do this. I'm going to get it working for EL first (smaller, simpler project) and then I can make the same changes to Servlet (and Pages and WebSocket). Timing TBD.