Add idempotence-s3fs-nio module for S3 bucket test resource generation

[Copilot]

Adds a new Maven module idempotence-s3fs-nio that enables the idempotence framework to copy test resources into Amazon S3 buckets (or S3-compatible stores like MinIO) via the carlspring/s3fs-nio NIO2 FileSystem provider.

New module: idempotence-s3fs-nio

• S3fsNioIdempotenceProperties — AbstractIdempotenceProperties impl; default basedir s3:///idempotence-test-resources, overridable via org.carlspring.testing.idempotence.s3fs.basedir
• S3fsNioPathTransformer — PathTransformer impl; strips Maven (target/test-classes) and Gradle (build/resources/test, build/classes/java/test) prefixes to produce relative S3 key paths
• S3ResourceCopier — copies classpath resources to S3 using Files.copy(InputStream, S3Path, REPLACE_EXISTING) via the NIO2 API; the basedir is resolved lazily per invocation so the endpoint can be overridden at test time
• S3TestResourceExtension — JUnit Jupiter @ExtendWith entry point; performs pre-test S3 cleanup via Files.walkFileTree() then delegates to S3ResourceCopier
• S3FileSystemUtils — shared utility to get or create the S3 FileSystem from a base URI (handles both AWS and custom endpoints)
• SPI registrations in META-INF/services/ for IdempotenceProperties and PathTransformer

Usage

@ExtendWith(S3TestResourceExtension.class)
class MyTest {

    @Test
    @TestResources(@TestResource(source = "classpath:/foo.txt"))
    void testSingleFile() {
        // Resource available at:
        // s3:///idempotence-test-resources/MyTest-testSingleFile/foo.txt
    }
}

Point at a local MinIO instance by setting:

-Dorg.carlspring.testing.idempotence.s3fs.basedir=s3://localhost:9090/my-bucket/test-resources

AWS credentials and region are resolved via the SDK's default provider chain.

Functional tests

S3BasicFunctionalityMinIOTest

End-to-end verification using a real MinIO instance started automatically via Testcontainers (skipped when Docker is unavailable):

• Starts a MinIOContainer, configures s3fs.* system properties, creates the target bucket, and wires IdempotencePropertiesService to point at the running instance.
• Three test methods (testSingleFile, testWithPatterns, testMultipleWithPatterns) use @TestResources/@TestResource annotations to trigger resource copying, then assert the uploaded S3 objects exist via Files.exists() on the NIO2 path.
• State (filesystem, system properties, IdempotencePropertiesService) is restored in @AfterAll.

S3BasicFunctionalityAWSTest

End-to-end verification against a real AWS S3 bucket (skipped unless the AWS_ACCESS_KEY_ID environment variable is set):

• Uses S3fsNioIdempotenceProperties defaults — reads the target S3 location from the org.carlspring.testing.idempotence.s3fs.basedir system property (defaults to s3:///idempotence-test-resources).
• AWS credentials and region are resolved via the SDK's default provider chain (env vars, ~/.aws/credentials, EC2 instance profile). The target bucket must already exist.
• Same three test methods as the MinIO variant, asserting that resources copied via annotations exist in the real S3 bucket.
• Cleans up all test S3 objects in @AfterAll and restores IdempotencePropertiesService.

Original prompt

---

This section details on the original issue you should resolve

<issue_title>Add support for generating test files in S3 buckets using s3fs-nio</issue_title>
<issue_description># Task Description

We need to add support for generating test files in S3 buckets using carlspring/s3fs-nio.

For this we need to add a new Maven module (idempotence-s3fs-nio) and make the necessary changes there.

Open Questions

Notes

Task List

The following tasks will need to be carried out:

• Investigate the issue.
• Provide a solution.
• Test the fix.
• Update the documentation.

Useful Commands

Incurred Costs

Task Relationships

This task:

• Is a sub-task of:
• Depends on:
• Is a follow-up of:
• Relates to:

Resources

Useful Links

Points of Contact

</issue_description>

Comments on the Issue (you are @copilot in this section)

• Fixes Add support for generating test files in S3 buckets using s3fs-nio #112

---

🔒 GitHub Advanced Security automatically protects Copilot coding agent pull requests. You can protect all pull requests by enabling Advanced Security for your repositories. Learn more about Advanced Security.

[github-actions]

🚀 Deployed to Netlify: View Preview

[carlspring]

@copilot :

This property should actually be org.carlspring.testing.idempotence.s3fs.basedir. Could you please also change this wherever else it's required?

[Copilot]

Done in e08094e. Renamed the system property to org.carlspring.testing.idempotence.s3fs.basedir in S3fsNioIdempotenceProperties, S3TestResourceExtension, and the docs.

[carlspring]

@copilot ,

There is no functionaity test that uses the annotations to generate the resource in the S3 bucket and then check if the resource really exists. Could you please implement this?

[Copilot]

@copilot ,

There is no functionaity test that uses the annotations to generate the resource in the S3 bucket and then check if the resource really exists. Could you please impleme...

Added in e5e88ef. S3BasicFunctionalityTest starts a MinIO container via Testcontainers, copies resources via @TestResources/@TestResource annotations, and verifies they exist in the bucket using Files.exists() on the NIO2 path. Three test methods mirror the Maven/Gradle pattern: single file, wildcard pattern, and multiple resources.

To wire this up correctly, S3TestResourceExtension was also refactored so that the S3ResourceCopier (and its basedir) is resolved lazily per invocation rather than eagerly at extension construction time — this lets @BeforeAll configure the MinIO endpoint before the extension's beforeEach runs.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

• dl.min.io
  • Triggering command: /usr/bin/minio minio server --console-address :9001 /data (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

• Configure Actions setup steps to set up my environment, which run before the firewall is enabled
• Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)

[github-actions]

🚀 Deployed to Netlify: View Preview

[carlspring]

@copilot :

Could you now, please:

• Rename S3BasicFunctionalityTest to S3BasicFunctionalityMinIOTest
• Add a new test called S3BasicFunctionalityAWSTest which actually connects to AWS

[Copilot]

@copilot :

Could you now, please:
...

Done in f936f21.

• S3BasicFunctionalityTest renamed to S3BasicFunctionalityMinIOTest (path strings inside the test updated accordingly).
• S3BasicFunctionalityAWSTest added — gated on @EnabledIfEnvironmentVariable(named = "AWS_ACCESS_KEY_ID", matches = "\\S+") so it is skipped when no explicit AWS credentials are present. Uses S3fsNioIdempotenceProperties defaults (reads basedir from org.carlspring.testing.idempotence.s3fs.basedir system property or s3:///idempotence-test-resources). Credentials and region are resolved via the SDK default provider chain. Cleans up the test S3 objects in @AfterAll. The target bucket must already exist.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

• dl.min.io
  • Triggering command: /usr/bin/minio minio server --console-address :9001 /data (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

• Configure Actions setup steps to set up my environment, which run before the firewall is enabled
• Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)

[github-actions]

🚀 Deployed to Netlify: View Preview