[FEA] Support Delta Lake 4.0.0

[nartal1]

Is your feature request related to a problem? Please describe.
This is an epic issue to track Delta-4.0.0 support.

Delta Lake-4.0.0 is built on Apache Spark-4.0. Hence, to support Delta tables on Spark-4.0 shim, we need to support Delta Lake-4.0.0 in the spark-rapids plugin. While doing so, we should try to consolidate delta-330 and delta-400.
Below is the list of tasks to consolidate and add support for delta-4.0

• Add Delta Lake 4.0.x module skeleton and build infrastructure - Add Delta Lake 4.0.x module skeleton and build infrastructure #13613
• Add shims layer, Delete command base classes and refactor delta-33x Delete - Add shims layer, Delete command base classes and refactor delta-33x Delete #13620
• Add Update command base classes and refactor delta-33x Update - Add Update/Merge command base classes and refactor delta-33x #13626
• Add Merge command base classes and refactor delta-33x Merge - Add Update/Merge command base classes and refactor delta-33x #13626
• Add Create/Optimize table command base classes and refactor delta-33x - Add Create/Optimize table command base classes and refactor delta-33x #13642
• Add FileFormat, Catalog, Provider, and RowIndexFilters base classes and refactor delta-33x - Add FileFormat, Catalog, Provider, and RowIndexFilters base classes and refactor delta-33x #13644
• Activate Delta Lake 4.0.x with full GPU acceleration support - Activate Delta Lake 4.0.x with full GPU acceleration [databricks] #13697
• Refactor DeltaDynamicPartitionOverwriteCommand - Refactor DeltaDynamicPartitionOverwriteCommand and enable Delta 4.0 integration tests [databricks] #13707
• Update integration tests for Delta 4.0 compatibility - Refactor DeltaDynamicPartitionOverwriteCommand and enable Delta 4.0 integration tests [databricks] #13707
• Enable Delta tests for Spark-4.0.x in CI/CD nightly jobs - Enable Delta Lake 4.0.x integration tests in nightly CI/CD jobs #13721

Additional context
Delta-4.0.0 release notes - https://github.com/delta-io/delta/releases/tag/v4.0.0
Documentation - https://docs.delta.io/4.0.0/index.html