fix(sagemaker-templates): Fix batch inference parameter resolution

[newman911]

The batch inference template was passing ParameterString objects directly to job_arguments, which caused them to be serialized as literal JSON strings instead of being resolved at runtime.

Changes:

• Use ProcessingInput with parameter as source for dynamic S3 paths
• Update preprocessing.py to handle both S3 URLs and local paths
• Follows the same pattern as xgboost_abalone template

This fixes the issue where InputDataUrl parameter was passed as '{"Get": "Parameters.InputDataUrl"}' instead of the actual S3 path, causing preprocessing jobs to fail with IndexError.

Resolves parameter resolution for batch inference pipelines.

Describe your changes

Issue ticket number and link

Checklist before requesting a review

• I updated CHANGELOG.MD with a description of my changes
• If the change was to a module, I ran the code validation script (scripts/validate.sh)
• If the change was to a module, I have added thorough tests
• If the change was to a module, I have added/updated the module's README.md
• If a module was added, I added a reference to the module to the repository's README.md
• I verified that my code deploys successfully using seedfarmer apply

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

We tested the batch inference pipeline end-to-end with:

Test case executed:

• Input: s3://mlops-workshop-model-training-757126856234/batch-inference-input/test.csv
• Pipeline: workshop-batch-inference
• Parameters: InputDataUrl, OutputsBucketName, instance types/counts

Results:

1. ✅ PreprocessData step - Parameter resolution works (ParameterString correctly resolved via ProcessingInput)
2. ✅ PreprocessData step - Sparse matrix conversion works (data outputs as proper CSV, not sparse format)
3. ✅ CreateModel step - Succeeded
4. ❌ Transformer step - Failed with feature mismatch (100 features vs 10 expected)

What we verified:

• Parameters can be passed to the pipeline and resolved correctly
• ProcessingInput downloads S3 files properly
• Preprocessing script finds and processes CSV files
• Sparse matrix converts to dense array before CSV output

What we didn't test:

• Different input data formats
• Different S3 paths/buckets
• Edge cases (empty files, malformed CSV, missing files)
• Full end-to-end success (blocked by pre-existing template design bug where preprocessing doesn't match model training)

The parameter resolution bug is fixed. The remaining Transformer failure is a separate pre-existing issue where the template's preprocessing doesn't match what the model expects.[1:57 PM][amzn-builder] 42% > what was the deploy command we tested?

bash
diw deploy-capability
--account $AWS_ACCOUNT_ID
--region us-east-1
--artifact-path ~/Downloads/test-batch-inference/awssolutions-innovation-workbench-solution-kit-ai-foundations-3.2.2-FIXED.tgz
--capability-name mlops-sagemaker-template-batch-inference
--config file://./batch-inference-config-fixed.json
--deployment-name corp-batch-inference

Where the artifact was modified to point to the fix/batch-inference-parameter-resolution branch instead of release/3.2.2.

Then executed the pipeline with:
bash
aws sagemaker start-pipeline-execution
--pipeline-name workshop-batch-inference
--pipeline-parameters
Name=InputDataUrl,Value=s3://mlops-workshop-model-training-757126856234/batch-inference-input/test.csv
Name=OutputsBucketName,Value=mlops-workshop-model-training-757126856234
Name=TransformInstanceCount,Value=1
Name=TransformInstanceType,Value=ml.m5.large
Name=ProcessingInstanceCount,Value=1
Name=ProcessingInstanceType,Value=ml.m5.large
--pipeline-execution-display-name "Sparse-Matrix-Fix"[1:57 PM][amzn-builder] 42% > cat the config file

I will run the following command: cat ~/Downloads/test-batch-inference/batch-inference-config-fixed.json 2>/dev/null || cat ./batch-inference-config-fixed.json 2>/dev/null || echo "File not found" (using tool: shell)
Purpose: Find and cat config file

{
"SAGEMAKER_PROJECT_ID": "workshop-batch-inference",
"SAGEMAKER_PROJECT_NAME": "workshop-batch-inference",
"SAGEMAKER_DOMAIN_ID": "d-egtvvpzmveem",
"SAGEMAKER_DOMAIN_ARN": "arn:aws:sagemaker:us-east-1:757126856234:domain/d-egtvvpzmveem",
"REPOSITORY_TYPE": "CodeCommit",
"BASE_JOB_PREFIX": "batch",
"MODEL_PACKAGE_GROUP_NAME": "workshop-model-training-workshop-model-training",
"MODEL_BUCKET_NAME": "mlops-workshop-model-training-757126856234"
}

[kukushking]

Nice catch!