Skip to content

Update download-hz-dist to support multiple repositories [DI-730] #67

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
JackPGreen wants to merge 3 commits into
base: master
Choose a base branch
from
+81 −81
Open

Update download-hz-dist to support multiple repositories [DI-730] #67

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
1d7148b
DI-730
JackPGreen Jun 18, 2026
95a4ac5
Update .github/workflows/test-download-hz-dist.yml
JackPGreen Jun 23, 2026
949cc02
https://github.com/hazelcast/docker-actions/pull/67#discussion_r34596…
JackPGreen Jun 23, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 13 additions & 27 deletions .github/workflows/test-download-hz-dist.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ on:
workflow_call:

jobs:
test-public:
test:
runs-on: ubuntu-latest
permissions:
id-token: write
Expand All @@ -13,6 +13,12 @@ jobs:
distribution:
- hazelcast
- hazelcast-enterprise
version:
- 5.1
- 5.6.1-SNAPSHOT
classifier:
- slim
- ''
steps:
- name: Checkout repository
uses: actions/checkout@v7
Expand All @@ -21,42 +27,22 @@ jobs:
uses: ./download-hz-dist
id: download
with:
repo-vars-as-json: "{\n \"MAVEN_EE_RELEASE_REPO\": \"https://repository.hazelcast.com/release\",\n \"MAVEN_OSS_RELEASE_REPO\": \" \"\n}"
environment: live
aws-role-to-assume: ${{ vars.AWS_HAZELCAST_OIDC_GITHUB_ACTIONS_ROLE_ARN }}
distribution: "${{ matrix.distribution }}"
hz_version: "5.0"
hz_version: "${{ matrix.version }}"
classifier: "slim"
packaging: "tar.gz"

- name: Check file exists
- name: Check file downloaded from `${{ steps.download.outputs.download_url }}`
run: |
if [[ ! -f "${{ steps.download.outputs.output_file }}" ]]; then
echo "${{ steps.download.outputs.output_file }} not found!"
exit 1
fi

test-private:
runs-on: ubuntu-latest
permissions:
id-token: write
steps:
- name: Checkout repository
uses: actions/checkout@v7

- name: Download
uses: ./download-hz-dist
id: download
with:
repo-vars-as-json: "{\n \"MAVEN_OSS_SNAPSHOT_REPO\": \"snapshot-internal::::https://repository.hazelcast.com/snapshot-internal\",\n \"MAVEN_OSS_RELEASE_REPO\": \" \"\n}"
aws-role-to-assume: ${{ vars.AWS_HAZELCAST_OIDC_GITHUB_ACTIONS_ROLE_ARN }}
distribution: hazelcast
hz_version: "5.6.1-SNAPSHOT"
classifier: "slim"
packaging: "tar.gz"

- name: Check file exists
run: |
if [[ ! -f "${{ steps.download.outputs.output_file }}" ]]; then
echo "${{ steps.download.outputs.output_file }} not found!"
# https://stackoverflow.com/a/2001750
if ! tar -tzf ${{ steps.download.outputs.output_file }} > /dev/null; then
Comment thread
JackPGreen marked this conversation as resolved.
Outdated
echo "${{ steps.download.outputs.output_file }} not valid!"
exit 1
fi
118 changes: 65 additions & 53 deletions download-hz-dist/action.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
name: Download Hazelcast Distribution
inputs:
repo-vars-as-json:
description: 'Repo Vars as JSON - not inherited in composite actions - see https://github.com/orgs/community/discussions/49689#discussioncomment-6398261'
environment:
required: true
aws-role-to-assume:
description: 'AWS Role to assume for accessing secrets'
Expand All @@ -25,61 +24,55 @@ inputs:
description: 'Path to the output file'
required: false
outputs:
maven_repo_url:
description: 'The Maven repository root URL used to download the distribution'
value: ${{ steps.get_repo_url.outputs.repo_url }}
download_url:
description: 'The URL used to download the distribution'
value: ${{ steps.download.outputs.url }}
output_file:
description: 'Path to the downloaded file'
value: ${{ steps.derive-output-file.outputs.file }}
runs:
using: "composite"
steps:
- name: Get repository URL
id: get_repo_url
- name: Derive repositories
id: derive-repositories
shell: bash
run: |
. ${GITHUB_ACTION_PATH}/../.github/scripts/logging.functions.sh

# Pick an appropriate key from "repo-vars-as-json"
case "${DISTRIBUTION}" in
"hazelcast")
if [[ "${HZ_VERSION}" == *"SNAPSHOT"* ]]; then
repo_var_name=MAVEN_OSS_SNAPSHOT_REPO
else
repo_var_name=MAVEN_OSS_RELEASE_REPO
fi
case "${ENVIRONMENT}" in
live)
repos=$(cat <<'EOF'
https://repo.maven.apache.org/maven2

@nishaatr nishaatr Jun 23, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not sure here but can we say whats the more frequent use case i.e. snapshots or release?
the list should favor that to minimize curl misses
we also do more PATCH releases so may be central should be last

not a big deal but thought I mention

@JackPGreen JackPGreen Jun 23, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's ordered alphabetically. I don't know what ordering would be more efficient, I did think about doing it in parallel but if more than one was successful (i.e. preprod + live) it'd overwrite the same file and be a mess so just left it.

https://repository.hazelcast.com/release
https://repository.hazelcast.com/snapshot
https://repository.hazelcast.com/snapshot-internal
EOF
)
;;
"hazelcast-enterprise")
if [[ "${HZ_VERSION}" == *"SNAPSHOT"* ]]; then
repo_var_name=MAVEN_EE_SNAPSHOT_REPO
else
repo_var_name=MAVEN_EE_RELEASE_REPO
fi
sandbox)
repos=$(cat <<'EOF'
https://repository.hazelcast.com/sandbox-mvn-prod
EOF
)
;;
*)
echoerr "Unsupported distribution type '${DISTRIBUTION}'" ; return 1
*)
echoerr "Unsupported environment (${ENVIRONMENT})" ; exit 1
Comment thread
JackPGreen marked this conversation as resolved.
Outdated
;;
esac

# Lookup up the value of the selected key in "repo-vars-as-json"
REPO_URL=$(jq --raw-output .${repo_var_name} <<< "${REPO_VARS_AS_JSON}")
echo "repo_url=${REPO_URL}" >> ${GITHUB_OUTPUT}

# Detect if the repo definition specifies authentication via a Maven server (e.g. "my-private-repo::::https://example.com/private-repo")
if [[ "${REPO_URL}" == *[!:]::::* ]]; then
echo "authentication-required=true" >> ${GITHUB_OUTPUT}
fi
{
echo "repositories<<EOF"
echo "${repos}"
echo "EOF"
} >> ${GITHUB_OUTPUT}
env:
REPO_VARS_AS_JSON: ${{ inputs.repo-vars-as-json }}
DISTRIBUTION: ${{ inputs.distribution }}
HZ_VERSION: ${{ inputs.hz_version }}
ENVIRONMENT: ${{ inputs.environment }}

# Should be a relative local path, but doesn't resolve when called from another repo
- uses: hazelcast/docker-actions/setup-maven-snapshot-internal@master
if: ${{ steps.get_repo_url.outputs.authentication-required == 'true' }}
- uses: hazelcast/docker-actions/get-jfrog-credentials@master
id: jfrog
with:
aws-role-to-assume: ${{ inputs.aws-role-to-assume }}
maven-settings-path: ${{ inputs.maven-settings-path }}
jfrog-oidc-provider-name: ${{ github.repository_owner }}-snapshot-internal

- name: Derive output file
id: derive-output-file
Expand All @@ -89,23 +82,42 @@ runs:
env:
FILE: ${{ inputs.output_file || format('distribution.{0}', inputs.packaging) }}

- name: Download via Maven
- name: Download
id: download
shell: bash
run: |
mvn \
org.apache.maven.plugins:maven-dependency-plugin:2.10:get \
-DremoteRepositories="${{ steps.get_repo_url.outputs.repo_url }}" \
-DgroupId="com.hazelcast" \
-DartifactId="${DISTRIBUTION}-distribution" \
-Dversion="${HZ_VERSION}" \
-Dclassifier="${CLASSIFIER}" \
-Dpackaging="${PACKAGING}" \
-Dtransitive=false \
-Ddest="${{ steps.derive-output-file.outputs.file }}" \
--batch-mode \
--no-transfer-progress
. ${GITHUB_ACTION_PATH}/../.github/scripts/logging.functions.sh

while IFS= read -r repository; do
url="${repository%/}/com/hazelcast/${ARTIFACT_ID}/${VERSION}/${ARTIFACT_ID}-${VERSION}${CLASSIFIER:+-$CLASSIFIER}.${PACKAGING}"

curl_args=(
--fail
--location
--output "${{ steps.derive-output-file.outputs.file }}"
--show-error
--silent
)

if [[ "${repository}" == *repository.hazelcast.com* ]]; then
curl_args+=(--user "${{ steps.jfrog.outputs.user }}:${{ steps.jfrog.outputs.token }}")
fi

echodebug "Downloading from ${url}..."

if curl "${curl_args[@]}" "${url}"; then
echodebug "Downloaded from ${url}"
echo "url=${url}" >> ${GITHUB_OUTPUT}
exit 0
else
echodebug "Failed to download from ${url}"

@nishaatr nishaatr Jun 23, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not a failure but just not found.
otherwise little misleading!

@JackPGreen JackPGreen Jun 23, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well it could be a failure... all it really means is curl returned a non-0 exit code. But you won't be able to see that, because --silent...

fi
done <<< "${{ steps.derive-repositories.outputs.repositories }}"

echoerr "Distribution not found in ${{ steps.derive-repositories.outputs.repositories }}"

@nishaatr nishaatr Jun 23, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
echoerr "Distribution not found in ${{ steps.derive-repositories.outputs.repositories }}"
echoerr "Distribution '${{ steps.derive-output-file.outputs.file }}' not found in ${{ steps.derive-repositories.outputs.repositories }}"

@nishaatr nishaatr Jun 23, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NIT

@JackPGreen JackPGreen Jun 23, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's not accurate as that's a user input (i.e. they could set the output file to anything). You could set it to inputs.distribution, but then you'd want to see the version etc and it just gets confusing.

exit 1
env:
DISTRIBUTION: ${{ inputs.distribution }}
HZ_VERSION: ${{ inputs.hz_version }}
ARTIFACT_ID: ${{ inputs.distribution }}-distribution
VERSION: ${{ inputs.hz_version }}
CLASSIFIER: ${{ inputs.classifier }}
PACKAGING: ${{ inputs.packaging }}
2 changes: 1 addition & 1 deletion get-supported-platforms/action.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ runs:
echo "platforms=linux/arm64,linux/amd64,linux/s390x,linux/ppc64le" >> ${GITHUB_OUTPUT}
;;
*)
echoerr "Unsupported platform behaviour for ${base_image_name}" ; return 1
echoerr "Unsupported platform behaviour for ${base_image_name}" ; exit 1

@nishaatr nishaatr Jun 23, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why change?

@JackPGreen JackPGreen Jun 23, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You return from a function, and exit from a shell - it's not in a function, so we shouldn't return.
I copied this switch when adding the repo switch statement to the new action and spotted it so thought I'd fix it there too.

;;
esac
env:
Expand Down
Loading