Skip to content

Commit 9882f9f

Browse files
erichareclaude
erichare
and
claude
authored
chore(ci): timeout bump + harden langflow grep + build/publish src/bundles/* wheels (#13199)
* chore: bump backend unit test timeout from 30 to 40 minutes * ci: anchor langflow version grep so new workspace packages can't break it The pattern `uv tree | grep 'langflow' | grep -v 'langflow-base' | grep -v 'langflow-sdk'` was used in 8 places to pluck the root `langflow` (or `langflow-nightly`) line out of `uv tree`. It relied on enumerating every other workspace package that starts with `langflow-`. As soon as a new one was added (`langflow-stepflow`, via #12015) the filter let two lines through and the downstream `awk` / `cut` produced a multi-line string, e.g.: Name langflow-stepflow langflow-nightly does not match langflow-nightly. Exiting the workflow. Replace the enumeration with an anchored regex `grep -E '^langflow(-nightly)?[[:space:]]'` so only the exact `langflow` or `langflow-nightly` root line matches, regardless of how many other `langflow-*` workspace packages exist now or in the future. * ci: build, test, and publish src/bundles/* wheels Langflow main pins exact versions of every bundle (e.g. `lfx-arxiv-nightly==0.1.0.dev38`, `lfx-arxiv==0.1.0`), but neither the nightly workflow nor the release workflow built or published those bundle wheels. So when cross-platform tests (or downstream consumers) ran `pip install langflow-nightly`, the resolver couldn't find `lfx-arxiv-nightly` / `lfx-duckduckgo-nightly` on PyPI and bailed: Because lfx-arxiv-nightly was not found in the package registry and langflow-nightly==1.10.0.dev38 depends on lfx-arxiv-nightly==0.1.0.dev38, we can conclude that langflow-nightly==1.10.0.dev38 cannot be used. Add a build → test → publish lane for every package under `src/bundles/*/` in both workflows, plus a matching bundle install step in cross-platform-test.yml. `release_nightly.yml` - `build-nightly-bundles`: iterates `src/bundles/*/pyproject.toml`, builds each wheel, uploads as `dist-nightly-bundles`. - `test-cross-platform` now passes `bundles-artifact-name` so the cross-platform test installs bundles before installing main. - `publish-nightly-bundles`: publishes each bundle wheel to PyPI after cross-platform passes and lfx is already published. - `publish-nightly-main` now waits on `publish-nightly-bundles` so main reaches PyPI only after its bundle deps do. `release.yml` - New `release_bundles` input (false by default). - `validate-dependencies` rejects `release_package_main=true` without `release_bundles=true`, mirroring the existing lfx/sdk gate. - `build-bundles`, `publish-bundles` mirror the nightly jobs. - `publish-main` now needs `publish-bundles`. `cross-platform-test.yml` - New optional `bundles-artifact-name` input; download step is gated on it being set. - `build-if-needed` (workflow_dispatch path) also builds bundles and publishes the artifact, so adhoc runs stay self-contained. - Each of the four "Install main package from wheel" steps gets a preceding "Install bundle packages from wheel" step that runs only when the artifact is present, leaving non-bundle callers (e.g. pre-bundle release branches) unaffected. * ci: drop mapfile from bundle install (macOS Bash 3.2 compat) `mapfile` is a Bash 4+ builtin and the macOS runners use the system Bash (3.2). The "Install bundle packages from wheel (Unix)" step failed on `Install & Run - macos amd64 3.12` with: mapfile: command not found find: stdout: Broken pipe Replace with the `shopt -s nullglob` + glob-array pattern already used in the bundle build/publish steps, which works on Bash 3.2. All artifact wheels live at the top level of ./bundles-dist, so the simple glob is equivalent to the previous `find` (which also wasn't recursing anywhere useful). --------- Co-authored-by: Claude <noreply@anthropic.com>
1 parent e1d79bb commit 9882f9f

7 files changed

Lines changed: 329 additions & 15 deletions

File tree

‎.github/workflows/cross-platform-test.yml‎

Lines changed: 109 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -26,6 +26,10 @@ on:
2626
description: "Name of the langflow-sdk package artifact"
2727
required: false
2828
type: string
29+
bundles-artifact-name:
30+
description: "Name of the bundles package artifact (contains every src/bundles/*/ wheel)"
31+
required: false
32+
type: string
2933
pre_release:
3034
description: "Whether this is a pre-release build"
3135
required: false
@@ -42,6 +46,7 @@ jobs:
4246
main-artifact-name: ${{ steps.set-names.outputs.main-artifact-name }}
4347
lfx-artifact-name: ${{ steps.set-names.outputs.lfx-artifact-name }}
4448
sdk-artifact-name: ${{ steps.set-names.outputs.sdk-artifact-name }}
49+
bundles-artifact-name: ${{ steps.set-names.outputs.bundles-artifact-name }}
4550
steps:
4651
- name: Checkout code
4752
uses: actions/checkout@v6
@@ -74,6 +79,25 @@ jobs:
7479
uv build --wheel --out-dir dist
7580
- name: Build main package
7681
run: make build_langflow args="--wheel"
82+
- name: Build bundle wheels
83+
id: build-bundles
84+
run: |
85+
shopt -s nullglob
86+
bundles=(src/bundles/*/pyproject.toml)
87+
if [ ${#bundles[@]} -eq 0 ]; then
88+
echo "No bundles found under src/bundles/*/"
89+
echo "bundles-found=0" >> $GITHUB_OUTPUT
90+
exit 0
91+
fi
92+
mkdir -p bundles-dist
93+
BUNDLES_DIST="$PWD/bundles-dist"
94+
for bundle_pyproject in "${bundles[@]}"; do
95+
bundle_dir=$(dirname "$bundle_pyproject")
96+
echo "Building wheel for $bundle_dir"
97+
(cd "$bundle_dir" && uv build --wheel --out-dir "$BUNDLES_DIST")
98+
done
99+
echo "bundles-found=1" >> $GITHUB_OUTPUT
100+
ls -la bundles-dist/
77101
- name: Upload lfx artifact
78102
uses: actions/upload-artifact@v6
79103
with:
@@ -94,13 +118,24 @@ jobs:
94118
with:
95119
name: adhoc-dist-main
96120
path: dist
121+
- name: Upload bundles artifact
122+
if: steps.build-bundles.outputs.bundles-found == '1'
123+
uses: actions/upload-artifact@v6
124+
with:
125+
name: adhoc-dist-bundles
126+
path: bundles-dist
97127
- name: Set artifact names
98128
id: set-names
99129
run: |
100130
echo "base-artifact-name=adhoc-dist-base" >> $GITHUB_OUTPUT
101131
echo "main-artifact-name=adhoc-dist-main" >> $GITHUB_OUTPUT
102132
echo "lfx-artifact-name=adhoc-dist-lfx" >> $GITHUB_OUTPUT
103133
echo "sdk-artifact-name=adhoc-dist-sdk" >> $GITHUB_OUTPUT
134+
if [ "${{ steps.build-bundles.outputs.bundles-found }}" = "1" ]; then
135+
echo "bundles-artifact-name=adhoc-dist-bundles" >> $GITHUB_OUTPUT
136+
else
137+
echo "bundles-artifact-name=" >> $GITHUB_OUTPUT
138+
fi
104139
105140
test-installation-stable:
106141
name: Install & Run - ${{ matrix.os }} ${{ matrix.arch }} ${{ matrix.python-version }}
@@ -215,6 +250,13 @@ jobs:
215250
name: ${{ inputs.main-artifact-name || needs.build-if-needed.outputs.main-artifact-name || 'adhoc-dist-main' }}
216251
path: ./main-dist
217252

253+
- name: Download bundles package artifact
254+
if: steps.install-method.outputs.method == 'wheel' && (inputs.bundles-artifact-name != '' || needs.build-if-needed.outputs.bundles-artifact-name != '')
255+
uses: actions/download-artifact@v7
256+
with:
257+
name: ${{ inputs.bundles-artifact-name || needs.build-if-needed.outputs.bundles-artifact-name }}
258+
path: ./bundles-dist
259+
218260
- name: Create fresh virtual environment
219261
run: |
220262
uv venv test-env --seed
@@ -278,6 +320,21 @@ jobs:
278320
fi
279321
shell: bash
280322

323+
- name: Install bundle packages from wheel (Windows)
324+
if: steps.install-method.outputs.method == 'wheel' && matrix.os == 'windows' && (inputs.bundles-artifact-name != '' || needs.build-if-needed.outputs.bundles-artifact-name != '')
325+
run: |
326+
ls -la ./bundles-dist/
327+
shopt -s nullglob
328+
BUNDLE_WHEELS=(./bundles-dist/*.whl)
329+
if [ ${#BUNDLE_WHEELS[@]} -gt 0 ]; then
330+
printf '%s\n' "${BUNDLE_WHEELS[@]}"
331+
uv pip install --prerelease=allow --python ./test-env/Scripts/python.exe "${BUNDLE_WHEELS[@]}"
332+
else
333+
echo "No bundle wheels found in ./bundles-dist/"
334+
exit 1
335+
fi
336+
shell: bash
337+
281338
- name: Install main package from wheel (Windows)
282339
if: steps.install-method.outputs.method == 'wheel' && matrix.os == 'windows'
283340
run: |
@@ -349,6 +406,21 @@ jobs:
349406
fi
350407
shell: bash
351408

409+
- name: Install bundle packages from wheel (Unix)
410+
if: steps.install-method.outputs.method == 'wheel' && matrix.os != 'windows' && (inputs.bundles-artifact-name != '' || needs.build-if-needed.outputs.bundles-artifact-name != '')
411+
run: |
412+
ls -la ./bundles-dist/
413+
shopt -s nullglob
414+
BUNDLE_WHEELS=(./bundles-dist/*.whl)
415+
if [ ${#BUNDLE_WHEELS[@]} -gt 0 ]; then
416+
printf '%s\n' "${BUNDLE_WHEELS[@]}"
417+
uv pip install --prerelease=allow --python ./test-env/bin/python "${BUNDLE_WHEELS[@]}"
418+
else
419+
echo "No bundle wheels found in ./bundles-dist/"
420+
exit 1
421+
fi
422+
shell: bash
423+
352424
- name: Install main package from wheel (Unix)
353425
if: steps.install-method.outputs.method == 'wheel' && matrix.os != 'windows'
354426
run: |
@@ -579,6 +651,13 @@ jobs:
579651
name: ${{ inputs.main-artifact-name || needs.build-if-needed.outputs.main-artifact-name || 'adhoc-dist-main' }}
580652
path: ./main-dist
581653

654+
- name: Download bundles package artifact
655+
if: steps.install-method.outputs.method == 'wheel' && (inputs.bundles-artifact-name != '' || needs.build-if-needed.outputs.bundles-artifact-name != '')
656+
uses: actions/download-artifact@v7
657+
with:
658+
name: ${{ inputs.bundles-artifact-name || needs.build-if-needed.outputs.bundles-artifact-name }}
659+
path: ./bundles-dist
660+
582661
- name: Create fresh virtual environment
583662
run: |
584663
uv venv test-env --seed
@@ -671,6 +750,21 @@ jobs:
671750
fi
672751
shell: bash
673752

753+
- name: Install bundle packages from wheel (Windows)
754+
if: steps.install-method.outputs.method == 'wheel' && matrix.os == 'windows' && (inputs.bundles-artifact-name != '' || needs.build-if-needed.outputs.bundles-artifact-name != '')
755+
run: |
756+
ls -la ./bundles-dist/
757+
shopt -s nullglob
758+
BUNDLE_WHEELS=(./bundles-dist/*.whl)
759+
if [ ${#BUNDLE_WHEELS[@]} -gt 0 ]; then
760+
printf '%s\n' "${BUNDLE_WHEELS[@]}"
761+
uv pip install --prerelease=allow --python ./test-env/Scripts/python.exe "${BUNDLE_WHEELS[@]}"
762+
else
763+
echo "No bundle wheels found in ./bundles-dist/"
764+
exit 1
765+
fi
766+
shell: bash
767+
674768
- name: Install main package from wheel (Windows)
675769
if: steps.install-method.outputs.method == 'wheel' && matrix.os == 'windows'
676770
run: |
@@ -713,6 +807,21 @@ jobs:
713807
fi
714808
shell: bash
715809

810+
- name: Install bundle packages from wheel (Unix)
811+
if: steps.install-method.outputs.method == 'wheel' && matrix.os != 'windows' && (inputs.bundles-artifact-name != '' || needs.build-if-needed.outputs.bundles-artifact-name != '')
812+
run: |
813+
ls -la ./bundles-dist/
814+
shopt -s nullglob
815+
BUNDLE_WHEELS=(./bundles-dist/*.whl)
816+
if [ ${#BUNDLE_WHEELS[@]} -gt 0 ]; then
817+
printf '%s\n' "${BUNDLE_WHEELS[@]}"
818+
uv pip install --prerelease=allow --python ./test-env/bin/python "${BUNDLE_WHEELS[@]}"
819+
else
820+
echo "No bundle wheels found in ./bundles-dist/"
821+
exit 1
822+
fi
823+
shell: bash
824+
716825
- name: Install main package from wheel (Unix)
717826
if: steps.install-method.outputs.method == 'wheel' && matrix.os != 'windows'
718827
run: |

‎.github/workflows/docker-build-v2.yml‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -124,7 +124,7 @@ jobs:
124124
- name: Determine version
125125
id: version
126126
run: |
127-
version=$(uv tree 2>/dev/null | grep '^langflow' | grep -v '^langflow-base' | grep -v '^langflow-sdk' | cut -d' ' -f2 | sed 's/^v//')
127+
version=$(uv tree 2>/dev/null | grep -E '^langflow(-nightly)?[[:space:]]' | cut -d' ' -f2 | sed 's/^v//')
128128
echo "Main version from pyproject.toml: $version"
129129
130130
if [ ${{inputs.pre_release}} == "true" ]; then
@@ -604,7 +604,7 @@ jobs:
604604
if [[ "${{ inputs.release_type }}" == "base" ]]; then
605605
version=$(uv tree 2>/dev/null | grep 'langflow-base' | awk '{print $3}' | sed 's/^v//' | head -n 1)
606606
else
607-
version=$(uv tree 2>/dev/null | grep '^langflow' | grep -v '^langflow-base' | grep -v '^langflow-sdk' | cut -d' ' -f2 | sed 's/^v//')
607+
version=$(uv tree 2>/dev/null | grep -E '^langflow(-nightly)?[[:space:]]' | cut -d' ' -f2 | sed 's/^v//')
608608
fi
609609
echo "Using version: $version"
610610
echo version=$version >> $GITHUB_OUTPUT

‎.github/workflows/docker-build.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -150,7 +150,7 @@ jobs:
150150
if: ${{ inputs.main_version == '' && (inputs.release_type == 'main' || inputs.release_type == 'main-ep' || inputs.release_type == 'nightly-main' || inputs.release_type == 'main-all' || inputs.release_type == 'nightly-main-all') }}
151151
id: get-version-main
152152
run: |
153-
version=$(uv tree | grep 'langflow' | grep -v 'langflow-base' | grep -v 'langflow-sdk' | awk '{print $2}' | sed 's/^v//')
153+
version=$(uv tree | grep -E '^langflow(-nightly)?[[:space:]]' | awk '{print $2}' | sed 's/^v//')
154154
echo version=$version
155155
echo version=$version >> $GITHUB_OUTPUT
156156
setup:

‎.github/workflows/docker-nightly-build.yml‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -168,7 +168,7 @@ jobs:
168168
id: version
169169
run: |
170170
echo "Extracting main version from pyproject.toml"
171-
version=$(uv tree 2>/dev/null | grep '^langflow' | grep -v '^langflow-base' | grep -v '^langflow-sdk' | cut -d' ' -f2 | sed 's/^v//')
171+
version=$(uv tree 2>/dev/null | grep -E '^langflow(-nightly)?[[:space:]]' | cut -d' ' -f2 | sed 's/^v//')
172172
173173
# Verify nightly tag format
174174
if [[ ! "$version" =~ ^[0-9]+\.[0-9]+\.[0-9]+\.dev[0-9]+$ ]]; then
@@ -263,7 +263,7 @@ jobs:
263263
id: version
264264
run: |
265265
echo "Extracting main version from pyproject.toml"
266-
version=$(uv tree 2>/dev/null | grep '^langflow' | grep -v '^langflow-base' | grep -v '^langflow-sdk' | cut -d' ' -f2 | sed 's/^v//')
266+
version=$(uv tree 2>/dev/null | grep -E '^langflow(-nightly)?[[:space:]]' | cut -d' ' -f2 | sed 's/^v//')
267267
268268
# Verify nightly tag format
269269
if [[ ! "$version" =~ ^[0-9]+\.[0-9]+\.[0-9]+\.dev[0-9]+$ ]]; then
@@ -353,7 +353,7 @@ jobs:
353353
if [[ "${{ inputs.release_type }}" == "nightly-base" ]]; then
354354
version=$(uv tree 2>/dev/null | grep 'langflow-base' | awk '{print $3}' | sed 's/^v//' | head -n 1)
355355
else
356-
version=$(uv tree 2>/dev/null | grep '^langflow' | grep -v '^langflow-base' | grep -v '^langflow-sdk' | cut -d' ' -f2 | sed 's/^v//')
356+
version=$(uv tree 2>/dev/null | grep -E '^langflow(-nightly)?[[:space:]]' | cut -d' ' -f2 | sed 's/^v//')
357357
fi
358358
echo "Using version: $version"
359359
echo version=$version >> $GITHUB_OUTPUT

‎.github/workflows/python_test.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -93,7 +93,7 @@ jobs:
9393
- name: Run unit tests
9494
uses: nick-fields/retry@v3
9595
with:
96-
timeout_minutes: 30
96+
timeout_minutes: 40
9797
max_attempts: 2
9898
command: make unit_tests args="-x -vv --splits ${{ matrix.splitCount }} --group ${{ matrix.group }} --reruns 5 --cov --cov-config=src/backend/.coveragerc --cov-report=xml --cov-report=html"
9999

0 commit comments

Comments
 (0)