Skip to content

Commit f7eee33

Browse files
committed
CI: Consolidate linting workflows into one
This patch consolidates the - `check-mem-files` - `dco-check` - `formatting-check` - `license-check` - `pr-title-check` - `pylint` workflows into a single `lint` workflow. Previously, each workflow had only one or two jobs, which were all meant to be run in parallel on each pull request. By including each of these jobs in a single workflow, they still run in parallel, but they do not clog up the Actions views with individual workflow runs. In addition, this patch takes the step of running these jobs on `ubuntu-slim` runners, which are containerized runners rather than VMs, which start more quickly. These runners are only 1 vCPU and 5 GB RAM, and jobs running on them can take a maximum of 15 minutes before they fail, which is well-suited for these lightweight checks. There is one remaining check that could reasonably go in this workflow, the `pytest-coverage` workflow. However, unlike all the workflows that were consolidated here, this one also runs on a cron job. It’s not clear to me whether it should or not, but this patch does not take the step of consolidating this workflow in case we do need this. Signed-off-by: Patrick M. Niedzielski <pniedzielski@bloomberg.net>
1 parent 85076ea commit f7eee33

7 files changed

Lines changed: 150 additions & 205 deletions

File tree

.github/workflows/check-mem-files.yaml

Lines changed: 0 additions & 47 deletions
This file was deleted.

.github/workflows/dco-check.yaml

Lines changed: 0 additions & 25 deletions
This file was deleted.

.github/workflows/formatting-check.yaml

Lines changed: 0 additions & 44 deletions
This file was deleted.

.github/workflows/license-check.yaml

Lines changed: 0 additions & 23 deletions
This file was deleted.

.github/workflows/lint.yaml

Lines changed: 150 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,150 @@
1+
name: Lint
2+
3+
on:
4+
pull_request:
5+
types:
6+
- "opened"
7+
- "reopened"
8+
- "synchronize"
9+
- "labeled"
10+
- "unlabeled"
11+
12+
jobs:
13+
commits_check_job:
14+
runs-on: ubuntu-slim
15+
name: DCO Check
16+
steps:
17+
- name: Get PR Commits
18+
id: "get-pr-commits"
19+
uses: tim-actions/get-pr-commits@198af03565609bb4ed924d1260247b4881f09e7d # 26 Feb 2024
20+
with:
21+
token: ${{ secrets.GITHUB_TOKEN }}
22+
- name: DCO Check
23+
uses: tim-actions/dco@f2279e6e62d5a7d9115b0cb8e837b777b1b02e21 # 10 Jun 2021
24+
with:
25+
commits: ${{ steps.get-pr-commits.outputs.commits }}
26+
27+
license_check:
28+
runs-on: ubuntu-slim
29+
name: License Check
30+
steps:
31+
- uses: actions/checkout@v4
32+
with:
33+
fetch-depth: 0
34+
- name: license check
35+
run: |
36+
python3 ${{ github.workspace }}/.github/workflows/ext/check_license.py --path=${{ github.workspace }} | tee missing_licenses.txt
37+
if [ -s missing_licenses.txt ]; then exit 1; fi
38+
39+
pr_title_check:
40+
runs-on: ubuntu-slim
41+
name: PR Title Check
42+
steps:
43+
- uses: actions/checkout@v4
44+
with:
45+
fetch-depth: 0
46+
- name: pr title check
47+
env:
48+
PR_TITLE: ${{ github.event.pull_request.title }}
49+
run: |
50+
python3 ${{ github.workspace }}/.github/workflows/ext/check_pr_title.py
51+
52+
check_mem_sorting:
53+
runs-on: ubuntu-slim
54+
name: Check .mem files sorting
55+
steps:
56+
- name: Checkout repository
57+
uses: actions/checkout@v4
58+
59+
- name: Check alphabetical order in .mem files
60+
run: |
61+
#!/usr/bin/env bash
62+
set -euo pipefail
63+
64+
echo "Checking all .mem files in src/ for sorted order..."
65+
66+
# Find all .mem files recursively under src/
67+
files=$(find src -type f -name "*.mem")
68+
69+
# Track whether we find any unsorted files
70+
unsorted=0
71+
72+
for f in $files; do
73+
# Compare file with its sorted version
74+
if ! diff -q <(sort "$f") "$f" > /dev/null; then
75+
echo "❌ File not sorted alphabetically: $f"
76+
echo " To fix, run: sort -o $f $f"
77+
unsorted=1
78+
fi
79+
done
80+
81+
if [ "$unsorted" -eq 1 ]; then
82+
echo
83+
echo "Some .mem files are not sorted alphabetically."
84+
exit 1
85+
fi
86+
87+
echo "✅ All .mem files are sorted."
88+
89+
cpp_formatting_check:
90+
name: C++ Formatting Check
91+
runs-on: ubuntu-24.04
92+
steps:
93+
- uses: actions/checkout@v4
94+
with:
95+
fetch-depth: 0
96+
- name: clang-format style check
97+
run: |
98+
git clang-format-18 --diff -q origin/main | tee format_diff.txt
99+
if [ -s format_diff.txt ]; then exit 1; fi
100+
101+
python_formatting_check:
102+
# There might be differences how the formatter refactors the code locally
103+
# and in the CI. If there is a problem with CI formatter mismatch, consider
104+
# wrapping a troublesome python code with comments:
105+
# "# fmt: off"
106+
# "<Python code we don't want to format>"
107+
# "# fmt: on"
108+
name: Python Formatting Check
109+
runs-on: ubuntu-slim
110+
steps:
111+
- uses: actions/checkout@v4
112+
with:
113+
fetch-depth: 0
114+
- uses: actions/setup-python@v5
115+
- name: ruff style check
116+
run: |
117+
pip3 install ruff
118+
ruff format .
119+
git diff -q | tee format_diff.txt
120+
if [ -s format_diff.txt ]; then exit 1; fi
121+
122+
python_linter_check:
123+
name: Python Linter Check
124+
runs-on: ubuntu-slim
125+
steps:
126+
- uses: actions/checkout@v4
127+
with:
128+
fetch-depth: 0
129+
- uses: actions/setup-python@v5
130+
with:
131+
python-version: '3.12.4'
132+
- name: Install dependencies
133+
run: |
134+
pip3 install -r ${{ github.workspace }}/src/python/requirements.txt
135+
pip3 install pylint lint-diffs
136+
ln -s ${{ github.workspace }}/src/python/pylintrc ${{ github.workspace }}/.pylintrc
137+
echo [pylint] > ${{ github.workspace }}/.lint-diffs
138+
echo extensions=.py >> ${{ github.workspace }}/.lint-diffs
139+
echo [clang-tidy] >> ${{ github.workspace }}/.lint-diffs
140+
echo extensions= >> ${{ github.workspace }}/.lint-diffs
141+
echo [rubocop] >> ${{ github.workspace }}/.lint-diffs
142+
echo extensions= >> ${{ github.workspace }}/.lint-diffs
143+
- name: Check the whole repo with pylint
144+
run: |
145+
export PYTHONPATH=${{ github.workspace }}/src/python:$PYTHONPATH
146+
pylint -j 8 src || true
147+
- name: Check the PR with pylint
148+
run: |
149+
export PYTHONPATH=${{ github.workspace }}/src/python:$PYTHONPATH
150+
git diff -U0 origin/main | lint-diffs

.github/workflows/pr-title-check.yaml

Lines changed: 0 additions & 25 deletions
This file was deleted.

.github/workflows/pylint.yaml

Lines changed: 0 additions & 41 deletions
This file was deleted.

0 commit comments

Comments
 (0)