Cache workspaces [ros2]#50
Conversation
Codecov Report
@@ Coverage Diff @@
## ros2 #50 +/- ##
=======================================
Coverage 71.91% 71.91%
=======================================
Files 5 5
Lines 242 242
=======================================
Hits 174 174
Misses 68 68 Continue to review full report at Codecov.
|
rhaschke
left a comment
rhaschke
left a comment
There was a problem hiding this comment.
I'm not yet sure about the workspace caching. Does it work as expected? As far as I can see:
- First you restore the
upstream_wscache (if available). I hope this also restore file timestamps. - Next,
industrial_ciupdates fromUPSTREAM_WORKSPACE. Hopefully, this doesn't touch files if they didn't change. Otherwise the cache would be useless.
I believe it does work. It seems to reduce the time to test this package by about half. |
tylerjw
force-pushed
the
cache-workspaces
branch
from
9118451 to
0c428c2
Compare
|
For context, before this caching a CI run of this on ros2 takes about 11min, with this it takes about 4.5min. |
|
For what it's worth, I started this as a way to quickly iterate on this idea as a way to speed up builds for moveit/moveit2. I'll clean this up when I'm satisfied with it and make it a PR here, although I'm not sure if the time difference here really matters. |
Sure. I was assuming that you take this repo as a quick-turnaround sandbox for your CI experiments. |
rhaschke
left a comment
rhaschke
left a comment
There was a problem hiding this comment.
Looks good from my side. I'm not sure you are done with experimenting though.
Please merge yourself if so.
|
I created this: moveit/moveit#2559 to show my current progress on moveit/moveit2. The newest issue I'm encountering is with running out of disk space in codecov test runs. Unfortunately that's not something I can do with this small repo. I'll clean this up and merge it. The clang-tidy script solution can nicely be tested locally so that one should be easy to do without having to use CI runs in the loop. |
tylerjw
force-pushed
the
cache-workspaces
branch
4 times, most recently
from
d9c081f to
3468305
Compare
tylerjw
force-pushed
the
cache-workspaces
branch
from
3468305 to
b6172b8
Compare
tylerjw
force-pushed
the
cache-workspaces
branch
from
e6b811d to
ba63866
Compare
rhaschke
left a comment
rhaschke
left a comment
There was a problem hiding this comment.
Some minor remarks, see below.
| - { | ||
| ROS_DISTRO: foxy, ROS_REPO: main, CCOV_UPLOAD: true, | ||
| CMAKE_ARGS: "-DCMAKE_BUILD_TYPE=Debug -DCMAKE_C_FLAGS='--coverage' -DCMAKE_CXX_FLAGS='--coverage'", | ||
| AFTER_RUN_TARGET_TEST: './.ci.prepare_codecov', | ||
| ADDITIONAL_DEBS: 'curl lcov grep' | ||
| IMAGE: 'foxy-ci', | ||
| CCOV: true, | ||
| AFTER_RUN_TARGET_TEST: './.ci.prepare_codecov' | ||
| } | ||
| - { | ||
| IMAGE: 'foxy-ci-testing', | ||
| } | ||
| - {ROS_DISTRO: foxy, ROS_REPO: testing, CCOV_UPLOAD: false} |
There was a problem hiding this comment.
Please don't mix in json syntax here (even though yaml is a superset of json). This reads much better (and is consistent) with the formatting of the remaining file:
env:
- ROS_DISTRO: ...
ROS_REPO: ...
- IMAGE: ...
| uses: pat-s/always-upload-cache@v2.1.3 | ||
| with: | ||
| path: ${{ env.BASEDIR }}/target_ws | ||
| key: target_ws-${{ env.CACHE_PREFIX }}-${{ hashFiles('**/CMakeLists.txt') }}-${{ hashFiles('**/package.xml') }}-${{ github.run_id }} |
There was a problem hiding this comment.
You can hash multiple file patterns at once: https://docs.github.com/en/actions/reference/context-and-expression-syntax-for-github-actions#example-with-multiple-patterns
${{ hashFiles('**/CMakeLists.txt', '**/package.xml') }}
tylerjw
force-pushed
the
cache-workspaces
branch
from
3897255 to
c48f6fc
Compare
tylerjw
force-pushed
the
cache-workspaces
branch
from
917bb09 to
8783a6f
Compare
2 participants
This is an attempt to use the caching features of GitHub Actions to further speed up builds by caching entire workspaces.