WIP: Bazel all the things#1591
Conversation
✅ Deploy Preview for aya-rs-docs ready!Built without sensitive environment variables
To edit notification comments on pull requests, go to your Netlify project configuration. |
|
|
||
| aya_rust_crate( | ||
| name = "aya-ebpf-macros", | ||
| proc_macro = True, |
There was a problem hiding this comment.
right now there's a bit of a "sniff for ebpf in the name" logic in the macro. should we just do a ebpf = True macro param instead?
| ], | ||
| ) | ||
| + | ||
| +source_stdlib( |
There was a problem hiding this comment.
todo: upstream to rules_rs in some form
| out = ctx.outputs.initrd | ||
| manifest = ctx.actions.declare_file(ctx.label.name + ".cpio.list") | ||
|
|
||
| lines = [ |
There was a problem hiding this comment.
todo: upstream initramfs builder to linux.bzl
| # --- end runfiles.bash initialization v3 --- | ||
| set -euo pipefail | ||
|
|
||
| runfile() {{ |
There was a problem hiding this comment.
todo: can we just use rootpath instead of runfiles for tests?
| use anyhow::Context as _; | ||
|
|
||
| #[cfg(target_os = "linux")] | ||
| fn set_loopback_up() -> anyhow::Result<()> { |
There was a problem hiding this comment.
i believe the previous kernel/distro setup we were using gave us the loopback "for free", now we do it manually
There was a problem hiding this comment.
i think it does not - we shouldn't depend on loopback being up, can you elaborate on what happens if this file is reverted?
| target_mode: Some(RXRXRX), | ||
| }, | ||
| Mount { | ||
| source: "debugfs", |
There was a problem hiding this comment.
i'm not sure why these changes are needed, perhaps another difference with the previous debian setup
| ) | ||
|
|
||
| def _c_bpf_compile(ctx, cc_toolchain, feature_configuration, name, defines = []): | ||
| _, compilation_outputs = cc_common.compile( |
There was a problem hiding this comment.
i was pretty happy with how this bit turned out :)
vadorovsky
left a comment
vadorovsky
left a comment
There was a problem hiding this comment.
Would be nice to have some explanation why do we want this and who would use it.
| @@ -1,5 +1,6 @@ | |||
| Cargo.lock | |||
There was a problem hiding this comment.
I overall agree with this (Aya is a set of library crates), but I think that deserves a separate commit with an explanation.
Will let @tamird take this one since he asked for help with this work :) But at a high level - typically projects use Bazel when they want very reproducible, very fast builds that can parallelize their actions across remote executors. For example I can run the e2e tests in around 4 minutes, which involves building all crates and linux kernel from source, spawning the VMs, and running the tests. Looks like it takes closer to 20 minutes in Github actions :) |
Got it, sounds cool. Could you please list some of these reasons in the pull request and commit message? I know nothing about Bazel, but I noticed that there is https://github.com/hermeticbuild/hermetic-llvm that caught my interest. We are doing some heavy cross-compilation stuff in https://github.com/aya-rs/bpf-linker (we build LLVM for different architectures and with musl), do you think it could benefit from the hermetic LLVM builds? |
Sure, will do before landing. Both the message and PR are still very WIP!
Yes! I'm actually one of the maintainers of hermetic-llvm project :) I only looked at bpf-linker briefly but am fairly confident if we just need to build some rust code and link against LLVM with some patches, hermetic-llvm likely should work out of the box and give ability to compile for all targets (including missing ones like windows!). |
tamird
left a comment
tamird
left a comment
There was a problem hiding this comment.
Very nice work here. What do I need to do to get this running in CI so we can look at the results?
@tamird reviewed 6 files and made 10 comments.
Reviewable status: 6 of 39 files reviewed, 15 unresolved discussions (waiting on dzbarsky and vadorovsky).
BUILD.bazel at r1 (raw file):
What does this empty file do?
Cargo.lock line 1 at r1 (raw file):
# This file is automatically @generated by Cargo.
this file needs a mention in the commit message and a CI staleness check
.bazelrc line 1 at r1 (raw file):
common --enable_platform_specific_config
this file needs some comments :)
test/kernel/README.md line 1 at r1 (raw file):
# Bazel Kernel Inputs
very nice. note that the current setup is undergoing some change in #1588 so it would be better to rewrite this without referencing the existing machinery.
test/kernel/aya_aarch64.config line 1 at r1 (raw file):
# Minimal allnoconfig-based aarch64 kernel for Aya integration VMs.
for both of these files it would be great to encode which configs are direct dependencies and which are transitive (or otherwise have a reproducible way to regenerate these)
MODULE.bazel.lock line 1 at r1 (raw file):
{
needs CI staleness check
MODULE.bazel line 1 at r1 (raw file):
module(
we need some maintenance instructions for this file + inline commentary for things that aren't obvious like the overrides
|
|
||
| aya_rust_crate( | ||
| name = "aya-ebpf-macros", | ||
| proc_macro = True, |
| use anyhow::Context as _; | ||
|
|
||
| #[cfg(target_os = "linux")] | ||
| fn set_loopback_up() -> anyhow::Result<()> { |
There was a problem hiding this comment.
i think it does not - we shouldn't depend on loopback being up, can you elaborate on what happens if this file is reverted?
3 participants
This PR converts the repo to building with Bazel. It does not yet add CI jobs. This is WIP for collaborating with @tamird .
Added/updated tests?
Tests added, to be enabled in CI
Checklist
cargo +nightly fmt.You can find failing lints with
cargo xtask clippy.cargo test.cargo xtask public-api --bless.(Optional) What GIF best describes this PR or how it makes you feel?
This change is