feat(proposal draft, submodule): manage repositories as submodules instead of vcstool.

[paulsohn]

Description

A design proposal of using git submodules to manage PRs instead of .repos files. Inspired from #6587 .

Mapping Rules

• repository aaa/bbb in xxx.repos -> submodule src/XXX/aaa/bbb
• If version is a branch, then submodule.<submodule>.branch = <version>.
  • exceptions are the nightly versions: submodule.<submodule>.branch is unset for nightly-able ones. update: branch is set to indicate nightly-able entries.
• If version is a tag or a commit hash, then submodule.<submodule>.update = none so that git submodule update --remote ignores updating. Otherwise, submodule.<submodule>.update is unset (which defaults to checkout option)

Mapping is done by copilot-generated python scripts:

`add_submodules.py`

#!/usr/bin/env python3
"""
Add git submodules listed in a .repos-like YAML file.

Usage:
  python3 ~/autoware/scripts/add_submodules.py <repos.yaml> <target_path>

Behavior:
  - Reads `repositories` mapping from the YAML.
  - For each key (e.g. "core/autoware_msgs"), reads `url` and `version`.
  - Adds a git submodule at <target_path>/<key> using the given url.
  - Immediately checks out the given version.
    - If the version resolves to a remote branch, the submodule is checked out to that branch
      and the branch is recorded in `.gitmodules` and repo config (equivalent to
      `git submodule set-branch`).
    - If the version is a tag or commit hash (or otherwise not a branch), the submodule
      is checked out to that tag/commit (detached) and `update = none` is set in `.gitmodules`.

Requires: PyYAML (pip install -r ~/autoware/scripts/requirements.txt)

This script does not create a final commit. It stages `.gitmodules` and the submodule
gitlink (via `git add <path>`), but leaves committing to the user.
"""

import argparse
import os
import subprocess
import sys
from typing import Optional

try:
    import yaml
except Exception:
    print("PyYAML is required. Install with: pip install pyyaml", file=sys.stderr)
    sys.exit(2)


def run(cmd, check=True, capture_output=False, **kwargs):
    # Print the command for transparency and debugging
    print("+", " ".join(cmd))
    return subprocess.run(cmd, check=check, capture_output=capture_output, text=True, **kwargs)


def is_remote_branch(url: str, version: str) -> bool:
    # Check if remote has a heads ref with this name
    try:
        r = subprocess.run(["git", "ls-remote", "--heads", url, version], check=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True)
        return bool(r.stdout.strip())
    except subprocess.CalledProcessError:
        return False


def is_remote_tag(url: str, version: str) -> bool:
    try:
        r = subprocess.run(["git", "ls-remote", "--tags", url, version], check=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True)
        return bool(r.stdout.strip())
    except subprocess.CalledProcessError:
        return False


def ensure_parent_dir(path: str):
    parent = os.path.dirname(path)
    if parent and not os.path.exists(parent):
        os.makedirs(parent, exist_ok=True)


def configure_gitmodules(submodule_name: str, branch: Optional[str] = None, update_none: bool = False):
    # Update .gitmodules and repository config to reflect branch/update settings.
    if branch:
        run(["git", "config", "-f", ".gitmodules", f"submodule.{submodule_name}.branch", branch])
        # Also set in the repo config similar to `git submodule set-branch`
        run(["git", "config", f"submodule.{submodule_name}.branch", branch])
    if update_none:
        run(["git", "config", "-f", ".gitmodules", f"submodule.{submodule_name}.update", "none"])
        run(["git", "config", f"submodule.{submodule_name}.update", "none"])


def add_submodule(name: str, url: str, version: str, target_root: str):
    path = os.path.join(target_root, name)
    print(f"\nProcessing submodule: {name}\n  url: {url}\n  version: {version}\n  path: {path}")

    ensure_parent_dir(path)

    # If path exists and is a git repo, skip adding and just try to configure
    if os.path.exists(path) and os.path.isdir(path) and os.path.exists(os.path.join(path, ".git")):
        print(f"Path {path} already exists and looks like a git repo. Skipping `git submodule add`.")
    else:
        # git submodule add
        try:
            run(["git", "submodule", "add", url, path])
        except subprocess.CalledProcessError as e:
            print(f"git submodule add failed: {e}. Trying to continue if possible.")

    # Try to fetch tags and refs so we can checkout the requested ref
    try:
        run(["git", "-C", path, "fetch", "--all", "--tags"])
    except subprocess.CalledProcessError as e:
        print(f"Warning: fetch failed in submodule {name}: {e}")

    # Determine ref type
    branch = None
    if is_remote_branch(url, version):
        branch = version
        print(f"Detected remote branch: {version}")
    elif is_remote_tag(url, version):
        print(f"Detected remote tag: {version}")
    else:
        # Could be commit hash. We'll treat as tag/commit (update = none)
        print(f"Treating version as tag/commit (not a remote branch): {version}")

    if branch:
        # Checkout branch from origin and set branch in .gitmodules and repo config
        try:
            run(["git", "-C", path, "fetch", "origin", branch])
        except subprocess.CalledProcessError:
            pass

        try:
            # Create or reset local branch to track origin/<branch>
            run(["git", "-C", path, "checkout", "-B", branch, f"origin/{branch}"])
        except subprocess.CalledProcessError:
            # Fallback: try to checkout the branch name directly
            run(["git", "-C", path, "checkout", branch], check=False)

        configure_gitmodules(path, branch=branch)
    else:
        # Checkout tag or commit (detached). Set update = none in .gitmodules
        try:
            run(["git", "-C", path, "checkout", version])
        except subprocess.CalledProcessError:
            # If checkout by name fails, try a detached checkout from FETCH_HEAD
            run(["git", "-C", path, "checkout", "--detach", "FETCH_HEAD"], check=False)

        configure_gitmodules(path, update_none=True)

    # Stage .gitmodules and submodule gitlink so the user can commit if desired
    try:
        run(["git", "add", ".gitmodules"])  # no-op if no changes
        run(["git", "add", path])
    except subprocess.CalledProcessError as e:
        print(f"Warning: git add failed: {e}")


def main(argv=None):
    p = argparse.ArgumentParser(description="Add submodules from a repos YAML file")
    p.add_argument("repos_yaml", help="Path to YAML file containing repositories mapping")
    p.add_argument("target_path", help="Target path under which to create submodule directories (prefix for keys)")
    args = p.parse_args(argv)

    if not os.path.exists(args.repos_yaml):
        print(f"YAML file not found: {args.repos_yaml}", file=sys.stderr)
        sys.exit(1)

    with open(args.repos_yaml, "r") as f:
        data = yaml.safe_load(f)

    repos = data.get("repositories") or {}
    if not repos:
        print("No repositories found in YAML.", file=sys.stderr)
        sys.exit(1)

    for key, meta in repos.items():
        if not isinstance(meta, dict):
            print(f"Skipping {key}: metadata is not a mapping")
            continue
        url = meta.get("url")
        version = meta.get("version")
        if not url or version is None:
            print(f"Skipping {key}: missing url or version")
            continue
        add_submodule(key, url, str(version), args.target_path)


if __name__ == "__main__":
    main()

`set_submodules_nightly.py`

#!/usr/bin/env python3
"""
Set git submodule nightly from a .repos-like YAML file.

Usage:
  python3 ~/autoware/scripts/set_submodules_nightly.py <repos.yaml> <target_path>

Behavior:
  - Reads `repositories` mapping from the YAML.
  - For each key (e.g. "core/autoware_msgs"), executes: `git submodule set-branch -d <target_path>/<key>`.

This script stages `.gitmodules` but does not create a commit.
"""

import argparse
import os
import subprocess
import sys

try:
    import yaml
except Exception:
    print("PyYAML is required. Install with: pip install pyyaml", file=sys.stderr)
    sys.exit(2)


def run(cmd, check=True, capture_output=False, **kwargs):
    # Print the command for transparency and debugging
    print("+", " ".join(cmd))
    return subprocess.run(cmd, check=check, capture_output=capture_output, text=True, **kwargs)


def set_submodule_nightly(name: str, target_root: str):
    path = os.path.join(target_root, name)
    print(f"\nSetting nightly for submodule: {name}\n  path: {path}")

    # Verify path exists
    if not os.path.exists(path):
        print(f"Warning: path {path} does not exist. Skipping.")
        return

    # Run git submodule set-branch
    try:
        run(["git", "submodule", "set-branch", "-d", path])
    except subprocess.CalledProcessError as e:
        print(f"git submodule set-branch failed: {e}")
        raise

    # Stage .gitmodules
    try:
        run(["git", "add", ".gitmodules"])
    except subprocess.CalledProcessError as e:
        print(f"Warning: git add .gitmodules failed: {e}")


def main(argv=None):
    p = argparse.ArgumentParser(description="Set submodule branches from a repos YAML file")
    p.add_argument("repos_yaml", help="Path to YAML file containing repositories mapping")
    p.add_argument("target_path", help="Target path prefix for submodule directories")
    args = p.parse_args(argv)

    if not os.path.exists(args.repos_yaml):
        print(f"YAML file not found: {args.repos_yaml}", file=sys.stderr)
        sys.exit(1)

    with open(args.repos_yaml, "r") as f:
        data = yaml.safe_load(f)

    repos = data.get("repositories") or {}
    if not repos:
        print("No repositories found in YAML.", file=sys.stderr)
        sys.exit(1)

    for key, meta in repos.items():
        if not isinstance(meta, dict):
            print(f"Skipping {key}: metadata is not a mapping")
            continue
        set_submodule_nightly(key, args.target_path)


if __name__ == "__main__":
    main()

And then I ran:

$ python3 scripts/add_submodules.py autoware.repos
$ python3 scripts/add_submodules.py extra-packages.repos
$ python3 scripts/add_submodules.py simulator.repos
$ python3 scripts/add_submodules.py tools.repos
$ python3 scripts/set_submodules_nightly.py autoware-nightly.repos
$ python3 scripts/set_submodules_nightly.py simulator-nightly.repos
$ python3 scripts/set_submodules_nightly.py tools-nightly.repos

Manipulations

More operations for this convention of git submodule are explained in my toy repository toy-advanced-submodule, made for testing vcstool-like repository management only using git submodule.

Cloning and Initialization

$ git clone https://github.com/paulsohn/autoware.git # no `--recursive`
$ cd autoware
$ git submodule update --init --checkout src/autoware # equivalent to `vcs import src < autoware.repos`

You can init other submodules or submodule groups at any time you want. Below commands work even if you have init'ed submodules partially.

Use nightly

$ git submodule update --remote # equivalent to importing all `*-nightly.repos`
$ git submodule update --remote src/autoware # equivalent to `vcs import src < autoware-nightly.repos`

Only nightly-able submodules (no update = none) are updated to the latest HEAD (or, to the latest branch HEAD).

Returning all submodules into the recorded commits

$ git submodule update --checkout
$ git submodule update --checkout src/autoware # equivalent to `vcs import src < autoware.repos` again

This can be used for:

• switching nightly to non-nightly
• checking out the superproject and syncing the submodules.

Checking the current submodule status at once

$ git submodule status

This displays commit hashes, and tags for whichever submodule that is pointing some tag.

How was this PR tested?

The building steps are same as usual. I built this on my local device and confirmed that the build passes without any problem, and build/ install/ directories look exactly the same as before.

Below is the justification of the versions I was building, and nightly demonstration.

`git submodule status`

$ git submodule status
 6c3faf2311db65dbd37a78904c9b96edad4b3bbd src/autoware/core/autoware_adapi_msgs (1.9.1)
 0e0794e034fe1b8fea6e3a0bbcb9d9b8cdba03ad src/autoware/core/autoware_cmake (1.1.0)
 19216fa7fe03b5b4669649ef5cfc3397dceb99f6 src/autoware/core/autoware_core (1.5.0)
 f5524e501775463f508cdb93fe6aec45e1fb4836 src/autoware/core/autoware_internal_msgs (1.12.0)
 5fadbb76b96083685cb36335c3c28d6554392d7e src/autoware/core/autoware_lanelet2_extension (0.10.0)
 588f00df3acca009ea709a74acd6538897eef424 src/autoware/core/autoware_msgs (1.11.0)
 bac3843c2f203feb28ff0963701b763207163ff6 src/autoware/core/autoware_rviz_plugins (0.3.0)
 10d45b89b9902382c56306cf2677b7db9c0846aa src/autoware/core/autoware_utils (1.4.2)
 4e8756ad53cc0cc53d80705e95de1d1d5d699175 src/autoware/launcher/autoware_launch (0.48.0)
 7bc17e97c462e578ddf8cfb15714c6b2be5ecfd9 src/autoware/middleware/external/agnocast (2.1.2)
 e4a491fe19694f3b72ec531c3e9997185217a61b src/autoware/sensor_component/external/nebula (v0.2.6)
 03ba094851ec90febfcfc0adb20b64b0e19df7a8 src/autoware/sensor_component/external/sensor_component_description (heads/main)
 e39a814180b03f00a5692b6951a5d4e9f0463486 src/autoware/sensor_component/ros2_socketcan (1.3.0-5-ge39a814)
 39ebd8afe1bb9760a6cd6272e428468480f6de90 src/autoware/sensor_component/transport_drivers (heads/main)
 c65fdc968b9b915069a43e87823818b4c401038e src/autoware/universe/autoware_universe (0.48.0)
 0cda477b09ed27cfc8484c792b4f5f3063cf425f src/autoware/universe/external/bevdet_vendor (heads/bevdet_vendor-ros2)
 1837689df2891f6223f07c178c21aed252566ede src/autoware/universe/external/cuda_blackboard (v0.2.0)
 c1919448336e86a8dd9c94a337032c05fcf6c381 src/autoware/universe/external/eagleye (v1.6.0-ros2-22-gc191944)
 ea36766fdc2ac8e8c8e3ac988ae69acd6d09bb30 src/autoware/universe/external/glog (v0.6.0-3-gea36766)
 07ad112b4f6b83eccd3a5f777bbe40ff01c67382 src/autoware/universe/external/llh_converter (remotes/origin/ros2)
 e235f65986890e7133f33176bc6be5c42b7f3dc7 src/autoware/universe/external/managed_transform_buffer (0.1.0)
 e2e75fc1603a9798773e467a679edf68b448e705 src/autoware/universe/external/morai_msgs (e2e75fc)
 c79e98fd5e658f4f90c06d93472faa977bc873b9 src/autoware/universe/external/muSSP (heads/tier4/main)
 eac198b55dcd052af5988f0f174902913c5f20e7 src/autoware/universe/external/negotiated (heads/master)
 d969ec699f84fad827fbadfa3001c9c657482fbe src/autoware/universe/external/pointcloud_to_laserscan (2.0.0-10-gd969ec6)
 ef094407bba4f475a8032972e0c60cbb939b51b8 src/autoware/universe/external/rtklib_ros_bridge (v0.1.0-11-gef09440)
 e3585b221df80387617e321509aa00953d0e496f src/autoware/universe/external/tier4_ad_api_adaptor (v0.40.0-27-ge3585b2)
 b9c385197baf1a2e77f088bea46e7287cd3ec367 src/autoware/universe/external/tier4_autoware_msgs (v0.40.0-20-gb9c3851)
 1a9df130b4a5d96c25019b5793abbfde42d237ac src/autoware/universe/external/trt_batched_nms (heads/main)
 fc939e9bfbf171986d2d558400a4a4fee97332c6 src/extra-packages/sensor_component/external/tamagawa_imu_driver (v0.11.0beta-9-gfc939e9)
 7da80108c801fb090e8f86df02c433f8f286fcb8 src/extra-packages/vehicle/external/pacmod_interface (heads/main)
 05bc0dde94436c0694cda925b70f6902dfb328b3 src/simulator/simulator/scenario_simulator (18.2.1)
 aa2674e1d7e15a46c64ce98339378f6be6691dfc src/tools/plotjuggler_ros (2.1.2-humble)
 84ac67f0d22caeafbabbe62ec124e72d00bc6915 src/tools/tools (0.4.0)
$ git submodule update --remote
Skipping submodule 'src/autoware/core/autoware_cmake'
Skipping submodule 'src/autoware/core/autoware_lanelet2_extension'
Skipping submodule 'src/autoware/core/autoware_msgs'
Skipping submodule 'src/autoware/middleware/external/agnocast'
Skipping submodule 'src/autoware/sensor_component/external/nebula'
Skipping submodule 'src/autoware/sensor_component/external/sensor_component_description'
Skipping submodule 'src/autoware/sensor_component/ros2_socketcan'
Skipping submodule 'src/autoware/sensor_component/transport_drivers'
Skipping submodule 'src/autoware/universe/external/cuda_blackboard'
Skipping submodule 'src/autoware/universe/external/eagleye'
Skipping submodule 'src/autoware/universe/external/glog'
Skipping submodule 'src/autoware/universe/external/llh_converter'
Skipping submodule 'src/autoware/universe/external/morai_msgs'
Skipping submodule 'src/autoware/universe/external/muSSP'
Skipping submodule 'src/autoware/universe/external/negotiated'
Skipping submodule 'src/autoware/universe/external/pointcloud_to_laserscan'
Skipping submodule 'src/autoware/universe/external/rtklib_ros_bridge'
Skipping submodule 'src/autoware/universe/external/trt_batched_nms'
Skipping submodule 'src/simulator/simulator/scenario_simulator'
Skipping submodule 'src/tools/plotjuggler_ros'
Submodule path 'src/autoware/core/autoware_core': checked out 'fc1b5255dd83fe762e1118eee88594dc3d9577cb'
Submodule path 'src/autoware/core/autoware_rviz_plugins': checked out '06a4426df4bbfe9a98e4b3240f59928f014b824e'
Submodule path 'src/autoware/core/autoware_utils': checked out '1ab84eca6b4c1eb10f54c28c98ea8f547b60495f'
Submodule path 'src/autoware/launcher/autoware_launch': checked out 'ce1b0a7cbe926c32fd0af44ad3733362bdd8db21'
Submodule path 'src/autoware/universe/autoware_universe': checked out '98c0d94ccaf5b66f490eac8d695b2de42b2447aa'
Submodule path 'src/autoware/universe/external/managed_transform_buffer': checked out '7de93a00769de0b0e3fd1b80aa5ec4e146115fe1'
Submodule path 'src/autoware/universe/external/tier4_ad_api_adaptor': checked out 'b80241598ec542621d59e0a1c7ae11f20a5181ae'
Submodule path 'src/tools/tools': checked out '2d93396275235daecac86890992e2eac651d670b'
$ git submodule status
 6c3faf2311db65dbd37a78904c9b96edad4b3bbd src/autoware/core/autoware_adapi_msgs (1.9.1)
 0e0794e034fe1b8fea6e3a0bbcb9d9b8cdba03ad src/autoware/core/autoware_cmake (1.1.0)
+fc1b5255dd83fe762e1118eee88594dc3d9577cb src/autoware/core/autoware_core (1.0.0-290-gfc1b5255)
 f5524e501775463f508cdb93fe6aec45e1fb4836 src/autoware/core/autoware_internal_msgs (1.12.0)
 5fadbb76b96083685cb36335c3c28d6554392d7e src/autoware/core/autoware_lanelet2_extension (0.10.0)
 588f00df3acca009ea709a74acd6538897eef424 src/autoware/core/autoware_msgs (1.11.0)
+06a4426df4bbfe9a98e4b3240f59928f014b824e src/autoware/core/autoware_rviz_plugins (0.3.0-1-g06a4426)
+1ab84eca6b4c1eb10f54c28c98ea8f547b60495f src/autoware/core/autoware_utils (1.4.2-9-g1ab84ec)
+ce1b0a7cbe926c32fd0af44ad3733362bdd8db21 src/autoware/launcher/autoware_launch (heads/main)
 7bc17e97c462e578ddf8cfb15714c6b2be5ecfd9 src/autoware/middleware/external/agnocast (2.1.2)
 e4a491fe19694f3b72ec531c3e9997185217a61b src/autoware/sensor_component/external/nebula (v0.2.6)
 03ba094851ec90febfcfc0adb20b64b0e19df7a8 src/autoware/sensor_component/external/sensor_component_description (heads/main)
 e39a814180b03f00a5692b6951a5d4e9f0463486 src/autoware/sensor_component/ros2_socketcan (1.3.0-5-ge39a814)
 39ebd8afe1bb9760a6cd6272e428468480f6de90 src/autoware/sensor_component/transport_drivers (heads/main)
+98c0d94ccaf5b66f490eac8d695b2de42b2447aa src/autoware/universe/autoware_universe (heads/main)
 0cda477b09ed27cfc8484c792b4f5f3063cf425f src/autoware/universe/external/bevdet_vendor (heads/bevdet_vendor-ros2)
 1837689df2891f6223f07c178c21aed252566ede src/autoware/universe/external/cuda_blackboard (v0.2.0)
 c1919448336e86a8dd9c94a337032c05fcf6c381 src/autoware/universe/external/eagleye (v1.6.0-ros2-22-gc191944)
 ea36766fdc2ac8e8c8e3ac988ae69acd6d09bb30 src/autoware/universe/external/glog (v0.6.0-3-gea36766)
 07ad112b4f6b83eccd3a5f777bbe40ff01c67382 src/autoware/universe/external/llh_converter (remotes/origin/ros2)
+7de93a00769de0b0e3fd1b80aa5ec4e146115fe1 src/autoware/universe/external/managed_transform_buffer (0.1.0-2-g7de93a0)
 e2e75fc1603a9798773e467a679edf68b448e705 src/autoware/universe/external/morai_msgs (e2e75fc)
 c79e98fd5e658f4f90c06d93472faa977bc873b9 src/autoware/universe/external/muSSP (heads/tier4/main)
 eac198b55dcd052af5988f0f174902913c5f20e7 src/autoware/universe/external/negotiated (heads/master)
 d969ec699f84fad827fbadfa3001c9c657482fbe src/autoware/universe/external/pointcloud_to_laserscan (2.0.0-10-gd969ec6)
 ef094407bba4f475a8032972e0c60cbb939b51b8 src/autoware/universe/external/rtklib_ros_bridge (v0.1.0-11-gef09440)
+b80241598ec542621d59e0a1c7ae11f20a5181ae src/autoware/universe/external/tier4_ad_api_adaptor (v0.40.0-41-gb802415)
 b9c385197baf1a2e77f088bea46e7287cd3ec367 src/autoware/universe/external/tier4_autoware_msgs (v0.40.0-20-gb9c3851)
 1a9df130b4a5d96c25019b5793abbfde42d237ac src/autoware/universe/external/trt_batched_nms (heads/main)
 fc939e9bfbf171986d2d558400a4a4fee97332c6 src/extra-packages/sensor_component/external/tamagawa_imu_driver (v0.11.0beta-9-gfc939e9)
 7da80108c801fb090e8f86df02c433f8f286fcb8 src/extra-packages/vehicle/external/pacmod_interface (heads/main)
 05bc0dde94436c0694cda925b70f6902dfb328b3 src/simulator/simulator/scenario_simulator (18.2.1)
 aa2674e1d7e15a46c64ce98339378f6be6691dfc src/tools/plotjuggler_ros (2.1.2-humble)
+2d93396275235daecac86890992e2eac651d670b src/tools/tools (0.4.0-2-g2d93396)

Partial init is tested as well.

Notes for reviewers

This PR is NOT intended to be merged as-is; although I updated most actions to keep consistency with the submodule convention, github bots bumping .repos versions, adapting every .repos in the submodules, and updating tools of AWF and all of its users such as Web.Auto to make this build are clearly out of this PR's scope.
Also, we should thoroughly discuss from whether we should accept this submodule design, to (if positive) directory and naming conventions of each submodules, git aliases to ease the learning curve of this particular submodule usage, and how can we migrate everyone's assets while retaining backward compatibility and cause minimal pain.

Effects on system behavior

Obviously CIs and build pipelines will be broken; it should not affect the actual binary built and hence the runtime behavior.

[github-actions]

Thank you for contributing to the Autoware project!

🚧 If your pull request is in progress, switch it to draft mode.

Please ensure:

• You've checked our contribution guidelines.
• Your PR follows our pull request guidelines.
• All required CI checks pass before marking the PR ready for review.

[paulsohn]

Context: Disabling webauto vcstool feature enables us to execute cloning -> dependency resolution -> build chain continuously. For clarity, I would like to have a source file in its place to be before cloning -> dependency resolution -> build chain.

After environment-setup phase, all build phases are done in $AUTOWARE_PATH directory.

[paulsohn]

Where is nightly?

git submodule update --remote src/xxx is equivalent to importing xxx-nightly.repos.

In this proposal, xxx-nightly.repos files are not materialized. Instead, one can always

• have nightly configuration by git submodule update --remote src/xxx (equivalent to importing xxx-nightly.repos)
• switch to "nightly" branch (which this workflow is assumed to be update daily), and the content of xxx.repos there is xxx.repos in main plus xxx-nightly.repos.

The nightly branch here must be branch-protected.

We can further create a PR to 'stable-nightly' branch (branch name is just an example) and manually updated so that the 'stable-nightly' branch always passes build and necessary tests.

[paulsohn]

This pre-commit checks:

• whether the indexed commit hash is contained in the branch specified in .gitmodule.

This takes some time as it fetches git history, and this might fail (for released repositories which may follow main branch in nightly configuration, tagged commit might be diverged from main branch)

It is not desirable to have a chronically broken check (broken window theory), so this check might get dropped.

[paulsohn]

This pre-commit checks:

• whether submodule name matches to submodule path
• whether each submodule entry in .gitmodules have either update = none or branch = <branch> (it might have both, but meaningless.)
• For each *.repos files contained in --repos-dir directory (i.e. repositories/*.repos), repositories.path/to/submodule entry in xxx.repos should be in 1-to-1 correspondence with a submodule at src/xxx/path/to/submodule, and version (commit hash), branch and git url should agree.
  • If there are any discrepancies, the repos files are overwritten. submodule status is the source of truth.
  • Note that even if there are submodules in src/xxx/, the check is skipped if repositories/xxx.repos is missing. In this way, we can remove repos files in the future.

[paulsohn]

Removed vcs import in the beginning, so we can take the full control on how we can clone the repositories (either vcs, or submodules)

[paulsohn]

After environment-setup phase, workdir should be on the $AUTOWARE_PATH for clarity.

[paulsohn]

Since we used autoware and simulator (corresponding to autoware.repos and simulator.repos), we include them in key_files. Not sure if webauto can handle gitlink.
.gitmodules just contains metadata for manipulation, and the contents of each repository are uniquely determined (up to hash collision) by the gitlink, so .gitmodules should not be a key file.