feat: Lustre charm documentation by dsloanm · Pull Request #155 · canonical/charmed-hpc-docs

dsloanm · 2026-06-26T14:23:42Z

Pre-submission checklist

I read and followed the CONTRIBUTING guidelines.
I have ensured that the documentation tests complete successfully.
I have added a metadata description at the top of any new page.

Summary of Changes

Adds documentation for the Charmed HPC Lustre offering. Includes an Explanation page detailing the Lustre filesystem and charm.

Related Issues, PRs, and Discussions

Code PR for the initial Lustre charm: canonical/filesystem-charms#36

dsloanm · 2026-08-20T12:46:03Z

@AshleyCliff This is now in a state where it could use a preliminary review. Biggest question IMO is how we reconcile the "Deploy Lustre" and "Deploy a shared filesystem" pages. Re-titling the latter may make sense since it actually describes integrating with an externally managed shared filesystem rather than deploying your own.

AshleyCliff

Looking good, thanks for all the effort! I've left a few suggested updates and a variety of open-ended questions.

AshleyCliff · 2026-08-20T13:24:34Z

+The `lustre-server` charm is in an experimental state and is not ready for production use.
+:::
+
+## Prerequisites


Are there any weird order-of-operations issues that could crop up if the Lustre server is deployed before Slurm? The deploy a shared filesystem lists 'A Slurm cluster' as a prereq. If not an explicit prereq, would it be advised?

Shouldn't be any issues with order-of-operations. Slurm or Lustre could come up first. You could even use Lustre without a Slurm cluster if you just wanted a standalone parallel filesystem so I've not included it as a prereq

AshleyCliff · 2026-08-20T18:07:00Z

+(howto-deploy-deploy-lustre-server)=
+## Deploy the `lustre-server` charm
+
+To deploy a default configuration, where the initial leader unit is a combined MGS+MDS and all remaining units are OSSes, run the following:


How well known are MGS, MDS, and OSS acronyms? Especially for those who may not be highly familiar with filesystems/Lustre? I'd suggest spelling them out with the acronym in parentheses the first time they're used on the page and then just acronyms after that.

Oh I see they're described on the explanation page. My other comments stands but maybe also a "Visit the Lustre explanation page for more details.." somewhere near this sentence.

I've spelled out the acronyms and added a "For an explanation of Lustre and related terminology, see the {ref}`Lustre explanation page `" to the top of the page

AshleyCliff · 2026-08-20T18:10:52Z

+(howto-deploy-deploy-lustre-server)=
+## Deploy the `lustre-server` charm
+
+To deploy a default configuration, where the initial leader unit is a combined MGS+MDS and all remaining units are OSSes, run the following:


initial leader

I'm not sure why but something about 'initial' keeps catching me here. Is there an expectation that the leader unit will change?

Yes, the leader unit can change at any time. When that happens, we don't want the MGS+MDS node to change with it so the service is fixed to the unit that happened to be leader as lustre-server was deploying

AshleyCliff · 2026-08-20T18:12:33Z

+  -n <number of units>
+:::
+
+A minimum of two units is required; a single unit deployment is not supported.


Two total - one leader and one OSS, or two OSS? (I'm assuming one leader one OSS but might be worth spelling it out).

AshleyCliff · 2026-08-25T20:24:19Z

+
+LNet is Lustre's network layer, responsible for communication between clients and server components. The `lustre-server` charm can automatically detect network interfaces and configure LNet or a custom configuration can be provided using the `lnet-networks` option.
+
+Both the `lustre-server` and `filesystem-client` charms perform network auto-detection by default, during the charm `install` hook. LNet is automatically configured with a network name of `tcp` using the default route Ethernet interface and, if RDMA hardware such as InfiniBand is detected, another network name of `o2ib` using every RDMA device, in a multi-rail setup if multiple devices are detected.


I believe this is the first mention of the filesystem-client charm on this page. Should there be a very brief statement added somewhere prior to this that describes how it fits into this process?

Agreed, I've moved the paragraph explaining filesystem-client from "Integrating with clients" to further up the page, as you suggested in another comment

AshleyCliff · 2026-08-25T20:28:53Z

+
+LNet is Lustre's network layer, responsible for communication between clients and server components. The `lustre-server` charm can automatically detect network interfaces and configure LNet or a custom configuration can be provided using the `lnet-networks` option.
+
+Both the `lustre-server` and `filesystem-client` charms perform network auto-detection by default, during the charm `install` hook. LNet is automatically configured with a network name of `tcp` using the default route Ethernet interface and, if RDMA hardware such as InfiniBand is detected, another network name of `o2ib` using every RDMA device, in a multi-rail setup if multiple devices are detected.


Suggested change

Both the `lustre-server` and `filesystem-client` charms perform network auto-detection by default, during the charm `install` hook. LNet is automatically configured with a network name of `tcp` using the default route Ethernet interface and, if RDMA hardware such as InfiniBand is detected, another network name of `o2ib` using every RDMA device, in a multi-rail setup if multiple devices are detected.

Both the `lustre-server` and `filesystem-client` charms perform network auto-detection by default, during the charm `install` hook. LNet is automatically configured with a network name of `tcp` using the default route Ethernet interface. If RDMA hardware such as InfiniBand is detected, another network name of `o2ib` is also configured, which uses every RDMA device in a multi-rail setup if multiple devices are detected.

Not sure that edit works but something needs to be tweaked, that sentence had way too much going and was hard to parse. The 'if multiple devices are detected' part still isn't fitting well.

I went with:

"If RDMA hardware such as InfiniBand is detected, an o2ib network is also configured. When multiple RDMA devices are present, all of them are configured in a multi-rail setup in the o2ib network"

which feels slightly blunt but more precise

AshleyCliff · 2026-08-25T20:32:09Z

+
+### Service placement
+
+The charm automatically places the MGS and MDS on the initial leader unit, with all subsequent units becoming OSSes. A minimum of two units is required in a deployment.


Same as the how-to question - two units total or two OSS units?

Clarified this

AshleyCliff · 2026-08-25T20:34:13Z

+
+A **ZFS dataset** is a subdivision of a zpool that can be configured like an independent filesystem. Datasets inherit properties from their parent pool but ZFS features can be individually tailored per dataset. For example, a particular dataset can be configured for more frequent snapshots than another dataset in the same zpool.
+
+TODO: how Lustre charm uses ZFS


Plans for updating this soon?

Yes, once we have disk detection and we've finalized exactly how we're using ZFS

AshleyCliff · 2026-08-25T20:35:38Z

+
+If all health checks pass while the unit is in a `BlockedStatus`, the unit is restored to `ActiveStatus`.
+
+### Integrating with clients


Putting this section earlier would resolve my prior comment about filesystem-client needed some description/context.

Co-authored-by: Ashley Cliff <ashley.cliff@canonical.com>

* remove admonitions * add additional explanation around LNet * provided additional context for why a user may want to override LNet auto-detection * clarify minimum unit counts

* add LNDs to wordlist * add "## Architecture" header to Explanation * move the "Integration with clients" section * reword LNet autodetection and minimum unit count descriptions * add sample `juju status` * add test to verify Lustre is functioning

dsloanm · 2026-08-28T15:10:00Z

Thanks for the comments @AshleyCliff. That should be all of them addressed and the docs ready for another review.

Getting disk detection functionality in the charm should be all that's needed to fix "TODO: how Lustre charm uses ZFS" and get this PR out of draft

Rewrite sections that reference the initial leader being set as the MGS+MDS as service placement is now done based on storage. Reword various sections for greater clarity. Signed-off-by: Dominic Sloan-Murphy <dominic.sloanmurphy@canonical.com> Assisted-by: opencode:gpt-5.6

dsloanm · 2026-09-09T13:22:46Z

Now out of draft!

benschwaller

Looks great! Left a few thoughts. A couple of the refs and code blocks loaded incorrectly on my github preview, affirm they are all correct.

benschwaller · 2026-09-14T17:42:55Z

+:::{code-block} shell
+juju add-storage lustre-server/1 ost=loop,4,1G
+:::


Might want to consider adding the 4 device minimum for OST here.

benschwaller · 2026-09-14T18:01:34Z

+--config lnet-networks="tcp=eth0; o2ib0=ib0,ib1"
+```
+
+configures LNet with a net name of `tcp` using the `eth0` interface, and a net name of `o2ib0` using the `ib0` and `ib1` interfaces.


Do we support RoCE interfaces? If not, should call that out here given it was discussed earlier.

benschwaller · 2026-09-14T18:03:15Z

+
+- Peer relation data is present and consistent.
+- Required kernel modules are loaded.
+- Lustre service mounts are active.


More info on "active" mount definition and how you test for it might be useful to properly set expectations.

dsloanm force-pushed the lustre-docs branch 2 times, most recently from d478e1f to 0f94923 Compare July 3, 2026 10:52

dsloanm mentioned this pull request Jul 17, 2026

feat: implement lustre-ops internal package and IB support canonical/filesystem-charms#42

Merged

4 tasks

jedel1043 mentioned this pull request Jul 24, 2026

docs: add documentation and contact links to charmcraft.yaml canonical/filesystem-charms#49

Merged

3 tasks

dsloanm requested a review from AshleyCliff August 20, 2026 12:46

dsloanm added 4 commits August 20, 2026 14:07

feat: add Lustre Explanation

3b3f774

feat: add draft Lustre How-to

f8e57a8

feat: link to relevant How-to in Lustre Explanation

876a90c

feat: add LNet info to Lustre Explanation and deployment How-to

535b551

dsloanm force-pushed the lustre-docs branch from 1c3b278 to 535b551 Compare August 20, 2026 13:07

AshleyCliff requested changes Aug 25, 2026

View reviewed changes

dsloanm and others added 3 commits August 27, 2026 14:54

fix: adjust wording in howto/deploy/deploy-lustre.md

2852679

Co-authored-by: Ashley Cliff <ashley.cliff@canonical.com>

fix: reword Lustre Explanation and How-to

c00d75d

* remove admonitions * add additional explanation around LNet * provided additional context for why a user may want to override LNet auto-detection * clarify minimum unit counts

dsloanm mentioned this pull request Sep 8, 2026

feat(lustre-server): add Juju storage support canonical/filesystem-charms#64

Open

4 tasks

dsloanm marked this pull request as ready for review September 9, 2026 13:22

dsloanm requested a review from a team as a code owner September 9, 2026 13:22

dsloanm requested review from AshleyCliff and NucciTheBoss September 9, 2026 13:22

benschwaller reviewed Sep 14, 2026

View reviewed changes


		LNet is Lustre's network layer, responsible for communication between clients and server components. The `lustre-server` charm can automatically detect network interfaces and configure LNet or a custom configuration can be provided using the `lnet-networks` option.

		Both the `lustre-server` and `filesystem-client` charms perform network auto-detection by default, during the charm `install` hook. LNet is automatically configured with a network name of `tcp` using the default route Ethernet interface and, if RDMA hardware such as InfiniBand is detected, another network name of `o2ib` using every RDMA device, in a multi-rail setup if multiple devices are detected.


		### Service placement

		The charm automatically places the MGS and MDS on the initial leader unit, with all subsequent units becoming OSSes. A minimum of two units is required in a deployment.


		A ZFS dataset is a subdivision of a zpool that can be configured like an independent filesystem. Datasets inherit properties from their parent pool but ZFS features can be individually tailored per dataset. For example, a particular dataset can be configured for more frequent snapshots than another dataset in the same zpool.

		TODO: how Lustre charm uses ZFS


		If all health checks pass while the unit is in a `BlockedStatus`, the unit is restored to `ActiveStatus`.

		### Integrating with clients

Uh oh!

Conversation

dsloanm commented Jun 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Pre-submission checklist

Summary of Changes

Related Issues, PRs, and Discussions

Uh oh!

dsloanm commented Aug 20, 2026

Uh oh!

AshleyCliff left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dsloanm Aug 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dsloanm Aug 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dsloanm commented Aug 28, 2026

Uh oh!

dsloanm commented Sep 9, 2026

Uh oh!

benschwaller left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

dsloanm commented Jun 26, 2026 •

edited

Loading

dsloanm Aug 28, 2026 •

edited

Loading

dsloanm Aug 28, 2026 •

edited

Loading