Make it harder to accidentily use an internal module

[yoshi-monster]

Recently, people have been starting to use internal modules inside of Lustre to get around limitations in its API that we put in place to guide them to make good APIs themselves.

Right now, while the LSP does not suggest importing an internal module, people have still been finding them by browsing the source, missing the internal_modules field set in the gleam.toml file.

I think it would be good when importing and/or using a module that is marked as internal and comes from another package would emit an extra warning. This should also include using field accessors from types defined in internal modules.

---

Sometimes people control multiple packages and want to use internal apis between them. I'm not 100% convinced that is good in the first place, maybe some research what folks are actually doing there would be needed.

Thank you!! ~ 💜

[giacomocavalieri]

Yeah I don't think raising for using internals is the right choice ever, after all if internal weren't meant to be used they would just be private. Though this PR might be fixing most of those issues making sure the LS won't expose those information inadvertently #5397.
If you think that's the case this could be closed as duplicate of #5388

[lpil]

Sorry @yoshi-monster, what exactly is the item of work for this ticket? 🙏

[yoshi-monster]

Hi!

I created this issue when multiple people reported that they unknowingly where using internal modules of Lustre.

This was probably in part because of #5388, and maybe also because we (Lustre) don't name our internal modules internal or use the @internal attribute much, but instead list modules explicitely in our gleam.toml.

As Jak explained being able to access internal moddules freely is intentional. However I believe this is not how they are used in practice. I don't think I'm alone with pretending @internal works like package-private (in Java) or internal in C# - allow access from within the same package, but do not allow access from other packages.

Since Gleam does not allow cyclic dependencies, I often find myself in a situation where I move things into an internal module to break these cycles or allow access from my FFI code, while also being able to provide the public API I want. I don't (technically) make those hidden-but-public because that is what I want - it's because that's the only option given my other goals.

From this perspective, being able to use internals from other packages is basically never intentional or supported within my own code. It's not just that the internal API might not follow SemVer and might be undocumented, it often breaks the invariants of the public API. A bunch of my packages are very easy to start panic-ing when you do.

We've seen that just hiding things from the documentation does not deter people enough to not use internals unintentionally even. I think this also gets worse with the popularity of AI tools, which in my experience just explore the contents of build/packages/ to figure out how a package works and which APIs it provides. Doing so ignores the internal configuration in gleam.toml entirely.

For these reasons, I'd personally like if there was an extra step required before you can use an internal module without a warning. For example, there could be a special type or flag on the dependency configuration in gleam.toml. A stronger version of that could be a friend-like modifier that I declare in my package instead, listing other packages that are allowed to use my internals.

Anyways, I think this should either be closed or maybe be moved into a discussion instead.