Add externally managed contract executable type

[mootz12]

Problem

The factory pattern is commonly used in smart contract protocols to deploy many contract instances that share the same WASM (a "fleet" of contracts). When that shared WASM needs to be upgraded, the admin must submit a separate upgrade transaction for every instance. This is fine for small fleets, but for large ones it is expensive and creates a window during the rollout where some instances run the old WASM and some run the new.

Because some upgrades carry security fixes, or are not backwards-compatible with the previous version, there is incentive for the upgrade to apply atomically across the entire fleet.

Ecosystems that manage reusable code via proxies, such as Ethereum, achieve this with the beacon proxy pattern.

Stellar has no equivalent today. Each contract instance defines its own WASM hash, and we intentionally do not expose the proxy primitives (fallback() or delegatecall) that beacon implementations rely on. This discussion proposes fixing this by extending the contract executable definition to support references to an updatable WASM hash.

Proposal

We would like to extend the contract instance executable definition to include an external reference variant:

ContractExecutable::ExternalRef { contract: Address, name: Symbol }

A contract can manage WASM entries via a new ScVal ledger key type ScVal::LedgerKeyWasmRef(Symbol) and a new host function that validates wasm_hash exists as a code entry:

set_contract_wasm_ref(name: Symbol, wasm_hash: BytesObject) -> Void

A contract can switch its executable to an external reference or update it's external reference via a new host function:

update_current_contract_executable_external_ref(contract: AddressObject, name: Symbol) -> Void

A factory contract can deploy contracts with an externally referenced WASM with the new host functions:

create_contract_with_external_ref(deployer: AddressObject, contract: AddressObject, name: Symbol, salt: BytesObject) -> AddressObject
create_contract_with_external_ref_and_constructor(deployer: AddressObject, contract: AddressObject, name: Symbol, salt: BytesObject, constructor_args: VecObject) -> AddressObject

Diagrams

Legend:

• Manager - a contract that manages contract wasm hashes
• Follower - a contract whose executable is an ExternalRef to the Manager
• Factory - a contract that deploys follower contracts
• Ledger - where everything is stored

Resolving an ExternalRef Contract during a contract invocation

sequenceDiagram
    actor U as Caller
    participant H as Host
    participant L as Ledger
    U->>H: invoke Follower.transfer(...)
    H->>L: read Follower instance
    L-->>H: executable = ExternalRef{ Manager, "token" }
    H->>L: read reserved entry (Manager, WasmRef("token"))
    L-->>H: wasm_hash
    H->>L: read ContractCode(wasm_hash)
    L-->>H: wasm code
    Note over H: instantiate VM, run transfer(...)
    H-->>U: result of transfer(...)

Loading

Manager upgrading a WASM it controls

sequenceDiagram
    actor Op as Admin
    participant M as Manager
    participant H as Host
    participant L as Ledger
    Op->>M: upgrade("token", new_hash)
    M->>M: governance checks (require_auth(admin), timelock, ...)
    M->>H: set_contract_wasm_ref("token", new_hash)
    H->>H: assert ContractCode(new_hash) exists
    H->>L: write (Manager, WasmRef("token")) = new_hash
    H->>H: emit wasm_ref_updated(Manager, "token", old_hash, new_hash)
    H-->>M: ok
    Note over L: every Follower with ExternalRef{ Manager, "token" }<br/>resolves to new_hash on its next call

Loading

Factory deploying a new follower contract

sequenceDiagram
    actor U as Deployer
    participant Fa as Factory
    participant H as Host
    participant L as Ledger
    U->>Fa: deploy(salt)
    Fa->>H: create_contract_with_external_ref(deployer, Manager, "token", salt)
    H->>H: derive Follower address from (deployer, salt)
    H->>L: read (Manager, WasmRef("token"))
    L-->>H: wasm_hash
    H->>H: assert ContractCode(wasm_hash) exists
    H->>L: write Follower instance, executable = ExternalRef{ Manager, "token" }
    H-->>Fa: Follower address
    Fa-->>U: Follower address
    Note over H,L: the _and_constructor variant also loads the code<br/>and runs the Follower's constructor against it

Loading

Alternatives

• Arbitrary storage slot. Rather than a reserved, typed key, the executable could point at any storage entry ExternalRef { contract: Address, key: Val }. This is the simplest protocol change, but potentially the easiest to go wrong as no additional host functions are added that can validate things like "this entry holds a deployed WASM hash".

Questions

• When a follower runs extend_current_contract_instance_and_code_ttl, what gets extended?
  • Follower instance -> yes
  • Manager WASM ref entry -> ?
  • WASM code -> ?

[dmkozh]

Thanks for posting this. I think another alternative that might be suggested is a direct ability for a contract to 'inherit' the executable of another contract directly. However, I think the proposed approach has several benefits compared to the direct inheritance approach:

• Proxying support must be intentional (there is no way to just naively inherit an arbitrary contract)
• The proxied contract logic is clearly separated from the manager contract logic (in case of inheritance both must be present in the same contract)
• One manager can manage an arbitrary number of Wasms

Some small design notes:

• create_contract_with_external_ref function doesn't seem necessary, we only need a version with constructor. Protocol already defines that contracts without explicit constructor have implicit constructor that accepts 0 arguments and does nothing, so at the host level it's sufficient to just pass an empty arguments vec object for such contracts

• extend_current_contract_instance_and_code_ttl question raises an interesting point: I think the behavior is quite clear here and we should bump instance, ref entry, and code itself (the intention here is 'bump everything necessary for my contract to execute'). But what we also need to consider is the ability to extend TTL of the reference itself. Adding a dedicated host function for that looks a bit annoying. Maybe we should actually consider the alternative you suggested - protocol can still verify that the reference you're pointing at contains a hash of an on-chain Wasm entry at the moment of the source update. This way we can cut the number of the new host functions greatly and thus make the proposal simpler.

[mootz12]

Thanks for the feedback @dmkozh!

direct ability for a contract to 'inherit' the executable of another contract directly

Yeah, considered this but agree with your reasoning. Primarily, it's very awkward to manage the instance executable data based on if you are the follower or the source.

create_contract_with_external_ref function doesn't seem necessary

Good catch, I missed that detail about explicit constructors. Will remove in the CAP.

extend_current_contract_instance_and_code_ttl question

Agree, bumping all 3 is the most obvious result. On first glance we should be able to bump the reference within the above host function.

But what we also need to consider is the ability to extend TTL of the reference itself

From the manager you mean? Yeah, we would need an additional host function here as is. Annoying - good catch.

Maybe we should actually consider the alternative you suggested (Arbitrary storage slot)

My main concern the host has assurances that WASM hashes map to contract code that exists. Allowing arbitrary data to be used would break that, even if we verify on source update, as the manager could just delete it or something. It looks like the code has enough error handling to manage this, but it felt like a more intrusive change than adding fairly simple host functions.

If you feel otherwise, this is simpler so happy to go this route. It's worth noting that the key is just a generic ScVal now. This isn't really an issue but forcing Symbol does allow them to be automatically readable by explorers.

---

Unrelated, but realized while responding but I think it makes sense to loosen the name: Symbol requirement to name: String if we stick with the reserved ScVal ledger key. Symbol doesn't allow : or . so common scope/version strings like token:latest or token:v1.2.3 would not be supported.

[dmkozh]

as the manager could just delete it or something.

Yeah, it's a bit of an issue, as we can't validate the updates to the arbitrary entry that happen after someone starts depending on it. Of course, the manager contract can always 'brick' every contract depending on it by e.g. pointing them at a completely irrelevant Wasm, but arbitrary ledger entry approach introduces some new 'brick' modes. Notably, none of them are terminal - it's possible to fix any issue by just overwriting the entry contents. So maybe this is not as big of a deal in reality?

Also, 'bricking' the dependent contracts is probably not the worst failure mode, as they'll just fail. It would be much worse to e.g. accidentally updating contracts to an old vulnerable (but compatible) Wasm, which is hard to detect and puts users at risk. We can't really do anything about that failure mode at the protocol level.

If we want to be really strict, I think we could introduce a new kind of ScVal/Object to indicate the 'Wasm key' entries, and then we could add correctness enforcement at the protocol level (making it non-removable, fail it when the value is not a valid Wasm hash). The benefit of that approach is that we only need a constructor host function for the new key type, and then we can really benefit from all the existing machinery (like storage writes, TTL extensions, existence checks etc.). This is a bit more work at the XDR level, with the benefit of keeping the host interface itself simpler. I think it's a better solution than bespoke functions (as has been proven by the huge set of bespoke functions for contract instance management, that we had to expand several times), if we need to do additional validation at all, that is.

[mootz12]

Also, 'bricking' the dependent contracts is probably not the worst failure mode, as they'll just fail

Agreed. I think having arbitrary entries is fine from this CAPs perspective. The reserved ScVal / host functions does not provide any security guarantees for users. I opted for this proposal just to keep the existing behavior that WASM code always exists / is actually WASM.

But, if we are OK relaxing that I agree that this is simpler.

I think we could introduce a new kind of ScVal/Object to indicate the 'Wasm key' entries, and then we could add correctness enforcement at the protocol level (making it non-removable, fail it when the value is not a valid Wasm hash)

E.g. contracts can build this new ScVal type themselves? And the protocol performs the additional checks when used as a storage key in the existing host functions?

Yeah this makes sense, and removes the redundant host functions.

if we need to do additional validation at all, that is.

The primary deciding factor here should be if we care that the contract address to WASM VM code path now has to handle the additional failure cases an arbitrary value would introduce.

Having a validated wasm ref doesn't do much for the usability or safety of the proposal from the user's perspective. There is a maybe that it could prevent managers from accidentally bricking wasm ref storage entries with non-wasm hashes, but IMO that scenario would be a complete failure of the manager. (and, it's easy to recover from).

[dmkozh]

I've drafted the CAP with the executable tag approach I've proposed above. I've sketched the new object addition, and I think it's rather small additional lift compared to the usability impact. The validation logic needs to live somewhere (either reference creation or resolution), so it doesn't make much difference complexity-wise. So it seems to me like the tag approach is a reasonable compromise.

[ozgunozerk]

I liked the approach, and I think it addresses a fair problem.

The only potential problem I could see is, there is no guarantee whether the updated code will be compatible with the proxy contract's storage.

If the upgrades were to be done manually, we could enforce the migration path easier. However, in this approach, I think the updates can happen, only if they do not require migration? Because, I couldn't see a way to migrate all the contracts, without calling them one by one (which is the exact thing this proposal is trying to avoid).

Although I'm pointing out a problem, I unfortunately don't have a good solution to it. This problem was the first thing came to my mind when I read the proposal. I'm hoping that raising this concern can ultimately be helpful for this proposal, and may spark further discussions on how to find a good solution to it (if it is possible).

Having said that, I don't think the problem I've described should be a blocker for this proposal. The value this proposal will bring, will be bigger than this problem, so I think overall, it is still good to go!

[dmkozh]

However, in this approach, I think the updates can happen, only if they do not require migration?

Given that by definition both old and new Wasms are owned by the manager contract, workarounds are available:

• Don't require the contracts to do explicit data migration at all.
  • This is going to be simpler as we're planning to also rollout a protocol change that would treat Option fields as truly optional, so that you could freely add/remove them to the contracttype structs.
• Do migration lazily:
  • In case of one-time-only migration that's going to be slightly wasteful (as you'd need to add the migration call to all the entry points), but still acceptable IMO given that it's just going to be an extra function call in Wasm - just a handful of instructions for cases when migration is not necessary.
  • In case of large-scale migration (say, update format of every token balance entry) the atomic change is not possible anyways due to the footprint limits, so it has to be lazy anyways

But I really hope that most of the contracts would be in 'no migration necessary bucket'; I think that's additional motivation for prioritizing the data compatibility change in this protocol.

[mootz12]

Thanks for taking a look @ozgunozerk !

I'd expect users that need to migrate would use the existing WASM hash contract executable type. An upgrade manager contract could then fetch the new WASM (either supplied by admin or fetched from a trusted source), upgrade the contract, then invoke the migration logic.

Contracts that use atomic fleet upgrades would need to be able to support some type of lazy migration if it is necessary as Dima mentioned.

[brozorec]

On the decision to have get_address_executable return only the resolved wasm hash rather than the external reference:

Returning the executable reference would not be helpful in this case, as the reference can't be resolved by a third-party contract as Soroban doesn't provide a way to perform lookups in other contract's storage.

I'd like to push back on this, because the ref provides information that's valuable independently of whether the caller can resolve it. When a large set of instances share a single owner-managed executable, the owner contract and/or the tag can be a signal for "membership", info that can be very useful to route or group calls.

I see the point about the backward compatibility issue if adding a new variant to the AddressExecutable, but what about a host function get_address_executable_ref returning the owner and the tag?

[dmkozh]

When a large set of instances share a single owner-managed executable, the owner contract and/or the tag can be a signal for "membership", info that can be very useful to route or group calls.

Could you please expand on this? You can't enumerate the contracts by executable, so the list of the actual contracts has to come from the storage or be supplied by the caller, at which point it's not clear why would you want to access the executable in the first place.

The motivation behind get_address_executable was to allow contracts to pin specific implementations, and the reference obfuscates the implementation. I realize that there can be a valid scenario where one would want to pin the reference, but there are possible workarounds for that, whereas there is no workaround to get the Wasm hash given a reference.

I'm not sure I like a new host function approach, it seems overly specific to me. I would at least like to understand the use case better.

[brozorec]

The scenario I had in mind was a manager that uses tags for versioning, ie it doesn't upgrade the whole fleet by changing the wasm, but by adding a new (tag2, wasm2) entry and letting instances from the fleet upgrade on their own pace, e.g. there could be some instances pointing to v1 and while some others to v2; with breaking changes in the interface. We can imagine a router that abstracts away the versioning by redirecting calls based on the tag. I'm thinking about this from the perspective of supportsInterface() in ERC-165 when some sort of run-time dispatch mechanisms are needed.

This is possible if the factory/manager contract reports the versions back via wasm tho.

@mootz12 I guess this could work in the scenario above, but it feels like an unnecessary round-trip: if we have (manager, tag) -> wasm, why not the other way around as well.

the reference obfuscates the implementation

@dmkozh I'd argue it's the other way around, it makes the management relationship invisible. The beacon proxy consumer trusts the owner's central upgrades, but with the current model the consumer can't verify that.

[dmkozh]

We can imagine a router that abstracts away the versioning by redirecting calls based on the tag. I'm thinking about this from the perspective of supportsInterface() in ERC-165 when some sort of run-time dispatch mechanisms are needed.

I feel like this is a completely separate problem from what we're trying to solve here and it's really orthogonal to the actual executable of the contract. There is really no good reason for why interface detection should hinge on using the tags, as tags are require much more trust than most of the protocols should need. E.g. in your scenario with manual updates of the fleet I'm not really sure why do we need tags in the first place - if there is a way to communicate the new tag, why it can't be used to communicate the new Wasm hash directly instead? You can also make this programmable if necessary by e.g. introducing a fn list_wasms() -> Vec<(VersionTag, Hash)> function in the manager contract that the downstream contracts can call on update if they want without giving up exclusive control of their contract.

As for the interface support, several ideas come to mind:

• Dispatch based on Wasm hash (that's well supported by the update to get_address_executable in this CAP - you don't need to tightly couple the detection mechanism to references)
• Use try_call to gracefully handle the errors (not super efficient, but also very general)
• Use a separate standard like ERC-165

None of these hinge on using the tags, and I would argue every one of them is better than tag-based approach.

I'd argue it's the other way around, it makes the management relationship invisible. The beacon proxy consumer trusts the owner's central upgrades, but with the current model the consumer can't verify that.

The point of pinning the implementation has been about preventing unexpected updates of the contract (which was updateable by definition, or else why would you need pinning in the first place?) - I don't see why it should matter how the update has been done. On the other hand, pinning implementation has very narrow guarantees and basically requires the pinned contract to have no (valuable) state. So I'm not sure if trust model should be materially different for externally managed contracts. I guess one can desire for the contract to be non-upgradeable, but that currently requires off-chain audit anyways - I would argue that if we wanted to solve this on-chain, we'd need a is_upgradeable function instead.

[brozorec]

I feel like this is a completely separate problem from what we're trying to solve here and it's really orthogonal to the actual executable of the contract.

Yes, I agree. What made me plunge into those scenarios was the comparison of the the EVM Beacon proxies and the mechanism in this CAP.

The EVM Beacon proxies are 1:N, i.e. there's one beacon that commands the upgrades of the whole fleet. Every proxy holds as an immutable variable the address of the beacon, which occupies a well-known storage slot. This makes the relationship between both contracts unambiguous and it allows block-explorers to recognize the contract as a beacon proxy and display the interface of the implementation it points to.

By using tags, this CAP proposes a model that's 1:N:M, i.e. one manager handling multiple categories of fleets. I see the practical utility of the tags in the context of upgrading, but also saw an opportunity for a tag as way to subscribe to a maintained version line. And I agree this is orthogonal to the current proposal.
Anyway, when thinking how CAP-85 would be used in practice, a model of a factory-like manager comes in mind, with the factory keeping track of all deployed instances. I still think an introspection mechanism for instances is a good call, but maybe this better be scoped in a SEP.

[dmkozh]

Every proxy holds as an immutable variable the address of the beacon, which occupies a well-known storage slot.

That's almost the same as we have for Soroban (executable field of the contract instance), with the caveat that Soroban contracts may be mutable. But the executable is known at any given point in time.

This makes the relationship between both contracts unambiguous and it allows block-explorers to recognize the contract as a beacon proxy and display the interface of the implementation it points to.

Isn't that the same for Soroban? A block explorer can resolve the reference and display the interface as well. Again, the interface of course may be updated, but that's true for any Soroban contract.

I still think an introspection mechanism for instances is a good call, but maybe this better be scoped in a SEP.

I don't disagree, but my intuition is that the executable resolution mechanism should be abstracted away for this, if the goal is to figure out whether a given contract conforms to a given interface. In any case, we can revisit the additional protocol support when we identify the exact gaps - right now it seems like SEP-only solution should be viable. If that's not the case, we can add more introspection mechanisms to the host.

[leighmcculloch]

Something observing during the SDK implementation in stellar/rs-soroban-sdk#2020, is that the significant behaviour difference introduced into auth::ContractExecutable vs address::Executable with regards to references makes for asymmetrical APIs in the SDK. The two types logically seem comparable and compatible but aren't.

To improve the UX of the APIs around executable, maybe we should consider expanding address::Executable to include the type and add a host function specifically for resolving refs.

stellar-protocol/core/cap-0085.md

Line 288 in 8091ab6

For contracts with executable references, the `get_address_executable` host function could return the executable reference itself. However, the point of this function is to allow developers to introspect the implementation for a given contract address (usually for the purpose of vetting 'trusted' implementations). Returning the executable reference would not be helpful in this case, as the reference can't be resolved by a third-party contract as Soroban doesn't provide a way to perform lookups in other contract's storage. Thus, instead of obfuscating the actual executable by returning the reference, we resolve the reference and return the actual Wasm hash.

I don't have a specific use case for this, other than improving the UX. So I don't expect this to move forward, but noting here that we have the assymetry.