Should SEPs define a storage model?

[aristidesstaffieri]

Today, most SEPs that aim to enshrine a class of contract or protocol do this by defining an interface for a Soroban contract and briefly describing the responsibility of each method. This allows for a lot of flexibility within a class of contract, for example SEP-41 contracts can model their balances as a simple sum of transfers or they can do something more exotic like implement a yield bearing token where your current balance has to be derived from several pieces of internal state. Both of these contracts can be compliant with SEP-41 today.

Downstream systems, like indexers, can rely on the stored contract code in order to classify a contract by parsing the contract code and matching the schema against the interface in question(SEP-41, SEP-50, etc). In order to index state associated to a protocol, the system can only rely on the SEPs interface in order to get that state. For example, you cannot reliably use a SEP-41 contracts events to derive the balance state for an account, you must invoke balance on the contract.

The impact of this is that systems that want to classify contracts and track state cannot use data emitted by core, like events. The Stellar docs state that Events are the mechanism that applications off-chain can use to monitor movement of value of any Stellar operation, as well as custom events in contracts on-chain. but I don't see this as being true because there is no guarantee at the standards level that you can recreate state from a set of events.

Sources:
https://developers.stellar.org/docs/learn/fundamentals/stellar-data-structures/events
https://github.com/stellar/stellar-protocol/blob/master/ecosystem/sep-0041.md

[tupui]

Events are just log files. In classical backend systems you would only use logs as a way to reconcile state between some actions and your DB. But you would never consider the log be the source of truth for the DB state. So yes I think storage layout is part of the public "contract".

[leighmcculloch]

To get a more complete understanding of the problem(s) this discussion is solving for:

you cannot reliably use a SEP-41 contracts events to derive the balance state for an account, you must invoke balance on the contract.

Can you expand on what you're seeing, and examples?

there is no guarantee at the standards level that you can recreate state from a set of events.

What events are missing? Which state cannot be recreated from them?

[aristidesstaffieri]

@leighmcculloch @tupui the easiest example is SEP-41, for tokens that implement different mechanics than your standard vanilla token, a whole different set of events would be needed to be able to derive state from past state + events. If we keep using the yield bearing example, in order to derive the correct balance state for an account you need an event log that exposes the share of the supply you own and the yield accrued. The set of events is different per flavor of token, while the standard that they implement is the same.

Systems that want to index and discover all SEP-41 tokens on the network have to make some trade-offs today, then can either:

• rely on events to derive state like balances, and accept that a subset of more exotic tokens will be incorrect for your baseline SEP-41 ingestion(requires special case and re-ingestion).
• invoke contracts during ingestion to ensure correctness for all tokens, at the cost of performance/complexity.

I do see the constraints that a storage model would put on a standard but my perspective is that this would lead to more focused standards, and more standards overall. For downstream systems, more standards with a narrower focus would improve the ability to index data from core outputs rather than relying on the invocation of the contract itself.

[tomerweller]

@aristidesstaffieri can you provide an example of a specific sep41 exotic token implementation that this would solve for?

The exotic example that often comes to mind is rebasing tokens, which tend to be very challenging to account for. Unfortunately, a fully defined storage layout would still be insufficient to describe such a token - a client would need to account for the correct multiplier and compute the balance.

Lucky, we don't have too many such tokens and they're largely frowned upon.

[aristidesstaffieri]

I'm picturing any of the yield bearing token patterns, there aren't many seeing real usage on Soroban today but an example could be aave's yield bearing token: https://github.com/aave/aave-protocol/blob/master/contracts/tokenization/AToken.sol

In the rebasing token example, would the multiplier not be derivable from storage also? I imagine it would be something like number of assets / number of shares in many cases?

I want to point out also that having a defined storage layout would be useful even in the case of a simple token where your balance is the sum of your credits/debits because it gives consumers a guarantee that they can read a storage slot(s) instead of executing the contract.

[leighmcculloch]

Should SEPs define a storage model?

There is nothing limiting a SEP from stating that an implementation must implement specific storage, but it has big tradeoffs.

Standards that lock developers to specific storage layouts will:

• Limit interoperability, make it impossible to implement certain types of contracts, as noted in the original post with the example of the yield bearing token.
• Make it difficult to introduce new patterns that involve multi-contract protocols.
• Prevent changing the internal data structures to optimise costs, innovate, etc.
• Will constrain future innovation with SDKs, compilers, and languages, that may structure or store types differently.

I suspect the problems described in the original post can be solved in other ways that do not share these tradeoffs. e.g. Better events? Better indexing? I've asked some questions at https://github.com/orgs/stellar/discussions/1991#discussioncomment-17875653 to better capture the problems.

Standards benefit from avoiding over-specification, and being written with a minimum necessary philosophy, only specifying what's required for interoperability. Internal storage isn't needed for on-chain interoperability.