Stellar
[SEP-48] Contract Interface Spec: Events #1724
Replies: 9 comments 32 replies
|
To experiment with the event schema xdr above, the following branches exist:
To generate and work with XDR for the schema, install the An example of the XDR-JSON for an event: {
"event_v0": {
"doc": "",
"lib": "",
"name": "transfer",
"fields": [
{
"doc": "",
"name": "from",
"type_":"address",
"location": "topic_list"
},
{
"doc": "",
"name": "to",
"type_":"address",
"location": "topic_list"
},
{
"doc": "",
"name": "amount",
"type_":"i128",
"location": "data"
}
],
"data_format": "single_value"
}
}The above XDR-JSON can be encoded into XDR with: If you have an IDE with a JSON LSP (e.g. VSCode), run the following command to generate a JSON Schema file, then open VSCode and create a new .json file containing the following, and you'll get auto-complete. {
"$schema": "./ScSpecEntry.json",
} |
The proposal in its current form makes it possible to define a contract event struct that will map to any of the three major shapes that events take:
The stellar asset contract uses all three shapes in its own events, and being able to describe the SAC with these event schemas is critical. However, that does increase the complexity of the events, how they work, and what a developers needs to know about to use them. Does anyone have alternative ideas? |
|
I see. So because SACs will always have the 3 types and should be supported themselves it means non-SACs should also be allowed to use the 3 types. (Technically we could change all events to be a map/key-value pairs but I don't think that is worth the effort) How would the proposed format look for a map and vec? Is it possible to write an example for each similar to the example in this comment |
|
Here are some examples from SEP-41. The first column is the XDR for an event emitted (existing). The second column is the event spec that would describe the event (as proposed by this proposal). The third column is the type of data model that could be derived from combining both (made possible by the proposal, but up to tooling adopting to decide what it looks like). Example 1: SEP-41
Example 2: SEP-41
Example 3: SEP-41
|
|
Thank you that's super helpful What would happen in the case of data being a map that also contains a map? Would the Note that this is an edge case that currently doesn't exist in events but it is theoretically possible because the scval type val in a map can be a map (I think) |
|
I understand the need to accommodate all three date types given the SAC implementation. However, this does seem like quite the leaky abstraction and it would be great to have sensible defaults. Can we introduce data type inference on the rust side macro where, in the absence of an explicit type, a single non-topic param implies a single param and multiple non-topic params imply a map*? the only way to get a vector would be to explicitly set it as the data type. *I assume that for multiple values a map is preferred due to being trivially inspectable, even in the absence of a spec. If we think that a vector is preferred because of the compactness we can switch the two. |
|
For defaults I think we can make the data a map, and any other data format requires a choice on the developers part. |
|
The proposal for event specs and how it enables dev tooling use cases later sounds good. Some observations:
|
|
I updated the proposal to use the term param instead of field. |
I think if this was to happen it should be considered holistically for all spec entries in the same way, not only for events. Implementation for doing it holistically would be outside of what's being proposed in this change. I'm thinking this could be a separate discussion. |
|
Another thing to consider with 2 is that a contract already has the SEPs it supports stored in contract meta. So a consumer will already be able to achieve the same effect by a combination of this proposal as-is plus SEP-47 Contract Interface Discovery. When ingesting an event, and inspecting the contract spec, also inspect the contract meta, and you'll have the same information. |
Yes, that sounds fine to address app protocol associativity separately. SEP-47 app protocol convention proposes a contract declares list of protocols. Which is great for some use cases which just need to know what app protocols a contract belongs. However, that list model makes it potentially ambiguous for consumers to identify which protocol an event or function of a contract belongs if/when contract defines more than one app protocol. |
|
I think SEP-47 will tide us over before we consider something more holistic.
SEP-47 is not that different to inheritance in most languages, where there's some ambiguity or constraints around multiple inheritance. i.e. It'd be confusing no matter the solution if a contract emits two identical looking events for different purposes to satisfy the requirements of different SEPs.
For a more holistic solution, I think we should consider introducing meta fields for all types of spec entries: types, functions, events. Meta fields would allow us to extend SEP-47 from only describing the contract to describing specific events. Meta fields would also enable whatever other meta might become useful, such as versioning. |
This is true for 95% of contract events. Soroswap events, however take a different shape, which accounts for ~1M events on network: In these examples, both topic names should be Can there be flexibility in how the topic name is set? If the developer could specify the data type of the topic name and location within the topic array, this interface would have better coverage for events because it could dynamically pull the topic name based on data type + location, with the default being first element, type Symbol. If this adds too much overhead or isn't possible, I don't feel strongly about it, just wanted to surface that there are relevant protocols that do not conform to this current interface and whether we want to accommodate that. |
|
I think it's fine to stick with the original proposal (1 topic name, type symbol) especially since you reached out to the developers of the primary contract that does not conform to the SEP. Let's stick with the original proposal and close this thread. |
|
Hello @leighmcculloch, @chowbao and @sydneynotthecity Yes, indeed, one of the events mentioned is a Soroswap event The other event mentioned, is part of a Phoenix AMM pool Regarding of supporting only symbols, I aggree. I added more info in the topic so it would be easier to write indexers like this one. But its true that we can filter the events later.... And if you take a look at our SubQuery implementation, the SDK consider both symbols and strings as strings. PS: Here an old discussion I, from where I based my code |
|
Maybe the issue of symbols or strings doesn't matter so much. I think it could be reasonable to expect a consumer to match on either the string or symbol type with specific contents in either. The SDK would be likely to use symbols, but the SEP can be written such that either can be used, and the type doesn't need specifying, consumers just look for both. Wdyt? |
I was imagining that the way contracts would define these prefix topics was it would be derived from the event structure, such as the example that follows, where the
This mirrors Solidity events:
But, Soroban has no actual concept of an event name, and in the proposal we're repurposing the first topic as the event name. Instead of repurposing the topic, we could instead specify fixed topics on the type, and that would be more true to Sorobans underlying data structure. For example:
|
|
Chatting to @tomerweller offline and he made the point that we can do both. We can define explicitly prefix topics for each event and by default it comes from the event structure name in the code but is overrideable. I think that works. |
|
Response above seems favourable and given that this proposal is not particularly controversial or novel, I suggest we move ahead and have opened this change to the Stellar XDR: |
|
I've pushed a change to the XDR to make the prefix topics an explicit value. diff --git a/Stellar-contract-spec.x b/Stellar-contract-spec.x
index c4942b4..ce6e030 100644
--- a/Stellar-contract-spec.x
+++ b/Stellar-contract-spec.x
@@ -244,6 +244,7 @@ struct SCSpecEventV0
string doc<SC_SPEC_DOC_LIMIT>;
string lib<80>;
SCSymbol name;
+ SCSymbol prefixTopics<2>;
SCSpecEventParamV0 params<50>;
SCSpecEventDataFormat dataFormat;
};The SDK may derive it from the event struct's name, but in the spec XDR it's better the list of prefix topics is explicit, and we can make that list 2 to accommodate Soroswap and other contracts that want to put two fields at the start. The list is symbols, but consumers can be tolerant of the fields being symbols or strings. I'll merge the XDR for protocol 23's XDR release in two days if there's no further feedback. |
I have a rather small suggestion: I think the SEP should specify that all the entries in map data are optional, or alternatively allow specifying that an entry is optional explicitly. In either case it should be able to represent optional entries. |
|
There are a multiple ways to achieve the optionality. 1️⃣ – Leveraging I think what you're describing is to omit the map entry when the value is type An upside of this approach is the spec is compact, we only need one spec entry for an event with many optional fields. Another upside is for events with many options and many possible combinations it is still possible to omit optional fields. In Another downside of optional and ommitted fields is that the data model has many shapes. Example of how this could be represented:
2️⃣ – Define multiple events An alternative would be that developers define multiple events with different shapes. This would be explicit, and more consistent with the behaviour of In this approach every event defined in code and in the spec has a single shape. However, events can overlap, which isn't something we could prevent even in the first case. A downside of this approach is a developer has to branch to omit fields, creating different structs instead of one. That would probably be inconvenient enough that folks might just not bother omitting and instead emit Example of defining a
Example of then also defining a
Example of instead defining a
Option 1️⃣ results in a single event spec with fluid shape. Embedded Option 2️⃣ results in two event specs with fixed shapes. Embedded Either approach doesn't change the definition of the XDR, but does change how we generate and interpret it, so we can defer this decision without blocking the XDR, but continue to collect thoughts from others on what would be best for consumers to build against. I'm inclined to go with 1️⃣ because:
Are there alternative approaches? @chowbao @sydneynotthecity @karthikiyer56 Thoughts on which approach would be better for consumers? |
Having worked for years with protobufs where all the fields are optional, I don't think this really is a downside, and it seems easy enough to solve on the client side once.
I think this just doesn't scale, as it would require 2^N event definitions for N optional fields. |
|
Agreed on both these items I'm just trying to capture the options. We might want to reconsider the contracttype behaviour with Option too, but we don't need to decide that here. I'll update the SEP with 1️⃣. The principle of 2️⃣ will need to be used too anyway because the SAC uses a single value data in some cases and a map in others. |
|
+1 for option1. |
|
An example of what's possible once contract event specs are available is here: It is a CLI that accepts a single event, and a list of specs to match against, and outputs a self-describing JSON view of the event by combining the internals of both. |
|
The content in the readme is super helpful, leigh.
|
A consumer wanting to generate self-describing events will need to decide what to do, in the context of how they are using the events. Maybe they discard those events, maybe they flag them for manual processing, maybe they map them to another format where numbers are used for the field names, maybe map them to a another top-level field such as `"unknown_params": [...]", etc.
I don't think SEP-48 should define an answer to those questions and a consumer can decide what to do. A consumer could consider events in the order they're included in the contract spec for a contract. The contract spec is a list/array/stream of specs. In most cases I hope that the prefixes will be sufficient at uniquely identifying an event. There will be some contracts, such as the Stellar Asset Contract with its transfer events and amount changing from a number to a map meaning there will be two schemas for the same prefix, and they should still be uniquely identifiable based on the top-level types in the params. I'm not sure it's worth doing much else beyond that. It's a similar problem to what if someone defines the same contract type twice in a contract. Problem exists today. |
|
There are other issues, such as what if the contract's wasm and spec changes over time and therefore a consumer can't figure out exactly which wasm-hash emitted which event. This is difficult to figure out. See this conversation for discussion: |
|
Linking out to other places where discussions are starting for how to incorporate the use of these event specifications: |
|
The pull request updating SEP-48 is here: |
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Hi all,
I propose we extend SEP-48 Contract Interface Spec to add support for events. I suggest this extension be made by modifying SEP-48 rather than writing a new SEP, but a new SEP would also work if folks think that is preferred.
Motivation
Today contract specs describe and document:
Events are only documented in comments and in SEPs, and not in the contract spec itself.
While this has little impact on contracts, since events are not consumable on chain by other contracts, it makes it more difficult to exchange information about the events with other developers or to offchain systems and applications.
Changes to XDR
A new
SCSpecEntrykind is added for events that stores the event name, fields, and data format.The event name maps to the first topic as a symbol. Reviewing all existing events on the network the majority of events take this shape. See events_topics_types query results.
Any other fields annotated to be located in the topics will have their value inserted into the topics in the order they appear in the struct.
The data format indicates whether the fields annotated to be located in the data are stored in a map, vec, or as a single value. This configuration allows for supporting all types of existing events that are seen on mainnet. See events_data_types query results.
If the data format is map, the data fields are stored in a map with each entry having its key be the symbol name of the field, and the value be the field value. Entries are sorted according to the rules for
ScMapentries.If the data format is vec, the data field values are stored in a vec in the order they appear in the struct.
If the data format is single-value, there must be only one data field. The value of that data field is stored as the data value.
Events and fields may be accompanied by docs.
The XDR changes are backwards compatible.
The XDR is expected to change as such:
Use Cases
There are a few use cases that can make use of events in the contract specs.
Contract Code
Contract code will better self describe the event being emitted. For example:
Today an event emitted appears in contract code as a very simple list:
And this change will require a type, either a struct or a function, so that there's a single location that defines the event once. That definition is what will cause the XDR to be built and written into the contract spec. Likely this will be modelled off the existing contract type struct definitions, but it could also be modelled as a function type.
Developer Products
Developer products such as stellar-cli, stellar.expert, lab, okashi.dev, etc will be able to display information about what events can be emitted.
For example, the tools will be able to include the shape of the events when inspecting the spec:
Products that Inspect and Map Events
Products that inspect and map events. For example, the recently released OpenZeppelin Monitor allows defining triggers based on events, and for EVM those triggers are defined based on event param names, where-as for Stellar today they are defined based on event param indexes, not names. For example: https://docs.openzeppelin.com/monitor/0.1.0/#stellar_variables cc @bramhoogenkamp @brozorec @janewang
Stellar SDKs / Event Bindings
Stellar SDKs can code generate event bindings in a variety of languages and data formats, such generating JS, Go, Python, Java, etc, with transformations from the XDR to those code types. For example:
Go:
Rust:
Data Products Dynamic Transformations
Data processing products such as stellar-cli, Go Stellar SDK's ingest pkg, Hubble, etc will be able to at runtime convert ContractEvent XDR into a simplified JSON format that renders the shape of the event itself, rather than what XDR-JSON does which is render the shape of the XDR.
For example, a ContractEvent XDR contains the following, rendered in XDR-JSON. Even though it is rendered in a human readable form, it is not self describing because the event structure does not contain the names of the fields.
{ "ext": "v0", "contract_id": "6994dcf9f3c32d623b91139d8906640387fb7ef8f6a8023a68774d23a075962e", "type_": "contract", "body": { "v0": { "topics": [ { "symbol": "transfer" }, { "address": "GBMBVAHBE6D4AJXJJVTBQTVU4G7SN4FEIJOL5YTOHZ4WCUMKQ52ANL2B" }, { "address": "GCAN5IE4PWMWSMFZCYQRCJM73MCPG5JD2R7WF5HIFHHHDBWSDJFIWX7X" } ], "data": { "i128": { "hi": 0, "lo": 10000 } } } } }With contract specs containing the shape of events, tooling could transform the ContractEvent XDR into a self-describing form.
{ "contract_id": "6994dcf9f3c32d623b91139d8906640387fb7ef8f6a8023a68774d23a075962e", "body": { "transfer": { "from": "GBMBVAHBE6D4AJXJJVTBQTVU4G7SN4FEIJOL5YTOHZ4WCUMKQ52ANL2B", "to": "GCAN5IE4PWMWSMFZCYQRCJM73MCPG5JD2R7WF5HIFHHHDBWSDJFIWX7X", "amount": { "i128": { "hi": 0, "lo": 10000 } } } } }Data Products Code Generated Static Transformations
Data processing products such as Go Stellar SDK's ingest pkg, etc can code generate event bindings in a variety of languages and data formats, such generating Proto IDL, Avro, etc, and generate transformations from the XDR to those formats. For example:
Or potentially a mapping to existing protos emitted by the TokenTransferProcessor (ref).
Downstream Impact
A type such as those demonstrated above in the use cases will need to be added to the Rust Soroban SDK, because each event must have a single location where it is described. It might be possible to annotate existing event call sites, but in any contract where the same event is published at multiple call sites duplicate schemas may be published, or worse inconsistent schemas may be published.
The above changes are the minimum work required to incorporate events into the contract specs. It will be possible and likely that other tools such as the stellar-cli, Go Stellar SDK ingest pkg, etc may wish to make use of the event specs to do ideas noted in the motivation section, but any adoption is independent of this proposal.
Prior Work
This proposal is based on concepts that have been discussed a few times in the last couple years, and this proposal takes the learnings from those prior conversations and prior work:
Other Discussions
Discord: https://discord.com/channels/897514728459468821/1368933406057955519/1368933406057955519
All reactions