[cmd/opampsupervisor] Support runtime config rollback (#47853)

Co-authored-by: Evan Bradley <11745660+evan-bradley@users.noreply.github.com>
Signed-off-by: Douglas Camata <159076+douglascamata@users.noreply.github.com>

Author: douglascamata

.chloggen/supervisor-config-rollback.yaml

@@ -0,0 +1,31 @@
+# Use this changelog template to create an entry for release notes.
+
+# One of 'breaking', 'deprecation', 'new_component', 'enhancement', 'bug_fix'
+change_type: enhancement
+
+# The name of the component, or a single word describing the area of concern, (e.g. receiver/filelog)
+component: cmd/opampsupervisor
+
+# A brief description of the change.  Surround your text with quotes ("") if it needs to start with a backtick (`).
+note: Add support for restoring the last working remote configuration when an
+  incoming remote configuration fails to apply
+
+# Mandatory: One or more tracking issues related to the change. You can use the PR number here if no issue exists.
+issues: [47192]
+
+# (Optional) One or more lines of additional information to render under the primary note.
+# These lines will be padded with 2 spaces and then inserted directly into the document.
+# Use pipe (|) for multiline entries.
+subtext: |
+  This behavior is controlled under the configuration option `agent.automatic_config_rollback`.
+  When an old remote configuration is restored, the agent will report an "applied" status again
+  for this configuraiton.
+
+# If your change doesn't affect end users or the exported elements of any package,
+# you should instead start your pull request title with [chore] or use the "Skip Changelog" label.
+# Optional: The change log or logs in which this entry should be included.
+# e.g. '[user]' or '[user, api]'
+# Include 'user' if the change is relevant to end users.
+# Include 'api' if there is a change to a library API.
+# Default: '[user]'
+change_logs: [user]

cmd/opampsupervisor/README.md

@@ -16,7 +16,7 @@ The design is still undergoing changes, and as such this implementation may chan
 
 Binary and container images for the Supervisor are available under tags starting with `cmd/opampsupervisor` [here](https://github.com/open-telemetry/opentelemetry-collector-releases/tags).
 
-## More information.
+## More information
 
 If you'd like to learn more about OpAMP, see the
 [OpAMP specification](https://github.com/open-telemetry/opamp-spec/blob/main/specification.md#open-agent-management-protocol).
@@ -121,6 +121,7 @@ agent:
 ## Persistent data storage
 
 The supervisor persists some data to disk in order to mantain state between restarts. The directory where this data is stored may be specified via the supervisor configuration:
+
 ```yaml
 storage:
   directory: "/path/to/storage/dir"
@@ -180,6 +181,24 @@ agent:
 
 See [examples/supervisor_fallback.yaml](./examples/supervisor_fallback.yaml) for a complete example configuration with fallback enabled.
 
+## Automatic Remote Configuration Rollback
+
+The Supervisor supports automatic rollback when a remote configuration received from
+the OpAMP backend causes the Collector to fail to start correctly. This feature is
+**disabled by default** and can be enabled in the Supervisor configuration file:
+
+```yaml
+agent:
+  automatic_config_rollback: true
+```
+
+When enabled, the Supervisor will watch the health of the Collector after applying remote
+configurations received from the OpAMP backend and it will cache the last working one in
+disk, besides the last received. If a remote configurations causes the Collector
+to fail to start correctly, the Supervisor will automatically swap to the last working one
+and restart the Collector. During this process, a new "APPLIED" status will be reported for
+the last working configuration.
+
 ## Status
 
 The OpenTelemetry OpAMP Supervisor is intended to be the reference