Registry and Plugin (Backend)

[psilabs-dev]

• Issue: [Feature request] Disable/hide metadata plugin #1146
• Plugins demo registry repo: https://github.com/psilabs-dev/lrr-plugins-demo

Under testing, lots of code may be subject to change. There are quite a number of avenues where code might break.

This backend-only PR removes all (non-local) plugins from LRR, and implements the registry system and plugin management.

• Multiple registry support, currently limited to single-registry with future compatibility
• Management features, such as installing/uninstalling/upgrading plugins, plugin conflict resolution, management APIs.
• Support for installing/uninstalling managed plugins
• Redis is now responsible for plugin package presence management (previously INC's). Which makes uninstall/install easier.
• Configuration-related features, such as metadata plugin priority, enablement, and dropdown hiding. Add metadata plugin priority support.

Obligatory security warning, as downloading untrusted arbitrary files and allowing code execution from the internet requires extra care which was not a consideration when plugins were the responsibility of LRR. The boundary for this PR is that LRR will not audit or review files/registries/licenses for the user, the user must trust registries and audit them personally/take responsibility for any bugs/vulnerabilities within plugins.

Some examples of ways to be compromised, which LRR cannot protect against: registry URL typos/typo squatting, DNS poison/certificate updates, hijacked registries/supply chain attacks

(tbh, with this I'm also just wondering whether plugin installations should require a restart altogether)

(PS: maybe we should support uninstalling sideloaded plugins?)

---

Below is a (partial) copy/paste from a local skill file and serves as the basis of this PR feature development. This may become outdated over time.

Registry System

The (plugin) registry is a centralized LRR manager. While it mainly manages plugins, the registry system is designed to be general purpose and support other resources as well.

Beginning with this PR, future plugins are no longer the responsibility of the LRR project. All plugins within LRR which do not operate only on files (i.e. they operate by performing API calls or fetch from other providers) will be cleared out, and downloaded exclusively through third party registries.

A registry may be either git-based, or local, and must contain a registry.json file which contains information about all plugins provided by the registry.

Plugin Management/Provenance and Configuration Feature

Plugins come in 3 archetypes (or types if not ambiguous with the 4 LRR plugin types i.e. "metadata"/"login"/"script"/"download"):

• default plugins, which exist prior to this PR.
• sideloaded plugins, which are added by the user via upload portal (under "Plugin/Sideloaded").
• managed plugins, which are solely managed by LRR's registry and plugin management system. This is the new type introduced by this PR.

Under this context, management also refers to the management of managed plugins. To put this in concrete terms and expressed as a capability-supported matrix:

action	default	sideloaded	managed
direct FS actions	no	yes	no/not recommended
install/upgrade	no	no	yes
uninstall	no	no	yes
location	Plugin/{TYPE}	Plugin/Sideloaded	Plugin/Managed

Where TYPE is one of "metadata", "login", "script", or "download". Module::Pluggable discovers all .pm files recursively under LANraragi::Plugin.

Two plugins conflict if any of the following are shared:

• namespace
• filepath
• package name

Installation/upgrading refers to downloading (and possibly replacing) a managed plugin to its appropriate location, and reloading perl plugin modules. Uninstallation refers to removing the managed plugin file and reloading modules.

If a managed plugin exists, it can be upgraded to another plugin as provided by the registry. Managed plugins cannot be installed if a conflicting plugin exists, but can be force-installed if the existing plugin is managed. Otherwise, if the plugin is sideloaded or default, then the user must manually remove it; force-install will not work here.

Default plugins are always accompanied by LRR installations; therefore, it is not recommended for users to install plugins which conflict with default plugin in some way, and for registry maintainers/developers to write conflicting plugins.

Plugin Provenance

All managed plugins have provenance/registry origin. A managed plugin may be orphaned, i.e. belong to a registry which does not exist in the LRR server. This can happen if the user adds a registry and installs a plugin, then removes the registry.

A plugin is upgradable if the version is different. That's it. The checksum does not contribute to determining whether a plugin can be upgraded; it serves as file integrity check during transit.

The PR only concerns single-registry systems, so we have the following cases to handle for a managed plugin and its case matrix (ignoring conflict with default/sideloaded plugins):

• Whether a plugin is available in the registry
• Whether a plugin is upgradable in the registry
  • If a plugin is not available, or does not exist, then clearly it is not upgradable.
• Whether a plugin exists in the file-system
• Whether a plugin has valid syntax/compiles

If a plugin's registry field is null, then it's either a default or sideloaded plugin.

If a plugin does not exist/exists in the filesystem but is available in the registry, then we can install/upgrade it, respectively.

Metadata Plugin Configuration

Plugin configuration refers to non-provenance properties of plugins, such as whether a plugin is enabled (auto-runs on new archives), among various arg settings under plugins, such as optional cookie, user agent, or whether to save title as Japanese/English.

Plugin configuration is not a property of a plugin, but that of how an administrator uses the plugin.

Two new features of metadata plugin configuration:

• Ability to hide/unhide plugins, ie. remove plugin from the dropdown under "Use plugin" dropdown under Batch Operations page, and "Import Tags from Plugin" dropdown under archive editor page.
• Ability to sort metadata plugin order, ie. control exactly how/where an enabled plugin is run relative to other enabled plugins.

These are orthogonal configurations: whether a plugin is hidden does not affect the plugin's execution order.

Explanation of properties:

• enabled is a property of existing metadata plugins (i.e. present in local FS). Uninstalled or non-metadata plugins cannot be enabled.
• priority is a property of enabled metadata plugins (obviously must exist). Disabled plugins or non-metadata plugins cannot have a priority/order, it would be meaningless. Priorities should be unique.
• hidden is a property of all existing metadata plugins when listed by LRR. A plugin which is not installed cannot be assigned a hidden value.

Plugin Ordering Functionality

A drag-and-drop functionality may be required for metadata plugin sorting. If the implementation goes this direction, two plugin pools must exist:

• A pool for disabled metadata plugins,
• A pool for enabled metadata plugins.

Dragging a plugin from disabled to enabled 1) enables the plugin and 2) sets a priority depending on its position among plugins in the enabled pool.

Dragging a plugin from enabled to disabled 1) disables the plugin and 2) updates the priorities of existing enabled plugins. The disabled plugins must then be sorted.

Only existing plugins may be dragged to the enabled pool. If a plugin is installed, it can be moved to the enabled pool too.