Use APCu as Composer level 2 optimisation strategy

[fredden]

Description

Composer offers two levels of autoloader optimisation. Currently level 1 is enabled; this pull request enables level 2 optimisations using the safest-available option. https://getcomposer.org/doc/articles/autoloader-optimization.md

Fixed Issues (if relevant)

None

Manual testing scenarios

1. Deploy to website on Magento Cloud using this version of ece-tools
2. Note that level 2 autoloader optimisations are enabled.

Release notes

Use APCu as Composer level 2 optimisation strategy. This is a performance improvement that comes with a dependency on the APCu extension. This extension is installed on Magento Cloud automatically.

Associated documentation updates

• Add APCu to 'always installed' list devdocs#8864

Contribution checklist

• Pull request has a meaningful description of its purpose
• Pull request introduces user-facing changes and includes meaningful updates for any required release notes and documentation changes
• All commits are accompanied by meaningful commit messages
• All new or changed code is covered with unit/integration tests (if applicable)
• All automated tests passed successfully (all builds on Travis CI are green)

[magento-cloud-ft-jenkins-svc]

Static Analysis & Code Style Results

❌  One or more static analysis or code style checks have failed.

PHP Codesniffer Output

Could not read phpcs output.

PHP Mess Detector Output

Could not read phpmd output.

PHPStan Output

Could not read phpstan output.

This comment was generated by Jenkins job ece-tools/static build 22.

[magento-cloud-ft-jenkins-svc]

Unit & Integration Test Results

❌  One or more unit or integration tests have failed.

Unit Test Output

Could not read unit test output.

Integration Test Output

Could not read integration test output.

This comment was generated by Jenkins job ece-tools/unit build 23.

[magento-cloud-ft-jenkins-svc]

Functional Acceptance Test Results

❌  One or more functional acceptance tests have failed.

PHP 7.1

❓  No scenario results found

PHP 7.2

❓  No scenario results found

PHP 7.3

❓  No scenario results found

PHP 7.4

❓  No scenario results found

Output for failed tests is below. If many tests have failed only the first 5 will be included. If you need additional information please reach out to the Magento Cloud team for more details.

This comment was generated by Jenkins job ece-tools/functional build 37.

[fredden]

@gabrieldagama How can we get the PHP extension APCu installed automatically on Magento Cloud which is a requirement for this change?

[navarr]

Does this provide any substantive benefits over Composer Level 1 + Opcache?

[fredden]

Yes. Opcache is an important but distinct optimisation.

Opcache will cache the compiled contents of a file in memory. Level 2 composer cache will cache which file-system path contains a class. Without a level 2 cache, the autoloader will scan several file-system locations until it finds a suitably named file for the corresponding class. When a file is found, it will be read & compiled (or pulled from opcache if enabled & available) and the class evaluated for use as normal. With a level 2 cache, the autoloader will read the file location from this cache, rather than having to test every file-system location in sequence until a match is found. File-system access can be expensive.

Let's talk through an example. Consider a scenario with an include path of "a, b, c, d, e, f, g" and a class existing in path "g." Without a level 2 cache, each time the class is required, the autoloader will:

• ask the file-system for "a/class.php" which will return no results, then
• ask the file-system for "b/class.php" which will return no results, then
• ask the file-system for "c/class.php" which will return no results, then
• ask the file-system for "d/class.php" which will return no results, then
• ask the file-system for "e/class.php" which will return no results, then
• ask the file-system for "f/class.php" which will return no results, then
• ask the file-system for "g/class.php" which will return a file, then
• this file will be read & compiled (or pulled from opcache if enabled & available).

With a primed* level 2 cache, each time the class is required, the autoloader will:

• inspect the level 2 cache for the file-system path for the requested class, then
• ask the file-system for "g/class.php" which will return a file, then
• this file will be read & compiled (or pulled from opcache if enabled & available).

* Depending on the level 2 cache method chosen, a class may need to be requested once in order to have its path cached. (This is the case for APCu.) In this example, we assume this has already happened or a method not requiring this has been chosen.

Hopefully this helps show why both optimisations are useful.

[navarr]

@fredden With composer dumpautoload -o which is what I believe Cloud uses already, composer will first check a hashmap in a PHP file that maps the class name to a file path, as opposed to searching all the potential paths. (vendor/composer/autoload_static.php). With that loaded into OpCache, does APCu provide any additional benefit?

[fredden]

This is the code being referring to:
https://github.com/composer/composer/blob/9a32bf9709960017f944753c68ab05c8de94add0/src/Composer/Autoload/ClassLoader.php#L361-L394

Yes, the level 1 / class map is always used first, and will often be effective for finding classes. The level 2 cache is useful for classes that did not exist at the time the autoloader was dumped. For example, when running php bin/magento without any arguments will search for several classes that do not exist. With a level 2 optimisation enabled, these will not need to touch the file-system. (Yes, using the APCu method for level 2 will not be effective for command-line usage. Marking the class-map as authoritative would be more efficient, although cannot be assumed to be safe in all scenarios.)

For reference, I've done a quick check this evening on a Magento v2.4.2-p1 website. When running php bin/magento (with no arguments), 384 classes that do not exist were requested matching Magento\{name}\Setup\ConfigOptionsList which lead to file_exists() being called on 1,536 paths in the autoloader.

See also https://getcomposer.org/doc/articles/autoloader-optimization.md

[phoenix-bjoern]

According to the Composer documentation the APCu optimization can be enabled in the composer.json too (https://getcomposer.org/doc/articles/autoloader-optimization.md#optimization-level-2-a-authoritative-class-maps). So adding

{
    "config": {
        "apcu-autoloader": true
    }
}

should enable APCu caching in Composer and makes this patch obsolete, correct?
If yes the Magento documentation (https://experienceleague.adobe.com/docs/commerce-operations/performance-best-practices/deployment-flow.html?lang=en#preprocess-dependency-injection-instructions) could be updated instead.

[fredden]

@phoenix-bjoern the required extension is not yet installed / available by default on Adobe Cloud.

[phoenix-bjoern]

@fredden I've created a ticket and referenced the DevDocs where Adobe explicitly recommends the installation of APCu (https://experienceleague.adobe.com/docs/commerce-operations/implementation-playbook/infrastructure/performance/recommendations.html?lang=en#php-fpm-settings). The extension got installed by the Adobe support team. Maybe just try it again ;-)