Added require-dev dependencies from mainline Magento 2.3.0

[ryantfowler]

Description

The Magento Cloud Template codebase does not include the same set of packages defined in the "on-prem" version of Magento 2

Action Taken

1. Added in the require-dev dependencies which are listed here in the same major/minor/patch versions of Magento2 on GitHub. composer require --dev friendsofphp/php-cs-fixer:~2.13.0 lusitanian/oauth:~0.8.10 magento/magento2-functional-testing-framework:2.3.9 --no-update

2. Attempted to update those specific dependencies. composer update friendsofphp/php-cs-fixer lusitanian/oauth magento/magento2-functional-testing-framework pdepend/pdepend phpmd/phpmd phpunit/phpunit sebastian/phpcpd squizlabs/php_codesniffer ( also tried appending --with-all-dependencies )

3. A conflict notification was presented, which attempted to install every satisfiable version of symfony/options-resolver, which Composer was unable to successfully do.

Dependency resolution completed in 0.618 seconds
Your requirements could not be resolved to an installable set of packages.

  Problem 1
    - Conclusion: don't install friendsofphp/php-cs-fixer v2.13.3
    - Conclusion: don't install friendsofphp/php-cs-fixer v2.13.2
    - Conclusion: don't install friendsofphp/php-cs-fixer v2.13.1
    - Conclusion: remove symfony/options-resolver v2.8.49
    - Installation request for friendsofphp/php-cs-fixer ~2.13.0 -> satisfiable by friendsofphp/php-cs-fixer[v2.13.0, v2.13.1, v2.13.2, v2.13.3].
    - Conclusion: don't install symfony/options-resolver v2.8.49
    - friendsofphp/php-cs-fixer v2.13.0 requires symfony/options-resolver ^3.0 || ^4.0 -> satisfiable by symfony/options-resolver[v3.0.0, v3.0.0-BETA1, v3.0.1, v3.0.2, v3.0.3, v3.0.4, v3.0.5, v3.0.6, v3.0.7, v3.0.8, v3.0.9, v3.1.0, v3.1.0-BETA1, v3.1.0-RC1, v3.1.1, v3.1.10, v3.1.2, v3.1.3, v3.1.4, v3.1.5, v3.1.6, v3.1.7, v3.1.8, v3.1.9, v3.2.0, v3.2.0-BETA1, v3.2.0-RC1, v3.2.0-RC2, v3.2.1, v3.2.10, v3.2.11, v3.2.12, v3.2.13, v3.2.14, v3.2.2, v3.2.3, v3.2.4, v3.2.5, v3.2.6, v3.2.7, v3.2.8, v3.2.9, v3.3.0, v3.3.0-BETA1, v3.3.0-RC1, v3.3.1, v3.3.10, v3.3.11, v3.3.12, v3.3.13, v3.3.14, v3.3.15, v3.3.16, v3.3.17, v3.3.18, v3.3.2, v3.3.3, v3.3.4, v3.3.5, v3.3.6, v3.3.7, v3.3.8, v3.3.9, v3.4.0, v3.4.0-BETA1, v3.4.0-BETA2, v3.4.0-BETA3, v3.4.0-BETA4, v3.4.0-RC1, v3.4.0-RC2, v3.4.1, v3.4.10, v3.4.11, v3.4.12, v3.4.13, v3.4.14, v3.4.15, v3.4.16, v3.4.17, v3.4.18, v3.4.19, v3.4.2, v3.4.20, v3.4.21, v3.4.22, v3.4.23, v3.4.3, v3.4.4, v3.4.5, v3.4.6, v3.4.7, v3.4.8, v3.4.9, v4.0.0, v4.0.0-BETA1, v4.0.0-BETA2, v4.0.0-BETA3, v4.0.0-BETA4, v4.0.0-RC1, v4.0.0-RC2, v4.0.1, v4.0.10, v4.0.11, v4.0.12, v4.0.13, v4.0.14, v4.0.15, v4.0.2, v4.0.3, v4.0.4, v4.0.5, v4.0.6, v4.0.7, v4.0.8, v4.0.9, v4.1.0, v4.1.0-BETA1, v4.1.0-BETA2, v4.1.0-BETA3, v4.1.1, v4.1.10, v4.1.11, v4.1.2, v4.1.3, v4.1.4, v4.1.5, v4.1.6, v4.1.7, v4.1.8, v4.1.9, v4.2.0, v4.2.0-BETA1, v4.2.0-BETA2, v4.2.0-RC1, v4.2.1, v4.2.2, v4.2.3, v4.2.4].
    - Can only install one of: symfony/options-resolver[v3.0.0, v2.8.49].
...
    - Can only install one of: symfony/options-resolver[v4.2.4, v2.8.49].
    - Installation request for symfony/options-resolver (locked at v2.8.49) -> satisfiable by symfony/options-resolver[v2.8.49].

4. I began investigation as to why symfony/options-resolver, with the version constraint required by friendsofphp/php-cs-fixer could not be installed. I found that it is due to msp/twofactorauth requiring endroid/qr-code, which then led to symfony/options-resolver:2.8.49 being installed, as at the time of msp/twofactorauth being included, friendsofphp/php-cs-fixer wasn't present ( nor was anything else in the codebase present which required symfony/options-resolver ) and therefore Composer didn't pull in a version that satisfied the requirements for both msp/twofactorauth and friendsofphp/php-cs-fixer.

composer why-not -r symfony/options-resolver "^3.0 || ^4.0"
magento/project-enterprise-edition  2.3.0  requires  magento/magento-cloud-metapackage (>=2.3.0 <2.3.1)  
magento/product-enterprise-edition  2.3.0  requires  magento/product-community-edition (2.3.0)           
magento/magento-cloud-metapackage   2.3.0  requires  magento/product-enterprise-edition (2.3.0)          
magento/product-community-edition   2.3.0  requires  msp/twofactorauth (^3.0.0)                          
magento/product-enterprise-edition  2.3.0  requires  msp/twofactorauth (^3.0.0)                          
msp/twofactorauth                   3.0.0  requires  endroid/qr-code (^2.5)                              
endroid/qr-code                     2.5.1  requires  symfony/options-resolver (^2.7) 

5. Visually verified what the previous command showed ( and what I had manually verified ).

composer why -t symfony/options-resolver

symfony/options-resolver v2.8.49 Symfony OptionsResolver Component
└──endroid/qr-code 2.5.1 (requires symfony/options-resolver ^2.7)
   └──msp/twofactorauth 3.0.0 (requires endroid/qr-code ^2.5)
      ├──magento/product-community-edition 2.3.0 (requires msp/twofactorauth ^3.0.0)
      │  └──magento/product-enterprise-edition 2.3.0 (requires magento/product-community-edition 2.3.0)
      │     └──magento/magento-cloud-metapackage 2.3.0 (requires magento/product-enterprise-edition 2.3.0)
      │        └──magento/project-enterprise-edition 2.3.0 (requires magento/magento-cloud-metapackage >=2.3.0 <2.3.1)
      └──magento/product-enterprise-edition 2.3.0 (requires msp/twofactorauth ^3.0.0)
         └──magento/magento-cloud-metapackage 2.3.0 (requires magento/product-enterprise-edition 2.3.0)
            └──magento/project-enterprise-edition 2.3.0 (requires magento/magento-cloud-metapackage >=2.3.0 <2.3.1)

Final Solution

1. Deleted the existing composer.lock file in the codebase. rm -f composer.lock

2. Just let Composer "figure it out" by resolving all the dependencies again, which now included friendsofphp/php-cs-fixer and it's requirements, namely symfony/options-resolver. composer install

Feedback Loop

If there's a more desirable approach to solving this problem, please let me know and I'll take action on that, but by looking at this commit, I figured that making an update to the entire codebase's dependencies, based on the configured version constraints, wasn't going to be a problem.

[hostep]

This was already attempted before in https://github.com/magento/magento-cloud/pull/200 and not approved (but the reasoning is weird, they should really run composer install --no-dev during build step, but that's just my opinion).

Also, depending on the outcome of https://github.com/magento/magento2/pull/21565 it might be that certain (or all) dev dependencies are being removed entirely from base Magento's composer.json file.

[ryantfowler]

@hostep yep, I agree that the --no-dev should be added to the build step. But I also know that there are some reasons for not doing this, such as developers complaining that some composer dependencies that were incorrectly added to the require-dev section weren't being deployed ... so the intent was to make sure that all dependencies were deployed. I'm not defending this, just offering up an explanation.

Regarding Fooman's PR, yea that'll be interesting to see how it goes.

Thanks