DoctrineMetadataCacheWarmer must load metadata first, check priority of you warmers when updated from 2.2.4 -> 2.3.0

[ivanbogomoloff]

Hello all. I got this message when run bin/console cache:clear or bin/console cache:warmup.

Conditions:

I have symfony 5.2.4 project out box Then i run composer update doctrine-bundle updated to 2.2.4 -> 2.3.0.

Then i got error.

DoctrineMetadataCacheWarmer must load metadata first, check priority of you warmers

[dmaicher]

I cannot reproduce this on any of my projects. I also tested it on symfony-demo and it works fine.

Can you provide a reproducer @ivanbogomoloff ?

[ivanbogomoloff]

I cannot reproduce this on any of my projects. I also tested it on symfony-demo and it works fine.

Can you provide a reproducer @ivanbogomoloff ?

No, i can't. It's magically gone. But now It occurs only in project with a lot of code, but when i copy composer.json and try to create raw new project (i mean out box is composer command create project) it works fine. Little notice, that error occurs only after rm -rf var/cache/prod and when APP_ENV=prod

[alexrozz]

I have the same issue on cache:clear, doctrine:migrations:migrate and any other. Could anybody explain the reason of this exception? What warmers must run before? In DoctrineMetadataCacheWarmer:47 there is a check if metadata exists, then crush. What the purpose, anybody knows?

[dmaicher]

Could anybody explain the reason of this exception? What warmers must run before? In DoctrineMetadataCacheWarmer:47 there is a check if metadata exists, then crush. What the purpose, anybody knows?

@alexrozz

With DoctrineBundle 2.3.0 we introduced a new and faster caching mechanism for the Doctrine meta data. For this to work properly we have to ensure that DoctrineMetadataCacheWarmer is the first to actually load all meta data so the cache can be properly warmed up.

Are you able to create a reproducer? Happy to take a look then

[jkabat]

I'm facing the issue as well. In my project it has relation to "msgphp/msgphp" package. In my understanding problematic is the change of warmers priority introduced in https://github.com/doctrine/DoctrineBundle/pull/1260 - other packages may be bitten as well if they depend on specific priority value.

[dmaicher]

In my understanding problematic is the change of warmers priority introduced in #1260

we did not change any priority of cache warmers :wink: We introduced a new DoctrineMetadataCacheWarmer with a high priority of 1000.

Again: if someone can provide a reproducer I can re-open this and take a look.

[jkabat]

I have created a reproducer, but I think it is related with the library Im using (it processes/loads entity metadata and should probably be run before doctrine warmup step which is not true anymore as new warmer has priority 1000, while msgphp one only 100).

https://github.com/jkabat/doctrine-bundle-test

bin/console cache:clear --env=prod --no-debug # http://prntscr.com/10zwd8u

In my application setup I had better insight with cache:warmup command: http://prntscr.com/10zwfkn. Unfortunately in reproducer I do not see this kind of error.

[ostrolucky]

Reproducer doesn't work for me

❯ /usr/local/Cellar/[email protected]/7.4.16/bin/php bin/console cache:clear --env=prod --no-debug -vvv

 // Clearing the cache for the prod environment with debug false

 // Clearing outdated warmup directory...

 // Warming up cache...

[info] User Deprecated: The "metadata_cache_driver" configuration key is deprecated. PHP Array cache is now automatically registered when %kernel.debug% is false.

[info] User Deprecated: The "metadata_cache_driver" configuration key is deprecated. PHP Array cache is now automatically registered when %kernel.debug% is false.


In MappingException.php line 65:

  [Doctrine\Persistence\Mapping\MappingException]
  No mapping file found named 'EmailPassword.orm.xml' for class 'MsgPhp\User\Credential\EmailPassword'.


Exception trace:
  at /Users/gabriel.ostrolucky/Documents/doctrine-bundle-test/vendor/doctrine/persistence/lib/Doctrine/Persistence/Mapping/MappingException.php:65
 Doctrine\Persistence\Mapping\MappingException::mappingFileNotFound() at /Users/gabriel.ostrolucky/Documents/doctrine-bundle-test/vendor/doctrine/persistence/lib/Doctrine/Persistence/Mapping/Driver/SymfonyFileLocator.php:230
 Doctrine\Persistence\Mapping\Driver\SymfonyFileLocator->findMappingFile() at /Users/gabriel.ostrolucky/Documents/doctrine-bundle-test/vendor/doctrine/persistence/lib/Doctrine/Persistence/Mapping/Driver/FileDriver.php:92
 Doctrine\Persistence\Mapping\Driver\FileDriver->getElement() at /Users/gabriel.ostrolucky/Documents/doctrine-bundle-test/vendor/doctrine/orm/lib/Doctrine/ORM/Mapping/Driver/XmlDriver.php:67
 Doctrine\ORM\Mapping\Driver\XmlDriver->loadMetadataForClass() at /Users/gabriel.ostrolucky/Documents/doctrine-bundle-test/vendor/doctrine/persistence/lib/Doctrine/Persistence/Mapping/Driver/MappingDriverChain.php:79
 Doctrine\Persistence\Mapping\Driver\MappingDriverChain->loadMetadataForClass() at /Users/gabriel.ostrolucky/Documents/doctrine-bundle-test/vendor/doctrine/orm/lib/Doctrine/ORM/Mapping/ClassMetadataFactory.php:149
 Doctrine\ORM\Mapping\ClassMetadataFactory->doLoadMetadata() at /Users/gabriel.ostrolucky/Documents/doctrine-bundle-test/vendor/doctrine/persistence/lib/Doctrine/Persistence/Mapping/AbstractClassMetadataFactory.php:306
 Doctrine\Persistence\Mapping\AbstractClassMetadataFactory->loadMetadata() at /Users/gabriel.ostrolucky/Documents/doctrine-bundle-test/vendor/doctrine/orm/lib/Doctrine/ORM/Mapping/ClassMetadataFactory.php:79
 Doctrine\ORM\Mapping\ClassMetadataFactory->loadMetadata() at /Users/gabriel.ostrolucky/Documents/doctrine-bundle-test/vendor/doctrine/persistence/lib/Doctrine/Persistence/Mapping/AbstractClassMetadataFactory.php:184
 Doctrine\Persistence\Mapping\AbstractClassMetadataFactory->getMetadataFor() at /Users/gabriel.ostrolucky/Documents/doctrine-bundle-test/vendor/msgphp/domain/Infrastructure/Doctrine/Event/ObjectMappingListener.php:163
 MsgPhp\Domain\Infrastructure\Doctrine\Event\ObjectMappingListener->getMetadata() at /Users/gabriel.ostrolucky/Documents/doctrine-bundle-test/vendor/msgphp/domain/Infrastructure/Doctrine/Event/ObjectMappingListener.php:100
 MsgPhp\Domain\Infrastructure\Doctrine\Event\ObjectMappingListener->processFieldMapping() at /Users/gabriel.ostrolucky/Documents/doctrine-bundle-test/vendor/msgphp/domain/Infrastructure/Doctrine/Event/ObjectMappingListener.php:77
 MsgPhp\Domain\Infrastructure\Doctrine\Event\ObjectMappingListener->processClassFields() at /Users/gabriel.ostrolucky/Documents/doctrine-bundle-test/vendor/msgphp/domain/Infrastructure/Doctrine/Event/ObjectMappingListener.php:81
 MsgPhp\Domain\Infrastructure\Doctrine\Event\ObjectMappingListener->processClassFields() at /Users/gabriel.ostrolucky/Documents/doctrine-bundle-test/vendor/msgphp/domain/Infrastructure/Doctrine/Event/ObjectMappingListener.php:62
 MsgPhp\Domain\Infrastructure\Doctrine\Event\ObjectMappingListener->loadClassMetadata() at /Users/gabriel.ostrolucky/Documents/doctrine-bundle-test/vendor/symfony/doctrine-bridge/ContainerAwareEventManager.php:64
 Symfony\Bridge\Doctrine\ContainerAwareEventManager->dispatchEvent() at /Users/gabriel.ostrolucky/Documents/doctrine-bundle-test/vendor/doctrine/orm/lib/Doctrine/ORM/Mapping/ClassMetadataFactory.php:239
 Doctrine\ORM\Mapping\ClassMetadataFactory->doLoadMetadata() at /Users/gabriel.ostrolucky/Documents/doctrine-bundle-test/vendor/doctrine/persistence/lib/Doctrine/Persistence/Mapping/AbstractClassMetadataFactory.php:306
 Doctrine\Persistence\Mapping\AbstractClassMetadataFactory->loadMetadata() at /Users/gabriel.ostrolucky/Documents/doctrine-bundle-test/vendor/doctrine/orm/lib/Doctrine/ORM/Mapping/ClassMetadataFactory.php:79
 Doctrine\ORM\Mapping\ClassMetadataFactory->loadMetadata() at /Users/gabriel.ostrolucky/Documents/doctrine-bundle-test/vendor/doctrine/persistence/lib/Doctrine/Persistence/Mapping/AbstractClassMetadataFactory.php:184
 Doctrine\Persistence\Mapping\AbstractClassMetadataFactory->getMetadataFor() at /Users/gabriel.ostrolucky/Documents/doctrine-bundle-test/vendor/doctrine/persistence/lib/Doctrine/Persistence/Mapping/AbstractClassMetadataFactory.php:90
 Doctrine\Persistence\Mapping\AbstractClassMetadataFactory->getAllMetadata() at /Users/gabriel.ostrolucky/Documents/doctrine-bundle-test/vendor/doctrine/doctrine-bundle/CacheWarmer/DoctrineMetadataCacheWarmer.php:51
 Doctrine\Bundle\DoctrineBundle\CacheWarmer\DoctrineMetadataCacheWarmer->doWarmUp() at /Users/gabriel.ostrolucky/Documents/doctrine-bundle-test/vendor/symfony/framework-bundle/CacheWarmer/AbstractPhpFileCacheWarmer.php:49
 Symfony\Bundle\FrameworkBundle\CacheWarmer\AbstractPhpFileCacheWarmer->warmUp() at /Users/gabriel.ostrolucky/Documents/doctrine-bundle-test/vendor/symfony/http-kernel/CacheWarmer/CacheWarmerAggregate.php:97
 Symfony\Component\HttpKernel\CacheWarmer\CacheWarmerAggregate->warmUp() at /Users/gabriel.ostrolucky/Documents/doctrine-bundle-test/vendor/symfony/http-kernel/Kernel.php:633
 Symfony\Component\HttpKernel\Kernel->initializeContainer() at /Users/gabriel.ostrolucky/Documents/doctrine-bundle-test/vendor/symfony/http-kernel/Kernel.php:136
 Symfony\Component\HttpKernel\Kernel->boot() at /Users/gabriel.ostrolucky/Documents/doctrine-bundle-test/vendor/symfony/http-kernel/Kernel.php:153
 Symfony\Component\HttpKernel\Kernel->reboot() at /Users/gabriel.ostrolucky/Documents/doctrine-bundle-test/vendor/symfony/framework-bundle/Command/CacheClearCommand.php:189
 Symfony\Bundle\FrameworkBundle\Command\CacheClearCommand->warmup() at /Users/gabriel.ostrolucky/Documents/doctrine-bundle-test/vendor/symfony/framework-bundle/Command/CacheClearCommand.php:129
 Symfony\Bundle\FrameworkBundle\Command\CacheClearCommand->execute() at /Users/gabriel.ostrolucky/Documents/doctrine-bundle-test/vendor/symfony/console/Command/Command.php:255
 Symfony\Component\Console\Command\Command->run() at /Users/gabriel.ostrolucky/Documents/doctrine-bundle-test/vendor/symfony/console/Application.php:1027
 Symfony\Component\Console\Application->doRunCommand() at /Users/gabriel.ostrolucky/Documents/doctrine-bundle-test/vendor/symfony/framework-bundle/Console/Application.php:97
 Symfony\Bundle\FrameworkBundle\Console\Application->doRunCommand() at /Users/gabriel.ostrolucky/Documents/doctrine-bundle-test/vendor/symfony/console/Application.php:273
 Symfony\Component\Console\Application->doRun() at /Users/gabriel.ostrolucky/Documents/doctrine-bundle-test/vendor/symfony/framework-bundle/Console/Application.php:83
 Symfony\Bundle\FrameworkBundle\Console\Application->doRun() at /Users/gabriel.ostrolucky/Documents/doctrine-bundle-test/vendor/symfony/console/Application.php:149
 Symfony\Component\Console\Application->run() at /Users/gabriel.ostrolucky/Documents/doctrine-bundle-test/bin/console:42

cache:clear [--no-warmup] [--no-optional-warmers] [-h|--help] [-q|--quiet] [-v|vv|vvv|--verbose] [-V|--version] [--ansi] [--no-ansi] [-n|--no-interaction] [-e|--env ENV] [--no-debug] [--] <command>

❯

[Eydamos]

I have the same issue. Unfortunately the project is private so I can't share it. I also, up to this point, failed to reproduce the behaviour in a new project. But after hours of debugging I can already give some insight why this is happening.

In my project I have several connections/entity manager and therefore multiple cache warmer are registered all with priority 1000.

For simplicity let's say we have the connections conn_a and conn_b which have the entity managers em_a and em_b and their cache warmers are also registered in this order. So we have two iterations of cache warmers first the cache warmer who gets the entity manager em_a and then the cache warmer who gets the entity manager em_b.

Now when I clear the cache the following happens:

Iteration 1 (em_a)

In DoctrineMetadataCacheWarmer in line 59 we have $metadataFactory->getAllMetadata(); which will load all the metadata for all entities of the entity manager em_a. During this method call the ClassMetadataFactory will call in line 244 if ($this->evm->hasListeners(Events::loadClassMetadata)) {.

And here comes the tricky part I was not able to reproduce yet as I don't know how symfony determines which event manager should get a certain subscriber but the following happens:

The event manager of em_a has a doctrine event subscriber assigned that I have written. This subscriber has a dependency on a custom repository. This custom repository will get the entity manager em_b and calls $entityManager->getRepository(SomeEntityFromConnB::class) in it's constructor. Now the entity_manager em_b will generate the metadata for SomeEntityFromConnB class.

Iteration 2 (em_b)

Now the cache warmer of entity manager em_b will do it's run and will fetch from it the meta data factory in line 47 of the DoctrineMetadataCacheWarmer and as the entity manager has already created metadata for SomeEntityFromConnB even though there are still several entities that do not have meta data generated the if condition in line 48 will throw an exception.

I will continue my quest to provide an example so you can debug it yourself but I hope I have already clearly outlined the cause for the bug.

[dmaicher]

@TimWerdin this sounds indeed like it could be the issue...

But then creating a reproducer should be possible when simply following your comments?

[Eydamos]

I guess it will not take too much time to be able to produce a demo but I'm currently busy with testing and deploying a big project. I will continue my work next Thuesday

[Eydamos]

I was a little bit busy last week but now I could prepare a demo application which shows the cause of the bug. You can find it here: https://github.com/TimWerdin/doctrine-bug-demo

[dmaicher]

The reproducer looks good to me. Thanks @TimWerdin

[Eydamos]

Is there any progress on this issue? This is limiting us to use older versions of doctrine to prevent conflicts

[dmaicher]

Is there any progress on this issue? This is limiting us to use older versions of doctrine to prevent conflicts

Did not have much time recently. You could test https://github.com/doctrine/DoctrineBundle/pull/1378 and give some feedback there

[DoobleD]

Anybody has a workaround for this? I cannot downgrade to 2.2.4 because of another issue in that version which was solved on 2.3.

[cywbl]

Hi all- just adding some info, I am experiencing this error on doctrine-bundle v2.5.5. Does anyone have a workaround?

[dmaicher]

A workaround is to define a cache pool for the metadata yourself.

See the older version of the recipe for example.

[pscheit]

note to me: make sure that the metadata cache key applies to all managers in the config (to apply the workaround) I had one manager triggering the autoloading of a repo in the other manager as well

[lin-lu]

A workaround is to define a cache pool for the metadata yourself.

See the older version of the recipe for example.

Hi @dmaicher, could you explain a bit more in detail how the workaround works? I have the same config in the doctrine.yaml config file, but still got the same error. Thanks!

[dmaicher]

A workaround is to define a cache pool for the metadata yourself. See the older version of the recipe for example.

Hi @dmaicher, could you explain a bit more in detail how the workaround works? I have the same config in the doctrine.yaml config file, but still got the same error. Thanks!

which version of DoctrineBundle are you using? latest?

If you configure your own psr6 metadata cache then the DoctrineMetadataCacheWarmer should not be doing anything.

[Eydamos]

which version of DoctrineBundle are you using? latest?

If you configure your own psr6 metadata cache then the DoctrineMetadataCacheWarmer should not be doing anything.

Lin is currently on vacation so I will continue the conversation for him. We are using doctrine/doctrine-bundle in version 2.6.3 Our config/packages/doctrine.yaml only defines the connections and entity managers

doctrine:
    dbal:
        connections:
            foo:
                driver: pdo_mysql
                host: '%env(HOST_FOO)%'
                user:  '%env(USER_FOO)%'
                password: '%env(PASSWORD_FOO)%'
                dbname: '%env(DB_NAME_FOO)%'
                server_version: '%env(SERVER_VERSION_FOO)%'
                keep_slave: true
            bar:
                driver: pdo_mysql
                host: '%env(HOST_BAR)%'
                user:  '%env(USER_BAR)%'
                password: '%env(PASSWORD_BAR)%'
                dbname: '%env(DB_NAME_BAR)%'
                server_version: '%env(SERVER_VERSION_BAR)%'
                keep_slave: true
    orm:
        entity_managers:
            foo:
                naming_strategy: doctrine.orm.naming_strategy.underscore
                connection: foo
            bar:
                naming_strategy: doctrine.orm.naming_strategy.underscore
                connection: bar

and our config/packages/prod/doctrine.yaml is the default one from the recipe:

doctrine:
    orm:
        auto_generate_proxy_classes: false
        metadata_cache_driver:
            type: pool
            pool: doctrine.system_cache_pool
        query_cache_driver:
            type: pool
            pool: doctrine.system_cache_pool
        result_cache_driver:
            type: pool
            pool: doctrine.result_cache_pool

framework:
    cache:
        pools:
            doctrine.result_cache_pool:
                adapter: cache.app
            doctrine.system_cache_pool:
                adapter: cache.system

Sorry if I still don't get it but what exactly do I have to do? Do I need to add an entry to framework.cache.pools and replace the value for doctrine.orm.metadata_cache_driver.pool then?

[dmaicher]

@TimWerdin so then you are only configuring the cache in config/packages/prod/doctrine.yaml for the default connection/manager? What about the other one?

[Eydamos]

We are not aware that we need to configure a cache per connection/entity_manager or that this is even possible. At least I can't find anything in the documentation I was of the impression that the cache is always used globally for all connections/entity_managers.

[barbuslex]

I have the same error message when i am execute symfony console c:c on production environnement. Have you a workaround to fix it temporarily ? Thanks

[barbuslex]

I have fixed it.

In config/packages/doctrine.yaml :

• Replace :

when@prod:
    doctrine:
        orm:
            auto_generate_proxy_classes: false
            query_cache_driver:
                type: pool
                pool: doctrine.system_cache_pool
            result_cache_driver:
                type: pool
                pool: doctrine.result_cache_pool

• With :

when@prod:
    doctrine:
        orm:
            auto_generate_proxy_classes: false
            metadata_cache_driver:
                type: pool
                pool: doctrine.system_cache_pool
            query_cache_driver:
                type: pool
                pool: doctrine.system_cache_pool
            result_cache_driver:
                type: pool
                pool: doctrine.result_cache_pool

Just add :

metadata_cache_driver:
    type: pool
    pool: doctrine.system_cache_pool

[mdevlamynck]

I've encountered the same issue in tests:

• we have multiple connections (different sqlite db in various states)
• we run with kernel.debug=false to speed up the tests

This results in the cache warmer being added twice during autoconfiguration:

$ bin/console debug:container --env=test --tag=kernel.cache_warmer

Symfony Container Services Tagged with "kernel.cache_warmer" Tag
================================================================

 --------------------------------------------------- ---------- -------------------------------------------------------------------------
  Service ID                                          priority   Class name
 --------------------------------------------------- ---------- -------------------------------------------------------------------------
  doctrine.orm.default_metadata_cache_warmer          1000       Doctrine\Bundle\DoctrineBundle\CacheWarmer\DoctrineMetadataCacheWarmer
  doctrine.orm.empty_database_metadata_cache_warmer   1000       Doctrine\Bundle\DoctrineBundle\CacheWarmer\DoctrineMetadataCacheWarmer
  cache_pool_clearer.cache_warmer                     64         Symfony\Bundle\FrameworkBundle\CacheWarmer\CachePoolClearerCacheWarmer
  ...

The workaround above works because the cache warmer is only added if no metadata_cache_driver is defined.

[hothson]

Remove APP_DEBUG=0 in .env file

[ivanbogomoloff]

i don't encountered this issue in version ^2.7 , so i decided to close issue

[justin-oh]

I'm commenting in case anyone encounters this specific error through a Google search. (Sorry in advance if this comment re-opens the issue.)

In my case, the error was caused by me initializing something in a service constructor that queries the database. Example:

<?php

namespace App\Service;

class MyService
{
    private $data;
    private OtherService $otherService;

    public function __construct(OtherService $otherService)
    {
        $this->otherService = $otherService;

        $this->data = $otherService->queryForData();
    }
}

My solution was to initialize $this->data in a separate function:

private $dataInit = false;

private function initData(): void
{
    if ($this->dataInit) {
        return;
    }

    $this->dataInit = true;

    $this->data = $otherService->queryForData();
}

Then I could call $this->initData() before I need to reference $this->data. The query is only performed once and only when absolutely necessary.

In hindsight, I should have took this approach regardless of any error. Especially in a Dependency Injection framework like Symfony.