doctrine1
doctrine1 copied to clipboard
FriendsOfSymfony1
serialize/unserialize of records with 'array' columns fails
This bug happens only on edge cases. Let me describe the scenario first:
schema.yml:
Model:
columns:
details: { type: array, notnull: true }
options:
symfony: { form: false, filter: false }
type: InnoDB
RelatedModel:
columns:
model_id: { type: integer(8), unsigned: true, notnull: true }
relations:
Model: { class: Model, foreign: id, local: model_id, foreignAlias: RelatedModels, type: one, foreignType: many }
options:
symfony: { form: false, filter: false }
type: InnoDB
Important here is that 'Model' contains a column of type 'array' and 'Model' has a 'RelatedModel'.
Now the database content: The database should contain at least one 'Model' (id: 1) connected with one 'RelatedModel' (id: 1). The 'Model'.'details' should contain an array with at least 20 entries.
Now lets provoke the error. I found these two methods:
Method 1: Load form database with cache If this query hits the cache the unserialize will fail.
$foo = ModelTable::getInstance()
->createQuery('m')
->leftJoin('m.RelatedModels rm')
->select('m.id, m.details, rm.id')
->where('m.id = ?', 1)
->useResultCache(true)
->execute();
Method 2: serialize/unserialize with references
$foo = ModelTable::getInstance()
->createQuery('m')
->leftJoin('m.RelatedModels rm')
->select('m.id, m.details, rm.id')
->where('m.id = ?', 1)
->execute();
$foo->serializeReferences(true);
unserialize(serialize($foo));
both of these examples will create an error similar to this:
>> sfWebDebugLogger Notice at /workdir/vendor/lexpress/doctrine1/lib/Doctrine/Collection.php on line 176 (unserialize(): Error at offset 596 of 657 bytes)
NOTICE |13:06:01: {sfWebDebugLogger} Notice at /workdir/vendor/lexpress/doctrine1/lib/Doctrine/Collection.php on line 176 (unserialize(): Error at offset 596 of 657 bytes)
Notice: unserialize(): Error at offset 596 of 657 bytes in /workdir/vendor/lexpress/doctrine1/lib/Doctrine/Collection.php on line 176
Call Stack:
0.0005 241824 1. {main}() /workdir/symfony:0
0.3185 25599144 2. sfSymfonyCommandApplication->run() /workdir/symfony:19
0.3207 25603696 3. sfTask->runFromCLI() /workdir/vendor/lexpress/symfony1/lib/command/sfSymfonyCommandApplication.class.php:76
0.3207 25605088 4. sfBaseTask->doRun() /workdir/vendor/lexpress/symfony1/lib/task/sfTask.class.php:98
0.3664 37410728 5. testTask->execute() /workdir/vendor/lexpress/symfony1/lib/task/sfBaseTask.class.php:70
0.3757 39949400 6. Doctrine_Query_Abstract->execute() /workdir/lib/task/testTask.class.php:45
0.6709 42200896 7. Doctrine_Query_Abstract->_constructQueryFromCache() /workdir/vendor/lexpress/doctrine1/lib/Doctrine/Query/Abstract.php:1085
0.6709 42200944 8. unserialize() /workdir/vendor/lexpress/doctrine1/lib/Doctrine/Query/Abstract.php:1235
0.6717 42437752 9. Doctrine_Collection->unserialize() /workdir/vendor/lexpress/doctrine1/lib/Doctrine/Query/Abstract.php:1235
0.6717 42437800 10. unserialize() /workdir/vendor/lexpress/doctrine1/lib/Doctrine/Collection.php:176
0.6718 42441352 11. Doctrine_Record->unserialize() /workdir/vendor/lexpress/doctrine1/lib/Doctrine/Collection.php:176
0.6718 42441968 12. unserialize() /workdir/vendor/lexpress/doctrine1/lib/Doctrine/Record.php:869
0.6718 42455632 13. Doctrine_Collection->unserialize() /workdir/vendor/lexpress/doctrine1/lib/Doctrine/Record.php:869
0.6718 42455680 14. unserialize() /workdir/vendor/lexpress/doctrine1/lib/Doctrine/Collection.php:176
>> sfWebDebugLogger Warning at /workdir/vendor/lexpress/doctrine1/lib/Doctrine/Collection.php on line 178 (Invalid argument supplied for foreach())
WARNING|13:06:01: {sfWebDebugLogger} Warning at /workdir/vendor/lexpress/doctrine1/lib/Doctrine/Collection.php on line 178 (Invalid argument supplied for foreach())
Warning: Invalid argument supplied for foreach() in /workdir/vendor/lexpress/doctrine1/lib/Doctrine/Collection.php on line 178
Call Stack:
0.0005 241824 1. {main}() /workdir/symfony:0
0.3185 25599144 2. sfSymfonyCommandApplication->run() /workdir/symfony:19
0.3207 25603696 3. sfTask->runFromCLI() /workdir/vendor/lexpress/symfony1/lib/command/sfSymfonyCommandApplication.class.php:76
0.3207 25605088 4. sfBaseTask->doRun() /workdir/vendor/lexpress/symfony1/lib/task/sfTask.class.php:98
0.3664 37410728 5. testTask->execute() /workdir/vendor/lexpress/symfony1/lib/task/sfBaseTask.class.php:70
0.3757 39949400 6. Doctrine_Query_Abstract->execute() /workdir/lib/task/testTask.class.php:45
0.6709 42200896 7. Doctrine_Query_Abstract->_constructQueryFromCache() /workdir/vendor/lexpress/doctrine1/lib/Doctrine/Query/Abstract.php:1085
0.6709 42200944 8. unserialize() /workdir/vendor/lexpress/doctrine1/lib/Doctrine/Query/Abstract.php:1235
0.6717 42437752 9. Doctrine_Collection->unserialize() /workdir/vendor/lexpress/doctrine1/lib/Doctrine/Query/Abstract.php:1235
0.6717 42437800 10. unserialize() /workdir/vendor/lexpress/doctrine1/lib/Doctrine/Collection.php:176
0.6718 42441352 11. Doctrine_Record->unserialize() /workdir/vendor/lexpress/doctrine1/lib/Doctrine/Collection.php:176
0.6718 42441968 12. unserialize() /workdir/vendor/lexpress/doctrine1/lib/Doctrine/Record.php:869
0.6718 42455632 13. Doctrine_Collection->unserialize() /workdir/vendor/lexpress/doctrine1/lib/Doctrine/Record.php:869
06.09.2018 01:06:01 - Task ./symfony, t:t caught exception of class Doctrine_Exception with message Couldn't find class /workdir/vendor/lexpress/doctrine1/lib/Doctrine/Table.php 310
#0 /workdir/vendor/lexpress/doctrine1/lib/Doctrine/Table.php(261): Doctrine_Table->initDefinition()
#1 /workdir/vendor/lexpress/doctrine1/lib/Doctrine/Connection.php(1148): Doctrine_Table->__construct(NULL, Object(Doctrine_Connection_Mysql), true)
#2 /workdir/vendor/lexpress/doctrine1/lib/Doctrine/Collection.php(182): Doctrine_Connection->getTable(NULL)
#3 [internal function]: Doctrine_Collection->unserialize('a:6:{s:4:"data"...')
#4 /workdir/vendor/lexpress/doctrine1/lib/Doctrine/Record.php(869): unserialize('a:15:{s:3:"_id"...')
#5 [internal function]: Doctrine_Record->unserialize('a:15:{s:3:"_id"...')
#6 /workdir/vendor/lexpress/doctrine1/lib/Doctrine/Collection.php(176): unserialize('a:6:{s:4:"data"...')
#7 [internal function]: Doctrine_Collection->unserialize('a:6:{s:4:"data"...')
#8 /workdir/vendor/lexpress/doctrine1/lib/Doctrine/Query/Abstract.php(1235): unserialize('a:3:{i:0;C:31:"...')
#9 /workdir/vendor/lexpress/doctrine1/lib/Doctrine/Query/Abstract.php(1085): Doctrine_Query_Abstract->_constructQueryFromCache('a:3:{i:0;C:31:"...')
#10 /workdir/lib/task/testTask.class.php(45): Doctrine_Query_Abstract->execute()
#11 /workdir/vendor/lexpress/symfony1/lib/task/sfBaseTask.class.php(70): testTask->execute(Array, Array)
#12 /workdir/vendor/lexpress/symfony1/lib/task/sfTask.class.php(98): sfBaseTask->doRun(Object(sfCommandManager), NULL)
#13 /workdir/vendor/lexpress/symfony1/lib/command/sfSymfonyCommandApplication.class.php(76): sfTask->runFromCLI(Object(sfCommandManager), NULL)
#14 /workdir/symfony(19): sfSymfonyCommandApplication->run()
#15 {main}
Couldn't find class
Fatal error: Call to a member function evictAll() on null in /workdir/vendor/lexpress/doctrine1/lib/Doctrine/Connection.php on line 1267
Call Stack:
0.0005 241824 1. {main}() /workdir/symfony:0
1.0430 42803488 2. sfDatabaseManager->shutdown() /workdir/vendor/lexpress/symfony1/lib/database/sfDatabaseManager.class.php:0
1.0430 42803616 3. sfDoctrineDatabase->shutdown() /workdir/vendor/lexpress/symfony1/lib/database/sfDatabaseManager.class.php:137
1.0430 42803792 4. Doctrine_Manager->closeConnection() /workdir/vendor/lexpress/symfony1/lib/plugins/sfDoctrinePlugin/lib/database/sfDoctrineDatabase.class.php:152
1.0430 42803976 5. Doctrine_Connection->close() /workdir/vendor/lexpress/doctrine1/lib/Doctrine/Manager.php:583
1.0430 42804560 6. Doctrine_Connection->clear() /workdir/vendor/lexpress/doctrine1/lib/Doctrine/Connection.php:1296
I could trace down the problem to the custom serialization of Doctrine_Record and Doctrine_Collection.
see https://github.com/LExpress/doctrine1/pull/54
To make it easier to understand the problem. Here is an abstract example of what happens in Doctrine:
$bar = new Model();
$str = serialize($bar);
$res = unserialize($str);
class RelatedModel
{
}
class Collection implements Serializable
{
private $data = null;
private $_snapshot = null;
public function __construct()
{
$this->data = $this->_snapshot = [new RelatedModel()];
}
public function serialize()
{
return serialize(get_object_vars($this));
}
public function unserialize($serialized)
{
$vars = unserialize($serialized);
foreach ($vars as $k => $v) {
$this->$k = $v;
}
}
}
class Model implements Serializable
{
private $_data = ['arrayField' => ['one', 'one', 'one', 'one', 'one', 'one', 'one', 'one', 'one',]];
private $relation = null;
public function __construct()
{
$this->relation = ['RelatedModels' => new Collection()];
}
public function serialize()
{
$vars = get_object_vars($this);
// fields of type array are serialized before the rest
$vars['_data']['arrayField'] = serialize($vars['_data']['arrayField']);
return serialize($vars);
}
public function unserialize($serialized)
{
$vars = unserialize($serialized);
foreach ($vars as $k => $v) {
$this->$k = $v;
}
$this->_data['arrayField'] = unserialize($this->_data['arrayField']);
}
}
This will result in this serialized string:
C:5:"Model":330:{a:2:{s:5:"_data";a:1:{s:10:"arrayField";s:132:"a:9:{i:0;s:3:"one";i:1;s:3:"one";i:2;s:3:"one";i:3;s:3:"one";i:4;s:3:"one";i:5;s:3:"one";i:6;s:3:"one";i:7;s:3:"one";i:8;s:3:"one";}";}s:8:"relation";a:1:{s:13:"RelatedModels";C:10:"Collection":82:{a:2:{s:4:"data";a:1:{i:0;O:12:"RelatedModel":0:{}}s:9:"_snapshot";a:1:{i:0;r:19;}}}}}}
The problem lies in r:19;. This is a reference which should point to O:12:"RelatedModel":0:{} because RelatedModel is reference twice in Collection but the number is wrong.
As far as I know serialize gives every object in the serialized sting a number to reference it later but the calculated number is wrong. I think the problem lies in Model::serialize().
On serialize PHP serializes $bar in this order
Model {
arrayField
Collection {
RelatedModel in data
RelatedModel in _snapshot (as Reference)
}
}
and every node in the result will get a number to reference it later
On unserialize we change the order (due to the custom serialization)
Model {
Collection {
RelatedModel in data
RelatedModel in _snapshot (fails to find the reference because arrayField was not handled yet)
}
arrayField
}
and fail because arrayField is out of order.
Attention: This bug can also lead to corrupt data. If the arrayField contains only a small array the reference will point to a node in the serialized string which exists but is wrong.
PHP does not support the double serialization, very weird.
<?php
$value = [['foo'], 'bar'];
$serialized = $value;
$serialized[0] = serialize($serialized[0]);
$serialized = serialize($serialized);
$unserialized = unserialize($serialized);
$unserialized[0] = unserialize($unserialized[0]);
$value == $unserialized // true
@alquerci your example should work and works.
The problem only exists if serialize uses references in the serialized string.
As in my abstract example we use in Collection::data and Collection::_snapshot the same object. This will create a reference in the serialized string (r:19;). But because we serialize/unserialize the array out of order the reference counter gets out of sync.
Here a shorter example to illustrate the problem
<?php
class RelatedModel
{ }
class Model implements Serializable
{
private $doubleSerialized = ['one', 'one', ];
private $obj1 = null;
private $obj2 = null;
public function __construct()
{
$this->obj1 = $this->obj2 = new RelatedModel();
}
public function serialize()
{
$vars = get_object_vars($this);
$vars['doubleSerialized'] = serialize($vars['doubleSerialized']);
return serialize($vars);
}
public function unserialize($serialized)
{
$vars = unserialize($serialized);
foreach ($vars as $k => $v) {
$this->$k = $v;
}
$this->doubleSerialized = unserialize($this->doubleSerialized);
}
}
$bar = new Model();
$str = serialize($bar); // 'C:5:"Model":122:{a:3:{s:16:"doubleSerialized";s:34:"a:2:{i:0;s:3:"one";i:1;s:3:"one";}";s:4:"obj1";O:12:"RelatedModel":0:{}s:4:"obj2";r:7;}}'
$res = unserialize($str); // throws error
Another prerequisite is that you use double serialization in a custom serialize function. Outside of Serializable::serialize() the reference counter gets reset. That's why this example works:
<?php
class RelatedModel
{ }
$object = new RelatedModel();
$value = [
'foo' => ['one', 'one'],
'obj1' => $object,
'obj2' => $object,
];
$value['foo'] = serialize($value['foo']);
$serialized = serialize($value); // 'a:3:{s:3:"foo";s:34:"a:2:{i:0;s:3:"one";i:1;s:3:"one";}";s:4:"obj1";O:12:"RelatedModel":0:{}s:4:"obj2";r:3;}'
$unserialized = unserialize($serialized);
$unserialized['foo'] = unserialize($unserialized['foo']);
$value === $unserialized; // true