implement Query::iterate

[dbu]

Implement the Query->iterate method which will return an IteratableResultSet as per the ORM.

[juzerali]

:+1: I am surprised it isn't there already. I keep hitting memory limits whenever I have to rebuild my elasticsearch index from the database. Surprisingly detaching while iterating during a foreach doesn't free up memory.

[dbu]

you are welcome to do a PR to implement the feature ;-)

[juzerali]

Of course I would, if I was familiar with the doctrine codebase. I'd probably have to take a deep dive over a week. Meanwhile can you tell me pointing out any other way of doing large data processing with doctrine odm, and shouldn't detach free up the document objects to get garbage collected?

Regards, Juzer Ali http://about.me/juzerali

On Thu, Jul 10, 2014 at 3:27 PM, David Buchmann [email protected] wrote:

you are welcome to do a PR to implement the feature ;-)

— Reply to this email directly or view it on GitHub https://github.com/doctrine/phpcr-odm/issues/425#issuecomment-48585752.

[dbu]

detach should free the memory normally. are you sure you don't keep any references elsewhere?

you could try to call $dm->clear() to wipe the whole document manager when you are doing batch processing. maybe we have an issue with the detach that we don't free everything. does clear work better?

[juzerali]

Its all the same.

[dbu]

if $dm->clear does not help then i strongly suspect that you keep references to the objects somewhere else in your code that prevent garbage collection from working.

[ElectricMaxxx]

I am :+1: for this Feature. Combined with the Paginator Class that is implemented in the ORM, we can save lots of memory.