Running a query on an observed collection causes memory growth

[dmorgereth]

trafficstars

How frequently does the bug occur?

All the time

Description

Our use case requires observing a Realm collection while running queries against it. We have noticed unrestricted memory growth in our app when the queries are executed. If we disable our observer, the memory use remains flat. A sample project that demonstrates the issue is attached. In general, this is what's happening:

Create a realm collection Observe the collection Run a query Observe memory usage Memory usage increases every time the query is run Stop the observer Run a query Observe memory usage Memory usage is flat when there is no observer

This behavior occurs when the collection is empty as well as when the collection has data.

Stacktrace & log output

No response

Can you reproduce the bug?

Yes, always

Reproduction Steps

See attached RealmMemory.zip for a sample project that demonstrates the issue. Note that all of the pertinent code is in the ViewController.swift file. It is a single view iphone application with 4 buttons: Load Data, Start Observing, Run Query and Stop observing.

To observe the problem:

1. open the project in xCode and launch the app

2. open the XCode debug pane

3. when the simulator appears and the app loads, tap the "Start observing" button

4. observe the memory usage as you tap the "Run query" button several times Expected Result: Memory usage is flat Actual Result: Memory usage increases every time the "Run query" button is tapped

5. tap the "Stop observing" button

6. observe the memory usage as you tap the "Run query" button several times Expected Result: Memory usage is flat Actual Result: Memory usage is flat

Attached please find a screenshot of an Instruments trace of heapshots using the allocations instrument, using the following process:

1. From xcode, open the project and select Profile
2. When Instruments appears, tap the record button
3. When the simulator appears and the app loads, Tap Start monitoring
4. (in Instruments) tap Heapshot
5. Tap Run query
6. Repeat steps 4 and 5 se Result: Each heapshot generation shows 17.90 MiB of memory growth and 51K persistent objects, all of which is attributed to Realm Note: I tried to attach the actual Instruments trace but it is too large

RealmMemory.zip

Version

10.30

What SDK flavour are you using?

Local Database only

Are you using encryption?

No, not using encryption

Platform OS and version(s)

iOS 15.5

Build environment

Xcode version: ... Dependency manager and version: ... Xcode 13.4.1 Cocoapods 1.11.3

[dmorgereth]

Here is the github repo for the sample project that illustrates the issue:

https://github.com/Sugarmate/RealmMemory

This is a huge blocker for us - any chance someone could look into this? Thank you

[ejm01]

@dmorgereth I see what you're talking about. Thanks for the .zip and detailed instructions. I'm looking into this more.

[dmorgereth]

Thank you Eric!

[dmorgereth]

Hi Eric, is there any update on this issue?

[ejm01]

Sorry no progress to report yet.

[tgoyne]

It turns out we only clean up some of the state involved in queries after either an observer with a callback is added or a write is performed on the Realm, so performing a large number of queries without doing either of those in between results in memory usage growing. In addition, this causes a cache to grow in size so even after performing a write and cleaning up the old notifier objects memory usage stays high even though nothing has been technically "leaked". https://github.com/realm/realm-core/pull/6007 fixes the problem with the cache so that memory usage will actually go back down.

[dmorgereth]

Thank you Thomas! We appreciate your help with this.

[tgoyne]

10.38.3 has some improvements here.

[dmorgereth]

Thanks Thomas – we’ll upgrade to that version

Dave Morgereth Principal Software Engineer

[signature_2368947587] 12400 High Bluff Drive San Diego, CA 92130 PH: (858) 255- 6655 | WWW.TANDEMDIABETES.COMhttp://WWW.TANDEMDIABETES.COM

The information in this email (including any attachments) may be privileged and/or confidential and is intended only for the recipient(s) listed above. Any use, disclosure, distribution or copying of this email is prohibited except by or on behalf of the intended recipient. If you have received this email in error, please notify me immediately and destroy all copies of the transmittal. Thank you.

From: Thomas Goyne @.> Date: Friday, April 28, 2023 at 6:33 PM To: realm/realm-swift @.> Cc: Dave Morgereth @.>, Mention @.> Subject: Re: [realm/realm-swift] Running a query on an observed collection causes memory growth (Issue #7978) Warning: This email came from outside of Tandem Diabetes Care. Be careful when responding, opening attachments, or clicking links.

10.38.3 has some improvements here.

— Reply to this email directly, view it on GitHubhttps://github.com/realm/realm-swift/issues/7978#issuecomment-1528420296, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AV2AYXXDRIE4ORZV7JI7JCDXDRVXHANCNFSM6AAAAAAQ2EV55M. You are receiving this because you were mentioned.Message ID: @.***>