Browsing history and site data causing excessive storage bloat

[KennethHeal]

Steps to reproduce

None in particular, just use fenix.

Expected behavior

Storage space used to remain stable and clearing browser cached images and files to yield some savings

Actual behavior

Storage space used keeps bloating and emptying browser cached images and files is useless. Deleting browser history and site data clears majority of the bloat, but results in loss of history.

Device information

About:support output

{ "application": { "name": "Firefox", "osVersion": "Linux 25 #1 SMP PREEMPT Wed Nov 15 16:06:34 KST 2017", "version": "103.0.2", "buildID": "20220802163236", "distributionID": "", "userAgent": "Mozilla/5.0 (Android 7.1.1; Mobile; rv:103.0) Gecko/103.0 Firefox/103.0", "safeMode": false, "memorySizeBytes": 1497272320, "diskAvailableBytes": 467644416, "supportURL": "https://support.mozilla.org/1/mobile/103.0.2/Android/en-US/", "osTheme": "", "numTotalWindows": 0, "numFissionWindows": 0, "numRemoteWindows": 0, "fissionAutoStart": false, "fissionDecisionStatus": "disabledByDefault", "remoteAutoStart": true, "keyLocationServiceGoogleFound": true, "keySafebrowsingGoogleFound": true, "keyMozillaFound": true }, "securitySoftware": { "registeredAntiVirus": "", "registeredAntiSpyware": "", "registeredFirewall": "" }, "processes": { "remoteTypes": { "web": 1, "gpu": 1 }, "maxWebContentProcesses": 2 }, "environmentVariables": {}, "modifiedPreferences": { "browser.search.region": "NL", "extensions.lastAppVersion": "103.0.2", "idle.lastDailyNotification": 1660082742, "media.gmp.storage.version.observed": 1, "network.dns.disablePrefetch": true, "network.http.speculative-parallel-limit": 0, "network.predictor.enabled": false, "network.prefetch-next": false, "privacy.purge_trackers.last_purge": "1660082755461", "privacy.purge_trackers.date_in_cookie_database": "0", "security.remote_settings.crlite_filters.checked": 1656588114, "security.remote_settings.intermediates.checked": 1656588114, "storage.vacuum.last.index": 0 }, "lockedPreferences": { "fission.autostart.session": false }, "printingPreferences": {}, "media": { "currentAudioBackend": "opensl", "currentMaxAudioChannels": 2, "currentPreferredSampleRate": 48000, "audioOutputDevices": [ { "name": "Default audio output device", "groupId": "", "vendor": "", "type": 2, "state": 2, "preferred": 15, "supportedFormat": 12336, "defaultFormat": 16, "maxChannels": 2, "defaultRate": 44100, "maxRate": 44100, "minRate": 44100, "maxLatency": 441, "minLatency": 128 } ], "audioInputDevices": [ { "name": "Default audio input device", "groupId": "", "vendor": "", "type": 1, "state": 2, "preferred": 15, "supportedFormat": 12336, "defaultFormat": 16, "maxChannels": 1, "defaultRate": 44100, "maxRate": 44100, "minRate": 44100, "maxLatency": 441, "minLatency": 128 } ] }, "accessibility": { "isActive": true, "forceDisabled": 0, "handlerUsed": false, "instantiator": "" }, "startupCache": { "DiskCachePath": "/data/user/0/org.mozilla.firefox/files/mozilla/ft7e13yv.default/startupCache/startupCache.4.little", "IgnoreDiskCache": false, "FoundDiskCacheOnInit": false, "WroteToDiskCache": true }, "libraryVersions": { "NSPR": { "minVersion": "4.34", "version": "4.34" }, "NSS": { "minVersion": "3.80", "version": "3.80" }, "NSSUTIL": { "minVersion": "3.80", "version": "3.80" }, "NSSSSL": { "minVersion": "3.80", "version": "3.80" }, "NSSSMIME": { "minVersion": "3.80", "version": "3.80" } }, "userJS": { "exists": false }, "intl": { "localeService": { "requested": [ "en-NL", "en-GB" ], "available": [ "an", "ar", "ast", "az", "be", "bg", "bn", "br", "bs", "ca", "cak", "cs", "cy", "da", "de", "dsb", "el", "en-CA", "en-GB", "en-US", "eo", "es-AR", "es-CL", "es-ES", "es-MX", "et", "eu", "fa", "ff", "fi", "fr", "fy-NL", "ga-IE", "gd", "gl", "gn", "gu-IN", "he", "hi-IN", "hr", "hsb", "hu", "hy-AM", "id", "is", "it", "ja", "ka", "kab", "kk", "kn", "ko", "lij", "lo", "lt", "lv", "ml", "mr", "ms", "my", "nb-NO", "ne-NP", "nl", "nn-NO", "oc", "pa-IN", "pl", "pt-BR", "pt-PT", "rm", "ro", "ru", "sk", "sl", "son", "sq", "sr", "sv-SE", "ta", "te", "th", "tr", "trs", "uk", "ur", "uz", "vi", "wo", "xh", "zam", "zh-CN", "zh-TW" ], "supported": [ "en-US", "en-CA", "en-GB" ], "regionalPrefs": [ "en-NL", "en-GB" ], "defaultLocale": "en-US" }, "osPrefs": { "systemLocales": [ "en-NL", "en-GB" ], "regionalPrefsLocales": [ "en-NL", "en-GB" ] } }, "crashes": { "submitted": [], "pending": 0 }, "remoteAgent": { "running": false, "url": "" }, "graphics": { "numTotalWindows": 2, "numAcceleratedWindows": 2, "windowLayerManagerType": "WebRender", "windowLayerManagerRemote": true, "adapterDescription": "Model: SM-T560NU, Product: gtelwifiue, Manufacturer: samsung, Hardware: qcom, OpenGL: Qualcomm -- Adreno (TM) 306 -- OpenGL ES 3.0 [email protected] AU@ (GIT@I750f7f2fa6)", "adapterVendorID": "Qualcomm", "adapterDeviceID": "Adreno (TM) 306", "adapterRAM": 0, "adapterDrivers": "", "driverVendor": "", "driverVersion": "OpenGL ES 3.0 [email protected] AU@ (GIT@I750f7f2fa6)", "driverDate": "", "targetFrameRate": 60, "webgl1Renderer": "Qualcomm -- Adreno (TM) 306", "webgl1Version": "OpenGL ES 3.0 [email protected] AU@ (GIT@I750f7f2fa6)", "webgl1DriverExtensions": "GL_AMD_compressed_ATC_texture GL_AMD_performance_monitor GL_AMD_program_binary_Z400 GL_EXT_debug_label GL_EXT_debug_marker GL_EXT_discard_framebuffer GL_EXT_robustness GL_EXT_texture_format_BGRA8888 GL_EXT_texture_type_2_10_10_10_REV GL_NV_fence GL_OES_compressed_ETC1_RGB8_texture GL_OES_depth_texture GL_OES_depth24 GL_OES_EGL_image GL_OES_EGL_sync GL_OES_EGL_image_external GL_OES_element_index_uint GL_OES_fbo_render_mipmap GL_OES_fragment_precision_high GL_OES_get_program_binary GL_OES_packed_depth_stencil GL_OES_depth_texture_cube_map GL_OES_rgb8_rgba8 GL_OES_standard_derivatives GL_OES_texture_3D GL_OES_texture_float GL_OES_texture_half_float GL_OES_texture_half_float_linear GL_OES_texture_npot GL_OES_vertex_half_float GL_OES_vertex_type_10_10_10_2 GL_OES_vertex_array_object GL_QCOM_alpha_test GL_QCOM_binning_control GL_QCOM_driver_control GL_QCOM_perfmon_global_mode GL_QCOM_extended_get GL_QCOM_extended_get2 GL_QCOM_tiled_rendering GL_QCOM_writeonly_rendering GL_EXT_sRGB GL_EXT_sRGB_write_control GL_EXT_texture_sRGB_decode GL_EXT_texture_filter_anisotropic GL_EXT_multisampled_render_to_texture GL_EXT_color_buffer_float GL_EXT_color_buffer_half_float GL_EXT_disjoint_timer_query ", "webgl1Extensions": "ANGLE_instanced_arrays EXT_blend_minmax EXT_color_buffer_half_float EXT_sRGB EXT_texture_filter_anisotropic MOZ_debug OES_element_index_uint OES_fbo_render_mipmap OES_standard_derivatives OES_texture_float OES_texture_half_float OES_texture_half_float_linear OES_vertex_array_object WEBGL_color_buffer_float WEBGL_compressed_texture_etc WEBGL_compressed_texture_etc1 WEBGL_debug_renderer_info WEBGL_debug_shaders WEBGL_depth_texture WEBGL_lose_context", "webgl1WSIInfo": "EGL_VENDOR: Qualcomm, Inc\nEGL_VERSION: 1.4\nEGL_EXTENSIONS: EGL_QUALCOMM_shared_image EGL_KHR_image EGL_KHR_image_base EGL_QCOM_create_image EGL_KHR_lock_surface EGL_KHR_lock_surface2 EGL_KHR_lock_surface3 EGL_KHR_fence_sync EGL_KHR_cl_event EGL_KHR_cl_event2 EGL_IMG_context_priority EGL_KHR_gl_texture_2D_image EGL_KHR_gl_texture_cubemap_image EGL_KHR_gl_texture_3D_image EGL_KHR_gl_renderbuffer_image EGL_EXT_create_context_robustness EGL_ANDROID_blob_cache EGL_ANDROID_framebuffer_target EGL_KHR_create_context EGL_KHR_wait_sync EGL_KHR_gl_colorspace EGL_ANDROIDX_no_config_context EGL_KHR_partial_update EGL_ANDROID_image_crop EGL_ANDROID_recordable EGL_ANDROID_native_fence_sync EGL_ANDROID_image_native_buffer EGL_KHR_gl_colorspace EGL_IMG_image_plane_attribs \nIsWebglOutOfProcessEnabled: 0", "webgl2Renderer": "Qualcomm -- Adreno (TM) 306", "webgl2Version": "OpenGL ES 3.0 [email protected] AU@ (GIT@I750f7f2fa6)", "webgl2DriverExtensions": "GL_AMD_compressed_ATC_texture GL_AMD_performance_monitor GL_AMD_program_binary_Z400 GL_EXT_debug_label GL_EXT_debug_marker GL_EXT_discard_framebuffer GL_EXT_robustness GL_EXT_texture_format_BGRA8888 GL_EXT_texture_type_2_10_10_10_REV GL_NV_fence GL_OES_compressed_ETC1_RGB8_texture GL_OES_depth_texture GL_OES_depth24 GL_OES_EGL_image GL_OES_EGL_sync GL_OES_EGL_image_external GL_OES_element_index_uint GL_OES_fbo_render_mipmap GL_OES_fragment_precision_high GL_OES_get_program_binary GL_OES_packed_depth_stencil GL_OES_depth_texture_cube_map GL_OES_rgb8_rgba8 GL_OES_standard_derivatives GL_OES_texture_3D GL_OES_texture_float GL_OES_texture_half_float GL_OES_texture_half_float_linear GL_OES_texture_npot GL_OES_vertex_half_float GL_OES_vertex_type_10_10_10_2 GL_OES_vertex_array_object GL_QCOM_alpha_test GL_QCOM_binning_control GL_QCOM_driver_control GL_QCOM_perfmon_global_mode GL_QCOM_extended_get GL_QCOM_extended_get2 GL_QCOM_tiled_rendering GL_QCOM_writeonly_rendering GL_EXT_sRGB GL_EXT_sRGB_write_control GL_EXT_texture_sRGB_decode GL_EXT_texture_filter_anisotropic GL_EXT_multisampled_render_to_texture GL_EXT_color_buffer_float GL_EXT_color_buffer_half_float GL_EXT_disjoint_timer_query ", "webgl2Extensions": "EXT_color_buffer_float EXT_texture_filter_anisotropic MOZ_debug WEBGL_compressed_texture_etc WEBGL_compressed_texture_etc1 WEBGL_debug_renderer_info WEBGL_debug_shaders WEBGL_lose_context", "webgl2WSIInfo": "EGL_VENDOR: Qualcomm, Inc\nEGL_VERSION: 1.4\nEGL_EXTENSIONS: EGL_QUALCOMM_shared_image EGL_KHR_image EGL_KHR_image_base EGL_QCOM_create_image EGL_KHR_lock_surface EGL_KHR_lock_surface2 EGL_KHR_lock_surface3 EGL_KHR_fence_sync EGL_KHR_cl_event EGL_KHR_cl_event2 EGL_IMG_context_priority EGL_KHR_gl_texture_2D_image EGL_KHR_gl_texture_cubemap_image EGL_KHR_gl_texture_3D_image EGL_KHR_gl_renderbuffer_image EGL_EXT_create_context_robustness EGL_ANDROID_blob_cache EGL_ANDROID_framebuffer_target EGL_KHR_create_context EGL_KHR_wait_sync EGL_KHR_gl_colorspace EGL_ANDROIDX_no_config_context EGL_KHR_partial_update EGL_ANDROID_image_crop EGL_ANDROID_recordable EGL_ANDROID_native_fence_sync EGL_ANDROID_image_native_buffer EGL_KHR_gl_colorspace EGL_IMG_image_plane_attribs \nIsWebglOutOfProcessEnabled: 0", "info": { "AzureCanvasBackend (UI Process)": "skia", "AzureFallbackCanvasBackend (UI Process)": "skia", "AzureContentBackend (UI Process)": "skia", "AzureCanvasBackend": "skia", "AzureContentBackend": "skia", "ApzTouchInput": 1, "ApzAutoscrollInput": 1, "ApzZoomingInput": 1, "CMSOutputProfile": "Empty profile data", "DisplayCount": 1, "Display0": "800x1280@0Hz scales:1.000000|1.000000" }, "failures": [ "CP+[GFX1-]: Receive IPC close with reason=AbnormalShutdown" ], "indices": [ 0 ], "featureLog": { "features": [ { "name": "HW_COMPOSITING", "description": "Compositing", "status": "force_enabled", "log": [ { "type": "default", "status": "available" }, { "type": "user", "status": "force_enabled", "message": "Force-enabled by pref" } ] }, { "name": "OPENGL_COMPOSITING", "description": "OpenGL Compositing", "status": "force_enabled", "log": [ { "type": "default", "status": "available" }, { "type": "user", "status": "force_enabled", "message": "Force-enabled by pref" } ] }, { "name": "GPU_PROCESS", "description": "GPU Process", "status": "available", "log": [ { "type": "default", "status": "available" } ] }, { "name": "WEBRENDER", "description": "WebRender", "status": "available", "log": [ { "type": "default", "status": "available" } ] }, { "name": "WEBRENDER_QUALIFIED", "description": "WebRender qualified", "status": "available", "log": [ { "type": "default", "status": "available" } ] }, { "name": "WEBRENDER_COMPOSITOR", "description": "WebRender native compositor", "status": "disabled", "log": [ { "type": "default", "status": "disabled", "message": "Disabled by default" } ] }, { "name": "WEBRENDER_PARTIAL", "description": "WebRender partial present", "status": "blocklisted", "log": [ { "type": "default", "status": "available" }, { "type": "env", "status": "blocklisted", "message": "Blocklisted by gfxInfo" } ] }, { "name": "WEBRENDER_SHADER_CACHE", "description": "WebRender shader disk cache", "status": "blocklisted", "log": [ { "type": "default", "status": "available" }, { "type": "env", "status": "blocklisted", "message": "Blocklisted by gfxInfo" } ] }, { "name": "WEBRENDER_OPTIMIZED_SHADERS", "description": "WebRender optimized shaders", "status": "available", "log": [ { "type": "default", "status": "available" } ] }, { "name": "WEBRENDER_ANGLE", "description": "WebRender ANGLE", "status": "unavailable", "log": [ { "type": "default", "status": "available" }, { "type": "env", "status": "unavailable", "message": "OS not supported" } ] }, { "name": "WEBRENDER_DCOMP_PRESENT", "description": "WebRender DirectComposition", "status": "unavailable", "log": [ { "type": "default", "status": "available" }, { "type": "user", "status": "disabled", "message": "User disabled via pref" }, { "type": "env", "status": "unavailable", "message": "Requires Windows 10 or later" }, { "type": "runtime", "status": "unavailable", "message": "Requires ANGLE" } ] }, { "name": "WEBRENDER_SOFTWARE", "description": "WebRender software fallback", "status": "available", "log": [ { "type": "default", "status": "available" } ] }, { "name": "WEBGPU", "description": "WebGPU", "status": "blocked", "log": [ { "type": "default", "status": "disabled", "message": "Disabled by default" }, { "type": "runtime", "status": "blocked", "message": "WebGPU cannot be enabled in release or beta" } ] }, { "name": "VP8_HW_DECODE", "description": "VP8 hardware decoding", "status": "available", "log": [ { "type": "default", "status": "available" } ] }, { "name": "VP9_HW_DECODE", "description": "VP9 hardware decoding", "status": "available", "log": [ { "type": "default", "status": "available" } ] } ], "fallbacks": [] }, "crashGuards": [] }, "addons": [ { "name": "Mozilla Android Components - Ads Telemetry", "type": "extension", "version": "103.0.10.0803084751", "isActive": true, "id": "[email protected]" }, { "name": "Mozilla Android Components - Browser Icons", "type": "extension", "version": "103.0.10.0803084737", "isActive": true, "id": "[email protected]" }, { "name": "Mozilla Android Components - Firefox Accounts WebChannel", "type": "extension", "version": "103.0.10.0803084742", "isActive": true, "id": "[email protected]" }, { "name": "Mozilla Android Components - ReaderView", "type": "extension", "version": "103.0.10.0803084737", "isActive": true, "id": "[email protected]" }, { "name": "Mozilla Android Components - Search Telemetry", "type": "extension", "version": "103.0.10.0803084751", "isActive": true, "id": "[email protected]" }, { "name": "Mozilla Android Components - Web Compatibility Interventions", "type": "extension", "version": "103.0.0", "isActive": true, "id": "[email protected]" }, { "name": "uBlock Origin", "type": "extension", "version": "1.43.0", "isActive": true, "id": "[email protected]" } ], "features": [] }

• Android device: SM-T560NU running 7.1.1
• Fenix version: 103.0.2

┆Issue is synchronized with this Jira Task

[amedyne]

@sv-ohorvath Can the team help validate this issue and see if clear cache and clear image are working properly? Thanks!

[KennethHeal]

Please let me know if I get any other useful data, force sqlite vacuum and/or browse filesystem of fenix data directory. Thanks.

[delia-pop]

We were not able to reproduce the issue so far on RC 103.2.0. The user data is reduced after deleting cached images and files data.

Devices used:

• Tab Sony Xperia (Android 6.0.1)
• LG G7 fit (Android 8.1.0)
• OnePlus Nord N10 (Android 11)
• Lenovo Yoga Tab 11 (Android 11)
• Xiaomi 12 Pro (Android 12)

[Dunexus]

I am able to reproduce the high amount of user data stored. This is an install of Fenix that I use daily and keep up to date for several years (since the beginning of fenix github releases).

[KennethHeal]

Thanks for the updates. In my case for a couple of weeks usage hovered around 150mb but in the last week ballooned to the 236mb you see here. Is there a way to browse the filesystem and/or check sqlite db sizes to establish what is taking up all that space?

[KennethHeal]

Storage is still bloating

[KennethHeal]

I conducted the following experiment of deleting just browsing history and site data. At the start of this this was the space usage:

So I went ahead and deleted browsing history and site data.

This took a few minutes. Having done so Fenix showed this

And storage space usage

[KennethHeal]

Unfortunately history failed to sync back so I lost all my history.... 😡

[KennethHeal]

Btw 101mb is also bloated for just ublock addon, cookies and logins

[KennethHeal]

It's still bloating away. At this stage it is near unusable.

[amedyne]

@KennethHeal thank you for providing updates and additional info on this ticket.

@mcarare Can you help investigate further what is going on here? Thanks!

[mcarare]

@KennethHeal Thank you for the investigations so far!

First, I would like to state that it is normal up to a point that upon visiting multiple web pages the "user data" (or simply "data" depending on OS) will increase continuously. That is caused by specific data to every website that is saved in order to optimize future visits to that page.

In your specific case, and that makes it harder for us to reproduce and investigate, it might be aggravated by specific pages you are visiting (as deleting site data decreases the space usage).

One thing that you could do that would help us investigate further, is to observe the "data" category size between navigating to pages that you visit regularly. If you notice a spike after opening one specific page that would be a piece of information that could help us reproduce and better diagnose the underlying problem.

Thank you!

[KennethHeal]

@mcarare thanks for the follow up. I'd agree that some usage would be normal, but this bloating is beyond that. Unfortunately this bug caused me to lose all site history. I've not been able to detect much of a pattern, other than for the first week it seems okay and then the bloat sets in. Unfortunately Fenix is a bit of a black box as there doesn't seem to be anyway to check the file system (no file:// or about:config), sqlite database size or where the data usage is coming from. Or am I missing any instrumentation or debug feature?

[mcarare]

@KennethHeal If you have access in your file explorer app, you can check files in data/data/org.mozilla.firefox ... folders to see if either one stands out in terms of size.

[KennethHeal]

@mcarare as my device is not rooted I don't see any of the files within my files app. Previously with file:// one could browse them.

[KennethHeal]

A couple days of light usage and its bloated to 230mb

[KennethHeal]

Bloaty McBloatfox... (updated to 104.1)

[KennethHeal]

As the bloat got out of hand I cleared all data, logged back into Firefox and resynced. Then went ahead and added uBlock Origin and logged into websites I commonly used. Having used it for half a day this is the usage. This is therefore a reasonable baseline. In a good functional scenario I would expect incremental growth from here. It's a pity there's no debugging tools and no progress yet, so I may be forced to switch browser.

[KennethHeal]

After a week of relative good behaviour Bloaty McBloatfox has resurfaced, 170mb data usage.

[Dunexus]

I did a fresh install on my new phone two weeks ago. The data usage is kept under 100MB since.

Everything seems normal on this new install. The old one may have suffer from running nightly for years of early development.

[KennethHeal]

I, too, did a fresh install a fortnight ago and synced back bookmarks, passwords arc. Unfortunately that has had no impact on the ever growing bloat which is now at 202mb. Dev seems to be at a complete loss and there seems to be no instrumentation to see where the bloat is being caused :(

[opusforlife2]

Fear not, mortals! With the power of root, I have pinpointed the incriminating folder!

@mcarare It's inside the profile folder: settings/security-state/cert-revocations/. Out of my total user data of 600 MB, a mind-boggling 480 MB is taken up solely by files inside this folder.

Just to be clear, this is separate from the security_state folder, which is also inside the profile folder, but outside the settings folder.

Does that help?

[KennethHeal]

Is there any way without root for me to confirm if that's the case for me too? It strikes me that it would be useful to reinstate the file:// browse mechanism Firefox used to have as this would allow a look through the fs.

[opusforlife2]

Without root, you can get as far as opening the /data/data folder and see the list of app folders using a file explorer. But you cannot go further than that.

[issue-dispenser]

This is another example where implementing #2080 would have helped: even non-root users would be able to manually inspect their profile directories and report specific sizes and even contents of otherwise-inaccessible files.

@mcarare:

First, I would like to state that it is normal up to a point that upon visiting multiple web pages the "user data" (or simply "data" depending on OS) will increase continuously. That is caused by specific data to every website that is saved in order to optimize future visits to that page.

Shouldn't that be under "Cache", though? Or is it some Android/integration quirk that puts this cache in the "Data" category?

[KennethHeal]

The bloat goes on, and this means Firefox is effectively unusable. It is concerning that apart from @issue-dispenser dev has little idea nor any tooling to troubleshoot this, leaving me unable to confirm if I'm having the same cause.

[mcarare]

As I stated before, this looks like the normal effects of using any browser. The size of the user data does not make the app unusable. IMO this works as expected.

[KennethHeal]

You stated up to a point this is normal, but unfortunately provided no workable steps to validate the behaviour here. This bloat causes the Android file system to become full and the device unusable. 216mb of unaccounted for data is excessive on a 10G filesystem! What makes no sense is that for a couple of weeks usage is around the 120mb mark and then it starts bloating. I'd expect normal behaviour to be moderate increases after the first few weeks, not a near doubling. I'd also expect this to be under cache rather than data, and of course cache can be emptied.

[opusforlife2]

@mcarare Did you view this root data I've provided above? One single folder taking up 80% of the app's data is unlikely to be normal.

Edit: Moreover, I use private browsing, and have all data set to delete when tapping Quit. So it's not even that the app is collecting my browsing and site data.

[KennethHeal]

I'd welcome some troubleshooting of the issue too. At a minimum, a way to see what is using the space on a non-rooted device, which used to be possible along with a method to reclaim it without deleting all history. Thanks.