Duplicates / Some images are repeated

[wanzh5]

Describe the bug When the app refreshes the list, some newly added images are duplicated, and it won't be fixed even if I reopen the app. It seems that this only happened after version 1.10.8.

To Reproduce Sorry, I don't know. It happens nearly 1 bugged image of 10 normal images.

Screenshots

System information and logs: aves-logs-20240410_132650.txt

Additional context If you move the image to other folder (whatever by file manager or by aves), it will be fixed. The "info" page of the duplicated images are totally same.

[deckerst]

Thanks for the report. I'm aware of the bug but i couldn't reproduce it myself (on my own device or emulators) so i was waiting for more reports and users willing to help me debug it.

Could you please paste the system info from About > Bug Report?

Do you use a custom ROM? A rooted device?

What is the app that creates the new media that gets duplicated (OEM camera, 3rd party camera, screenshot, file transfer, etc)?

Do you switch to Aves right after they are created, or later?

Related:

• #880
• #891

[wanzh5]

Could you please paste the system info from About > Bug Report?

here is the systen info: Package: deckers.thibault.aves.libre Installer: dev.imranr.obtainium Aves version: 1.10.8-libre, build 11702 Flutter: stable 3.19.5 Android version: 10, API 29 Android build: TAS-AN00 2.0.0.263(C00E230R5P6) Device: HUAWEI TAS-AN00 Support: dynamic colors=false, geocoder=true, HDR=true Mobile services: not available System locales: zh_Hans_CN, en_CN, zh_Hant_CN Storage volumes: /storage/emulated/0/ Storage grants: Error reporting: false

What is the app that creates the new media that gets duplicated?

Until now, I have seen screenshots and downloads.

Do you switch to Aves right after they are created, or later?

Most of the time I switch to Aves a few minutes later.

The bug seems to usually happen after I add more than 5 images at a time.

[deckerst]

Could you please do the following:

1. install this test build (20240805_1930). It will install as Aves [Profile]. Please let the default settings as they are.
2. launch Aves [Profile] and let it run its initial media scan. It might take a while if you have many media items.
3. try to reproduce the issue by creating new media and checking in Aves [Profile] if you see duplicates
4. when you spot a duplicate, please note the URI and path of the duplicate (available in the media Info page), and save the app logs (from About > Bug Report)
5. attach here the logs, and let me know the URI/path of the duplicate.

[wanzh5]

I tried to reproduce it and saw some more stranger bugs. I take a lot of screenshots and there are no duplicates any more. However, there are some images whose date change to 2202. You can see it in this screenshot. picture1: content://media/external/images/media/1002706 /storage/emulated/0/Pictures/Screenshots/Screenshot_20240411_220042_com.android.deskclock.jpg⁩ picture2: content://media/external/images/media/1002732⁩ ⁨/storage/emulated/0/Pictures/Screenshots/Screenshot_20240411_220209_com.android.deskclock.jpg⁩

Then I opened the normal aves and see all of the pictures are duplicated. However, I can't reproduce it. You can see it in this screenshot. an example: content://media/external/images/media/1002715⁩ /storage/emulated/0/Pictures/Screenshots/Screenshot_20240411_220104_com.android.deskclock.jpg⁩

Here is the log. normal aves aves-logs-20240411_220519.txt debug version aves-logs-20240411_220426.txt

[wanzh5]

I found that when I took screenshots, aves (debug version) was still taking its initial media scan. I thought that it may help you.

[deckerst]

Thanks for taking the time to try it, but for now I'd like to focus on reproducing the duplicate issue with the test version. And it's important to let the initial scan complete first. Please let me know if you can reproduce the duplication with the debug version (with the debug logs and duplicate name).

[deckerst]

Actually, I've noticed that the log files attached here are incomplete. I need to find another way to log that's compatible with your device, so no need to test further for now.

[deckerst]

About the screenshot with the wrong date, if it's still the case, could you open one of them (in the debug app) in the viewer, then select in the menu > Debug.

Then please take screenshots showing the Entry and Android icon, as follows:

[wanzh5]

Then please take screenshots showing the Entry and Android icon, as follows:

[deckerst]

Thanks, so it's really an issue with cataloguing ^^ Could you please send me that file with the wrong date (without modification or resize)?

[wanzh5]

It is in the zip file. Screenshot_20240318_132517_com.google.android.apps.translate.jpg.zip

[deckerst]

Weird, i can't reproduce. On my device, it has the correct date. Could you please export the metadata and attach it here?

[wanzh5]

Screenshot_20240318_132517_com.google.android.apps.translate-metadata.txt

I thought that it may be the same as the bug about duplicate, because when I move the image to another folder, it will be fixed.

[deckerst]

Maybe you're right. The metadata is fine, so cataloguing should be accurate. Maybe at first Aves sees the file with a wrong date reported by Android, and thinks it's a different file later... Hard to say without full logs.

I'll try and figure out a way to dig deeper. Thank you for now!

[kmcbest]

I came across the same problem with Redmi K60 Ultra. Delete either dupe will delete them both. Move either one to another location will result in a single one. So I often move it and move back to solve the problem. But it is not very reproducible. My experience is that the dupes are created upon tagging multiple (more than 50) images.

[deckerst]

@kmcbest could you please follow the instructions at https://github.com/deckerst/aves/issues/971#issuecomment-2048108266 ?

[bodypilot]

Could you please do the following:

1. install this [test build](https://drive.google.com/file/d/1KfsSUqS21BnfkCnk4KtI3CbQv6ZtmPcx/view?usp=sharing). It will install as `Aves [Profile]`. Please let the default settings as they are.

After months, possibly years, of using Aves without issues, I now have the problem of duplicates appearing as well. I wanted to try the test build, but can't download it ("No preview available. File is in owners trash" and "Viewers cannot download")

Duplicates started to appear in the last few days from downloads from Mull browser, 3rd party camera app (Google Cam by BSG, 8.7.250) and Whatsapp. Screenshots don't appear to be affected (yet?)

I changed only two things just before this issue appeared: I set GoogleCam to also save the raw image files (*.dng) and I set Aves to hide those files. I've now set Aves to not hide the raw files to see if that helps.

I'm using the LineageOS for Microg rom.

System info: Package: deckers.thibault.aves.libre Installer: org.fdroid.fdroid.privileged Aves version: 1.11.8-libre, build 12702 Flutter: stable 3.22.3 Android version: 13, API 33 Android build: lineage_alioth-userdebug 13 TQ3A.230901.001 eng.root.20240524.072639 dev-keys Device: Xiaomi M2012K11AG Support: dynamic colors=true, geocoder=true, HDR=true Mobile services: not available Connectivity: wifi System locales: en_US Storage volumes: /storage/emulated/0/ Storage grants: /storage/emulated/0/Pictures/, /storage/emulated/0/DCIM/, /storage/emulated/0/Download/Youtube/, /storage/emulated/0/Download/Fairemail/, /storage/emulated/0/Android/media/ Error reporting: false

Logs attached. aves-logs-20240805_182439.txt

[deckerst]

Thanks for the report.

Regarding the test build, I'll come back to you with a new build from the latest source.

As for the issue, can you reliably make it happen? Or is it randomly happening once in a blue moon?

Do you allow notifications for Aves, so you can see when the media scanning service is running? If so, do you see it running when you experience the issue?

When duplicates appear, is it when opening the app very soon after the media is created on the device?

[bodypilot]

Thanks for the report.

Regarding the test build, I'll come back to you with a new build from the latest source.

As for the issue, can you reliably make it happen? Or is it randomly happening once in a blue moon?

It's random, I haven't been able to determine a pattern yet.

Do you allow notifications for Aves, so you can see when the media scanning service is running? If so, do you see it running when you experience the issue?

I didn't, but have now allowed Aves to show notifications.

When duplicates appear, is it when opening the app very soon after the media is created on the device?

I don't think I've opened Aves immediatly after creating media but will pay more attention going forward.

[deckerst]

@bodypilot I edited the test build link in the comment above.

[bodypilot]

Thanks. I installed the test build and right of the bat it doesn't show the duplicates the regular version still shows. Have taken some pics which don't duplicate, not in the test build, nor in the regular build. I'll keep monitoring this in the next few days.

[deckerst]

The duplicate issue is a corruption internal to the app, so it's normal that the duplicates in one app do not show in the other.

[krazykelv]

The test build seems to have fixed it for me as well, thank you!

[deckerst]

The test build is not supposed to fix anything. It's just a build with additional logging, so that if the issue happens, logs should help me diagnose the root cause.

[bodypilot]

I've shot about 3 dozen pictures these last couple of days, and have seen no duplicates, neither in the test build, nor the "normal" build. I'll keep using the test build and will report back when I see a duplicate appearing.

[krazykelv]

I've got 3 or 4 duplicates showing again on the Profiling build. I've attached my current app logs for you @deckerst aves-logs-20240816_123818.txt

[deckerst]

@krazykelv thanks for following up. Sadly, I don't see my additional logs here and it seems the exported logs only contain the very last session. I'll need to think of a smarter way to debug this. I'll come back to you, but in a while (probably end of the month).

[krazykelv]

Ok. For now, if I notice the problem straight after taking a photo, I'll send the debug logs through again and see if it captures the current session

[bodypilot]

I've shot hundreds of pictures in the last couple of weeks: no duplicates were seen. I'll be taking a lot less pictures going forward, but if I see a duplicate I'll attache the log to this thread. I'm also willing to try any new test builds you might come up with.

[deckerst]

tl;dr: I'm releasing a "fix".

Long version: I couldn't find the root cause and it's too difficult to reproduce the issue. So I didn't actually fix the issue but instead took measures to detect and remove duplicates as they appear (assuming they appear in the code paths I think they do). It will also report the issue (for users who allowed automatic error reporting). I hope to get insights from these reports, and fix (or narrow down) the issue in the following releases, but that would be transparent for users.