Fujifilm X-E3: Device scan failed

[davidak]

When i connect the Fujifilm X-E3 via USB with the PC in "USB Card" mode and open rapid-photo-downloader, i get an error. This is probably similar to the other reported Fuji errors.

Terminal output:

[davidak@gaming:~]$ rapid-photo-downloader 
ERROR    Unexpected exception while scanning X-E3
Traceback (most recent call last):
  File "/nix/store/rgd7qzjzsz6nxbmwg94grfa4kwvap7qz-rapid-photo-downloader-0.9.34/lib/python3.10/site-packages/raphodo/scan.py", line 208, in do_work
    self.do_scan()
  File "/nix/store/rgd7qzjzsz6nxbmwg94grfa4kwvap7qz-rapid-photo-downloader-0.9.34/lib/python3.10/site-packages/raphodo/scan.py", line 367, in do_scan
    self.camera.free_camera()
  File "/nix/store/rgd7qzjzsz6nxbmwg94grfa4kwvap7qz-rapid-photo-downloader-0.9.34/lib/python3.10/site-packages/raphodo/camera.py", line 787, in free_camera
    self.camera.exit(self.context)
gphoto2.GPhoto2Error: [-1] Unspecified error

rpd-bug-report-20231029.tar.gz

System journal output when connecting the camera:

Oct 29 22:49:19 gaming kernel: usb 1-8: new high-speed USB device number 11 using xhci_hcd
Oct 29 22:49:19 gaming kernel: usb 1-8: New USB device found, idVendor=04cb, idProduct=02d6, bcdDevice= 1.30
Oct 29 22:49:19 gaming kernel: usb 1-8: New USB device strings: Mfr=0, Product=2, SerialNumber=3
Oct 29 22:49:19 gaming kernel: usb 1-8: Product: USB PTP Camera
Oct 29 22:49:19 gaming kernel: usb 1-8: SerialNumber: *****
Oct 29 22:49:19 gaming mtp-probe[311864]: checking bus 1, device 11: "/sys/devices/pci0000:00/0000:00:14.0/usb1/1-8"
Oct 29 22:49:19 gaming mtp-probe[311864]: bus: 1, device: 11 was not an MTP device
Oct 29 22:49:19 gaming mtp-probe[311867]: checking bus 1, device 11: "/sys/devices/pci0000:00/0000:00:14.0/usb1/1-8"
Oct 29 22:49:19 gaming mtp-probe[311867]: bus: 1, device: 11 was not an MTP device

---

Version: 0.9.34 (apparently one patch version ahead of https://damonlynch.net/rapid/download.html) OS: NixOS 22.11.4588.93fddcf640c

[davidak]

I'm able to download the files using gphoto2 CLI.

[nix-shell:~]$ gphoto2 --list-files
There is no file in folder '/'.                                                
There is no file in folder '/store_10000001'.
There is no file in folder '/store_10000001/DCIM'.
There are 14 files in folder '/store_10000001/DCIM/115_FUJI'.
#1     DSCF5573.JPG               rd 13459 KB 6000x4000 image/jpeg 1698618458
#2     DSCF5574.JPG               rd 13342 KB 6000x4000 image/jpeg 1698618458
#3     DSCF5575.JPG               rd 13442 KB 6000x4000 image/jpeg 1698618458
#4     DSCF5576.JPG               rd 13448 KB 6000x4000 image/jpeg 1698618458
#5     DSCF5577.JPG               rd  8916 KB 6000x4000 image/jpeg 1698618890
#6     DSCF5578.JPG               rd  9978 KB 6000x4000 image/jpeg 1698618942
#7     DSCF5579.JPG               rd  9917 KB 6000x4000 image/jpeg 1698618962
#8     DSCF5580.JPG               rd 10352 KB 6000x4000 image/jpeg 1698618976
#9     DSCF5581.JPG               rd  9907 KB 6000x4000 image/jpeg 1698618980
#10    DSCF5584.JPG               rd  9995 KB 6000x4000 image/jpeg 1698619038
#11    DSCF5587.JPG               rd 10909 KB 6000x4000 image/jpeg 1698619166
#12    DSCF5588.JPG               rd 11323 KB 6000x4000 image/jpeg 1698619170
#13    DSCF5589.JPG               rd 10984 KB 6000x4000 image/jpeg 1698619170
#14    DSCF5590.JPG               rd  9712 KB 6000x4000 image/jpeg 1698619368

[nix-shell:~]$ gphoto2 --get-all-files
Saving file as DSCF5573.JPG                                                    
Saving file as DSCF5574.JPG
Saving file as DSCF5575.JPG
Saving file as DSCF5576.JPG
Saving file as DSCF5577.JPG
Saving file as DSCF5578.JPG
Saving file as DSCF5579.JPG
Saving file as DSCF5580.JPG
Saving file as DSCF5581.JPG
Saving file as DSCF5584.JPG
Saving file as DSCF5587.JPG
Saving file as DSCF5588.JPG
Saving file as DSCF5589.JPG
Saving file as DSCF5590.JPG

gphoto2-2.5.28 here uses libgphoto2-2.5.30 (full nix package path: /nix/store/12phfpnmdbnc5m4qx5xs8rx98kvbwc6w-libgphoto2-2.5.30)

rapid-photo-downloader uses the exact same package. so it does not seem to be an issue with libgphoto2

[damonlynch]

What is "USB Card" mode?

[davidak]

data transfer mode, allowing data to be copied to the computer

https://fujifilm-dsc.com/en/manual/x-e3/menu_setup/connection_setting/index.html#pc_connection_mode

It sounds like the camera would act as a usb card reader and is a standard mass storage device for the computer (like a usb thumb drive), but it seem to be MTP (PTP).

[damonlynch]

The bug might be because of changes in the python interface to gPhoto2. You have Python gPhoto2 2.3.4 installed, which is a very recent version that I have not tested.

But it might be because of your camera being a Fuji. In my experience with Rapid Photo Downloader, historically they have been the manufacturer most likely to deviate from industry standards. If it is a problem caused by using Fuji, it's extremely difficult to test, given I do not have one of those cameras.

If it's a problem because of the recent version of Python gPhoto2, I can investigate that myself. If it is caused by the camera, that is more difficult. Either I need to somehow get hold of one of these cameras so I can revise and test the code, or else you'll need to spend time doing that, with my help as needed.

[Skoubidou]

I have the same issue with my FujiFilm X-S10. I'm a new Fedora user. I didn't have any problem with Linux Mint. rpd-bug-report-20231217.tar.gz

[damonlynch]

@davidak or @Skoubidou are you interested in testing a possible fix?

[ciaron]

I have the same issue with X100V and X-T3. I'd be happy to test fixes on these cameras.

[damonlynch]

@ciaron great. Here is how to test:

1. Download a zip archive of the main branch for Rapid Photo Downloader from GitHub. On the home page for this project, you should see a green button that says Code. Use that. There is no need to use git unless you really want to.
2. I'm going to assume you have Rapid Photo Downloader already installed, meaning all its dependencies are already met.
3. Extract the code into a subfolder. I will assume it is called rapid-photo-downloader, located in your home folder.
4. Within a terminal window, set the following temporary environment variable to use the new subfolder containing the Rapid Photo Downloader code: export PYTHONPATH="$HOME/rapid-photo-downloader"
5. Within the same terminal window, run the program: python3 ~/rapid-photo-downloader/raphodo/rapid.py --debug. On your system you may need to use python instead of python3, depending on how your system is configured.
6. If you are still seeing the bug, export another temporary environment variable: export RPD_SCAN_DEBUG=1
7. Replicate the problem after setting that second temporary environment variable, which will fill the log files with additional debug information.
8. Upload the debug tarball that the program generates here. With that 2nd environment variable, the log files will have more detailed information that may help diagnose any problem.
9. You can delete the subfolder rapid-photo-downloader when you are done.

[ciaron]

Tested, but the error persists:

rpd-bug-report-20240110-2.tar.gz

It seems to be line 764 (self.camera.exit()) in free_camera (raphodo/camera.py). I commented this line out as a test and the error disappears, so this seems to be a problem in gphoto2?

I also tried a newer python-gphoto2 (pip install in a conda environment), this also failed.

(similar issue reported here: https://github.com/jim-easterbrook/python-gphoto2/issues/128, I tried the small test script from this issue as well)

Let me know if I can do more tests.

Edit: just as another data point: The above error manifests on a standard Ubuntu 23.10 installation:

libgphoto2-6/mantic,now 2.5.30-1 amd64 [installed,automatic]
  gphoto2 digital camera library

libgphoto2-port12/mantic,now 2.5.30-1 amd64 [installed,automatic]
  gphoto2 digital camera port library

python3-gphoto2/mantic,now 1.9.0-1build4 amd64 [installed,automatic]
  Python interface to libgphoto2 (Python 3)

I tested on Ubuntu 22.04.3 LTS, the error does not appear:

libgphoto2-6/jammy,now 2.5.27-1build2 amd64 [installed,automatic]
  gphoto2 digital camera library

libgphoto2-port12/jammy,now 2.5.27-1build2 amd64 [installed,automatic]
  gphoto2 digital camera port library

python3-gphoto2/jammy,now 1.9.0-1build3 amd64 [installed,automatic]
  Python interface to libgphoto2 (Python 3)

[damonlynch]

I suspect the problem is in python-gphoto2, but I am certainly not sure about that at this moment. Sometimes bugs manifest themselves in strange ways.

One user has found that his X-S10 has been detected as a GFX-100. That indicates the root cause of the problem is occurring well before the free_camera() call, and that this exception is merely a symptom of that root cause.

Regarding the small test script you tried: are you saying that you used a python-gphoto2 test script (which has no Rapid Photo Downloader code), but still manifests a problem? If so, that would narrow down the root problem.

[ciaron]

Yes, I ran a python-gphoto2 test script, with no Rapid Photo Downloader code, and eventually traced the problem back to libgphoto2. I've opened a ticket there (https://github.com/gphoto/libgphoto2/issues/950). I'm now not certain that the X-E3 issue is the same as the X100V issue, but for the X100V it appeared when the detection code for the camera was added to the libgphoto2 code. (Before that it was detected as a generic PTP device)