[Bug]: can't find NEF files

[ymarkus]

As Immich supports NEF files, but right now there's no bulk upload...

[alextran1502]

What do you mean by this? Can you elaborate?

[ymarkus]

Oh, I just realized it is supposed to support it, sorry! For some reason, it doesn't find the files in my directory. I'll upload some screenshots: This also doesn't work:

[ymarkus]

Ok, I think I've found the issue: It seems like this CLI looks for these mime types: "image/nef", "image/x-nikon-nef", These are not standard mime types according to https://www.iana.org/assignments/media-types/media-types.xhtml and are also not "custom filetypes" in mime-db. Did I miss something or do I need to open a PR there to get it registered as "custom filetype"?

[alextran1502]

Is there an example file that you can share for testing? I can update the file mime type once I got the correct one

[ymarkus]

Yes, of course. I'll send it by e-mail if it's ok for you. So, I've been having a conversation at https://github.com/jshttp/mime-db/pull/219 with the maintainer there. This is the mime-db package mime-types depends on as explained in https://www.npmjs.com/package/mime-types. It seems like they are not able to add those mime types to their database. Maybe we need a different way of recognizing mime types / file types. Another indication that "nef" mime types are hit or miss:

❯ mimetype raw/2020/Island/100D5000/DSC_0001.NEF
raw/2020/Island/100D5000/DSC_0001.NEF: image/x-nikon-nef
❯ file --mime-type raw/2020/Island/100D5000/DSC_0001.NEF
raw/2020/Island/100D5000/DSC_0001.NEF: image/tiff

[ymarkus]

Ok, so I did some more investigation. The mime-types package returns false as mimeType for all NEF or nef files. Do you think checking for filename extensions would be fine for those mime types missing in mime-db?

[n3u1r0n]

I do have the same issue with dng files. The mime-types package simply returns false and they do not get uploaded.

I tried removing the mime-type check, as I knew that all the files in the directory are safe, but then I get

Failed to upload 1 files  [
  {
    file: '/home/USER/test/IMG20150320_000007.dng',
    reason: Error: Request failed with status code 400
        at createError (/usr/lib/node_modules/immich/node_modules/axios/lib/core/createError.js:16:15)
        at settle (/usr/lib/node_modules/immich/node_modules/axios/lib/core/settle.js:17:12)
        at IncomingMessage.handleStreamEnd (/usr/lib/node_modules/immich/node_modules/axios/lib/adapters/http.js:322:11)
        at IncomingMessage.emit (node:events:525:35)
        at endReadableNT (node:internal/streams/readable:1359:12)
        at process.processTicksAndRejections (node:internal/process/task_queues:82:21) {
      config: [Object],
      request: [ClientRequest],
      response: [Object],
      isAxiosError: true,
      toJSON: [Function: toJSON]
    },
    response: {
      statusCode: 400,
      message: 'Unsupported file type .dng',
      error: 'Bad Request'
    }
  }
]

[n3u1r0n]

As a workaround (obviously not a good solution) and a lead to where the bug lies: I commented out and adapted the mime-type check in the CLI

// if (SUPPORTED_MIME.includes(mimeType)) {
    if (mimeType === false) {
        mimeType = "image/x-adobe-dng";
    }
    const fileStat = fs.statSync(filePath);
    localAssets.push({
        id: `${path.basename(filePath)}-${fileStat.size}`.replace(/\s+/g, ""),
        filePath,
    });
// }

and adapted the getAssetType function in the CLI

function getAssetType(filePath) {
    var mimeType = mime.lookup(filePath);
    if (mimeType === false) {
        mimeType = "image/x-adobe-dng";
    }
    return mimeType.split("/")[0].toUpperCase();
}

which then gave me the 400 response above.

So I went inside the docker container of the Immich server I commented out the mime-type check in /usr/src/app/dist/apps/immich/apps/immich/src/config/asset-upload.config.js

//if (file.mimetype.match(/\/(jpg|jpeg|png|gif|mp4|webm|x-msvideo|quicktime|heic|heif|dng|x-adobe-dng|webp|tiff|3gpp|nef|x-nikon-nef)$/)) {
    cb(null, true);
//}
//else {
//    logger.error(`Unsupported file type ${(0, path_1.extname)(file.originalname)} file MIME type ${file.mimetype}`);
//    cb(new common_1.BadRequestException(`Unsupported file type ${(0, path_1.extname)(file.originalname)}`), false);
//}

When I now upload my .dng's everything works fine.

[uhthomas]

Yeah, I observed this too. The cli tool will need some more custom mime type stuff given the upstream library doesn't have an api to add custom mime types(!?). I'll probably take a look at this soon.

[kmaid]

Just chiming in here that I have noticed that NEFS were not added when imported via an external library