Remote image download error: `ENAMETOOLONG: name too long`

[beschler]

Hi there! Very much appreciate your work on this package.

I'm running into an issue using this package to download remote images from an AWS S3 bucket using pre-signed URLs. My S3 bucket is private, and my goal is to download the remote images from AWS at build time as static assets for my static Next.js website.

Here is the contents of my remoteOptimizedImages.js file:

require("dotenv").config();
const { Pool } = require("pg");
const { S3, GetObjectCommand } = require("@aws-sdk/client-s3");
const { getSignedUrl } = require("@aws-sdk/s3-request-presigner");

const dbconnect = {
  host: process.env.DB_HOST,
  port: process.env.DB_PORT,
  user: process.env.DB_USER,
  database: process.env.DB_NAME,
};

const db = new Pool(dbconnect);

async function getStaticImages() {
  // get all photos from PostgreSQL database
  const client = await db.connect();
  const all_photos = await client.query(`SELECT filename FROM photos`);
  client.release();

  // connect to AWS S3 bucket
  const s3 = new S3({
    region: process.env.AWS_S3_REGION,
    credentials: {
      accessKeyId: process.env.AWS_ACCESS_KEY_ID,
      secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY,
    },
  });

  let photos = [];

  for (const row of all_photos.rows) {
    // get photo object from AWS S3 bucket
    const getObject = new GetObjectCommand({
      Bucket: process.env.NEXT_PUBLIC_AWS_S3_BUCKET,
      Key: row.filename,
    });

    // get pre-signed URL for AWS S3 bucket object
    const signedUrl = await getSignedUrl(s3, getObject, {
      expiresIn: 3600,
    });

    photos.push(signedUrl);
  }

  console.log(photos);

  return photos;
}

module.exports = getStaticImages();

However, I'm receiving an error stating the file name of the remote image is too long:

---- next-image-export-optimizer: Begin with optimization... ---- 
Found 62 remote images...
Error: Unable to save /Users/beschler/git/MY_REPO/remoteImagesForOptimization/MY_AWS_S3_BKT.s3.us-west-1.amazonaws.com_3af9282b9cac.webp_X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=AKIA4MTWG2XAGGQA57F5_2F20240515_2Fus-west-1_2Fs3_2Faws4_request&X-Amz-Date=20240515T033806Z&X-Amz-Expires=3600&X-Amz-Signature=89aa0609baee3266e81d5103c92a23f8cb7eb1f6859d7391919efdd2fc2152f1&X-Amz-SignedHeaders=host&x-id=GetObject.jpg (ENAMETOOLONG: name too long, open '/Users/beschler/git/MY_REPO/remoteImagesForOptimization/MY_AWS_S3_BKT.s3.us-west-1.amazonaws.com_3af9282b9cac.webp_X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=AKIA4MTWG2XAGGQA57F5_2F20240515_2Fus-west-1_2Fs3_2Faws4_request&X-Amz-Date=20240515T033806Z&X-Amz-Expires=3600&X-Amz-Signature=89aa0609baee3266e81d5103c92a23f8cb7eb1f6859d7391919efdd2fc2152f1&X-Amz-SignedHeaders=host&x-id=GetObject.jpg').
Error: Unable to download remote images (ENAMETOOLONG: name too long, open '/Users/beschler/git/MY_REPO/remoteImagesForOptimization/MY_AWS_S3_BKT.s3.us-west-1.amazonaws.com_3af9282b9cac.webp_X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=AKIA4MTWG2XAGGQA57F5_2F20240515_2Fus-west-1_2Fs3_2Faws4_request&X-Amz-Date=20240515T033806Z&X-Amz-Expires=3600&X-Amz-Signature=89aa0609baee3266e81d5103c92a23f8cb7eb1f6859d7391919efdd2fc2152f1&X-Amz-SignedHeaders=host&x-id=GetObject.jpg').
node:internal/process/promises:289
            triggerUncaughtException(err, true /* fromPromise */);
            ^

[Error: ENAMETOOLONG: name too long, open '/Users/beschler/git/MY_REPO/remoteImagesForOptimization/MY_AWS_S3_BKT.s3.us-west-1.amazonaws.com_3af9282b9cac.webp_X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=AKIA4MTWG2XAGGQA57F5_2F20240515_2Fus-west-1_2Fs3_2Faws4_request&X-Amz-Date=20240515T033806Z&X-Amz-Expires=3600&X-Amz-Signature=89aa0609baee3266e81d5103c92a23f8cb7eb1f6859d7391919efdd2fc2152f1&X-Amz-SignedHeaders=host&x-id=GetObject.jpg'] {
  errno: -63,
  code: 'ENAMETOOLONG',
  syscall: 'open',
  path: '/Users/beschler/git/MY_REPO/remoteImagesForOptimization/MY_AWS_S3_BKT.s3.us-west-1.amazonaws.com_3af9282b9cac.webp_X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=AKIA4MTWG2XAGGQA57F5_2F20240515_2Fus-west-1_2Fs3_2Faws4_request&X-Amz-Date=20240515T033806Z&X-Amz-Expires=3600&X-Amz-Signature=89aa0609baee3266e81d5103c92a23f8cb7eb1f6859d7391919efdd2fc2152f1&X-Amz-SignedHeaders=host&x-id=GetObject.jpg'
}

Node.js v20.11.1

I'm not able to modify the names of the files of the remote image URLs, as they're generated automatically from AWS. If it's helpful, here's an example of an image file URL generated by the @aws-sdk/s3-request-presigner package (that I'm passing into an array in remoteOptimizedImages.js):

https://MY_AWS_S3_BKT.s3.us-west-1.amazonaws.com/IMG_FILENAME.webp?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=AKIA4MTWG2XAGGQA57F5%2F20240515%2Fus-west-1%2Fs3%2Faws4_request&X-Amz-Date=20240515T040052Z&X-Amz-Expires=3600&X-Amz-Signature=5bb7c6b44b78482dbe9a5b5bc69b32b90f3d4552193401aa336bf7bbfcac2ffa&X-Amz-SignedHeaders=host&x-id=GetObject

Is there some way to modify the downloaded names of these images, or truncate them if they're longer than Node or my file system can handle?

Thanks in advance for your help!

---

System information:

Device: MacBook Pro 16-inch, 2023, M2 Pro OS: macOS Ventura 13.6.6 Browser: Google Chrome v124.0.6367.201 Node.js Version: 20.11.1 Next.js Version: 14.2.2 next-image-export-optimizer Version: 1.12.3