Cache/Listing issues

[bstein-clever]

I'm working on upgrading our site from Craft2 to Craft3, and have been experiencing trouble getting S3 buckets to work, which we use heavily.

Upgrading Craft and installing the aws-s3 plugin all went fine, and our settings were migrated, or so it seemed. The site was erroring at first (I didn't think to save that, unfortunately), which was remedied by viewing and re-saving all of the s3 asset volumes.

However, now the site doesn't actually display any of the images, and trying to preview (or download) it spinwaits for a long time and then displays "an unknown error occurred". No logs seem to be emitted in storage/logs.

I tried to make a new asset volume to recreate one, but it doesn't work properly, either (no files indexed). I'm able to list all my buckets, but after creating it, there are no files indexed - and I've tried both with and without public URLs, since some of the other issues implied there might be something to that.

This led me to think that maybe it was an issue with indexing, and when I try to refresh the asset index in the CP manually, I'm rewarded with an interesting error:

Error executing "ListObjects" on "https://s3.us.amazonaws.com/assets.clever.com?prefix=website%2Ffiles%2F&encoding-type=url"; AWS HTTP error: cURL error 6: Could not resolve host: s3.us.amazonaws.com (see http://curl.haxx.se/libcurl/c/libcurl-errors.html)

Where website_files is the subfolder under the assets.clever.com s3 bucket. Strangely, I can find no references to the s3.us.amazonaws.com url anywhere in the code, so I'm not clear where it's coming from, or how best to fix it.

Any suggestions would be greatly appreciated!

[bstein-clever]

Also worth mentioning: I'm fairly certain that this isn't a permissions issue, because all the asset volumes worked perfectly fine before the upgrade. But I would very much like to be able to update the volumes in-place, so that we don't have to redo the linking of all of these assets to the pages where they're used, which would be quite an undertaking.

[andris-sevcenko]

Hmm. This is not the first time I see this cropping up, but this is the first time I see Craft 2 mentioned, so maybe there's an issue in the upgrade path. The real problem here is that the regions are saved incorrectly in the volume configuration.

To resolve this, what you should do is open up each Volume, refresh the bucket list and then save. That should fix this for you.

I'm leaving this issue open since I definitely have to look into the migration path from Craft 2 to Craft 3, to see if there is an issue there.

[bstein-clever]

Thanks, that helped! As an extra data point, clicking refresh alone was insufficient, I actually had to change the bucket and then change it back (I didn't have to save in between) which then overwrote my public URL path (which I changed back). That got the caching working, and I was able to uncover a separate bug in my template related to the new .all() and .one() style, but that was on my end. :)

If you need a tester after fixing the upgrade path, feel free to let me know - I'm doing a slow dry-run making sure everything works while there are other modifications being made to our production site, and then I'll end up redoing it again once I've ironed out all the bugs. But it means I'll have to deal with re-editing all the buckets again...

[andris-sevcenko]

Seems like this largely has been fixed with 1.1.0

[aaronbushnell]

Hi! I seem to still be having this issue with the latest version of the plugin. My volumes in Craft 2 have a US location and they remain US when upgrading to Craft 3.

What I end up having to do is—once upgraded—manually changing each volume to use the us-east-1 region.

Not sure if there's way to resolve that during the upgrade path or not!

[andris-sevcenko]

@aaronbushnell thanks for the information!

I'm reponing this issue and will test this upgrade path from Craft 2 breaking things.

[aaronbushnell]

No problem, @andris-sevcenko! Let me know if I can assist by sending you any logs/database info/etc! 🙌

[aaronbushnell]

Here's the JSON data within the settings column of my assetsources table. Just in case that helps here.

{
	"keyId": "REDACTED",
	"secret": "REDACTED",
	"bucket": "BUCKET_NAME",
	"subfolder": "SUBFOLDER_NAME",
	"publicURLs": "1",
	"urlPrefix": "URL",
	"expires": "1hours",
	"location": "US"
}

[mildlygeeky]

I'm working on a C2 -> C3 upgrade, and can report the same issue (region stayed as "US" after upgrading to C3) — manually updating the volumes table's settings for the volume fixed things for me.

[kmgdevelopment]

Same issue. Migrated from C2 to C3, can't update indexes and get the 'Error executing "ListObjects"' error for s3.us.amazonaws.com. Refreshing the bucket list worked for one of my sources, but not for the others.

I was able to fix it by going into the craft_volumes database table and editing the settings for each volume. Under "region" where it says US I changed it to us-east-1 or whatever AWS region code your bucket falls under.

After doing that I was able to successfully index my volumes.

[eablokker]

Same issue here, still happening when upgrading from Craft 2 to Craft 3. What solved it for me is refreshing the bucket list, and manually switching buckets, then saving the asset source.

[pdaleramirez]

This has most likely been fixed on the newer version of Craft. We’ll close this issue due to inactivity. Please comment if you still have problems with the more recent version.