Having trouble with endpoints

[twekberg]

When I try to get a file with the AWS credentials exports set I get this error:

PermanentRedirectThe bucket you are attempting to access must be addressed using the specified endpoint. Please send all future requests to this endpoint.dlmp-oltg-data.s3.amazonaws.comdlmp-oltg-dataJ6PHT3BQ1M16WQPBa25dkSf+MfTKjZDflgi4PvbERAtKnuf3wJj5wJwpvWlISxHCjSB0F9UkPxmHWMx77g8QF+k+GXk=t

Not sure how to proceed.

[paulhammond]

Hi!

This script is really really old and used to use "path-style" URLs, which were deprecated by AWS in 2020. I've just pushed 7d5e11a which uses "virtual hosted" urls which I think should fix your problem.

If not, the easiest fix is probably to copy the file, and edit line 74 until it works for you. You might need to add endpoint. into the URL, for example.

Hope this is helpful!

[paulhammond]

It's also possible that the problem here is the deprecation of S3 Signature V2 which also affects any buckets created since 2020. If so you'll probably get an error message saying as such. I'm reopening this to see if that's the problem.

[twekberg]

I figured out that the problem was using the wrong region in the endpoint. I fixed my version to do that. I also wrote another script that grabs the default region from ~/.aws/config and the 2 AWS settings from a specified section in ~/.aws/credentials, and calls a modified version of your script. Feel free that take that and my changes. I work for the state of Washington so all of the stuff I write is free. I had to tack on a .txt extension to the filenames to paste them here.

s3_simple_transfer.sh.txt
aws_transfer.txt

[paulhammond]

I'm glad you got it working - and thanks for sharing your changes!

I think they might be too specific to merge into main but if I'm wrong and anyone else is reading this and thinks they're useful please leave a comment so I can reconsider.

[paulhammond]

It looks like someone else wanted region support so it's getting implemented - see #6 for details.