Support for eu-central-1 and IAM role authentication

[mattheworiordan]

I believe there has been a protocol change for IAM auth that is required in eu-central-1 that is incompatible with the current version of aws. See below:

$ aws describe-tags --region eu-central-1
+-------------+--------------------------------------------------------------+
|    Code     |                           Message                            |
+-------------+--------------------------------------------------------------+
| AuthFailure | AWS was not able to validate the provided access credentials |
+-------------+--------------------------------------------------------------+

This command works fine for other regions, and works with the official AWS cli tool.

[mattheworiordan]

FYI, see https://forums.aws.amazon.com/thread.jspa?messageID=582707

[timkay]

I recall implementing AWS V4 signatures. Please try adding --AWS4 to the command line. I don't recall if the support is for all AWS products.

[mattheworiordan]

Sorry, no, that doesn't work:

$ ./aws describe-tags --region eu-central-1 --AWS4
+-------------+--------------------------------------------------------------+
|    Code     |                           Message                            |
+-------------+--------------------------------------------------------------+
| AuthFailure | AWS was not able to validate the provided access credentials |
+-------------+--------------------------------------------------------------+

[timkay]

I just updated the code with support for Signature Version 4 across almost all other services. (S3 was already supported. Now EC2, etc.) Feedback, please.

To use, add --AWS4 to all commands or in your ~/.awsrc file.

Should it become the default?

[mattheworiordan]

Hi @timkay

I am not sure if I am doing something wrong, but I downloaded the latest version and tried it with --AWS4 and I see the same error unfortunately.

$ wget https://raw.github.com/timkay/aws/master/aws

$ ./aws describe-tags --region eu-central-1 --AWS4
+-------------+--------------------------------------------------------------+
|    Code     |                           Message                            |
+-------------+--------------------------------------------------------------+
| AuthFailure | AWS was not able to validate the provided access credentials |
+-------------+--------------------------------------------------------------+

Am I doing something wrong? When I remove --region eu-central-1 it works

[timkay]

Hmmm. Works for me:

./aws describe-tags --region eu-central-1 --AWS4

f9b4a3b4-cb79-4cfa-be74-cfff9d6379fb

Please send the output with -vv (two v's).

[mattheworiordan]

Here is the output

$ ./aws describe-tags --region eu-central-1 --AWS4 -vv
aws versions: (ec2: 2013-10-15, sqs: 2012-11-05, elb: 2011-11-15, sdb: 2009-04-15, iam: 2010-05-08, ebn: 2010-12-01, cfn: 2010-05-15, rds: 2013-09-09)
curl version: 7.35.0
HTTP/1.1 200 OK
x-amz-id-2: 0k7SCYmB9hOIhOl2JVzJla+ZTRc0qp4pce0Hy+iiJ6V3IsTchPgFqdZaAYaZJZAL
x-amz-request-id: 29CF96BB304ECD8C
Date: Tue, 14 Jul 2015 09:11:31 GMT
Last-Modified: Tue, 17 Mar 2009 15:15:37 GMT
ETag: "4108ecce80045c0c38bbc77a3bc600e5"
Accept-Ranges: bytes
Content-Type: text/plain
Content-Length: 28
Server: AmazonS3

aws sanity-check succeeded!
ec2(Action, DescribeTags)
data = []
https://ec2.eu-central-1.amazonaws.com/?AWSAccessKeyId=ASIAJ334H7HOLAZGIHQA&Action=DescribeTags&Expires=2015-07-14T09%253A12%253A01Z&SecurityToken=AQoDYXdzENL%252F%252F%252F%252F%252F%252F%252F%252F%252F%252FwEa
4AMni%252BntH7r0r9TLrQg5hKTS4c2eUf0Pf%252BaPQg3u5o9KHd4C3iueXssGrpRiG%252F%252BF%252FvQR37cMa2Kkd929d6wFgVgB6v9Uno4nJk2%252BNkkoUCbyuzcmu7jO2lHZg%252BDDXK1znsKHh9QGEnJpbpI%252Fm3cZEXgHpJj%252BmI9D3gGJVbuVYvtpThO
30evfDRvEK6eMY9oAa4fvgxnPTZfAz5t9mBO8aepd0ER8dteu8ChC6tWqfU3hf7XTm0zdofMvsj2yqyxWx9P%252BxbcH%252Fdz47ddCPtPcUKadereOHGjZQVgyKkLLLFZ855DvXwemW%252BYAVdCKZO9F%252F%252BKVXBQASDBbr8rkU38s3ozuSNZDk4%252FEoX7SO8%252
FcMf4lWbWArC%252BL%252BH1aWnFxahpEiV16OMCshWQyQe1IF9JGqx5OEJCPnA7nievfNlN9IM%252FofLBX8Sz4LV3U0zV5hvIRxVeUEWnBBPQGL65Iqwpj0KcxrNGNb2W1lmGDLBZhXEkm1f0zXCFuq9amkT8uaOwj28iGoky7uAepM8jXyRr7%252FW8XsiYi0PhRULIJI%252
BUKDeQZAbuNHjZnARzobYNhLmyi07E%252BbFOR5VGp%252FWHgjCpJw5eBpgGhGDaUaGzHIxI8s0rBMkDLhnjv1G1qm3keKPe%252FJH0gkqKTrQU%253D&SignatureMethod=HmacSHA1&SignatureVersion=4&Version=2013-10-15&X-Amz-Algorithm=AWS4-HMAC-SH
A256&X-Amz-Credential=ASIAJ334H7HOLAZGIHQA%2F20150714%2Feu-central-1%2Fec2%2Faws4_request&X-Amz-Date=20150714T091130Z&X-Amz-Expires=30&X-Amz-SignedHeaders=host&X-Amz-Signature=d6440df17a61954a4f670190d5909e7d4bb
faab5073dcd9434b058b1866b396f
cmd=[curl  -q -g -S --remote-time --retry 3 --verbose -s  'https://ec2.eu-central-1.amazonaws.com/?AWSAccessKeyId=ASIAJ334H7HOLAZGIHQA&Action=DescribeTags&Expires=2015-07-14T09%253A12%253A01Z&SecurityToken=AQoDY
XdzENL%252F%252F%252F%252F%252F%252F%252F%252F%252F%252FwEa4AMni%252BntH7r0r9TLrQg5hKTS4c2eUf0Pf%252BaPQg3u5o9KHd4C3iueXssGrpRiG%252F%252BF%252FvQR37cMa2Kkd929d6wFgVgB6v9Uno4nJk2%252BNkkoUCbyuzcmu7jO2lHZg%252BDD
XK1znsKHh9QGEnJpbpI%252Fm3cZEXgHpJj%252BmI9D3gGJVbuVYvtpThO30evfDRvEK6eMY9oAa4fvgxnPTZfAz5t9mBO8aepd0ER8dteu8ChC6tWqfU3hf7XTm0zdofMvsj2yqyxWx9P%252BxbcH%252Fdz47ddCPtPcUKadereOHGjZQVgyKkLLLFZ855DvXwemW%252BYAVdC
KZO9F%252F%252BKVXBQASDBbr8rkU38s3ozuSNZDk4%252FEoX7SO8%252FcMf4lWbWArC%252BL%252BH1aWnFxahpEiV16OMCshWQyQe1IF9JGqx5OEJCPnA7nievfNlN9IM%252FofLBX8Sz4LV3U0zV5hvIRxVeUEWnBBPQGL65Iqwpj0KcxrNGNb2W1lmGDLBZhXEkm1f0zXC
Fuq9amkT8uaOwj28iGoky7uAepM8jXyRr7%252FW8XsiYi0PhRULIJI%252BUKDeQZAbuNHjZnARzobYNhLmyi07E%252BbFOR5VGp%252FWHgjCpJw5eBpgGhGDaUaGzHIxI8s0rBMkDLhnjv1G1qm3keKPe%252FJH0gkqKTrQU%253D&SignatureMethod=HmacSHA1&Signatu
reVersion=4&Version=2013-10-15&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=ASIAJ334H7HOLAZGIHQA%2F20150714%2Feu-central-1%2Fec2%2Faws4_request&X-Amz-Date=20150714T091130Z&X-Amz-Expires=30&X-Amz-SignedHeade
rs=host&X-Amz-Signature=d6440df17a61954a4f670190d5909e7d4bbfaab5073dcd9434b058b1866b396f']
* Hostname was NOT found in DNS cache
*   Trying 54.239.54.28...
* Connected to ec2.eu-central-1.amazonaws.com (54.239.54.28) port 443 (#0)
* successfully set certificate verify locations:
*   CAfile: none
  CApath: /etc/ssl/certs
* SSLv3, TLS handshake, Client hello (1):
} [data not shown]
* SSLv3, TLS handshake, Server hello (2):
{ [data not shown]
* SSLv3, TLS handshake, CERT (11):
{ [data not shown]
* SSLv3, TLS handshake, Server finished (14):
{ [data not shown]
* SSLv3, TLS handshake, Client key exchange (16):
} [data not shown]
* SSLv3, TLS change cipher, Client hello (1):
} [data not shown]
* SSLv3, TLS handshake, Finished (20):
} [data not shown]
* SSLv3, TLS change cipher, Client hello (1):
{ [data not shown]
* SSLv3, TLS handshake, Finished (20):
{ [data not shown]
* SSL connection using AES128-SHA
* Server certificate:
*        subject: C=US; ST=Washington; L=Seattle; O=Amazon.com, Inc.; CN=ec2.eu-central-1.amazonaws.com
*        start date: 2015-06-18 00:00:00 GMT
*        expire date: 2015-10-18 23:59:59 GMT
*        subjectAltName: ec2.eu-central-1.amazonaws.com matched
*        issuer: C=US; O=VeriSign, Inc.; OU=VeriSign Trust Network; OU=Terms of use at https://www.verisign.com/rpa (c)10; CN=VeriSign Class 3 Secure Server CA - G3
*        SSL certificate verify ok.
> GET /?AWSAccessKeyId=ASIAJ334H7HOLAZGIHQA&Action=DescribeTags&Expires=2015-07-14T09%253A12%253A01Z&SecurityToken=AQoDYXdzENL%252F%252F%252F%252F%252F%252F%252F%252F%252F%252FwEa4AMni%252BntH7r0r9TLrQg5hKTS4c2eUf0Pf%252BaPQg3u5o9KHd4C3iueXssGrpRiG%252F%252BF%252FvQR37cMa2Kkd929d6wFgVgB6v9Uno4nJk2%252BNkkoUCbyuzcmu7jO2lHZg%252BDDXK1znsKHh9QGEnJpbpI%252Fm3cZEXgHpJj%252BmI9D3gGJVbuVYvtpThO30evfDRvEK6eMY9oAa4fvgxnPTZfAz5t9mBO8aepd0ER8dteu8ChC6tWqfU3hf7XTm0zdofMvsj2yqyxWx9P%252BxbcH%252Fdz47ddCPtPcUKadereOHGjZQVgyKkLLLFZ855DvXwemW%252BYAVdCKZO9F%252F%252BKVXBQASDBbr8rkU38s3ozuSNZDk4%252FEoX7SO8%252FcMf4lWbWArC%252BL%252BH1aWnFxahpEiV16OMCshWQyQe1IF9JGqx5OEJCPnA7nievfNlN9IM%252FofLBX8Sz4LV3U0zV5hvIRxVeUEWnBBPQGL65Iqwpj0KcxrNGNb2W1lmGDLBZhXEkm1f0zXCFuq9amkT8uaOwj28iGoky7uAepM8jXyRr7%252FW8XsiYi0PhRULIJI%252BUKDeQZAbuNHjZnARzobYNhLmyi07E%252BbFOR5VGp%252FWHgjCpJw5eBpgGhGDaUaGzHIxI8s0rBMkDLhnjv1G1qm3keKPe%252FJH0gkqKTrQU%253D&SignatureMethod=HmacSHA1&SignatureVersion=4&Version=2013-10-15&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=ASIAJ334H7HOLAZGIHQA%2F20150714%2Feu-central-1%2Fec2%2Faws4_request&X-Amz-Date=20150714T091130Z&X-Amz-Expires=30&X-Amz-SignedHeaders=host&X-Amz-Signature=d6440df17a61954a4f670190d5909e7d4bbfaab5073dcd9434b058b1866b396f HTTP/1.1
> User-Agent: curl/7.35.0
> Host: ec2.eu-central-1.amazonaws.com
> Accept: */*
>
< HTTP/1.1 401 Unauthorized
< Transfer-Encoding: chunked
< Date: Tue, 14 Jul 2015 09:11:30 GMT
* Server AmazonEC2 is not blacklisted
< Server: AmazonEC2
<
{ [data not shown]
* Connection #0 to host ec2.eu-central-1.amazonaws.com left intact
+-------------+--------------------------------------------------------------+
|    Code     |                           Message                            |
+-------------+--------------------------------------------------------------+
| AuthFailure | AWS was not able to validate the provided access credentials |
+-------------+--------------------------------------------------------------+

[mattheworiordan]

@timkay Note I am running this command from an instance in eu-central-1 region.

[timkay]

I modified my ~.awssecret file to contain an invalid key, and I got

aws describe-tags --region eu-central-1 --AWS4

+-------------+--------------------------------------------------------------+
| Code | Message
|
+-------------+--------------------------------------------------------------+
| AuthFailure | AWS was not able to validate the provided access
credentials |
+-------------+--------------------------------------------------------------+

You do need new keys for each new region. Please take a look at the
possibility that your keys are no good.

...Tim

On Tue, Jul 14, 2015 at 2:13 AM, mattheworiordan notifications@github.com
wrote:

@timkay https://github.com/timkay Note I am running this command from
an instance in eu-central-1 region.

—
Reply to this email directly or view it on GitHub
#97 (comment).

Tim Kay
m: +1-650-248-0123
Skype: timkay

[mattheworiordan]

I am only using IAM authentication though, and the command works with the Amazon aws CLI tool.

[timkay]

Yes, but the credentials are stored in a different place. Make sure you
have valid credentials in ~/.awssecret
On Jul 16, 2015 5:30 AM, "mattheworiordan" notifications@github.com wrote:

I am only using IAM authentication though, and the command works with the
Amazon aws CLI tool.

—
Reply to this email directly or view it on GitHub
#97 (comment).

[mattheworiordan]

Sorry @timkay I am not following you. We never store any credentials on the instance themselves and rely entirely on IAM

[timurb]

Do you mean IAM role attached to the instance?

[mattheworiordan]

Yes, we use CloudFormation and IAM assigned to the instance

Sent from my phone

On 16 Jul 2015, at 15:36, Timur Batyrshin notifications@github.com wrote:

Do you mean IAM role attached to the instance?

—
Reply to this email directly or view it on GitHub.

[timkay]

I hadn't realized that you are using role-based authentication. In that case, you have to add --role. However, then we get a different error, which I am working on now.