Introduction

Each of the AWS tools has slightly different expectations about the location and naming of the credentials file and the various properties within it. It seems like the Python tools are moving closer to the Java standard as they iterate through releases, but it is still necessary to use a patchwork solution to be able to have a unified credentials file.

Java SDK

Version: “com.amazonaws” % “aws-java-sdk” % “1.8.2”
Installation: sbt
Link: AWS Java SDK Class ProfilesConfigFile


The standard location for the credentials file is ~/.aws/credentials, which can be overridden with the AWS_CREDENTIAL_PROFILES_FILE environment variable or by specifying an alternate file location in the constructor. The format of this file is described below:

[default]
 aws_access_key_id=testAccessKey
 aws_secret_access_key=testSecretKey
 aws_session_token=testSessionToken

 [test-user]
 aws_access_key_id=testAccessKey
 aws_secret_access_key=testSecretKey
 aws_session_token=testSessionToken

Java Command Line Tools (e.g. ec2-api-tools)

Version: 1.6.13.0
Installation: brew install ec2-api-tools
Link: Setting Up the Amazon EC2 Command Line Interface Tools on Linux/Unix and Mac OS X


The standard configuration is to use environment variables, since these tools have not been updated to read the standard AWS credentials file. Add the following to your ~/.bash_profile, to link the required data to the standard credentials file and allow for session tokens:

export AWS_CREDENTIAL_FILE="$HOME/.aws/credentials"
export AWS_ACCESS_KEY="$(grep aws_access_key_id $AWS_CREDENTIAL_FILE |cut -d= -f2)"
export AWS_SECRET_KEY="$(grep aws_secret_access_key $AWS_CREDENTIAL_FILE |cut -d= -f2)"
export AWS_DELEGATION_TOKEN="$(grep aws_session_token $AWS_CREDENTIAL_FILE |cut -d= -f2)"

Python Boto

Version: boto==2.31.1
botocore==0.56.0
Installation: pip install boto
Link: Boto Config


The latest version of boto needs to have aws_security_token defined, rather than aws_session_token, in the credentials file. The simplest solution for this is to duplicate the token between both names; the Java SDK will throw the following log message when reading the extra property, but will work as expected: INFO: Skip unsupported property name aws_security_token in profile [default]. Boto will not throw log messages about the existence of the aws_session_token property.

AWS CLI

Version: aws-cli/1.3.22
Installation: pip install awscli
Link: Configuring the AWS Command Line Interface


The standard location for the credentials file is ~/.aws/config, which can be overridden with the AWS_CREDENTIAL_FILE environment variable. The latest version of this tool accepts the Java SDK credential file format as-is, including the use of aws_session_token, whereas previous versions wanted aws_security_token instead. When you have multiple profiles in the credentials file, you can select a profile with the tool like so:

aws --profile test-user s3 ls

Unified Solution

The best approach for creating a unified credentials file is to follow the Java credentials file format as closely as possible, while redirecting the Python tools to that file and adding properties to cover the corner cases.

To do this, create a ~/.aws/credentials file that duplicates the necessary properties:

[default]
 aws_access_key_id=testAccessKey
 aws_secret_access_key=testSecretKey
 aws_session_token=testSessionToken
 aws_security_token=testSessionToken

 [test-user]
 aws_access_key_id=testAccessKey
 aws_secret_access_key=testSecretKey
 aws_session_token=testSessionToken
 aws_security_token=testSessionToken

 [prod-user]
 aws_access_key_id=testAccessKey
 aws_secret_access_key=testSecretKey
 aws_session_token=testSessionToken
 aws_security_token=testSessionToken

And add a section to your ~/.bash_profile:

export AWS_CREDENTIAL_FILE="$HOME/.aws/credentials"
export AWS_ACCESS_KEY="$(grep aws_access_key_id $AWS_CREDENTIAL_FILE |cut -d= -f2)"
export AWS_SECRET_KEY="$(grep aws_secret_access_key $AWS_CREDENTIAL_FILE |cut -d= -f2)"
export AWS_DELEGATION_TOKEN="$(grep aws_session_token $AWS_CREDENTIAL_FILE |cut -d= -f2)"

With this configuration, you should be able to move seamlessly between the various Java and Python tools available for AWS.



blog comments powered by Disqus

Published

14 July 2014

Category

aws

Tags