January 27, 2014 / by Christian Albrecht / sysadm, dba, bba, smo / @cal_040

bling bling shiny new awscli

We run several systems in the cloud. Our provider of choice is Amazon. To interact with the services (ec2, s3, route53, etc) one could use either the web based console or the api provided by Amazon. As you could guess the power comes with the api. I will describe the basic setup of the awscli tool and show a little use case for it.

Installation

The awscli is written in Python. We have to make sure that at least Python 2.6.3 is installed on our mac. By the love of god, Mountain Lion and Maverick ship Python 2.7 and we don’t have to struggle with Python.

UPDATE: By now you can install awscli with homebrew! Just run

$ brew install awscli

You can skip the cheese shop part now!

$ python --version
Python 2.7.6

We choose the cheese shop to install awscli.

$ sudo pip install awscli

No big deal. To finish the installation we have to provide our AWS credentials. The configuration file is stored in our home directory ($HOME/.aws/config).

$ aws configure
AWS Access Key ID [None]: <your access key id>
AWS Secret Access Key [None]: <your secret access key>
Default region name [None]: eu-west-1
Default output format [None]: json

Another possible way is to export the following environment variables

$ export AWS_DEFAULT_REGION="eu-west-1"
$ export AWS_DEFAULT_OUTPUT="json"
$ export AWS_ACCESS_KEY_ID="<your id here>"
$ export AWS_SECRET_ACCESS_KEY="<your id here>"

The next command enables tab autocompletion.

$ complete -C aws_completer aws  

To persist this setting, put this command into your $SHELL profile file.

Congratulations! We have setup the aws cli.

Real world example

Problem: You have a hugh amount of unneeded ebs snapshots in your aws account and don’t want to delete them manually on the web based console.

Solution: Run this nifty oneliner. Be careful! This command may delete your data! Run it on your own risk!

$ aws ec2 describe-volumes | \
jq '.Volumes[] | select(.State == "available") | .VolumeId' | \
xargs -I % aws ec2 delete-volume --output text --volume-id %

aws ec2 describe-volumes reports all volumes as JSON output. jq is a lightwight and flexible command-line JSON processor (imagine a grep for JSON). We query the VolumeId out of the Volumes array in an available state. This list is passed to xargs which invokes aws ec2 delete-volume to delete all volumes one by one.