2 handy AWS S3 public-read commands

I needed to host a simple public html file. S3 is easy.

To make a file publicly readable in S3 is easy without having to give up whole bucket access. This method also uses the newer ONE_IA storage option to save a few cents.

It uses the S3 and S3api commands from awscli for this.

Update the FILE and BUCKET variables to make it yours. Tested on Ubuntu.

FILE=privacy_policy.html
BUCKET=myAwsomeBucketName 
aws s3 cp $FILE s3://$BUCKET/ --storage-class ONE_IA 
aws s3api put-object-acl --bucket $BUCKET --key $FILE --acl public-read

HTH

Advertisements

Streamlined Security for AWS Well-Architected solutions using Inspector & Guard Duty

Improving security is always a good idea. Some people still think only about static security, such as Security Groups, SSL or VPNs.

Consideration should also be given to the application vulnerability .  Adding a regular scanning process adds another layer of defence. Remember; Test is best.

AWS Inspector

AWS Inspector  has been around since October 2015 so its not new, but its implementation and use make it easy to get started – even if you haven’t tried it yet.

It uses an awsagent installed inside the AMI to allow the Inspector service to connect and perform the scan. Results are rated as High, Medium, Low and Informational severity, via the Inspector console and Findings tab.

Benefits for each Well-Architected pillar are;

  • Improve Security, easy and early detection of security issues for your applications, a streamlined compliance approach and an overall better security position
  • Gain Performance Efficiency, integrate with DevOps through your CI/CD pipeline and increase your development agility
  • Ensure Reliability, as Inspector can be run it as often as needed, in parallel (but only 1 concurrent scan per agent), with repeatable and measurable results
  • Improve Operational Excellence, by using a suitable managed service, your operating burden is a reduced and by leveraging expertise of security experts with preconfigured tests. You also save time by not having to submit a request to AWS Support for approval to vulnerability scan whic is needed as part of the Acceptable Use Policy for Security.
  • Cost Optimisation, its cost is certainly justified in most cases compared with a roll-your-own solution

A Use Case study – Hardening a custom AMI to verify CIS Benchmark compliance

  • Build your own custom AMI
  • Patch and secure it using CIS Benchmark best practices.
  • Tag instances (I used InspectorReady:yes)
  • Download and install inspector agent with these commands
# install Inspector Agent software
wget https://d1wk0tztpsntt1.cloudfront.net/linux/latest/install 
sudo bash install 
sudo /opt/aws/awsagent/bin/awsagent status    # check agent status
  • Go to Inspector console, create a IAM Role for Inspector to use (if not already done)
  • Create/Select targets for assessment based on the tag you created
  • Create Assessment Template, adding multiple Rules Packages (will take longer to scan the more you add)  
    • CIS Operating System Security Configuration Benchmarks-1.0
    • Security Best Practices-1.0
  • Run Inspector Assessment on Assessment Targets
  • Review Findings, they can be downloaded as CSV as well
  • Remediate High risk issues as a priority using the Findings as a task list.

Using the Common Vulnerability and Exposure 1.1 rules package, a scan took around an hour to complete, using a (poor choice!) t2.micro instance.  A faster instance type (M5 or C5) would yield quicker results.

Inspector Findings, on a new Bitnami WordPress install showed (52 overall) 30 High risk issues.

inspectorFindings

Pricing is reasonable. 25 runs x $0.30 = $7.50. You also get the first 250 free in the initial 90 days as part of the AWS Free Tier service. So it’s basically free to give it a try and then start to incorporate it into your CI/CD pipeline.

AWS Guard Duty

Further compliment your good work with Inspector by enabling AWS Guard Duty for all your accounts. Unlike Inspector which checks for threats within the AMI or at the OS level via an agent, Guard Duty does the same for your AWS account activity – continuously without agents.  Within 30 minutes of enabling Guard Duty (and launching a new test instance with unrestricted ports), I checked the Findings.  It showed these results; GuardDuty Results

Guard Duty has associated costs, based on number of CloudTrail events and GB/logs assessed.  Once you sign up you get 30 days free.  Via the Guard Duty console, you can check you Free Tier usage progress as well as a estimate of monthly costs you could expect after your free tier offer expires.  Helps you to make informed cost decisions.

Another good idea is to subscribe to a Vulnerability Notification list for your applications to ensure you are staying up to date with potential issues. It’s good practice to patch security often.

As a champion for a Well-Architected system, these tools ticks all the boxes for the 5 pillars, not just security.

HTH

cloud9 IDE http://c9.io

If you need a Lnux Virtual Desktop for developers, then check out http://c9.io

Within 5 minutes, I had registered, logged in, view the sample code, deployed it and an Apache web server and run the code. Super Easy!!

Some of the constraints of other solutions may be;

* constraints
VMWare View 6 (v1.0 linux desktops)
Hardware purchases
Building and Managing infrastructure
Short cycle project work

Easy way to rotate AWS access keys

We all know we should change passwords often, well same goes for access keys.

This handy INTERACTIVE bash script walks you through to create a new AWS Access Key, save the .pem file in your .ssh directory.  And gives you the option to delete the old keys.

You can download from my gitlab – here

Hope this helps someone 🙂

#!/bin/bash
# @shallawell
# Program Name: aws-iam-access-keys.sh
# Purpose: Manage AWS access keys
# version 0.1

# new key content will be created in this file.
FILE=new_aws_key.txt
#remove the old file first
rm $FILE
### create a key
echo -n "Do you want to create a new Access/Secret key. (y/n) [ENTER]: "
#get user input
read response2
if [ "$response2" == "y" ]; then
echo "Ok.. Creating a new keys !!!"
aws iam create-access-key --output json | grep Access | tail -2 | tee -a $FILE
#Alternative create key command
#KEY=myIndiaAWSKeytest
#REGION=ap-south-1
#aws ec2 create-key-pair --key-name=$KEY --region $REGION --query="KeyMaterial" --output=text > ~/.ssh/$KEY.pem
#readonly the key
#chmod 400 ~/.ssh/$KEY.pem
echo "key created."
echo "REMEMBER: You should rotate keys at least once a year! Max of 2 keys per user."
echo "$FILE created for Access and Secret Keys"
echo "HINT: Run aws configure to update keys. (you just rotated your keys!)"
else [ "$response2" == "n" ]
echo "Not creating keys."
exit 0
fi

### list a key, save to DELKEY var
#this command LISTS the access keys for current user, sorts by CreateDate,
#gets the latest AccessKeyId result. awk grabs the Access key (excludes date field)
DELKEY=$(aws iam list-access-keys \
--query 'AccessKeyMetadata[].[AccessKeyId,CreateDate]' \
| sort -r -k 2 | tail -1 | awk {'print $1'})

echo "list-Access-key sorted to find OLDEST key."
echo -n "Key Found : $DELKEY. Do you want to delete this key. (y/n) [ENTER]: "
#get user input
read response
if [ "$response" == "y" ]; then
echo "you said yes. Deleteing Key in 3 secs!!!"
sleep 3
echo "delete-access-key disabled, NO REAL DELETE OCCURRED"
### delete a key, uncomment to activate the delete function.
#aws iam delete-access-key --access-key-id $DELKEY
echo "deleted $DELKEY"
else [ "$response" == "n" ]
echo "you said no. Not Deleting"
fi

echo "done."

Nice list of Aliases for .bashrc

$cat .bashrc
# .bashrc

# Source global definitions
if [ -f /etc/bashrc ]; then
 . /etc/bashrc
fi

# Uncomment the following line if you don't like systemctl's auto-paging feature:
# export SYSTEMD_PAGER=

# User specific aliases and functions
alias cp='cp -iv' # Preferred 'cp' implementation
alias mv='mv -iv' # Preferred 'mv' implementation
alias mkdir='mkdir -pv' # Preferred 'mkdir' implementation
alias ll='ls -FGlAhp' # Preferred 'ls' implementation
alias la='ll -FGlAhpa' # Preferred 'ls -a' implementation
alias less='less -FSRXc' # Preferred 'less' implementation
cd() { builtin cd "$@"; ll; } # Always list directory contents upon 'cd'
alias cd..='cd ../' # Go back 1 directory level (for fast typers)
alias ..='cd ../' # Go back 1 directory level
alias ...='cd ../../' # Go back 2 directory levels
alias .3='cd ../../../' # Go back 3 directory levels
alias .4='cd ../../../../' # Go back 4 directory levels
alias .5='cd ../../../../../' # Go back 5 directory levels
alias .6='cd ../../../../../../' # Go back 6 directory levels
alias findall='find / -name' # find on the whole filesystem
alias sudo="sudo " # A trailing space in value causes the next word to be checked for alias substitution when the alias is expanded

# lr: Full Recursive Directory Listing
# ------------------------------------------
alias lr='ls -R | grep ":$" | sed -e '\''s/:$//'\'' -e '\''s/[^-][^\/]*\//--/g'\'' -e '\''s/^/ /'\'' -e '\''s/-/|/'\'' | less'

# showa: to remind yourself of an alias (given some part of it)
# ------------------------------------------------------------
 showa () { /usr/bin/grep --color=always -i -a1 $@ ~/Library/init/bash/aliases.bash | grep -v '^\s*$' | less -FSRXc ; }

alias mypubip='curl -s ipinfo.io/ip' # mypubip: Public facing IP Address

alias openPorts='ss -t -a -l' # openPorts: List Listening TCP ports

# ii: display useful host related informaton
# -------------------------------------------------------------------
 ii() {
 echo -e "\nYou are logged on ${RED}$HOST"
 echo -e "\nAdditionnal information:$NC " ; uname -a
 echo -e "\n${RED}Users logged on:$NC " ; w -h
 echo -e "\n${RED}Current date :$NC " ; date
 echo -e "\n${RED}Machine stats :$NC " ; uptime
 echo -e "\n${RED}Public facing IP Address :$NC " ;mypubip
 echo
 }

# httpDebug: Download a web page and show info on what took time
# -------------------------------------------------------------------
 httpDebug () { /usr/bin/curl $@ -o /dev/null -w "dns: %{time_namelookup} connect: %{time_connect} pretransfer: %{time_pretransfer} starttransfer: %{time_starttransfer} total: %{time_total}\n" ; }