About aws-lab

Terraform AWS free tier, EC2/ECR/RDS/EFS/DynamoDB/Lambda/S3. Docker running on EC2, Traefik reverse proxy, Lets Encrypt, dynamic DNS, ZeroTier VPN(TODO)

r

Published by

rogeriomm

Visit View Profile

README.md

View on GitHub

Terraform AWS Free Tier

Getting started with the Terraform for managing a base free-tier AWS resources.

Project description

This is a Terraform project for managing AWS resources.

It can build the next infrastructure:

VPC
Public Subnet in the VPC
IGW to enable access to or from the Internet for VPC
Route Table to associate IGW, VPC and Subnet
RDS in the private Subnet
EC2 in the public Subnet with the HTTPS Lets Encrypt certificate & SSH access with free dynamic DNS provider
- Docker running on EC2: docker-compose.yaml
  - Free dynamic DNS host on AWS
  - Web application sample
    - My personal URL: https://aws-dashboard.duckdns.org/whoami
  - Traefik Reverse proxy, Lets Encrypt, Traefik Lets Encrypt
    - My personal URL: https://aws-dashboard.duckdns.org/traefik
S3
Lambda
- Samples
EFS
- Mount NFS filesystem on AWS EC2
DynamoDB
ECR
GLUE

Architecture Diagram

Install

Pre steps

Install software

Install infrastructure tools
```
brew install terraform awscli
```
Install general development tools
```
brew install make yq zip curl gradlle
```
Install PYTHON
```
brew install pyenv
```
Install GO
```
brew install golang
```
Install JAVA
```
brew install java11 jenv
```

Install RUST: https://rustup.rs

curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
rustup target add x86_64-unknown-linux-gnu
brew tap cargo-lambda/cargo-lambda
brew install cargo-lambda

Create AWS account
If the file ~/.aws/credentials doesn't exist, create it and add you Terraform profile to the file. For example:
```
[terraform]
aws_access_key_id = Your access key
aws_secret_access_key = Your secret access key 
```
Check AWS account
```
aws sts get-caller-identity
```

Create S3 bucket to store Terraform state

aws s3api create-bucket --bucket world-terraform --region us-east-1

Create config file config.tf that will contain information how to store state in a given bucket. See example.
Create SSH key pair to connect to EC2 instance:
```
cd ./src/free-tier/provision/access
```

it creates "free-tier-ec2-key" private key and "free-tier-ec2-key.pub" public key

ssh-keygen -f free-tier-ec2-key


9. Adds SSH private key identities to the authentication agent
```shell
ssh-add src/free-tier/provision/access/free-tier-ec2-key

Build infrastructure

Install Python

pyenv install 3.8.16
cd ./src/free-tier/python
pyenv local 3.8.16

Install Java
```
jenv enable-plugin gradle
```

Install RUST lambda

brew tap cargo-lambda/cargo-lambda
brew install cargo-lambda

Build lambda samples

cd ./src/free-tier/lambda/samples
make all

cd ./src/free-tier
terraform init -backend-config="./backend/config.tf"

cd ./src/free-tier
terraform plan

cd ./src/free-tier
terraform apply

Post install

ip=$(aws ec2 describe-instances | 
      yq 'select(.Reservations[].Instances[].State.Code == 16) | .Reservations[].Instances[].NetworkInterfaces[].PrivateIpAddresses[].Association.PublicIp')
echo $ip

Edit /etc/hosts add "aws" host name

sudo bash -c "echo $ip aws >> /etc/hosts"

Add SSH public key from EC2 server

ssh-keygen -R aws
ssh-keyscan -H aws >> ~/.ssh/known_hosts

Install EPEL, Postgres 14 packages

ssh ec2-user@aws "sudo amazon-linux-extras install epel postgresql14 -y"

Upgrade Linux and install packages

ssh ec2-user@aws "sudo yum update && sudo yum upgrade -y && sudo yum install -y netcat openvpn postgresql docker python3-pip htop"

Install Docker compose, configuration

ssh ec2-user@aws "sudo usermod -a -G docker ec2-user && sudo pip3 install docker-compose"
ssh ec2-user@aws "sudo systemctl enable docker.service && sudo systemctl start docker.service && systemctl status docker.service"

Edit src/docker/env-duckdns.sh

SUBDOMAINS=your-subdomain
DUCKDNS_TOKEN=your-token
TOKEN=$DUCKDNS_TOKEN

Edit src/docker/docker-compose.yaml, set email and Duckdns subdomain TODO
Edit src/docker/conf/users.txt TODO

Copy Docker files

ssh ec2-user@aws "mkdir -p docker/conf"
scp src/docker/docker-compose.yaml src/docker/env-duckdns.sh ec2-user@aws:./docker/
scp src/docker/conf/dynamic_conf.yml ec2-user@aws:./docker/conf/dynamic_conf.yml
scp src/docker/conf/users.txt ec2-user@aws:./docker/conf/users.txt

Start Docker containers

ssh ec2-user@aws "cd docker && docker-compose up -d"

Services

Duckdns

https://www.duckdns.org/

./src/docker/env-duckdns.sh

SUBDOMAINS=sub-domain-1,sub-domain-2,sub-domain-3,sub-domain-4,sub-domain-5
DUCKDNS_TOKEN=your-token
TOKEN=your-token

Duckdns logs

ssh ec2-user@aws "cd docker && docker-compose logs duckdns"

Traefik

Traefik logs

ssh ec2-user@aws "cd docker && docker-compose logs traefik"

AWS RDS

aws rds describe-db-instances | yq

Get Postgres endpoint

address=$(aws rds describe-db-instances | yq '.DBInstances[] | select(.DBName=="labdb") | .Endpoint.Address')
port=$(aws rds describe-db-instances | yq '.DBInstances[] | select(.DBName=="labdb") | .Endpoint.Port')
echo $address:$port

Check Postgres routing from EC2
```
ssh ec2-user@aws "nc -v $address $port"
```

Postgres cli

ssh ec2-user@aws psql --host $address --port $port --username postgres

AWS DynamoDB

aws dynamodb list-tables | yq

AWS EFS

aws efs describe-file-systems | yq

aws efs describe-access-points | yq

filesystem_id=$(aws efs describe-file-systems | yq '.FileSystems[] | select(.Name=="free-tier-efs") | .FileSystemId')
echo $filesystem_id

aws efs describe-mount-targets --file-system-id $filesystem_id | yq

Get NFS IP address

nfs_ip=$(aws efs describe-mount-targets --file-system-id $filesystem_id | yq '.MountTargets[] | select (.AvailabilityZoneName=="us-east-1a") | .IpAddress')
echo $nfs_ip

Check NFS routing from EC2
```
ssh ec2-user@aws "nc -v $nfs_ip 2049"
```

AWS API Gateway

aws apigatewayv2 get-apis | yq

endpoint=$(aws apigatewayv2 get-apis | yq '.Items[] | select (.Name=="free-tier-api-gateway") | .ApiEndpoint')
echo $endpoint

Run lambda Python
```
curl "$endpoint/api/python"
```

Zero Tier VPN (TODO)

https://www.zerotier.com/

Destroy infrastructure

cd ./src/free-tier
terraform destroy

aws-lab