Introduction
We will cover How To Setup AWS EC2 Instance Using AWS CDK Python.
Did you know that AWS CDK gives you the flexibility to create AWS resources in your Python applications?
I will break this review in the following sections:
- Why setting up an AWS EC2 Instance using CDK is useful
- Walk through step by step on How To Setup AWS EC2 Instance Using AWS CDK Python
I have been using this code successfully on all my Python AWS CDK deployments to launch new EC2 Instances.
Going to keep things simple and straight to the point on how to get up and running in less than 5 minutes.
All source found in this guide can be found in the GitHub repo here.
AWS CDK Python vs AWS Terraform for Creating EC2 Instance
There are several reasons as to why you may want to use AWS CDK Python compared to Terraform to deploy your EC2 instances, I will list them below and see if one of them suits your needs or you find yourself still needing to do that.
- Terraform is more of a descriptive language so it doesn’t allow a lot of flexibility on adding conditional stuff
- AWS CDK Python can be incorporated into an existing coding project and blend in without calling external libraries
- You can deploy and destroy a lot of EC2 instances fairly fast using loops (the terraform looping constructs are less intuitive than Python code)
- You are from a programming background rather than DevOps so programming makes more sense to you
- You can make libraries that are re-useable across an existing Python project or inherit from others
If any of the above reasons is you then keep reading and I’ll help you get started and going super fast.
How To Setup AWS CDK Python Environment
I have written a detailed step by step guide here which you can follow and get up and running in less than a few minutes. Please follow this before proceeding into writing code. If you already have AWS CDK setup in your system there’s no need to do this.
How To Setup AWS CDK With Python
How To Create/Destroy AWS EC2 Instances Using AWS CDK Python
Now that we have the environment all setup we will do the following:
- Create a new project
- Initialize our Environment file
- Write the Code that Sets up our Stack
Create New EC2 Instance using AWS CDK Python
If followed the previous guide and want to re-use the project you have created that’s fine. We will be doing a slight twist to this here and basically initializing it with a different name since it’s going to be dedicated for an AWS S3 creation using AWS CDK Python.
$ cdk init app --language python ... Enjoy! Please run 'python3 -m venv .venv'! Executing Creating virtualenv... ✅ All done!
Setting Up AWS Environment Credentials For CDK
In order to do this I have made a detailed guide which you can find here:
Boto3 Session: Setup Profile, Create, Close and Mock sessions
You can follow the steps listed above to add a new iam user and configure your .env file accordingly.
The permission you want to use is the following in this case:
Other than the change above you can follow the guide as is to initialize the AWS SDK permissions. Do note you may not need the EC2 and Cloudformation permission.
Once this is done you also need to setup your credentials sections in your config, in this example I will be using a profile named test_deploy_ec2.
The typical values such as the secret key and access keys need to be entered. Mine looks something like this:
$ cat credentials [test_deploy_ec2] aws_access_key_id=YOUR ID aws_secret_access_key=YOUR KEY aws_default_region=us-east-1 $ cat config [test_deploy_ec] region=us-east-1
Modifying The AWS CDK Environment File For Python
Besides that we need to add two more attributes to our AWS CDK environment file and those are related to the CDK.
More specifically this would look like this:
CDK_DEFAULT_ACCOUNT=YOURACCOUNT CDK_DEFAULT_REGION=us-east-1 VPC_NAME=VPCNAME
Make sure to replace the CDK_DEFAULT_ACCOUNT and CDK_DEFAULT region accordingly with your information which you can retrieve from the AWS console.
One note here you need to find your VPC from the AWS Console. If you navigate to your VPCs section under EC2 you should be able to see the ID of an existing VPC. If you don’t have one simply create one from there. This can be seen in the screenshot below.
In my case the vpc is shown in blue you can go ahead and fill it in the env file, this can also be found in the Github repo.
After that don’t forget to initialize your virtual environment and update your requirements.txt file as it’s shown in the Github repo here.
$ pip install -r requirements.txt Collecting aws-cdk-lib==2.15.0 Using cached aws_cdk_lib-2.15.0-py3-none-any.whl (64.0 MB) Collecting constructs<11.0.0,>=10.0.0 Using cached constructs-10.0.84-py3-none-any.whl (54 kB) Collecting python-dotenv Using cached python_dotenv-0.19.2-py2.py3-none-any.whl (17 kB) Collecting jsii<2.0.0,>=1.54.0 Using cached jsii-1.55.0-py3-none-any.whl (383 kB) Collecting publication>=0.0.3 Using cached publication-0.0.3-py2.py3-none-any.whl (7.7 kB) Collecting attrs~=21.2 Using cached attrs-21.4.0-py2.py3-none-any.whl (60 kB) Collecting python-dateutil Using cached python_dateutil-2.8.2-py2.py3-none-any.whl (247 kB) Collecting cattrs<1.11,>=1.8 Using cached cattrs-1.10.0-py3-none-any.whl (29 kB) CDK_DEFAULT_ACCOUNT= Collecting typing-extensions<5.0,>=3.7 Using cached typing_extensions-4.1.1-py3-none-any.whl (26 kB) Collecting six>=1.5 Using cached six-1.16.0-py2.py3-none-any.whl (11 kB) Installing collected packages: six, attrs, typing-extensions, python-dateutil, cattrs, publication, jsii, constructs, python-dotenv, aws-cdk-lib Successfully installed attrs-21.4.0 aws-cdk-lib-2.15.0 cattrs-1.10.0 constructs-10.0.84 jsii-1.55.0 publication-0.0.3 python-dateutil-2.8.2 python-dotenv-0.19.2 six-1.16.0 typing-extensions-4.1.1
One important part to note here is that we are adding in our requirements.txt file the python-dotenv library as this is not added by default and we need to use it to access the environment variables for AWS CDK.
How To Bootstrap and Deploy Your EC2 Instance Using AWS CDK Python
The final step is to basically run our code to:
- Bootstrap
- Deploy
How To Bootstrap Your AWS EC2 Instance formation Using AWS CDK Python
The first step as mentioned above is to Deploy your environment with code.
This can be done in three steps:
- Modify the app.py initialization file
- Run bootstrap command
How To Modify app.py To Initialize AWS CDK
Lets examine the code that implements this.
#!/usr/bin/env python3
import os
from dotenv import load_dotenv
# load our env file
print ('Loading env file')
load_dotenv()
import aws_cdk as cdk
from ec2_deploy.ec2_deploy_stack import Ec2DeployStack
print ('Creating environment')
cdk_env = cdk.Environment(account=os.getenv('CDK_DEFAULT_ACCOUNT'), region=os.getenv('CDK_DEFAULT_REGION'))
app = cdk.App()
app = cdk.App()
Ec2DeployStack(
app,
'Ec2DeployStack',
env=cdk_env
)
# synthesize it
print ('Synthesizing stack')
app.synth()
As noted we are using python-dotenv to load the environment variables we had created previously in this guide. This is to let the CDK know where we will be deploying our code. Also we will be naming in our case the stack EC2DeployStack.
How To Implement Code To Describe AWS EC2 Instance Using AWS Python CDK
The next step we need to do is to implement the code that will be deploying our AWS EC2 Instance formation.
import os
from aws_cdk import (
Stack,
aws_ec2
)
from constructs import Construct
class Ec2DeployStack(Stack):
def __init__(self, scope: Construct, construct_id: str, **kwargs) -> None:
super().__init__(scope, construct_id, **kwargs)
self.instance_name = 'unbiased_coder_instance'
self.instance_type = 't2.micro'
self.ami_name = 'ubuntu/images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20211129'
self.vpc_name = os.getenv('VPC_NAME')
print (f'Looking up AMI: {self.ami_name}')
ami_image = aws_ec2.MachineImage().lookup(name=self.ami_name)
if not ami_image:
print ('Failed finding AMI image')
return
print (f'Looking up instance type: {self.instance_type}')
instance_type = aws_ec2.InstanceType(self.instance_type)
if not instance_type:
print ('Failed finding instance')
return
print (f'Using VPC: {self.vpc_name}')
vpc = aws_ec2.Vpc.from_lookup(self, 'vpc', vpc_id=self.vpc_name)
if not vpc:
print ('Failed finding VPC')
return
print ('Creating security group')
sec_grp= aws_ec2.SecurityGroup(self, 'ec2-sec-grp', vpc=vpc, allow_all_outbound=True)
if not sec_grp:
print ('Failed finding security group')
return
print ('Creating inbound firewall rule')
sec_grp.add_ingress_rule(
peer=aws_ec2.Peer.ipv4('0.0.0.0/24'),
description='inbound SSH',
connection=aws_ec2.Port.tcp(22))
if not sec_grp:
print ('Failed creating security group')
return
print (f'Creating EC2 Instance: {self.instance_name} using {self.instance_type} with ami: {self.ami_name}')
ec2_inst = aws_ec2.Instance(
self, 'ec2_inst',
instance_name=self.instance_name,
instance_type=instance_type,
machine_image=ami_image,
vpc=vpc,
security_group=sec_grp)
if not ec2_inst:
print ('Failed creating ec2 instance')
return
There are a few high level steps we need to do in order to start running our EC2 formation code:
- Find a suitable AMI from the AWS AMI store. In the example below I’m using the latest at the time Ubuntu Server flavor but you can use whatever you like.
- Select the instance type we will be deploying, in this case we are just going with the free eligible tier t2.micro instance.
- Selecting and getting an object for our VPC. As mentioned earlier for this you will need the name of your VPC in order to call this function to identify and associate an object with that. It can be seen in the code above where the search takes place.
- Next we need a security group to associate with our EC2 instance. This is simply something we will be creating here and attaching to it port 22 as inbound and allowing all outbound data.
- The final and last step is to create the EC2 instance. It’s important to pass it in all the objects we previously identified and created in order to deploy it in the AWS cloud.
How To Bootstrap Your AWS EC2 Instance Using AWS CDK Python
The next step in the process is starting to bootstrap the AWS EC2 Instance using our AWS CDK Python code that we wrote earlier in this guide.
$ cdk bootstrap --profile test_deploy_ec2 Loading env file Creating environment Synthesizing stack ⏳ Bootstrapping environment aws://ACCOUNT/us-east-1... Trusted accounts for deployment: (none) Trusted accounts for lookup: (none) Using default execution policy of 'arn:aws:iam::aws:policy/AdministratorAccess'. Pass '--cloudformation-execution-policies' to customize. ✅ Environment aws://ACCOUNT/us-east-1 bootstrapped (no changes).
As it can be seen above the bootstrapping was successful. If you are following the code from the Github repo you should be good.
In order to verify this we can check the before and after of how the AWS console looked in the CloudFormation part.
As it can be seen the bootstrap command added a CDKToolkit to it which it will use to deploy our CDK code.
Also another thing the AWS Python CDK creates is an S3 bucket to store metadata and any necessary files needed for deploying. If you navigate in the AWS console into the S3 section you can see it has created a temporary bucket.
Now that we went over to what changes the bootstrapping did to our AWS account we can proceed with deploying our code into the infrastructure.
How To Deploy Your AWS EC2 Instance Using AWS CDK Python
The final step is to actually deploy our code into the AWS Stack. This is a lengthy process and syncs all the changes you made in your code to the AWS server infrastructure.
To do this run the cdk deploy command as shown below.
$ cdk deploy --profile test_deploy_ec2
Loading env file
Creating environment
Looking up AMI: ubuntu/images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20211129
Looking up instance type: t2.micro
Using VPC: vpc-af9c5bd2
Creating security group
Creating inbound firewall rule
Creating EC2 Instance: unbiased_coder_instance using t2.micro with ami: ubuntu/images/hvm-ssd/ubuntu-focal-20.04-amd64-server-20211129
Synthesizing stack
✨ Synthesis time: 3.34s
This deployment will make potentially sensitive changes according to your current security approval level (--require-approval broadening).
Please confirm you intend to make the following modifications:
IAM Statement Changes
┌───┬──────────────────────────────┬────────┬────────────────┬───────────────────────────┬───────────┐
│ │ Resource │ Effect │ Action │ Principal │ Condition │
├───┼──────────────────────────────┼────────┼────────────────┼───────────────────────────┼───────────┤
│ + │ ${ec2_inst/InstanceRole.Arn} │ Allow │ sts:AssumeRole │ Service:ec2.amazonaws.com │ │
└───┴──────────────────────────────┴────────┴────────────────┴───────────────────────────┴───────────┘
Security Group Changes
┌───┬────────────────────────┬─────┬────────────┬─────────────────┐
│ │ Group │ Dir │ Protocol │ Peer │
├───┼────────────────────────┼─────┼────────────┼─────────────────┤
│ + │ ${ec2-sec-grp.GroupId} │ In │ TCP 22 │ 0.0.0.0/24 │
│ + │ ${ec2-sec-grp.GroupId} │ Out │ Everything │ Everyone (IPv4) │
└───┴────────────────────────┴─────┴────────────┴─────────────────┘
(NOTE: There may be security-related changes not in this list. See https://github.com/aws/aws-cdk/issues/1299)
Do you wish to deploy these changes (y/n)? y
Ec2DeployStack: deploying...
[0%] start: Publishing 4d58d5324b774aa150b9bf2446b0431c51232dff8dc0c829da168fa1048bce17:-us-east-1
[100%] success: Published 4d58d5324b774aa150b9bf2446b0431c51232dff8dc0c829da168fa1048bce17:-us-east-1
Ec2DeployStack: creating CloudFormation changeset...
✅ Ec2DeployStack
✨ Deployment time: 201.68s
initializing repo
Stack ARN:
arn:aws:cloudformation:us-east-1::stack/Ec2DeployStack/6e95dec0-a474-11ec-ba06-0e8f90e012a7
✨ Total time: 205.02s
As it can be seen above the EC2DeployStack was successfully deployed.
In order to do an extra check and verify everything works we need to visit the AWS console and see if our EC2 was created.
To do this navigate into the AWS EC2 section and we should see our newly created
As it can be seen the AWS CDK has created the unbiased_coder_instance as we dictated to it in our code earlier.
Finally we also need to take a note that AWS CDK installed a new cloud formation.
If you navigate to the Cloud formation section in the AWS console you will see the newly created cloud formation that were created by the AWS CDK Python deployment code.
And if we investigate a few more details from the AWS console we will see the resources associated with it which is our EC2 instance we told it to create when deploying it.
Since there were more than one things involved on deploying our EC2 instance like the:
- Creation of a new role
- Creation of firewall rules
- Creation of security group
- EC2 Instance
All those actions are included in our CloudFormation. This is great because as we will demonstrate later on when we call the destroy command everything will be cleaned up gracefully.
Potential Error Creating an EC2 Instance Using AWS CDK Python
Do note that if you previously had a CloudFormation deployed by the AWS you may get a potential error and this may be silently suppressed in the AWS bootstrap command.
The error you may see is the following:
‘fail: No bucket named ‘cdk-assets’. Is account bootstrapped?’
This means there was previously a created bootstrapped cloud formation.
In order to fix this simply navigate to the Cloud Formation section in AWS console and remove the previously created CDKToolkit as shown in the screenshot above.
Once this is done you can proceed into running the ‘cdk bootstrap‘ command as explained earlier in the article to re-create the CDKToolkit CloudFormation to fix the error ‘No bucket named ‘…’ Is account bootstrapped’.
Another error you may see is the following:
‘Error No AMI found that matched the search criteria’
This means the AMI name you selected is not typed properly and you may need to find the full name from the AWS AMI store and specify it properly. If unsure you can use the Ubuntu image I used earlier in the example code.
Finally you may run into another error condition such as:
‘Could not find any VPCs matching {“account”:””,”region”:””,”filter”:{},”returnAsymmetricSubnets”:true,”lookupRoleArn”:”arn:aws:iam::…’
To resolve the above error you need to make sure you typed in the code properly the correct VPC name in your account. To find this I have explained earlier how you can see it in the EC2 section under My VPCs.
How To Destroy Your AWS EC2 Instance Formation Using AWS CDK Python
As a bonus we will go over how to clean up and destroy any resources you setup in your infrastructure up to now.
To do that CDK offers a very simple destroy command which basically will remove everything we added.
If we were to execute this would look like this:
$ cdk destroy --profile test_deploy_ec2 Loading env file Creating environment Synthesizing stack Are you sure you want to delete: EC2DeployStack (y/n)? y EC2DeployStack: destroying... ✅ EC2DeployStack: destroyed
AWS CDK goes into our AWS infrastructure and deletes everything that it had previously created.
In order to verify this we go also in our AWS console and see if the EC2 Instance and Cloudformations are gone.
An updated view looks like this:
It must be noted here that the user you created earlier and associated roles to it will not be deleted. This is something we did outside this context and you will have to navigate to the iam section and delete the user manually.
To further verify we do one last check which is to check the AWS EC2 section to ensure our instance has been terminated from the destroy command. To do this navigate to the EC2 section which has the instance listing. This should look like this:
Conclusion
We were able to successfully show you How To Setup AWS EC2 Instance Using AWS CDK Python and why you may want to use it over using Terraform.
If you found this useful and you think it may have helped you please drop me a cheer below I would appreciate it.
If you have any questions, comments please post them below or send me a note on my twitter. I check periodically and try to answer them in the priority they come in. Also if you have any corrections please do let me know and I’ll update the article with new updates or mistakes I did.
Do you prefer to use CDK or the AWS console to create EC2 Instances?
My background is Python so I generally prefer to stick to my strengths so AWS CDK Python is my favorite way to deploy EC2 Instances.
If you would like to find more on AWS CDK check the resources below: