Introduction
We will go over Python AWS Boto3 Subnet Guide.
Did you know that Boto3 offers a fully managed way to control your Subnets programmatically?
We will break down this in the following sections:
- Why Is it Important To Control AWS Subnets Programmatically
- How to setup your environment to control AWS Subnets programmatically
- How to list all AWS Subnets
- How To Add and Delete AWS Subnets
I have used this successfully in various projects and it works very well and has saved me a ton of time getting going with managing my Subnets in a programmatic way. We will cover how to do this in Python using the Boto3 library framework.
We will go point by point on getting you up and running in less than 5mins, you do not need to have any programming knowledge to use the tool we are going to be developing here but know Python is preferable if you would like to modify the code for your needs.
This is a complete guide and should cover all your questions on managing Subnets in an AWS environment.
All code and examples on how to do this can be found in the Github link here.
When To Programmatically Control Your AWS Subnet Networks
Before I dive into any code and implementation I want to give you a list of reasons of why this may or may not be useful for your needs. Basically whenever programming is involved you need to understand that the nature of your requirements but also capabilities increase so lets go over some reasons for this.
- You can setup conditional Subnet creations allowing Python to extend the logic
- You can integrate this in a system that performs devops operations such as a pipeline
- You can do bulk operations which are hard to do manually over the console such as deleting mass Subnets in your AWS account
- This can be part of a batch job that’s scheduled to run periodically
- You can use this code to automatically check certain things in your AWS account and set filters or rules on how your Subnets are created using the enumeration method we will describe below
- You can set your own limits on how many Subnets and what network setup you want to use and enforce them by periodically checking and even removing them automatically if they do not satisfy your requirements.
As you can see the list is rather big but there’s some cases where you may not want to use programmatic access to manage your Subnet networks.
A list of those is below:
- You have a small amount of Subnets that you rarely change
- Your environment is highly critical and all changes should be done manually
- You do not need to perform bulk operations or add programming logical conditions to it
- You are just happy with the default Subnets that AWS has created automatically for your account
I’m sure I may have missed some on the lists above and if you do spot something please send me a note and I will be sure to update it.
How To Setup Boto3 To Manage Subnets
We will start by going over how to setup your system in order to run the Boto3 code which we will be outlining below. Since the Boto3 AWS library is a Python library we will be leveraging here a virtual environment to install our Python libraries to keep the main system clean from any dependencies.
For this I have made the process fairly easy for you by including the needed requirements file which contains all the packages you need to get going. Below you will find an outline of the commands you need to execute to get the environment setup and ready to go. You can also use conda if you prefer that but to keep things simple and lean I’m using directly virtualenv here.
$ virtualenv venv
created virtual environment CPython3.9.12.final.0-64 in 208ms
creator CPython3Posix(dest=/Users/alex/code/unbiased/python-boto3-load-balancer/venv, clear=False, no_vcs_ignore=False, global=False)
seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/Users/alex/Library/Application Support/virtualenv)
added seed packages: pip==22.1.2, setuptools==62.6.0, wheel==0.37.1
activators BashActivator,CShellActivator,FishActivator,NushellActivator,PowerShellActivator,PythonActivator
$ source venv/bin/activate
$ pip install -r requirements.txt
Collecting boto3
Downloading boto3-1.24.37-py3-none-any.whl (132 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 132.5/132.5 kB 2.1 MB/s eta 0:00:00
....
Installing collected packages: urllib3, six, python-dotenv, jmespath, python-dateutil, botocore, s3transfer, boto3
Successfully installed boto3-1.24.37 botocore-1.27.37 jmespath-1.0.1 python-dateutil-2.8.2 python-dotenv-0.20.0 s3transfer-0.6.0 six-1.16.0 urllib3-1.26.11
$ python
Python 3.9.12 (main, Mar 26 2022, 15:44:31)
[Clang 13.1.6 (clang-1316.0.21.2)] on darwin
>>> import dotenv
>>> dotenv.__all__
['get_cli_string', 'load_dotenv', 'dotenv_values', 'get_key', 'set_key', 'unset_key', 'find_dotenv', 'load_ipython_extension']
>>> import boto3
>>> boto3.__version__
'1.24.37'
As you can see from the code above we basically installed two packages:
- dotenv: This contains all the environment variables were will be using in our code such as the AWS access and secret key and also the region of AWS we will be using.
- boto3: This is the AWS library that contains the code we need to connect and interact with our Subnet setup in the AWS account.
To test the installation we simply run the version and some functions for dotenv to ensure it got installed properly, you can skip that test if you want it’s just a precaution here.
Now that we have the Python packages installed we need to go over some helper libraries we have implemented in order to wrap the Boto3 Subnet setup. Lets start by checking the code which is shown below and then we will do an analysis on the function helpers we wrote.
import os
import boto3
import pprint
from dotenv import load_dotenv
def get_aws_keys():
load_dotenv()
return os.getenv('AWS_ACCESS_KEY'), os.getenv('AWS_SECRET_KEY')
def init_aws_session():
access_key, secret_key = get_aws_keys()
return boto3.Session(aws_access_key_id=access_key, aws_secret_access_key=secret_key, region_name=os.getenv('AWS_REGION'))
def ec2_get_subnet_list():
session = init_aws_session()
ec2 = session.client('ec2')
response = ec2.describe_subnets()
return response['Subnets']
def ec2_add_subnet(vpc_id, ip_block):
session = init_aws_session()
ec2 = session.client('ec2')
response = ec2.create_subnet(VpcId=vpc_id, CidrBlock=ip_block)
return response['Subnet']
def ec2_delete_subnet(subnet_id):
session = init_aws_session()
ec2 = session.client('ec2')
response = ec2.delete_subnet(SubnetId=subnet_id)
return response
The first few functions that initialize the AWS session have been discussed extensively in a previous article that I wrote which you can find below:
So for simplicity and to avoid repetition here I’m going to cover the new functions that are related to Subnet. The function names are made in such as a way to be self explanatory on what they do so I’m going to focus mainly on the functionality.
The first thing you would notice that’s common in all the functions is that we initialize an AWS session which we use later to perform the Subnet operations. This is shown in the init_aws_session function. Once the session is successfully initialized (this is where authentication happens too) then we need to acquire a client for the EC2 code base from the Boto3 library. The EC2 portion has a subset in it which contains the Subnet related wrappers which we will be using below.
ec2_add_subnet: The first function we will be talking about is adding a Subnet in AWS using Boto3. This function basically takes as an argument the very much needed IP address block we will be using to allocate to it. This can be a local network within AWS along with the net mask it has. Furthermore we need to also specify the VPC ID to make the association on where this subnet will live.
ec2_delete_subnet: Similarly to adding we also implement the Subnet deletion function. This function basically does the opposite and just removes a Subnet network from our list. Each Subnet network has a unique identifier associated with it which is called Subnet ID. Using the Subnet ID we can easily manage a particular Subnet including deleting it. When a Subnet gets first added you can see what unique identifier AWS associated for it and save it, or you can use the method which we will talk about next.
ec2_get_subnet_list: The Get Subnet list lets you list all AWS Subnet networks in your AWS account using Boto3. Basically this is useful if you are trying to find particular details about a Subnet such as the IP address block or association with a VPC network via it’s VPC ID. You can also find the Subnet ID based on some filter requirements to use later in your code for operations such as deletion.
How To List Subnets With Boto3
Now that we covered the basic Boto3 Subnet function wrappers lets start implementing some examples and interacting with our AWS account. For this you may see specific IDs associated in the code but that’s just for simplicity and only work with my AWS account. In order to run this in your system you will need to keep track of what exists and save it or use the Subnet list Boto3 code below to enumerate everything and pick one to manipulate.
import pprint from boto3_helper import ec2_get_subnet_list result = ec2_get_subnet_list() pprint.pprint(result)
The code above basically uses the function we implemented earlier that lists all Subnet networks in your AWS account using Boto3. We simply run the function and print out the results of our Subnet networks. An example execution of this is in my AWS account and can be seen below.
$ python ./boto3-subnet-list.py
[{'AssignIpv6AddressOnCreation': False,
'AvailabilityZone': 'us-east-1e',
'AvailabilityZoneId': 'use1-az3',
'AvailableIpAddressCount': 4091,
'CidrBlock': '172.31.48.0/20',
'DefaultForAz': True,
'EnableDns64': False,
'Ipv6CidrBlockAssociationSet': [],
'Ipv6Native': False,
'MapCustomerOwnedIpOnLaunch': False,
'MapPublicIpOnLaunch': True,
'OwnerId': '033533902081',
'PrivateDnsNameOptionsOnLaunch': {'EnableResourceNameDnsAAAARecord': False,
'EnableResourceNameDnsARecord': False,
'HostnameType': 'ip-name'},
'State': 'available',
'SubnetArn': 'arn:aws:ec2:us-east-1:033533902081:subnet/subnet-7ebb454f',
'SubnetId': 'subnet-7ebb454f',
'VpcId': 'vpc-af9c5bd2'},
...
To cut the output I just showed the first few networks that exist in my account. As you can see Subnet have a lot more information associated with them besides the IP address block, the VPC ID associated with them and a unique identifier. Since this is beyond the scope of this article I’m not going to go over every single one but if you want me I can do it in the comments below so drop me a line. However I do want to explain a few fields here that may be useful for you.
- State: The state of your Subnet is important to know because sometimes it may be disabled or turned off and this gives you visibility to it.
- Owner: The Identifier of the owner is also good to know for accounting reasons and if you are trying to debug something.
- Availability Zone: This allows us to see where that subnet is available. Knowing which zone it belongs too you can easily setup later your policies or anything else related to your EC2 instances.
The rest of the options while important I do not believe play a major role for your basic management of your Subnet setup using Boto3.
How To Create A Subnet With Boto3
Moving on we are going to cover how to add a Subnet using Boto3. Similar to before the logic to this is basically to invoke the function we implemented on our Boto3 Subnet function wrapper. The only difference here is that when invoking our function to add a Subnet network to our AWS account we are also going to need to pass in the IP address block to it so AWS knows how to allocate it.
import pprint
from boto3_helper import ec2_add_subnet
result = ec2_add_subnet('vpc-af9c5bd2', '172.31.1.0/24')
pprint.pprint(result)
In the code above you can see we are basically adding the network IP range with 172.31.0.0 and with a netmask of a class B network (16). This is enough to cover most of your needs generally. We also pass in the VPC identifier that we will be associating this network with.
To further demonstrate how this works and see the response that AWS sends back to us when it gets added we will be executing the code to create this new Subnet with the Boto3 code wrapper we implemented.
$ python ./boto3-subnet-add.py
{'AssignIpv6AddressOnCreation': False,
'AvailabilityZone': 'us-east-1b',
'AvailabilityZoneId': 'use1-az4',
'AvailableIpAddressCount': 251,
'CidrBlock': '172.31.1.0/24',
'DefaultForAz': False,
'EnableDns64': False,
'Ipv6CidrBlockAssociationSet': [],
'Ipv6Native': False,
'MapPublicIpOnLaunch': False,
'OwnerId': '033533902081',
'PrivateDnsNameOptionsOnLaunch': {'EnableResourceNameDnsAAAARecord': False,
'EnableResourceNameDnsARecord': False,
'HostnameType': 'ip-name'},
'State': 'available',
'SubnetArn': 'arn:aws:ec2:us-east-1:033533902081:subnet/subnet-0604756b09f4a2dcd',
'SubnetId': 'subnet-0604756b09f4a2dcd',
'VpcId': 'vpc-af9c5bd2'}
$ python ./boto3-subnet-list.py|grep "subnet-0604756b09f4a2dcd"
'SubnetArn': 'arn:aws:ec2:us-east-1:033533902081:subnet/subnet-0604756b09f4a2dcd',
'SubnetId': 'subnet-0604756b09f4a2dcd',
As you can see above AWS successfully added the Subnet to our list and returned to us the unique Subnet identifier to allocated to it, in this case: subnet-0604756b09f4a2dcd. As an extra verification we will be leveraging the code to list all Subnets in our system and basically grep specifically for this new Subnet ID to see if it exists in our system. Above it’s visible that the Subnet ID exists and it was successfully added to it.
How To Delete Subnet With Boto3
In this section we will demonstrate how to delete a Subnet using Boto3. Since we already added a test Subnet earlier we will basically proceed into deleting it now and checking to see if it still exists in the list.
Similarly to the two earlier examples we will use the Boto3 delete Subnet wrapper function we wrote and we will be passing to it the Subnet ID that we want to mark for deletion.
import pprint
from boto3_helper import ec2_delete_subnet
result = ec2_delete_subnet('subnet-0128e4cd0a6ebca54')
pprint.pprint(result)
We basically send the command to the AWS service to delete the Subnet and then print out the result. The execution of this is shown below.
$ python ./boto3-subnet-delete.py
{'ResponseMetadata': {'HTTPHeaders': {'cache-control': 'no-cache, no-store',
'content-length': '225',
'content-type': 'text/xml;charset=UTF-8',
'date': 'Wed, 27 Jul 2022 19:04:42 GMT',
'server': 'AmazonEC2',
'strict-transport-security': 'max-age=31536000; '
'includeSubDomains',
'x-amzn-requestid': '8f47c3b0-6790-4df7-903e-9219f01cc447'},
'HTTPStatusCode': 200,
'RequestId': '8f47c3b0-6790-4df7-903e-9219f01cc447',
'RetryAttempts': 0}}
$ python ./boto3-subnet-list.py|grep "subnet-0604756b09f4a2dcd"
[empty]
As you can see above the command succeeded with a 200 HTTP return code meaning the record of the Subnet ID we provided was deleted from the AWS account. Since we are adding extra verification we will again check our list for the Subnet ID to see if it exists in the list. Running the command shows that the Subnet has disappeared and is no longer there indicating we were successfully able to demonstrate how to remove a Subnet network using Boto3 from the AWS account.
Conclusion
We were able to successfully go over Python AWS Boto3 Subnet Guide, hopefully I answered any questions you may have had and helped you get started on your quest on managing your Subnets programmatically in your AWS account.
If you found this useful and you think it may have helped you please drop me a cheer below I would appreciate it.
If you have any questions, comments please post them below or send me a note on my twitter. I check periodically and try to answer them in the priority they come in. Also if you have any corrections please do let me know and I’ll update the article with new updates or mistakes I did.
Do you prefer to use the AWS console or Boto3 to manage your Subnets?
I personally still think both have their place and complement each other when working from the terminal or wanting to do customized stuff that require programming and needing to add logic to it.
If you would like to visit the official Python Boto3 documentation here.
If you would like to find more articles related to AWS services: