AWS Lambda vs AWS Step Functions

Introduction

Today we will discuss on everything you need to know about AWS Lambda vs AWS Step Functions in simple and easy to follow guide.

Did you know that both Lambdas and Step functions can work in a complementary way?

Today I will break down:

• Briefly discuss how each technology works
• Explain the differences between the two in regards to:
  • Cost
  • Which and When

I have been working in the Software industry for over 23 years now and I have been a software architect, manager, developer and engineer. I am a machine learning and crypto enthusiast with emphasis in security. I have experience in various industries such as entertainment, broadcasting, healthcare, security, education, retail and finance.

If you have used Lambda’s or Step functions before this may help you understand some small important differences between the two but if you are starting out this may help you make a decision on what and when to use it.

What Is the Difference Between Lambda and Step Functions

If you don’t want to go over all the details of how the two technologies are different you can preview the data in the table below and read on specific points further below if you would like to understand more on each topic.

I will try to break down all the details on each section of the table below as to what I think the reasoning behind things is.

Setup Time

The setup time for the AWS Lambda is pretty much instant. You follow the AWS guidelines such as selecting your language, versioning and environment info and it’s ready to go.

However for the AWS Step functions the process is a little more involved as you are creating a workflow process. First you need to setup your workflow and outline and then you need to associate each step of it with the equivalent actions which could be a previously created lambda function.

Difficulty To Learn

For AWS Lambda functions learning them is pretty straight forward, if you have programmed in the language you want your lambda to be in for example Python or NodeJS then you just select it from the drop down menu and you are ready to run. The only thing you need to define is your function entry point which is where execution will begin.

For Step functions however you need to learn how to build workflows on top of associating them with your lambda’s. While this adds an extra step in the process it’s not as difficult to do.

Workflow Support

AWS Step functions are basically built to extend the functionality of extended isolated lambdas into a workflow. While you can have your workflow within the lambda itself like code logic this is not ideal as someone reviewing your business logic that is not a programmer will not be able to understand it. Furthermore the visualization aspect of the Step functions allows an easy and quick glance to see what’s happening and when it’s happening. If necessary then you can always drill down to the details of the isolated step and understand more about it.

Cost

The cost of a Lambda function is always going to be lower as it’s a bare bones lambda pricing scheme. When you add Step functions you are basically adding a new service to your infrastructure costs and this basically increases what you have to pay for it. Overall the difference is not major and pricing could be a factor you look at as a last thing when trying to make a decision between the two technologies.

Maintenance

DevOps

Maintenance is a crucial aspect of any system regardless of the technologies. Since Lambdas are fairly simple maintaining them is also very simple. The idea of just adjusting the attributes of a single lambda function are as trivial as going to the AWS console and modifying them. However when it comes to AWS Step functions the process gets more involved as now you are maintaining state and potentially multiple smaller lambdas which would otherwise would just be one piece of code.

Code / Debugging

If we look at the code aspect of things maintaining a Lambda function is more work. Since you have to put all the business logic into one single monolithic piece of code this will also be harder to maintain even if you break it down in small pieces such as functions or classes depending on what programming language you use. The introduction of AWS Lambda layers can potentially help out but it won’t be significant when you are dealing with the entirety of the business functionality into one piece of code.

Inversely with AWS Step functions you can break up your logic as much as you want and still have a clear conceptual view with a graphical interface. Not only does this make your flaw easier to navigate but it allows you to break up things in micro-modules making debugging easier too. Furthermore debugging an AWS Lambda function component can have various helpers such as IDE extensions and such but at the end of the day you are debugging code not components so if one piece fails that means you need to get a traceback and identify the cause of your function failure. On the other hand you see a component failing and you can instantly switch over to that and analyze it.

Execution Speed

The execution speed is relative in both aspects. If you consider that Step functions may require initialization and getting various Lambda functions hot there’s an overhead that’s associated with that. As such while you can still get pretty close a monolithic AWS Lambda function will always be faster than an AWS Step function because one only requires a single component to be initialized and executed.

Similarly this will hold true with other resources such as:

• Memory
• Disk
• Networking

All of the above components will be amplified for every single initialization for a new AWS Lambda your AWS Step function workflow logic has.

What Is a Benefit of Leveraging AWS Step Functions with AWS Lambda

The main benefit of leveraging AWS Step Functions is that they work perfectly alongside AWS Lambda functions. This means that within your AWS Step function workflow you can invoke as many AWS Lambda functions as you want. This is important as the alternative of this is to basically cascade AWS Lambda function invocations using the AWS Lambda Invoke method.

While this makes for a much cleaner implementation it’s also easier to manage because you have a high level overview of how the AWS Step functions look and which step it’s executing in.

Chaining

• Lambda: If you are doing this within a Lambda function the chaining will basically be a synchronous invocation using the AWS Lambda Invoke function and waiting for it to complete. Note here you are limited with the Max execution time of your original Lambda which is 15minutes currently so this may be a big problem for you and you may need to do it setting up a periodic monitor task rather than a waiting task.
• Step: This process is abstracted for you from an invocation part so you basically just choose your Lambda functions and execute them.

Winner: AWS Step Functions

Parallelism

• Lambda: This is where things get a bit more complex. Since the Invoke method of the Lambda function allows you to run tasks asynchronously you don’t necessarily need to wait for the Lambda to complete. Which basically allows you to bypass the original limitation of the 15minute execution time you would otherwise have if you were running the task synchronously. Which means it’s possible however maintaining and keeping track of them will be a bit difficult to visualize.
• Step: As mentioned above this can be accomplished using both methods however things are much more clear here. The only disadvantage you have is that you can’t dynamically create the allocation of your parallelism speed or based on code conditions, which using a Lambda allows you to do this in any instance of the process. So the advantage goes out to AWS Lambda functions.

Winner: AWS Lambda

Branching

• Lambda: Branching is a very similar process to chaining the only difference is that it also combines some form of parallelism. Since we have the original limitation that when you chain from your original Lambda you need to wait for the execution time of the sub-tasks to complete you will always be bound by it unless you create an AWS Lambda supervisor which would mean now you have to write a lot of boilerplate logic on top of your original code.
• Step: Since chaining is an issue the Step functions can accommodate both here so it’s easier to branch out using Step functions.

Winner: AWS Step functions

As a closing note there’s also catching issues and retrying which are very hard to implement on your own and you get this for free if you were to use AWS Step functions. If you have complicated conditional logic on this and your catch and retry couldn’t be abstracted beyond the basic things such as a timeout, error etc. then it’s easier to do it using your own AWS Lambda supervisor code. The winner on this is out for a judge as it’s highly dependent on what you are trying to accomplish with your code.

Is AWS Step Functions Serverless

The very nature of running AWS Step Functions is basically based on a serverless concept. There’s no servers involved in the whole process and it’s just based on Lambda functions as a natural extension to the concept. To make it easier for your AWS has now moved the whole State based as a section under the Lambda functions. Keep in mind that because this is more like a workflow organizer in the future AWS can easily extend the same concept to things such as EC2 Instances. If you would like to learn more about AWS EC2 instances I have an article below that may help you get started:

Boto3 EC2 Create, Launch, Stop, List and Connect to instances

Workflow management as a concept it’s not limited to a serverless only technology, it’s just that at this current moment it’s mainly an extension of the a serverless component of AWS Lambda functions.

What Can Trigger AWS Step Functions

AWS has a good article explaining this in detail which you can find here.

The idea however of triggering an AWS Step function is very similar to triggering an AWS Lambda function.

These are the most common methods you can invoke one (this may change in the future and more may get added so please reference the article above for a more up to date source of it).

• AWS Lambda
• Amazon API Gateway
• Amazon EventBridge
• AWS CodePipeline
• AWS IoT Rules Engine
• AWS Step Functions

How Long Can AWS Step Function Wait vs Lambda Functions

Step functions can execute currently up to 1 year. As you can understand this is much higher than simply running an AWS Lambda function which has a maximum execution time of 15minutes. As these quotas may change over time I recommend you always refer to Amazon’s official documentation on AWS Step function quotas which you can find here.

If you check that you will also be able to find other limits such as:

• Naming
• Event history
• Bucket Size
• Max and Min parallel executions
• Max request sizes

Off course all of the above are easily adjustable to your needs and usually scale to thousands.

Can Lambda Trigger a Step Function

Yes AWS Lambda’s are one of the ways you can trigger the execution of a Step Function using the StartExecution method. A detailed document on how this can be done can be found on Amazon’s website here.

The syntax of it is as simple as giving it input parameters and the invocation parameters for the AWS Step Function as it can be seen below:

{
   "input": "string",
   "name": "string",
   "stateMachineArn": "string",
   "traceHeader": "string"
}

Similarly a JSON response comes in when it is invoked giving you the ARN number and the start of it’s execution time.

{
   "executionArn": "string",
   "startDate": number
}

Conclusion

If you found AWS Lambda vs AWS Step Functions useful and you think it may have helped you please drop me a cheer below I would appreciate it.

If you have any questions, comments please post them below I check periodically and try to answer them in the priority they come in. Also if you have any corrections please do let me know and I’ll update the article with new updates or mistakes I did.

Do you think the two technologies synnergize well?

I personally think they do. The two technologies even though can be used on their own especially Lambdas, they work well together too. The Step functions build upon the concept and add a good organization and workflow perspective to your existing work.

If you would like to learn more about AWS resources I have written many articles which you can check below:

• AWS Fargate vs AWS Lambda Functions
• How to Setup an AWS Lambda Python Function From Scratch
• How To Create An AWS Python Lambda Layer
• Fix AWS Python No module named lambda_function