Lambda and Event Source Concurrency

AWS Lambda and Event Source Concurrency is a very important concept that determines the number of instances of a function that can run simultaneously. Managing concurrency properly ensures optimized performance, efficient scaling and of course cost control.

In this blog, we’ll understand AWS Lambda concurrency but also take a look at the event source mapping. We will also cover the difference between Provisioned and Reserved Concurrency and how Event Source Mapping Concurrency interacts with Lambda Concurrency.

How AWS Lambda Concurrency Works

First let’s explain what is Lambda’s default behaviour. When you create new AWS Lambda, AWS applies “Unreserved Concurrency Limit” which means:

If no concurrency settings are configured, Lambda scales based on available account-level concurrency.
If the account-wide concurrency limit is reached, additional invocations are throttled.

AWS Lambda concurrency limit is 1000 concurrent executions per AWS region and when Lambda is invoked AWS automatically provisions and scales instances based on incoming requests. This means if you create 5 Lambdas in the same region and do not set reserved concurrency, they will all share a 1000 concurrent execution limit and if one of those is getting close to the limit additional invocations would throttle.

To control this behaviour, AWS provides two key concurrency management options:

Reserved vs. Provisioned Concurrency

AWS Lambda supports two types of concurrency control:

Reserved Concurrency

Reserved Concurrency ensures a specific lambda always has dedicated capacity
If reserved_concurrent_executions = 10, then 10 concurrent executions are always available for this function at all times.
No other function can consume these reserved executions.
Setting it to 0 completely disables the function from running.
Example:

resource "aws_lambda_function" "example" {
  function_name = "my_lambda_function"
  reserved_concurrent_executions = 10  # Ensures 10 concurrent executions
}

Use case for which configuration should be used:

Critical function availability like payment processing lambda.
Low priority function like logging or analytics – setting reserved concurrency to 2 would ensure it doesn’t consume too many resources.
Add limit to DB writing operations.
Other

Provisioned Concurrency

Keeps function instances warm and ready to respond instantly.
Ideal for low-latency applications where cold starts are a concern.
Works well with API Gateway and other synchronous workloads.
Example:

resource "aws_lambda_provisioned_concurrency_config" "example" {
  function_name = aws_lambda_function.example.function_name
  provisioned_concurrent_executions = 10  # Keeps 10 instances ready to go
}

Use case for which configuration should be used:

Interactive web applications with high traffic spikes.
When response to system needs to be consistent and fast.
Actions that should be completed near real time.
Other

Key Difference:

Reserved Concurrency guarantees execution slots but doesn’t keep them warm.
Provisioned Concurrency ensures warm starts but is only useful for synchronous invocations.

Lambda Concurrency with Event Source Mapping

When using event-driven architectures (e.g., SQS, Kinesis, DynamoDB streams), Event Source Mapping controls how messages are processed.

Default Behavior

When Lambda is triggered by an event source (e.g., SQS or Kinesis), it automatically scales based on available concurrency.
If concurrency limits are reached, events may be delayed or retried.

Using maximum_concurrency with Event Source Mapping

You can limit how many concurrent executions are used for a specific event source by configuring maximum_concurrency in Terraform.

Example with Kinesis (General Case):

resource "aws_lambda_event_source_mapping" "this" {
  count                              = var.needs_event_source_mapping ? 1 : 0
  enabled                            = var.lambda_event_source_mapping_enabled
  event_source_arn                   = var.lambda_kinesis_event_source_arn
  function_name                      = aws_lambda_function.this.function_name
  starting_position                  = var.lambda_event_source_mapping_starting_position
  batch_size                         = var.lambda_event_source_mapping_batch_size
  maximum_retry_attempts             = var.lambda_event_source_mapping_maximum_retry_attempts
  parallelization_factor             = var.lambda_event_source_mapping_parallelization_factor
  maximum_batching_window_in_seconds = var.lambda_event_source_maximum_batching_window_in_seconds
}

Example with SQS :

resource "aws_lambda_event_source_mapping" "retry_sqs_map" {
  event_source_arn                   = aws_sqs_queue.spti_retry_sqs.arn
  function_name                      = module.retry_lambda.lambda_function_name
  batch_size                         = var.retry_lambda.batch_size
  maximum_batching_window_in_seconds = var.retry_lambda.maximum_batching_window_in_seconds
  scaling_config {
    maximum_concurrency = var.retry_lambda.maximum_concurrency
  }
  depends_on = [module.retry_lambda]
}

Key Takeaways from This Setup:

Maximum concurrency prevents Lambda from consuming too many messages at once, reducing pressure on downstream systems.
For SQS, Lambda automatically polls and scales, but with maximum_concurrency, you can control how aggressively it scales.
For Kinesis or DynamoDB Streams, parallelization_factor also plays a role in scaling across shards.

Best Practices for Managing AWS Lambda Concurrency

As you might noticed we can configure AWS Lambda concurrency but also maximum concurrency for event source mapping when triggering lambda. Reserved Concurrency is a hard cap, meaning you can not override that with maximum concurrency on event source mapping.

Scenario A:
- Reserved concurrency is set to 10 and maximum concurrency to 50 – function cannot exceed 10 concurrent executions.
Scenario B:
- Reserved concurrency is set to 50 and maximum concurrency to 10 – the event source will trigger only 10 concurrent execution and rest of the “slots” can be used for other event sources.
Scenario C:
- Provisioned concurrency is set to 10 and maximum concurrency to 50 – function can exceed 10 concurrent execution but 10 will always be warm.
Scenario D:
- Provisioned concurrency is set to 50 and maximum concurrency to 10 – only 10 concurrency executions are possible but all warm

General advices

If you do not have many Lambdas that will compete for concurrency you should let AWS handle scaling and executions to eliminate any chance of throttling.
If your Lambda need more time to process then you should configure maximum concurrency to avoid Lambda overloading downstream services
If you service to which Lambda is sending/writing/communicating can handle let’s say only 20 concurrent actions then maximum concurrency on event source map should be set.
Monitor with AWS CloudWatch to fine-tune concurrency settings based on actual usage.

Conclusion

AWS Lambda concurrency is a powerful tool to optimize function execution and prevent overloading your AWS environment. By combining Reserved, Provisioned, and Event Source Mapping Concurrency, you can fine-tune performance, reduce latency, and control costs effectively.