When trying to run multiple API Gateway instances locally with SAM, you may encounter numerous issues, especially when invoking API Gateways using the warm containers parameter.
sam build --paralle && sam local start-api --warm-containers EAGER -p 8090 --docker-network blowstack-private -n env.json
The exact issues depend on the project itself, the resources and parameters you declare, and whether you use Layers, etc. Personally, I encountered the following issues when I tried to start multiple independent API Gateways locally with warm containers:
- Source code has been changed; terminate its warm container. The new container will be created in lazy mode
- Lambda function container initialization failed due to an error (UnrecognizedClientException) when calling the GetLayerVersion operation: The security token included in the request is invalid
- Lambda function container initialization failed because of 'default_config_resolver'
However, all the issues listed above in my case were related to the way SAM builds and runs templates with warm containers.
Prerequisites
If you have already decoupled API Gateway into separate templates (can be nested templates) or separate projects, skip to the Build and Run paragraph. We need decoupled APIs because each of them will be built and run separately, so we can exclude any interference.
However, if you have multiple API Gateways in one template, you need to separate each API Gateway into a nested templates to run them in parallel locally without interference.
Nested Templates Structure
Remove any API Gateways from the root template so that the root template will serve only the purpose of bonding resources together in the cloud. Locally, you will omit the root template altogether and run each API Gateway in a nested template separately. Take a look at the example below.
Root Template
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: >
BlowStack Multi API Gateway
Resources:
Service1ApiGateway:
Type: AWS::Serverless::Application
Properties:
Location: template_service1.yaml
Service2ApiGateway:
Type: AWS::Serverless::Application
Properties:
Location: template_service2.yaml
Service 1 Template
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: >
BlowStack Service 1 example
Resources:
Service1API:
Type: AWS::Serverless::HttpApi
Properties:
Name: Service 1 API
StageName: 'dev'
CorsConfiguration:
AllowMethods:
- GET
- OPTIONS
- POST
- PUT
- HEAD
- PATCH
- DELETE
AllowHeaders:
- "*"
AllowOrigins:
- "*"
FailOnWarnings: true
Example1Function:
Type: AWS::Serverless::Function
Properties:
Events:
ApiEvent:
Type: HttpApi
Properties:
Path: /example1/{id}
Method: post
ApiId:
Ref: Service1API
Runtime: nodejs18.x
Handler: handler.example1Function
Metadata:
BuildMethod: esbuild
BuildProperties:
Minify: true
Target: "es2020"
EntryPoints:
- "./lambdas/service1/handler.js"
Service 2 template
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: >
BlowStack Service 2 example
Resources:
Service1API:
Type: AWS::Serverless::HttpApi
Properties:
Name: Service 2 API
StageName: 'dev'
CorsConfiguration:
AllowMethods:
- GET
- OPTIONS
- POST
- PUT
- HEAD
- PATCH
- DELETE
AllowHeaders:
- "*"
AllowOrigins:
- "*"
FailOnWarnings: true
Example2Function:
Type: AWS::Serverless::Function
Properties:
Events:
ApiEvent:
Type: HttpApi
Properties:
Path: /example2/{id}
Method: post
ApiId:
Ref: Service2API
Runtime: nodejs18.x
Handler: handler.example1Function
Metadata:
BuildMethod: esbuild
BuildProperties:
Minify: true
Target: "es2020"
EntryPoints:
- "./lambdas/service2/handler.js"
Build and Run
In the example, we have two API Gateways in two different templates. Let's build each of them separately, providing the exact location of templates and, importantly, different locations for build outputs. Notice that the parallel parameter is not required but greatly increases build speed for larger APIs.
// service 1
sam build --template_service1.yaml --parallel --build-dir .aws-sam-service1/build
// service 2
sam build --template_service2.yaml --parallel --build-dir .aws-sam-service2/build
Thus, we have 100% assurance that none of the services will update the build of the other.
Next, run the API Gateways, each in a separate terminal or in the background (which IMHO hinders debugging). What is important this time is to pass the template parameter as the output target (notice that each path has to refer to template.yaml
) and provide different ports where APIs will be available on your localhost.
// service 1
sam local start-api --warm-containers EAGER --docker-network blowstack-private -n env.json -p 8090 --template .aws-sam-service1/build/template.yaml --skip-pull-image --debug
// service 2
sam local start-api --warm-containers EAGER --docker-network blowstack-private -n env.json -p 8091 --template .aws-sam-service2/build/template.yaml --skip-pull-image --debug
Summary
By having separate build folders and different ports where APIs run, you will have full assurance that running APIs won't interfere with each other. However, if you drop warm containers (which greatly negatively impacts local performance), you can just build the whole project from the root template if it's not dispersed into smaller microservices.