How to use AWS Lambda Layers with SAM
How to create a project structure and template with layers that work in local TypeScript and NodeJS environment.
GOLDEN RULES
There are 4 rules you have to follow in order to use lambda layers:
- in local development
- in the same template as the rest of resources
RULE #1
Nodejs folder have to be created as a parent folder for layer dependencies (node_modules).
RULE #2
Create layer resources and reference them in lambda functions.
RULE #3
Mark layer dependencies as external during build
RULE #4
Build your project using containers: sam build --use--container
Read further if you need more explanation.
Do you really need lamba layers?
Using lambda layers can be troublesome but at the same time beneficial or even necessary in a project.
So what can lambda layers do in the first place?
- share libraries between different lambda functions (in different projects, the same account)
- lower lambdas size
- environment consistency (global version control)
- faster development
- can overlap default installed packages (in lambda environment) like aws-sdk
- easy to upgrade or downgrade any library/package globally
- some big packages like Sharp are easier to install that way
- can be utilized as a custom runtime, data or configuration file
This post is focused on layers as a library.
Project structure with lambda layers
Basically there are two options that you can choose from.
Keep or not keep layers in the same SAM template where you define other resources.
The main reason for not keeping layers in the same template is when you are going to use them in different projects.
Then even keeping them in different repositories makes a lot of sense.
Unfortunately there are many people pointing out technical difficulties as the reason to always choose a multi template option.
For example one of the articles about nested SAM stacks states that:
“When working with a nested SAM stack, you cannot import a Lambda Layer exported from another stack.”
But it's not true.
You can nest layers in SAM templates and test them locally as dependencies as long as you stick to some rules.
So what are the rules? Proper structure is one of them. So take a look at the example project.

- the initial structure is based on SAM init “Hello World template”
- there is only one SAM template in the root of the project
- both layers sit in layers directories and have separate node_modules folders
- there is only one lambda function `thumbnail-post-cover-creator` which uses dependencies from both layers
- node_modules are in nodejs folders
RULE #1
Nodejs folders have to be created as a parent folder for layer dependencies (node_modules).
Node modules
I also post contents of all three packages.json files to make it clear where specific dependencies are installed.layers/sharp-layer/nodejs/package.json
{
"name"
:
"sharp-layer"
,
"version"
:
"1.0.0"
,
"description"
:
""
,
"main"
:
"index.js"
,
"scripts"
:
{
"test"
:
"echo
\"
Error: no test specified
\"
&& exit 1"
}
,
"keywords"
:
[]
,
"author"
:
"BlowStack
,
"license"
:
"MIT-0"
,
"dependencies"
:
{
"sharp"
:
"^0.30.7"
}
}
layers/aws-layer/nodejs/package.json
{
"name"
:
"aws-layer"
,
"version"
:
"1.0.0"
,
"description"
:
""
,
"main"
:
"index.js"
,
"scripts"
:
{
"test"
:
"echo
\"
Error: no test specified
\"
&& exit 1"
}
,
"keywords"
:
[]
,
"author"
:
"BlowStack
,
"license"
:
"MIT-0"
,
"dependencies"
:
{
"sharp"
:
"^0.30.7"
}
}
package.json
{
"name"
:
"blowstack_website_sam"
,
"version"
:
"1.0.0"
,,
"main"
:
"handler.js"
,
"scripts"
:
{
"unit"
:
"jest"
,
"lint"
:
"eslint '*.ts' --quiet --fix"
,
"compile"
:
"tsc"
,
"test"
:
"npm run compile && npm run unit"
}
,
"homepage"
:
"https://github.com/blowstack"
,
"devDependencies"
:
{
"@types/aws-lambda"
:
"^8.10.92"
,
"@types/jest"
:
"^27.4.0"
,
"@types/node"
:
"^17.0.13"
,
"@typescript-eslint/eslint-plugin"
:
"^5.10.2"
,
"@typescript-eslint/parser"
:
"^5.10.2"
,
"esbuild"
:
"^0.14.14"
,
"esbuild-jest"
:
"^0.5.0"
,
"eslint"
:
"^8.8.0"
,
"eslint-config-prettier"
:
"^8.3.0"
,
"eslint-plugin-prettier"
:
"^4.0.0"
,
"jest"
:
"^27.5.0"
,
"prettier"
:
"^2.5.1"
,
"ts-node"
:
"^10.4.0"
,
"typescript"
:
"^4.5.5"
}
}
Template with nested layers
Take a look at the template. There are three important things to focus on.
- layer resource creation and its proper content uri path
- layer referencing in lambda function
- marking layers dependencies as external
template.yml
AWSTemplateFormatVersion
:
'2010-09-09'
Transform
: AWS::Serverless-2016-10-31
Description
: Layer test SAM
Resources
:
SharpLayer
:
Type
: AWS::Serverless::LayerVersion
Properties
:
LayerName
: SharpLayer
Description
: Sharp NPM package.
ContentUri
:
'layers/sharp-layer/'
CompatibleRuntimes
:
- nodejs12.x
AwsLayer
:
Type
: AWS::Serverless::LayerVersion
Properties
:
LayerName
: AwsLayer
Description
: AWS NPM package.
ContentUri
:
'layers/aws-layer/'
CompatibleRuntimes
:
- nodejs12.x
ThumbnailPostCoverCreator
:
Type
: AWS::Serverless::Function
Properties
:
Handler
: handler.thumbnailPostCoverCreator
Runtime
: nodejs14.x
Architectures
:
- x86_64
Layers
:
- !Ref SharpLayer
- !Ref AwsLayer
Metadata
:
BuildMethod
: esbuild
BuildProperties
:
Minify
: true
Target
:
"es2020"
Sourcemap
: true
EntryPoints
:
- handler.js
External
:
-
'aws-sdk'
-
'sharp'
Now take a look at the lambda code which uses both layers.
Notice that aws-sdk and sharp are required as usual (as if they were in root node_modules but they are not).
thumbnail-post-cover-crator.ts
import { S3Event } from 'aws-lambda';
const AWS = require('aws-sdk');
const sharp = require('sharp');
const path = require('path');
const util = require('util');
const s3 = new AWS.S3();
export const thumbnailPostCoverCreator = async (event: S3Event): Promise<void> => {};
In fact there is no sharp package in package.json in the root folder of the project.
But because the rules were applied it's possible to use layers dependencies automatically.
RULE #2
Create layer resources and reference them in lambda functions.
SharpLayer
:
Type
: AWS::Serverless::LayerVersion
Properties
:
LayerName
: SharpLayer
Description
: Sharp NPM package.
ContentUri
:
'layers/sharp-layer/'
CompatibleRuntimes
:
- nodejs12.x
Layers
:
- !Ref SharpLayer
- !Ref AwsLayer
RULE #3
Mark layer dependencies as external during build
External
:
-
'aws-sdk'
-
'sharp'
POSSIBLE ERROR
If you don't mark layer dependencies as external the following error will pop up during building.
Could not resolve "sharp"
thumbnail-post-cover-creator/app.ts:6:22:
6 │ const sharp = require('sharp');
╵ ~~~~~~~
You can mark the path "sharp" as external to exclude it from the bundle, which will remove this error. You can also surround this "require" call with a try/catch block to handle this failure at run-time instead of bundle-time.
Handler.js
Just to make everything as clear as possible I show contents of handler.js too.
import { thumbnailPostCoverCreator } from './thumbnail-post-cover-creator/app.ts';
module.exports.thumbnailPostCoverCreator = thumbnailPostCoverCreator;
Building
To make the whole thing work together properly you need to build your project with docker containers.
RULE #4
Build your project using containers: sam build --use--container
After building the project is ready for local testing and deployment.
Summary and BONUS TIP
In this article you learned how to create lambda layers in NodeJS and TypeScript that can be
- used in other lambdas as dependencies
- tested locally with other lambdas without build problems
But there is more you can learn.
If you read “Do you really need lambda layers?” part you know that layers can also:
- overlap default installed packages
- make some of them easier to install
I show how to use aws-sdk
and sharp
packages as layers in this article. They are both great choices to explain the above features.
For example aws-sdk
is already installed in the lambda environment by default. You can refer to this package in your code immediately without installing anything. The downside is the current managed aws-sdk version, it's not the latest. If you want to use the specific aws-sdk, wrapping it as a layer is a good choice. Of course you can try to add aws-sdk without a layer but then does it make sense to install it multiple times? The bundle size limits are also not too high!
More interesting things happen when you try to install a sharp package not as a layer. The following error will likely occur.
2022-07-07T20:04:57.198Z undefined ERROR Uncaught Exception {"errorType":"Error","errorMessage":"\nSomething went wrong installing the \"sharp\" module\n\nCannot find module '../build/Release/sharp-linux-x64.node'\nRequire stack:\n- /var/task/handler.js\n- /var/runtime/UserFunction.js\n- /var/runtime/Runtime.js\n- /var/runtime/index.js\n\nPossible solutions:\n- Install with verbose logging and look for errors: \"npm install --ignore-scripts=false --foreground-scripts --verbose sharp\"\n- Install for the current linux-x64 runtime: \"npm install --platform=linux --arch=x64 sharp\"\n- Consult the installation documentation: https://sharp.pixelplumbing.com/install","stack":["Error: ","Something went wrong installing the \"sharp\" module","","Cannot find module '../build/Release/sharp-linux-x64.node'","Require stack:","- /var/task/handler.js","- /var/runtime/UserFunction.js","- /var/runtime/Runtime.js","- /var/runtime/index.js","","Possible solutions:","- Install with verbose logging and look for errors: \"npm install --ignore-scripts=false --foreground-scripts --verbose sharp\"","- Install for the current linux-x64 runtime: \"npm install --platform=linux --arch=x64 sharp\"","- Consult the installation documentation: https://sharp.pixelplumbing.com/install"," at /var/task/handler.js:32:1753110"," at /var/task/handler.js:1:35"," at /var/task/handler.js:33:121"," at /var/task/handler.js:1:35"," at /var/task/handler.js:33:70531"," at /var/task/handler.js:1:35"," at Object.<anonymous> (/var/task/handler.js:33:70661)"," at Module._compile (internal/modules/cjs/loader.js:1085:14)"," at Object.Module._extensions..js (internal/modules/cjs/loader.js:1114:10)"," at Module.load (internal/modules/cjs/loader.js:950:32)"]}
You can workaround this error but then ask yourself is it worth it? Using layers in this case can save you a lot of time and effort.