I encountered an issue with local resource communication when trying to access a local frontend application from the API Gateway using Puppeteer on AWS SAM. The plan was to navigate to a specific page on my local website using Puppeteer and then send a callback to the API Gateway to retrieve the necessary data for generating a PDF.
When I initially used the default frontend app and AWS SAM API Gateway addresses, respectively localhost:3001 and 127.0.0.1:3000, the issues with communication were visible in the console:
Error: net::ERR_CONNECTION_REFUSED at http://127.0.0.1:3001/*
and
failed: timeout 30000ms exceeded
However, in the production environment on the AWS cloud, where domain addresses were used, everything worked as expected. Because AWS SAM is intended to work and test locally, I decided to write an article that aims to help anyone seeking correct addressing also in a local environment.
Accessing local resources with AWS SAM API Gateway
By external I refer to the API Gateway accessing my local website through the proxy of Puppeteer. It's considered external because the local API Gateway operates within Docker, necessitating a different approach to access local HOST resources. To connect to frontend website, which runs on the default address localhost:3001, you need to use a different address than the default, due to the encapsulated Docker environment.
How to find the correct address to access localhost services on the host machine from within a Docker container?
The address used to access localhost services from within a Docker container can vary depending on the Operating System. Typically, for Linux, it might be 172.17.0.1, while for Mac and Windows, it is usually host.docker.internal. Instead of guessing, you can easily verify this using the Docker CLI. Just execute the following command:
ip addr show docker0
Then, use the address displayed after the "inet" prefix.
Therefore, to access my frontend website from Puppeteer running on a Lambda function, facilitated by the local API Gateway, I would use:
await page.goto('http://172.17.0.1:3001', {waitUntil: 'networkidle0'});
To summarize, replace `localhost:3001` (or `127.0.0.1:3001`) with `http://172.17.0.1:3001` or whichever address is designated for accessing localhost services from Docker containers on your host machine.
Changing the address on the backend is not enough though. You will have to run your frontend website using the 0.0.0.0 address; the implementation depends on the framework. For example, for Nuxt3, it suffices to add the following code in nuxt.config.ts:
export default defineNuxtConfig({
devServer: {
host: '0.0.0.0',
port: 3001
},
...
So, calling 172.17.0.1 on port 3001 from a Dockerized Lambda will target 0.0.0.0:3001 on the host, hence communication will be successful.
Accessing AWS SAM API Gateway Using isolated Local Resources
Puppeteer can now correctly access my frontend website within a Docker encapsulation. However, a new problem arises: the page wants to access the same or another API Gateway to fetch some data required to generate a PDF. If you use the default address that AWS SAM runs the API Gateway on (usually localhost:3000), you will encounter a net::ERR_CONNECTION_REFUSED or 404 error - NOT FOUND.
This issue occurs because Puppeteer, running within Lambda, once again doesn't have access to your API running on localhost. To make your API Gateway accessible to Docker, you will have to change the way it's run. This means adding the host interface 0.0.0.0 and hosting the entire API on the same open address. Use the following command to start the API:
sam local start-api --container-host-interface '0.0.0.0' --host '0.0.0.0'
Next, in the frontend website, change the address invoking data fetching from the original API Gateway (localhost:3000) to the one displayed after starting the API, usually 192.168.0.101:XXXX or similar.
From now on, your Lambda, specifically, can access local resources, as well as local resources encapsulated in Lambda, can access Lambda.