Understanding API Gateway Authorizer Caching (REST)

When using API Gateway authorizers, specifically Lambda (custom) authorizers, customers often wish to use the built-in caching mechanism to save Lambda costs and reduce latency.

This article explains how the cache applies to different resources and methods in API Gateway REST APIs, something which is commonly misunderstood.

Setting up a Lambda authorizer cache

There are two types of custom authorizer:

  1. TOKEN - simplest option where you specify a header (normally Authorization) that is used to authenticate the request. This is usually a bearer JWT token.
  2. REQUEST - receives the caller’s identity in a combination of headers, query strings, stageVariables and $context variables (we will come onto why this is important later). Most of the information about the request will be passed to the Lambda function.

Both options can be seen side-by-side in the console in the below screenshot.

API GW Authorizer ComparisonComparison of API Gateway Lambda Authorizers

Both TOKEN and REQUEST have the option to enable Authorization Caching with a TTL from 0-3600 seconds. 3600 seconds (60 mins) is a hard limit that cannot be increased.

The cache key will be the defined Token Source or Identity Sources.

What is stored in the cache?

The Lambda authorizer function’s output, which includes the policyDocument used to determine whether the caller is allowed or denied.

{
  "principalId": "yyyyyyyy", // The principal user identification associated with the token sent by the client.
  "policyDocument": {
    "Version": "2012-10-17",
    "Statement": [
      {
        "Action": "execute-api:Invoke",
        "Effect": "Allow|Deny",
        "Resource": "arn:aws:execute-api:{regionId}:{accountId}:{apiId}/{stage}/{httpVerb}/[{resource}/[{child-resources}]]"
      }
    ]
  },
  "context": {
    "stringKey": "value",
    "numberKey": "1",
    "booleanKey": "true"
  },
  "usageIdentifierKey": "{api-key}"
}

For example, the below output denies calls to the GET method for the dev stage of API ymy8tbxw7b of account 123456789012:

{
  "principalId": "user",
  "policyDocument": {
    "Version": "2012-10-17",
    "Statement": [
      {
        "Action": "execute-api:Invoke",
        "Effect": "Deny",
        "Resource": "arn:aws:execute-api:us-west-2:123456789012:ymy8tbxw7b/dev/GET/"
      }
    ]
  }
}

What level is the cache applied?

This is a common misunderstanding. The cache operates per API stage (not per resource, or per method). For example the following calls will return the same cached authorizer response for the prod stage:

GET https://myapi.com/prod/pets/cats    [trigger Lambda, generate policy and store in cache for TTL]
GET https://myapi.com/prod/pets/cats    [return policy from cache]
GET https://myapi.com/prod/pets/dogs    [return policy from cache]
POST https://myapi.com/prod/pets/cats   [return policy from cache]

As a result you can only flush the entire API stage cache using the FlushStageAuthorizersCache API.

For example, using the AWS CLI: aws apigateway flush-stage-authorizers-cache --rest-api-id 1234123412 --stage-name dev

What is the common mistake?

The documentation says:

To enable caching, your authorizer must return a policy that is applicable to all methods across an API. To enforce method-specific policy, you can set the TTL value to zero to disable policy caching for the API.

Customers often just return an allow statement for the specific resource/method being called at that time. Because the cache will apply to the entire stage this policy will be returned for subsequent calls to different resources/method, which means an implicit deny is applied. This means that where customers expect an allow they may receive a deny!

The example Lambda code from the documentation uses the event.methodArn to generate the policy returned. This will cause a problem in the following scenario (assume user should be allowed to both cats and dogs resources):

  1. User calls https://myapi.com/prod/pets/cats
  2. Policy generated allows the user and is stored in the cache
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Action": "execute-api:Invoke",
      "Effect": "Allow",
      "Resource": "arn:aws:execute-api:eu-west-1:123456789012:s2gse9wr12/prod/GET/pets/cats"
    }
  ]
}
  1. User calls https://myapi.com/prod/pets/dogs
  2. User is implicitly denied by the cached policy as it does not allow the /pets/dogs resource.

If the cache is disabled the user would be allowed correctly, as the Lambda authorizer would dynamically generate the allow policy using the methodArn on every call.

How do I handle the cache correctly?

There are two options:

  1. Include all resources and methods for the API stage in the returned policyDocument. This can be done using wildcards where appropriate.
  2. Include httpMethod and resourcePath available in $context when configuring Identity Sources.

Tell me more about the $context caching key

REQUEST based authorizers support adding $context variables to the identity source. This allows you to use the httpMethod and resourcePath as part of the cache key and effectively create a cache per method, per resource.

REQUEST authorizer with context variables

Using these context variables our earlier example will look like this:

GET https://myapi.com/prod/pets/cats    [trigger Lambda, generate policy and store in cache for TTL]
GET https://myapi.com/prod/pets/cats    [return policy from cache]
GET https://myapi.com/prod/pets/dogs    [trigger Lambda, generate policy and store in cache for TTL]
POST https://myapi.com/prod/pets/cats   [trigger Lambda, generate policy and store in cache for TTL]

Can I only do that with REQUEST authorizers?

Yes. The $context caching key method is only supported for REQUEST authorizers. If you are using a TOKEN authorizer you will need to ensure you return a policyDocument that applies to subsequent resources/methods the client may call within the cache TTL.

One option is to explicitly list all the allowed/denied actions.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Action": "execute-api:Invoke",
      "Effect": "Allow",
      "Resource": "arn:aws:execute-api:eu-west-1:123456789012:s2gse9wr12/prod/GET/pets/cats"
    },
    {
      "Action": "execute-api:Invoke",
      "Effect": "Allow",
      "Resource": "arn:aws:execute-api:eu-west-1:123456789012:s2gse9wr12/prod/GET/pets/dogs"
    },
    {
      "Action": "execute-api:Invoke",
      "Effect": "Allow",
      "Resource": "arn:aws:execute-api:eu-west-1:123456789012:s2gse9wr12/prod/POST/pets/cats"
    }
  ]
}

Another option is to use wildcards within the policy statement. This is not as secure as the previous option as you are not explicitly considering what access the caller should have to each resource and method within your API. This could lead to unintentionally giving access to a new resource added to the API at a later date. It does, however, make the logic easier and will help avoid the 8 KB limit.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Action": "execute-api:Invoke",
      "Effect": "Allow",
      "Resource": "arn:aws:execute-api:eu-west-1:123456789012:s2gse9wr12/prod/GET/*"
    },
    {
      "Action": "execute-api:Invoke",
      "Effect": "Allow",
      "Resource": "arn:aws:execute-api:eu-west-1:123456789012:s2gse9wr12/prod/POST/pets/cats"
    }
  ]
}

For more information about valid policies for calling an API, see Statement reference of IAM policies for executing API in API Gateway