My issue: We are running nginx as an API gateway in a multi-layer caching infrastructure. API calls from clients have to pass trough 2 layers of cache, namely:
- a CDN operated by cloudflare
- Nginx acting as API gateway
The cache feature on nginx is missing the Age header when serving assets from its cache, not respecting RFC 7234 section 4.2.3.
As per the RFC, this header helps cloudflare (downstream client) to calculate the time left to cache the assets on its side. However, this header not being present causes sometimes cloudflare to cache the assets for twice the max-age set in the Cache-Control header.
Nginx also does not take into account the Age header it receives from upstream when calculating the left max-age to cache assets.
How I encountered the problem:
Some assets on cloudflare could be cached for a maximum of twice the max-age when the Age header is missing and depending on when cloudflare is asking for the assets to nginx.
Nginx as well might store the assets loner than expected, not considering upstream Àge` header which tells him how long the upstream cache stored the asset already.
Solutions I’ve tried: N/A
Version of NGINX or NGINX adjacent software (e.g. NGINX Gateway Fabric): 1.27
Deployment environment:
Docker compose environment
Summary
services:
curl:
image: nicolaka/netshoot
restart: always
command: sleep 10000
nginx:
image: "nginx:1.27"
volumes:
- ./nginx/nginx.conf:/etc/nginx/nginx.conf
ports:
- "8080:8080"
upstream:
image: ealen/echo-server:0.9.2
environment:
PORT: 80
Nginx configuration
Summary
pid /tmp/nginx.pid;
worker_processes 1;
worker_rlimit_nofile 8192;
events {
worker_connections 4096;
}
http {
include /etc/nginx/mime.types;
index index.html index.htm index.php;
log_format custom '$remote_addr - $remote_user [$time_local] '
'"$request" $status $body_bytes_sent '
'"$http_referer" "$http_user_agent" '
'"cache:$upstream_cache_status" '
'"age:$upstream_http_age"';
proxy_cache_path /etc/nginx/cache keys_zone=mycache:10m;
server {
listen 80;
server_name _;
access_log /dev/stdout custom;
proxy_cache mycache;
location / {
proxy_pass http://upstream;
add_header 'x-cache-status' $upstream_cache_status always;
}
}
}
#1 Nginx caching the asset and not sending Age header
Using the curl container we issue the following commands
$ curl http://nginx -X GET -I -H 'X-ECHO-HEADER: cache-control:public,max-age=10'
HTTP/1.1 200 OK
Server: nginx/1.27.5
Date: Fri, 01 Aug 2025 14:31:16 GMT
Content-Type: application/json; charset=utf-8
Content-Length: 536
Connection: keep-alive
cache-control: public,max-age=10
ETag: W/"218-YdPQfL3An/Yz++FIFyecImCtdIk"
x-cache-status: MISS
$ curl http://nginx -X GET -I -H 'X-ECHO-HEADER: cache-control:public,max-age=10'
HTTP/1.1 200 OK
Server: nginx/1.27.5
Date: Fri, 01 Aug 2025 14:31:18 GMT
Content-Type: application/json; charset=utf-8
Content-Length: 536
Connection: keep-alive
cache-control: public,max-age=10
ETag: W/"218-YdPQfL3An/Yz++FIFyecImCtdIk"
x-cache-status: HIT
nginx-1 | 172.19.0.4 - - [01/Aug/2025:14:31:16 +0000] "GET / HTTP/1.1" 200 536 "-" "curl/8.14.1" "cache:MISS" "age:-"
nginx-1 | 172.19.0.4 - - [01/Aug/2025:14:31:18 +0000] "GET / HTTP/1.1" 200 536 "-" "curl/8.14.1" "cache:HIT" "age:-"
Nginx is not sending any Age header to the downstream during a HIT. We have no way of knowing how long the asset’s time has left.
#2 Nginx caching the asset and not respecting upstream Age header
Using the curl container we issue the following commands
$ curl http://nginx -X GET -I -H 'X-ECHO-HEADER: cache-control:public,max-age=10, age:9'
HTTP/1.1 200 OK
Server: nginx/1.27.5
Date: Fri, 01 Aug 2025 14:34:43 GMT
Content-Type: application/json; charset=utf-8
Content-Length: 543
Connection: keep-alive
cache-control: public,max-age=10
age: 9
ETag: W/"21f-CNUY6UVipFWmDQJaPkebkFa2DD4"
x-cache-status: EXPIRED
$ curl http://nginx -X GET -I -H 'X-ECHO-HEADER: cache-control:public,max-age=10, age:9'
HTTP/1.1 200 OK
Server: nginx/1.27.5
Date: Fri, 01 Aug 2025 14:34:44 GMT
Content-Type: application/json; charset=utf-8
Content-Length: 543
Connection: keep-alive
cache-control: public,max-age=10
age: 9
ETag: W/"21f-CNUY6UVipFWmDQJaPkebkFa2DD4"
x-cache-status: HIT
# Sent 5s later
$ curl http://nginx -X GET -I -H 'X-ECHO-HEADER: cache-control:public,max-age=10, age:9'
HTTP/1.1 200 OK
Server: nginx/1.27.5
Date: Fri, 01 Aug 2025 14:34:49 GMT
Content-Type: application/json; charset=utf-8
Content-Length: 543
Connection: keep-alive
cache-control: public,max-age=10
age: 9
ETag: W/"21f-CNUY6UVipFWmDQJaPkebkFa2DD4"
x-cache-status: HIT
nginx-1 | 172.19.0.4 - - [01/Aug/2025:14:34:43 +0000] "GET / HTTP/1.1" 200 543 "-" "curl/8.14.1" "cache:EXPIRED" "age:9"
nginx-1 | 172.19.0.4 - - [01/Aug/2025:14:34:44 +0000] "GET / HTTP/1.1" 200 543 "-" "curl/8.14.1" "cache:HIT" "age:9"
nginx-1 | 172.19.0.4 - - [01/Aug/2025:14:34:49 +0000] "GET / HTTP/1.1" 200 543 "-" "curl/8.14.1" "cache:HIT" "age:9"
Here even if we issue the 3rd curl 5s after the 2nd curl we still have a HIT. However nginx should have computed the Age header sent by the upstream and only cached the assets for 10 - 9 = 1s.
NGINX access/error log: (Tip → You can usually find the logs in the /var/log/nginx directory.) N/A
.