To enable keepalive connections to upstream, the keepalive directive must be included in upstream{} blocks, and in the location{} blocks you need to switch HTTP version to 1.1 and clear the default Connection: close request header set by NGINX. See Headers in NGINX.
proxy_http_version 1.1;
proxy_set_header "Connection" "";
It should be particularly noted that the
keepalivedirective does not limit the total number of connections to upstream servers that an nginx worker process can open. The_connections_parameter should be set to a number small enough to let upstream servers process new incoming connections as well.
Also note that if you have keepalive set in the upstream and max_conns set on a server,
multiple workers, and the shared memory are enabled, the total number of active and idle connections to the proxied server may exceed the
max_connsvalue.
So choose the number of keepalive connections carefully.
You may also want to customize keepalive_requests and keepalive_timeout, but the defaults are reasonable enough.
References
https://nginx.org/en/docs/http/ngx_http_upstream_module.html#keepalive