NGINX thundering herd problem

Solution 1: EPOLLEXCLUSIVE flag

If EPOLLEXCLUSIVE (Linux 4.5, glibc 2.24) is defined when compiling NGINX, NGINX could make use of it to reduce resource usage when volume of new connection is low.

NGINX discards EPOLLRDHUP if EPOLLEXCLUSIVE is enabled, keeping EPOLLIN and EPOLLOUT compatible with the CentOS 7 backport.

#if (NGX_HAVE_EPOLLEXCLUSIVE && NGX_HAVE_EPOLLRDHUP)
    if (flags & NGX_EXCLUSIVE_EVENT) {
        events &= ~EPOLLRDHUP;
    }
#endif

For every 16 requests handled, NGINX would re-add the socket in ngx_reorder_accept_events() to balance request across workers.

    if (c->requests++ % 16 != 0
        && ngx_accept_disabled <= 0)
    {
        return;
    }
 
    if (ngx_del_event(c->read, NGX_READ_EVENT, NGX_DISABLE_EVENT)
        == NGX_ERROR)
    {
        return;
    }
 
    if (ngx_add_event(c->read, NGX_READ_EVENT, NGX_EXCLUSIVE_EVENT)
        == NGX_ERROR)
    {
        return;
    }

Solution 2: listen reuseport

SO_REUSEPORT could significantly increase the max latency in a degraded state. See https://blog.cloudflare.com/the-sad-state-of-linux-socket-balancing/.

As a side effect, total length of pending connection queue (backlog) is multiplied by the number of workers in this case.