From pnz.stalker at mail.ru Mon Dec 16 13:59:12 2019 From: pnz.stalker at mail.ru (pnz.stalker at mail.ru) Date: Mon, 16 Dec 2019 15:59:12 +0300 Subject: [mpm-itk] fork bomb or.. Message-ID: Hi! Need help. 2 servers with mpm-itk and apache 2.4.38-3+deb10u3/2.4.25-3+deb9u9 (debian buster and stretch). Some random time (1-2 times a month) i get over 500 processes apache2 with uid==www-data with 100% cpu usage According to the graphs of growth of requests to the server is not fixed at this moment 1)apache setting's: ServerLimit 40 StartServers 10 MinSpareServers 5 MaxSpareServers 25 MaxRequestWorkers 500 MaxConnectionsPerChild 0 2) for all users set limit's for processes and memory in /etc/security/limits.conf On old servers with apache 2.4.10 (Debian jessie) with some setting server's work fine From steinar+mpmitk at gunderson.no Mon Dec 16 14:03:04 2019 From: steinar+mpmitk at gunderson.no (Steinar H. Gunderson) Date: Mon, 16 Dec 2019 14:03:04 +0100 Subject: [mpm-itk] fork bomb or.. In-Reply-To: References: Message-ID: <20191216130304.yuwij4niwfavorbg@sesse.net> On Mon, Dec 16, 2019 at 03:59:12PM +0300, pnz.stalker at mail.ru wrote: > Some random time (1-2 times a month) i get over 500 processes apache2 > with uid==www-data with 100% cpu usage What are they spending that CPU usage on? Have you collected a profile? > 2) for all users set limit's for processes and memory in > /etc/security/limits.conf mpm-itk doesn't call PAM after doing setuid(), so pam_limits doesn't apply, and limits.conf will thus not be read. /* Steinar */ -- Homepage: https://www.sesse.net/ From pnz.stalker at mail.ru Mon Dec 16 14:16:30 2019 From: pnz.stalker at mail.ru (pnz.stalker at mail.ru) Date: Mon, 16 Dec 2019 16:16:30 +0300 Subject: [mpm-itk] fork bomb or.. In-Reply-To: <20191216130304.yuwij4niwfavorbg@sesse.net> References: <20191216130304.yuwij4niwfavorbg@sesse.net> Message-ID: 16.12.2019 16:03, Steinar H. Gunderson ?????: >> Some random time (1-2 times a month) i get over 500 processes apache2 >> with uid==www-data with 100% cpu usage > What are they spending that CPU usage on? Have you collected a profile? I try attach to processes with starce.. empty output or like == pwrite64(238, "\334\334\337\337\343\343\0\0", 8, 249312) = 8 pwrite64(238, "\334\334\337\337\343\343\0\0", 8, 256248) = 8 === Load average on server's more that 300.. > >> 2) for all users set limit's for processes and memory in >> /etc/security/limits.conf > > mpm-itk doesn't call PAM after doing setuid(), so pam_limits doesn't apply, > and limits.conf will thus not be read. > > /* Steinar */ > From steinar+mpmitk at gunderson.no Mon Dec 16 14:22:48 2019 From: steinar+mpmitk at gunderson.no (Steinar H. Gunderson) Date: Mon, 16 Dec 2019 14:22:48 +0100 Subject: [mpm-itk] fork bomb or.. In-Reply-To: References: <20191216130304.yuwij4niwfavorbg@sesse.net> Message-ID: <20191216132248.u2k55mdwzuc42ncv@sesse.net> On Mon, Dec 16, 2019 at 04:16:30PM +0300, pnz.stalker at mail.ru wrote: >> What are they spending that CPU usage on? Have you collected a profile? > I try attach to processes with starce. strace is not a profiler. Try perf. /* Steinar */ -- Homepage: https://www.sesse.net/ From pnz.stalker at mail.ru Mon Dec 16 14:35:43 2019 From: pnz.stalker at mail.ru (pnz.stalker at mail.ru) Date: Mon, 16 Dec 2019 16:35:43 +0300 Subject: [mpm-itk] fork bomb or.. In-Reply-To: <20191216132248.u2k55mdwzuc42ncv@sesse.net> References: <20191216130304.yuwij4niwfavorbg@sesse.net> <20191216132248.u2k55mdwzuc42ncv@sesse.net> Message-ID: <974ba66a-793d-0401-0260-fc964ca89d1f@mail.ru> 16.12.2019 16:22, Steinar H. Gunderson ?????: > On Mon, Dec 16, 2019 at 04:16:30PM +0300, pnz.stalker at mail.ru wrote: >>> What are they spending that CPU usage on? Have you collected a profile? >> I try attach to processes with starce. > > strace is not a profiler. Try perf. OK. When the next occurrence of the problem, I?ll start perf. In log I see === AH00161: server reached MaxRequestWorkers setting, consider raising the MaxRequestWorkers setting === increase to more that 1000 not solve problem... and in top I see more that 1000 processes and more LA. And now I set loglevel to debug for apache...