Automatic restart of apache

[tobixen]

We have experienced three times that apache has been eating too much memory, last time the server got so bloated that it was impossible to do anything for a whole hour, until the OOM killer finally kicked in.

My ad-hoc workaround idea is to create a script and doing something like:

while true:
  if apache is bloated:
     restart apache
     send email about it
     sleep 5 minutes
  sleep 30 seconds

The script should probably run with negative nice value, or protected from swapping somehow.

The algorithm and treshold level for "apache is bloated" has to be refined.

The "restart apache"-part has to make sure we do "kill -9" on all pids that won't terminate cleanly; probably it's best to run "/etc/init.d/httpd restart", sleep for a while, and then do kill -9 on all old apache processes.

Of course, we should also investigate why apache eats so much, and get it fixed - but that's another ticket.

[anonymous]

apache bloat detection could be a bit hard. What do you think?

[tobixen]

I haven't given it that much a thought, but it shouldn't be that difficult to sum up the amount of memory used by apache.

Another idea (as commented in #73) is to periodically restart apache, i.e. 00:00 UTC every day.

Or a combination (the ordinary restart script to run every night, and the extraordinary restart script would hopefully never run).

Periodically restarts is a breeze to set up, I'll just throw it into /etc/cron.daily/ ... done.

I still think we should have that extraordinary script to actually check the memory consumption of apache.