How to make Apache2 server faster - Part-01 [Theory]

Making Apache Suck Less -1

• Don't use apache default configurations for hosting python web applications.
• Don't use the prefork MPM unless you know how to configure Apache Properly,use the worker MPM, it is more forgiving.
• Don't allow apache to automatically scale out the number of processes over too great a range.
• Don't try and use a single Apache instance to host Python,PHP,Perl Web application at the same time.

Making Apache Suck Less -1

• Ensure MaxSpareServers is greater than MinSpareServers.If you don't,Apache will set MaxSpareServers to be MinSpareServers+1 for you anyway.
• Don't set StartServers to be less than MinSpareServers as it will delay start up of processes so as to reach minimum spare required.
• Don't set StartServers to be greater than MaxSpareServers as processes will start to be killed off immediately.

Overhead of Process Creation

• Initialization of Python interpreter
• Loading of the WSGI application
• Loading of required standard library modules
• Loading of required third party modules
• Initialization of the WSGI application

What MPM are you using?

$ apachectl -V | grep 'Server MPM'

or

$ /user/sbin/httpd -V | grep 'Server MPM'

Pre-loading is a security Risk

• You don't necessarily know which WSGI application to load until the first request arrives for it.
• Python web application aren't usually designed properly for preloading prior to forking of worker processes.
• All code run in the Apache parent process is run as root.

When using mod_wsgi five initialization is deferred until after the processes forked, any application code required python modules are then lazily loaded, if initializing python and loading wsgi application in the worker process can be expensive,

Why can't we preload everything in Apache parent process before the worker process were forked.

Even if a python web application were designed to be able to pre-loaded and run properly after the processes was forked the key issue that uses application ocuured on startup would run as root, if executed in the parent process and this is one very big security risk.

Preloading causes memory leaks

• The python interpreter will leak memory into the parent process when apache restart occurs.
• No Saving in memory usage from copy on write

when the interpreter is destroyed combined with the way in which Apache reloads mod_wsgi when restarting the python interpreter will leak memory into the Apache parent process if Apache restarts have done on a regular basis the size of the Apache process will grow over time and thus so will the forked worker processes effectively have a memory leak.

Handling large number of clients

• MaxClients 256

The only reason that the default for Apache specifies such a large value for max clients and thus a large number of processes when threading is used is because of slow clients and keep-alive, a high number is required to support concurrent session from many users if using single threaded processes this means, there should be much more memory available.

A much better solution is to put is Nginx proxy in front of Apache, the nginx server will isolate apache from slow clients as a leaning forwarded request, when it is completely available and can be handled immediately the nginx server can also handle keep-alive connections, then keep-alive can be turned off on apache, this will allow us to significantly reduce the number of processes needded for Apache to handle the same ammount of traffic as before.

Making Apache Suck Less - 3

• Set MaxSpareProcesses at a Level above the typical number of concurrent requests you would need to handle.
• Do not use MaxRequestsPerChild, especially at a low count which would cause frequent process churn.
• Remember that if you don't set MaxRequestsPerChild explicitly, it defaults to 10000.
• Use nginx as front-end proxy to isolate Apache from slow clients.
• Turn off Keep-Alive in apache when using nginx front-end.

Making Apache Such Less - 4

• Ensure MaxSpareThreads is at least MinSpareThread+ThreadsPerChild.If you don't, Apache will set it to that for you anyway.
• Suggested that MinSpareThreads and MaxSpareThreads be set as Multiples of ThreadsPerChild
• Don't set StartServers to be less thab MinSpareThreads/ThreadsPerChild as it will delay start up of processes so as to reach minimum spare required.
• Don't set StartServers to be greater than MaxSpareThreads/ThreadsPerChild as Processes will start to be killed off immediately.

Reducing Per Thread Memory Use

• MaxMemFree - Maximum amount of memory that the main allocator is allowed to hold without calling free()

MaxMemFree 256 # KB

2.4 --> MB

Even at 2 MB in Apache 2. this could mean that 25 threads >>> 50 MB can be hold by the persistent memory pools at eash process when running mod_wsgi and normal circumstances this should not be much call for memory to be allocated from the per request memory pool.

Making Apache Such Less -5

• Ensure that MaxMemFree is set and not left to be unbounded.
• Even on Apache2.4 where is 2MB, consider reducing the value further.

Daemon Mode of Apache/mod_wsgi

Now the configuration for frefork or worker MPM are princiapply an issue when using ehat is called embedded mode of mod_wsgi, that is wsgi application runs inside of the Apache server child processes, the dynamics scaling algorithm in Apache being what can cause us grief when doing this.

Use mod_wsgi Daemon mode in this case wsgi application runs in a separate set of managed proccesses the main difference when using daemon mode is that there is no automatic scaling on the number of processes, the number of processes and the thread is instead fixed, being fixed everything is more predictable and then you only need to ensure you have sufficient resources, using daemon mode the need to have nginx as frontend proxy is reduced as the apache server child worker process are serving much the same purpose in isolating wsgi application from the clients.

Because the apache server process is now only acting as proxy forwarding request to the mod_wsgi daemon mode process as well as serving static files we don't need to initialize the python interpreter in the Apache server processes.

Process creation is again lightweight and we have to sidestep the need to have to pay so much attention to the apache MPM settings.

Configuration :

WSGIDaemonProcess myapp processes=3 threads=5

WSGIScriptAlias / /some/path/wsgi.py  process-group=myapp application-group=%{GLOBAL}

Exclusively Using Daemon Mode:

• WSGIRestrictedEmbedded - Controls whether the Python interpreter is initialised in Apache server worker processes.

WSGIRestrictedEmbedded On

The thing that make Apache Suck

• An algorithm for dynamically scaling processes which isn't particularly suited to embedded Python web applications.
• Default MPM settings which magnify the issues which can arise with dynamic scaling when running python web applications.
• A concurrency mechanism that can use a lot of memory for a higher of concurrent requests,especially around handling of keep alive connections.
• Defaults for memory pool sizes which cause Apache to be heavyweight on memory usage

Conclusion :

• Use daemon mode.
• Disable embedded mode using WSGIRestrictEmbedded On.
• Use at most 5 threads per process, unless excessively I/O bound.
• Use processes over threads, but don't get carried away with processes either.
• Scale across multiple machines.
• Instrument the WSGI server and application so now how it performs for real system.

References:
https://youtu.be/k6Erh7oHvns