The Ops Community ⚙️: Tikam Alma

Deploy Backend Server Using Apache Webserver

Tikam Alma — Thu, 26 May 2022 04:52:13 +0000

How do the backend servers work?

What is a backend server?

The backend servers are the main component of any functional website which uses API, databases, and other features.
What the user sees on the browser is the frontend website and the processing of the login, signup, storing credentials, and data are done by backend servers on cloud services.
Frontend servers are rendered on the client-side, and backend servers are rendered on the server-side, and on the server, it processes all the complex functions and receives data from the database so that the frontend server can use it.

Hosting backend servers in any cloud system requires different configurations for each of the, there are different programming languages and the most popular programming languages which are used for writing backend servers are Python, Go, Javascript, PHP, RUBY Java.
Each language has its own framework to write backend servers, for python language Django framework is used widely, for javascript Node is used, for Go, Gin framework is used.

Dependency and required configurations for hosting backend servers

Different backend servers are built on different languages and different languages have their own frameworks and each framework has its own required dependency and tools to work on web servers.

Python - WSGI (Web Server Gateway Interface)

Building projects in Django creates auto-generated files by default which are the skeleton or the structure of the Django framework, wsgi.py is one such important file that is used to communicate with web servers. WSGI is Web Server Gateway Interface, it describes the communication between web servers and Python web application servers or frameworks.
WSGI (Web Server Gateway Interface) is a standard interface between web server software and web applications written in Python.
It covers how a web server connects with Python web applications/frameworks, as well as how web applications/frameworks may be chained together to execute a request.

The apache webservers are popular for their customization and multi-platform support, it has a specific module for Django servers, called mod_wsgi. The mod WSGI module provides a WSGI compliant interface for executing and running a python based web application server within apache.

Node - Proxy Module

In an apache web server, the node or javascript servers are hosted or run using the proxypass and reverse proxy method and its modules.

A reverse proxy serves as a link between the client and the server, forwarding requests to the main web server or application webservers. It manages the requests and responses between clients and servers and offers security and a high level of abstraction.

The ProxyPass directive defines how inbound requests are routed to the backend server (or a cluster of servers known as a Balancer group).

And another directive used is the ProxyPass directive, which defines how the inbound requests are sent to the Node/Javascript backend servers, it manages any requests or responses under the root URL (/) that should be sent or routes to the Node/Javascript backed servers localhost address or the root default address.

Hosting backend servers in the cloud

Django server configurations

Setup and install the Django dependencies and packages.

The basic requirements for hosting the Django server are its requiremnets.txt file in which all of the dependency packages are written and connecting it to the database and running its migrations.

For installing the dependency and packages, check the python version which the Django project application needs to be set and make a virtual environment upon which the Django server will run.

In any cloud service, Linux servers are mostly used, use these commands to check the python version and virtual environment.

Install all Python 3 packages. These will ensure we have everything for Python - Django framework application server.

Update the Linux/ubuntu system

$ sudo apt-get update && sudo apt-get dist-upgrade

$ sudo apt install -y python3-pip python3 python3-setuptools build-essential libssl-dev libffi-dev python3-dev python3-venv

Make a virtual environment

$ mkdir project_env
## Directory where environment files can be stored

$ cd project_env
$ python3 -m venv penv
## penv can be named anything

Activate virtulenv and it should look like this :

$ cd project_env
$ source penv/bin/activate
(penv)[user]$ / directory /

Install the packages inside the virtualenv

$ pip install package_name
OR
$ pip install -r requiremenrs.txt

Configuring WSGI.py

Django autogenerates the wsgi.py file inside the main directory of django alonside settings.py and other files.

Open the wsgi.py and configure the project directory path.


"""
WSGI config for lgraph project.

It exposes the WSGI callable as a module-level variable named ``application``.

For more information on this file, see
https://docs.djangoproject.com/en/2.0/howto/deployment/wsgi/
"""

import os
import sys
from django.core.wsgi import get_wsgi_application

## Add this line to add project directory.
sys.path.append('/home/your_project')

os.environ.setdefault("DJANGO_SETTINGS_MODULE", "project.settings")

application = get_wsgi_application()

Installing and enabling "Mod WSGI"

sudo apt-get update
sudo apt-get install libapache2-mod-wsgi-py3

Enabling the mod_wsgi module server wide.

$ a2enmod wsgi

The Django doundation has offical docmentation about how to used mod_wsgi with django - https://docs.djangoproject.com/en/4.0/howto/deployment/wsgi/modwsgi/

Running and testing Django server in localhost

After setting up the django application, now start the server, check if everything is working, start the django server with these commands:

To migrate the database, get static files ready and to test run the Django application server use these commands:

$ python manage.py makemigrations
$ python manage.py migrate
$ python manage.py collectstatic
$ python manage.py runserver 0.0.0.0:8000

Node server configuration

Setup Setup and install the Node dependencies and packages

Running and testing node server in localhost

Install and enable proxy_modules

Configuring apache virtualhost to serve the backend servers.

Configuring Virtualhost for Django Backend Server

After enabling the mod_wsgi, now it's time to configure the virtualhost to deploy the application server to expose it to internet

enable mod wsgi

$ a2enmod wsgi

Open the /etc/apache2/site-available directory and edit the project's configuration file.

$ cd /etc/apache2/sites-available
$ /etc/apache2/sites-available / ~ ls
$ ls

$ nano django-project.conf
OR 
$ vim django-project.conf

Configure and add the following line:

WSGIScriptAlias
WSGIDaemonProcess
WSGIProcessGroup

<VirtualHost *:80> 
 ServerName mysite.example.com 
 DocumentRoot /var/www/vhosts/mysite 
 WSGIScriptAlias / /var/www/vhosts/mysite/myproject/wsgi.py 

 # adjust the following line to match your Python path 
 WSGIDaemonProcess mysite.example.com processes=2 threads=15 display-name=%{GROUP} python-home=/var/www/vhosts/mysite/venv/lib/python3.5 
 WSGIProcessGroup mysite.example.com 

 <directory /var/www/vhosts/mysite> 
   AllowOverride all 
   Require all granted 
   Options FollowSymlinks 
 </directory> 

 Alias /static/ /var/www/vhosts/mysite/static/ 

 <Directory /var/www/vhosts/mysite/static> 
  Require all granted 
 </Directory> 
</VirtualHost>

Configuring Virtualhost for Node Backend Server

Dry run and testing the live servers

To check the application server is running or not

Check the apache webserver status
Check the application server status
Curl localhost:80
If in the cloud get the public IP and open the web application using the IP or domain set in the configuration file.

How to make Apache2 server faster - Part-02 [Implementation]

Tikam Alma — Wed, 25 May 2022 21:20:34 +0000

I will explain here how to increase latency of apache2 server,using Django deployment example.At the end of the article you'll get to know that this can be applies to any backend server be it Django or node server.

Installing mod_wsgi the easy way

pip install mod_wsgi

mod_wsgi - module for apache web server

No Apache Configuration Required

$ mod_wsgi-express start-server hello.wsgi

Django Framework Integration

INSTALLED_APPS = [

'django.contrib.admin',

'django.contrib.auth',

.....

'mod_wsgi.server',

]

Send logging to terminal

settings.py

LOGGING = {
    #  ... omitting the formatters and handlers for brevity ...
    'loggers': {
        # ...  you may have other loggers here as well ...
        'django': {
            'handlers': ['name_of_your_file_handler_goes_here'],
            'level': 'DEBUG',
            'propagate': True,
        }
    }

LOGGING = {
        'version':1,
        'disable_existing_loggers': False,
        'handlers': {
                'console': {
                        'class' : 'logging.StreamHandler',
                 },
            },
            'loggers' : {
                    'django' : {
                            'handlers' : ['console'],
                            'level' : os.getenv('DJANGO_LOG_LEVEL','INFO'),
                    },
                },
    }

Run Django With Management command

python [manage.py](http://manage.py) runmodwsgi

Automatic code reloading

python [manage.py](http://manage.py) runmodwsgi —reload-on-changes

Django DEBUG settings

settings.py

if os.environ.get('MOD_WSGI_DEBUG_MODE'):
        DEBUG = True

if os.environ.get('MOD_WSGI_DEBUGGER_ENABLED'):
        DEBUG_PROPAGATE_EXCEPTIONS = True

Interactive Debugger

python [manage.py](http://manage.py) runmodwsgi —enable-debugger

Production configuration

python [manage.py](http://manage.py) runmodwsgi \
--server-root /etc/wsgi-port-80  \
--user www-data --group www-data\
--port 80  --setup-only

/etc/wsgi-port-80/apachectl start 
/etc/wsgi-port-80/apachectl restart 
/etc/wsgi-port-80/apachectl stop

Use daemon mode of mod_wsgi

WSGIRestrictedEmbedded On
WSGIDaemonProcess myapp python-home=/../env

WSGIScriptAlias / /../src/wsgi.py \ 
        process-group=myapp application-group=%{GLOBAL}

<Directory /../src>
        <Files wsgi.py>
        Require all granted
        </Files>
</Directory>

Just add whatever the backend server Django or Node anything, putting this above virtualhost on line-1 will boost your latency.

WSGIRestrictedEmbedded On

Resources:

https://youtu.be/H6Q3l11fjU0

mod_wsgi - mod_wsgi 4.8.0 documentation

How to make Apache2 server faster - Part-01 [Theory]

Tikam Alma — Wed, 25 May 2022 21:19:06 +0000

Making Apache Suck Less -1

Don't use apache default configurations for hosting python web applications.
Don't use the prefork MPM unless you know how to configure Apache Properly,use the worker MPM, it is more forgiving.
Don't allow apache to automatically scale out the number of processes over too great a range.
Don't try and use a single Apache instance to host Python,PHP,Perl Web application at the same time.

Making Apache Suck Less -1

Ensure MaxSpareServers is greater than MinSpareServers.If you don't,Apache will set MaxSpareServers to be MinSpareServers+1 for you anyway.
Don't set StartServers to be less than MinSpareServers as it will delay start up of processes so as to reach minimum spare required.
Don't set StartServers to be greater than MaxSpareServers as processes will start to be killed off immediately.

Overhead of Process Creation

Initialization of Python interpreter
Loading of the WSGI application
Loading of required standard library modules
Loading of required third party modules
Initialization of the WSGI application

What MPM are you using?

$ apachectl -V | grep 'Server MPM'

$ /user/sbin/httpd -V | grep 'Server MPM'

Pre-loading is a security Risk

You don't necessarily know which WSGI application to load until the first request arrives for it.
Python web application aren't usually designed properly for preloading prior to forking of worker processes.
All code run in the Apache parent process is run as root.

When using mod_wsgi five initialization is deferred until after the processes forked, any application code required python modules are then lazily loaded, if initializing python and loading wsgi application in the worker process can be expensive,

Why can't we preload everything in Apache parent process before the worker process were forked.

Even if a python web application were designed to be able to pre-loaded and run properly after the processes was forked the key issue that uses application ocuured on startup would run as root, if executed in the parent process and this is one very big security risk.

Preloading causes memory leaks

The python interpreter will leak memory into the parent process when apache restart occurs.
No Saving in memory usage from copy on write

when the interpreter is destroyed combined with the way in which Apache reloads mod_wsgi when restarting the python interpreter will leak memory into the Apache parent process if Apache restarts have done on a regular basis the size of the Apache process will grow over time and thus so will the forked worker processes effectively have a memory leak.

Handling large number of clients

MaxClients 256

The only reason that the default for Apache specifies such a large value for max clients and thus a large number of processes when threading is used is because of slow clients and keep-alive, a high number is required to support concurrent session from many users if using single threaded processes this means, there should be much more memory available.

A much better solution is to put is Nginx proxy in front of Apache, the nginx server will isolate apache from slow clients as a leaning forwarded request, when it is completely available and can be handled immediately the nginx server can also handle keep-alive connections, then keep-alive can be turned off on apache, this will allow us to significantly reduce the number of processes needded for Apache to handle the same ammount of traffic as before.

Making Apache Suck Less - 3

Set MaxSpareProcesses at a Level above the typical number of concurrent requests you would need to handle.
Do not use MaxRequestsPerChild, especially at a low count which would cause frequent process churn.
Remember that if you don't set MaxRequestsPerChild explicitly, it defaults to 10000.
Use nginx as front-end proxy to isolate Apache from slow clients.
Turn off Keep-Alive in apache when using nginx front-end.

Making Apache Such Less - 4

Ensure MaxSpareThreads is at least MinSpareThread+ThreadsPerChild.If you don't, Apache will set it to that for you anyway.
Suggested that MinSpareThreads and MaxSpareThreads be set as Multiples of ThreadsPerChild
Don't set StartServers to be less thab MinSpareThreads/ThreadsPerChild as it will delay start up of processes so as to reach minimum spare required.
Don't set StartServers to be greater than MaxSpareThreads/ThreadsPerChild as Processes will start to be killed off immediately.

Reducing Per Thread Memory Use

MaxMemFree - Maximum amount of memory that the main allocator is allowed to hold without calling free()

MaxMemFree 256 # KB

2.4 --> MB

Even at 2 MB in Apache 2. this could mean that 25 threads >>> 50 MB can be hold by the persistent memory pools at eash process when running mod_wsgi and normal circumstances this should not be much call for memory to be allocated from the per request memory pool.

Making Apache Such Less -5

Ensure that MaxMemFree is set and not left to be unbounded.
Even on Apache2.4 where is 2MB, consider reducing the value further.

Daemon Mode of Apache/mod_wsgi

Now the configuration for frefork or worker MPM are princiapply an issue when using ehat is called embedded mode of mod_wsgi, that is wsgi application runs inside of the Apache server child processes, the dynamics scaling algorithm in Apache being what can cause us grief when doing this.

Use mod_wsgi Daemon mode in this case wsgi application runs in a separate set of managed proccesses the main difference when using daemon mode is that there is no automatic scaling on the number of processes, the number of processes and the thread is instead fixed, being fixed everything is more predictable and then you only need to ensure you have sufficient resources, using daemon mode the need to have nginx as frontend proxy is reduced as the apache server child worker process are serving much the same purpose in isolating wsgi application from the clients.

Because the apache server process is now only acting as proxy forwarding request to the mod_wsgi daemon mode process as well as serving static files we don't need to initialize the python interpreter in the Apache server processes.

Process creation is again lightweight and we have to sidestep the need to have to pay so much attention to the apache MPM settings.

Configuration :

WSGIDaemonProcess myapp processes=3 threads=5

WSGIScriptAlias / /some/path/wsgi.py  process-group=myapp application-group=%{GLOBAL}

Exclusively Using Daemon Mode:

WSGIRestrictedEmbedded - Controls whether the Python interpreter is initialised in Apache server worker processes.

WSGIRestrictedEmbedded On

The thing that make Apache Suck

An algorithm for dynamically scaling processes which isn't particularly suited to embedded Python web applications.
Default MPM settings which magnify the issues which can arise with dynamic scaling when running python web applications.
A concurrency mechanism that can use a lot of memory for a higher of concurrent requests,especially around handling of keep alive connections.
Defaults for memory pool sizes which cause Apache to be heavyweight on memory usage

Conclusion :

Use daemon mode.
Disable embedded mode using WSGIRestrictEmbedded On.
Use at most 5 threads per process, unless excessively I/O bound.
Use processes over threads, but don't get carried away with processes either.
Scale across multiple machines.
Instrument the WSGI server and application so now how it performs for real system.

References:
https://youtu.be/k6Erh7oHvns

Harden Apache2 Server

Tikam Alma — Wed, 25 May 2022 20:23:54 +0000

1. Install Mod_Security2

sudo apt install libapache2-mod-security2 -y

Alternatively install from Github official repository

[https://github.com/SpiderLabs/ModSecurity(https://github.com/SpiderLabs/ModSecurity)

After installing ModSecurity, enable the Apache 2 headers module :

sudo a2enmod headers

After installing ModSecurity and enabling the header module,restart the apache2 service :

    sudo systemctl restart apache2

2. Get OWASP CRS and Configure it

ModSecurity is a firewall and therefore requires rules to function.

So we add the OWASP's CRS - Core Rule Set to harden our server

Remove the default .recommended extension from the ModSecurity configuration file name

sudo cp /etc/modsecurity/modsecurity.conf-recommended /etc/modsecurity/modsecurity.con

With a text editor such as vim, open /etc/modsecurity/modsecurity.conf and change the value for SecRuleEngine to On:

sudo nano /etc/modsecurity/modsecurity.conf

Add this

SecRuleEngine On

Restart Server

sudo systemctl restart apache2

The OWASP ModSecurity Core Rule Set (CRS) is a set of generic attack detection rules for use with ModSecurity or compatible web application firewalls. The CRS aims to protect web applications from a wide range of attacks, including the OWASP Top Ten, with a minimum of false alerts. The CRS provides protection against many common attack categories, including SQL Injection, Cross Site Scripting, and Local File Inclusion.

To set up the OWASP-CRS, follow the procedures outlined below.

First, delete the current rule set that comes prepackaged with ModSecurity by running the following command:
```
sudo rm -rf /usr/share/modsecurity-crs
```
Ensure that git is installed:
```
sudo apt install git
```

Clone the OWASP-CRS GitHub repository into the /usr/share/modsecurity-crs directory:

sudo git clone https://github.com/coreruleset/coreruleset /usr/share/modsecurity-crs

Rename the crs-setup.conf.example to crs-setup.conf:

sudo mv /usr/share/modsecurity-crs/crs-setup.conf.example /usr/share/modsecurity-crs/crs-setup.conf

Rename the default request exclusion rule file:

sudo mv /usr/share/modsecurity-crs/rules/REQUEST-900-EXCLUSION-RULES-BEFORE-CRS.conf.example /usr/share/modsecurity-crs/rules/REQUEST-900-EXCLUSION-RULES-BEFORE-CRS.conf

You should now have the OWASP-CRS setup and ready to be used in your Apache configuration.

3. Configure your server to use Mod_security

Edit the /etc/apache2/mods-available/security2.conf file to include the OWASP-CRS

cd /etc/apache2/mods-available/security2.conf

<IfModule security2_module>
        SecDataDir /var/cache/modsecurity
        Include /usr/share/modsecurity-crs/crs-setup.conf
        Include /usr/share/modsecurity-crs/rules/*.conf
</IfModule>

Include the SecRuleEngine directive set to On.

cd to /etc/apache2/your.website.conf

<VirtualHost *:80>
        ServerAdmin webmaster@localhost
        DocumentRoot /var/www/html

        ErrorLog ${APACHE_LOG_DIR}/error.log
        CustomLog ${APACHE_LOG_DIR}/access.log combined

        SecRuleEngine On
</VirtualHost>

Or

<IfModule security2_module>
SecRuleEngine On
</IfModule>

Restart the apache2 service to apply the configuration

sudo systemctl restart apache2

Testing ModSecurity

curl http://<SERVER-IP/DOMAIN>/index.php?exec=/bin/bash

You'll get 404

4. Enabling HTTP Policy and Paranoia

cd to /usr/share/modsecurity-crs

$ cd /usr/share/modsecurity-crs
or
$ sudo nano /usr/share/modsecurity-crs/crs-setup.conf

Change Paranoia Level on Paranoia Level Initialization section.

Find this section on

/usr/share/modsecurity-crs/crs-setup.conf

[[ Paranoia Level Initialization ]]

The Paranoia Level (PL) setting allows you to choose the desired level of rule checks that will add to your anomaly scores.With each paranoia level increase, the CRS enables additional rules giving you a higher level of security. However, higher paranoia levels also increase the possibility of blocking some legitimate traffic due to false alarms (also named false positives or FPs). If you use higher paranoia levels, it is likely that you will need to add some exclusion rules for certain requests and applications receiving complex input.

A paranoia level of 1 is default.

In this level, most core rules are enabled. PL1 is advised for beginners, installations covering many different sites and applications, and for setups with standard security requirements.At PL1 you should face FPs rarely. If you encounter FPs, please open an issue on the CRS GitHub site and don't forget to attach your complete Audit Log record for the request with the issue.

Paranoia level 2

This level Includes many extra rules, for instance enabling many regexp-based SQL and XSS injection protections, and adding extra keywords checked for code injections. PL2 is advised for moderate to experienced users desiring more complete coverage and for installations with elevated security requirements. PL2 comes with some FPs which you need to handle.

Paranoia level 3

It enables more rules and keyword lists, and tweaks limits on special characters used. PL3 is aimed at users experienced at the handling of FPs and at installations with a high security requirement.

Paranoia level 4

It further restricts special characters.The highest level is advised for experienced users protecting installations with very high security requirements. Running PL4 will likely produce a very high number of FPs which have to be treated before the site can go productive.

SecAction \
"id:900000,\
phase:1,\
nolog,\
pass,\
t:none,\
setvar:tx.paranoia_level=1"

5. HTTP Policy Settings on HTTP Policy Settings Section

$ sudo nano /usr/share/modsecurity-crs/crs-setup.conf

Section on - [[ HTTP Policy Settings ]]

This section defines your policies for the HTTP protocol, such as: allowed HTTP versions, HTTP methods, allowed request Content-Types forbidden file extensions (e.g. .bak, .sql) and request headers (e.g. Proxy)

These variables are used in the following rule files:

REQUEST-911-METHOD-ENFORCEMENT.conf

REQUEST-912-DOS-PROTECTION.conf

REQUEST-920-PROTOCOL-ENFORCEMENT.conf

HTTP methods that a client is allowed to use.

Default: GET HEAD POST OPTIONS

Example: for RESTful APIs, add the following methods: PUT PATCH DELETE

Example: for WebDAV, add the following methods: CHECKOUT COPY DELETE LOCK

MERGE MKACTIVITY MKCOL MOVE PROPFIND PROPPATCH PUT UNLOCK

Uncomment this rule to change the default.

SecAction \
"id:900200,\
phase:1,\
nolog,\
pass,\
t:none,\
setvar:'tx.allowed_methods=GET HEAD POST OPTIONS PUT PATCH DELETE'"

Configuring Domains and Sub-Domains in Apache webserver.

Tikam Alma — Wed, 25 May 2022 20:00:56 +0000

What are the Sub-domains?Why are Domains and Sub-Domains required?

Domains are well website name or the identity or a website, that one buys from a domain registrar website like google or GoDaddy and that is the address of your personal or professional website.
And subdomains are the subset or part of a personal or professional website, where users can add and extend the main domain's functions and features.
For example, a user wants to start his Food tech business and also wants to write blogs about food, and also wants to tracks the business transactions of his business.

Each feature has different functionality on the application side, the calculation, tracking, and analysis on the dashboard, the text editor and content management for the blog, etc.

Here, comes the subdomains part, why we need it.
The user will create his home page on a different directory, blog features have different code and stores on the different directory and same for dashboard also, and now the apache webserver will be able to manage all these directories and able to handle the requests on domains and sub-domains using configuration on the virtual host.

Configuring VirtualHost to Enable Domains and Sub-Domains

Change directory to the /etc/apache2/ and select the configuration file which you want to edit for adding subdomains.
There must be two files one is my-domain.conf and the other is default configuration 000-default.conf

$ sudo vi /etc/apache2/sites-available/my-domain.conf
OR 
$ sudo vi /etc/apache2/sites-available/000-default.conf

Add Wildcard Subdomain

Home Page - mysite.com (Main Domain)

 <VirtualHost *:80>
   DocumentRoot /var/www/home
   ServerName mysite.com
 </VirtualHost>

Blog Page - blog.mysite.com (Sub-Domain)

<VirtualHost *:80>
   DocumentRoot /var/www/blog
   ServerName blog.mysite.com
 </VirtualHost>

Dashboard - dashboard.mysite.com (Sub-domain)

<VirtualHost *:80>
   DocumentRoot /var/www/dash
   ServerName dashboard.mysite.com
 </VirtualHost>

Wildcard Domain

<VirtualHost *:80>
     ServerAlias *.mysite.com
     DocumentRoot /var/www/html/wildcard/
 </VirtualHost>

Final Configuration

## HomePage
 <VirtualHost *:80>
   DocumentRoot /var/www/home
   ServerName mysite.com
 </VirtualHost>

## Blog Page
<VirtualHost *:80>
   DocumentRoot /var/www/blog
   ServerName blog.mysite.com
 </VirtualHost>

## Dashboard Page
<VirtualHost *:80>
   DocumentRoot /var/www/dash
   ServerName dashboard.mysite.com
 </VirtualHost>

## Wild Card
<VirtualHost *:80>
     ServerAlias *.mysite.com
     DocumentRoot /var/www/html/wildcard/
 </VirtualHost>

Testing and Dry Run the Configuration

Check the configuration by running the command:

sudo apachectl configtest

Reload and Restart the server

# Reload
$ sudo service apache2 reload

# Restart
$ sudo service apache2 restart