The Ops Community ⚙️: Daniel Pepuho

Automating Flask & PostgreSQL Deployment on KVM with Terraform & Ansible

Daniel Pepuho — Tue, 10 Dec 2024 22:48:38 +0000

😀 Intro

Hi, in this post, we will use Libvirt with Terraform to provision 2 KVM locally and after that, we will Deploy Flask App & PostgreSQL using Ansible.

Content

Project Architecture
Requirements
Create KVM
Create Ansible Playbook
- Playbook to install Docker
- Playbook to install and configure postgresql
- Playbook to deploy Flask App
- Run Playbook and testing
Conclusion

📝 Project Architecture

So we will create 2 VMs using Terraform, then deploying a flask project and the database using Ansible.

🔨 Requirements

I used Ubuntu 22.04 LTS as the OS for this project. If you're using a different OS, please make the necessary adjustments when installing the required dependencies.

The major pre-requisite for this setup is KVM hypervisor. So you need to install KVM in your system. If you use Ubuntu you can follow this step:

$ sudo apt -y install bridge-utils cpu-checker libvirt-clients libvirt-daemon qemu qemu-kvm

Execute the following command to make sure your processor supports virtualisation capabilities:

$ kvm-ok

INFO: /dev/kvm exists
KVM acceleration can be used

Install Terraform

$ wget -O - https://apt.releases.hashicorp.com/gpg | sudo gpg --dearmor -o /usr/share/keyrings/hashicorp-archive-keyring.gpg
$ echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/hashicorp-archive-keyring.gpg] https://apt.releases.hashicorp.com $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/hashicorp.list
$ sudo apt update && sudo apt install terraform -y

Verify installation:

$ terraform version

Terraform v1.9.8
on linux_amd64

Install Ansible

$ sudo apt update
$ sudo apt install software-properties-common
$ sudo add-apt-repository --yes --update ppa:ansible/ansible
$ sudo apt install ansible -y

Verify installation:

$ ansible --version

ansible [core 2.15.1]
...

Create KVM

we will use the libvirt provider with Terraform to deploy a KVM Virtual Machine.

Create main.tf, just specify the provider and version you want to use:

terraform {
  required_providers {
    libvirt = {
      source = "dmacvicar/libvirt"
      version = "0.8.1"
    }
  }
}

provider "libvirt" {
  uri = "qemu:///system"
}

Thereafter, run terraform init command to initialize the environment:

$ terraform init

Initializing the backend...
Initializing provider plugins...
- Reusing previous version of hashicorp/template from the dependency lock file
- Reusing previous version of dmacvicar/libvirt from the dependency lock file
- Reusing previous version of hashicorp/null from the dependency lock file
- Using previously-installed hashicorp/template v2.2.0
- Using previously-installed dmacvicar/libvirt v0.8.1
- Using previously-installed hashicorp/null v3.2.3

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.

Now create our variables.tf. This variables.tf file defines inputs for the libvirt disk pool path, the Ubuntu 20.04 image URL as OS for the VMs , and a list of VM hostnames.

variable "libvirt_disk_path" {
  description = "path for libvirt pool"
  default     = "default"
}

variable "ubuntu_20_img_url" {
  description = "ubuntu 20.04 image"
  default     = "https://cloud-images.ubuntu.com/releases/focal/release/ubuntu-20.04-server-cloudimg-amd64.img"
}

variable "vm_hostnames" {
  description = "List of VM hostnames"
  default     = ["vm1", "vm2"]
}

Let's update our main.tf:

resource "null_resource" "cache_image" {
  provisioner "local-exec" {
    command = "wget -O /tmp/ubuntu-20.04.qcow2 ${var.ubuntu_20_img_url}"
  }
}

resource "libvirt_volume" "base" {
  name   = "base.qcow2"
  source = "/tmp/ubuntu-20.04.qcow2"
  pool   = var.libvirt_disk_path
  format = "qcow2"
  depends_on = [null_resource.cache_image]
}

resource "libvirt_volume" "ubuntu20-qcow2" {
  count          = length(var.vm_hostnames)
  name           = "ubuntu20-${count.index}.qcow2"
  base_volume_id = libvirt_volume.base.id
  pool           = var.libvirt_disk_path
  size           = 10737418240  # 10GB
}

data "template_file" "user_data" {
  count    = length(var.vm_hostnames)
  template = file("${path.module}/config/cloud_init.yml")
}

data "template_file" "network_config" {
  count    = length(var.vm_hostnames)
  template = file("${path.module}/config/network_config.yml")
}

resource "libvirt_cloudinit_disk" "commoninit" {
  count          = length(var.vm_hostnames)
  name           = "commoninit-${count.index}.iso"
  user_data      = data.template_file.user_data[count.index].rendered
  network_config = data.template_file.network_config[count.index].rendered
  pool           = var.libvirt_disk_path
}

resource "libvirt_domain" "domain-ubuntu" {
  count  = length(var.vm_hostnames)
  name   = var.vm_hostnames[count.index]
  memory = "1024"
  vcpu   = 1

  cloudinit = libvirt_cloudinit_disk.commoninit[count.index].id

  network_interface {
    network_name   = "default"
    wait_for_lease = true
    hostname       = var.vm_hostnames[count.index]
  }

  console {
    type        = "pty"
    target_port = "0"
    target_type = "serial"
  }

  console {
    type        = "pty"
    target_type = "virtio"
    target_port = "1"
  }

  disk {
    volume_id = libvirt_volume.ubuntu20-qcow2[count.index].id
  }

  graphics {
    type        = "spice"
    listen_type = "address"
    autoport    = true
  }
}

the script will provisions multiple KVM VMs using the Libvirt provider. It downloads an Ubuntu 20.04 base image, clones it for each VM, configures cloud-init for user and network settings, and deploys VMs with specified hostnames, 1GB memory, and SPICE graphics. The setup dynamically adapts based on the number of hostnames provided in var.vm_hostnames.

As I've mentioned, I'm using cloud-init, so lets setup the network config and cloud init under the config directory:

$ mkdir config/

Then create our config/cloud_init.yml, just make sure that you configure your public ssh key for ssh access in the config:

#cloud-config
runcmd:
  - sed -i '/PermitRootLogin/d' /etc/ssh/sshd_config
  - echo "PermitRootLogin yes" >> /etc/ssh/sshd_config
  - systemctl restart sshd
ssh_pwauth: true
disable_root: false
chpasswd:
  list: |
    root:cloudy24
  expire: false
users:
  - name: ubuntu
    gecos: ubuntu
    groups:
      - sudo
    sudo:
      - ALL=(ALL) NOPASSWD:ALL
    home: /home/ubuntu
    shell: /bin/bash
    lock_passwd: false
    ssh_authorized_keys:
      - ssh-rsa AAAA...

And then network config, in config/network_config.yml:

version: 2
ethernets:
  ens3:
    dhcp4: true

Our project structure should look like this:

$ tree
.
├── config
│   ├── cloud_init.yml
│   └── network_config.yml
├── main.tf
└── variables.tf

Now run a plan, to see what will be done:

$  terraform plan

data.template_file.user_data[1]: Reading...
data.template_file.user_data[0]: Reading...
data.template_file.network_config[1]: Reading...
data.template_file.network_config[0]: Reading...
...

Plan: 8 to add, 0 to change, 0 to destroy

And run terraform apply to run our deployment:

$ terraform apply

...
null_resource.cache_image: Creation complete after 10m36s [id=4239391010009470471]
libvirt_volume.base: Creating...
libvirt_volume.base: Creation complete after 3s [id=/var/lib/libvirt/images/base.qcow2]
libvirt_volume.ubuntu20-qcow2[1]: Creating...
libvirt_volume.ubuntu20-qcow2[0]: Creating...
libvirt_volume.ubuntu20-qcow2[1]: Creation complete after 0s [id=/var/lib/libvirt/images/ubuntu20-1.qcow2]
libvirt_volume.ubuntu20-qcow2[0]: Creation complete after 0s [id=/var/lib/libvirt/images/ubuntu20-0.qcow2]
libvirt_domain.domain-ubuntu[1]: Creating...
...

libvirt_domain.domain-ubuntu[1]: Creation complete after 51s [id=6221f782-48b7-49a4-9eb9-fc92970f06a2]

Apply complete! Resources: 8 added, 0 changed, 0 destroyed

Verify VM creation using virsh command:

$ virsh list

 Id   Name   State
----------------------
 1    vm1    running
 2    vm2    running

Get instances IP address:

$ virsh net-dhcp-leases --network default

Expiry Time           MAC address         Protocol   IP address          Hostname   Client ID or DUID
-----------------------------------------------------------------------------------------------------------------------------------------------
2024-12-09 19:50:00   52:54:00:2e:0e:86   ipv4       192.168.122.19/24   vm1        ff:b5:5e:67:ff:00:02:00:00:ab:11:b0:43:6a:d8:bc:16:30:0d
2024-12-09 19:50:00   52:54:00:86:d4:ca   ipv4       192.168.122.15/24   vm2        ff:b5:5e:67:ff:00:02:00:00:ab:11:39:24:8c:4a:7e:6a:dd:78

Try to access the vm using ubuntu user:

$ ssh ubuntu@192.168.122.15

The authenticity of host '192.168.122.15 (192.168.122.15)' can't be established.
ED25519 key fingerprint is SHA256:Y20zaCcrlOZvPTP+/qLLHc7vJIOca7QjTinsz9Bj6sk.
This key is not known by any other names
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added '192.168.122.15' (ED25519) to the list of known hosts.
Welcome to Ubuntu 20.04.6 LTS (GNU/Linux 5.4.0-200-generic x86_64)
...

ubuntu@ubuntu:~$

Create Ansible Playbook

Now let's create the Ansible Playbook to deploy Flask & Postgresql on Docker. First you need to create ansible directory and ansible.cfg file:

$ mkdir ansible && cd ansible

[defaults]
inventory = hosts
host_key_checking = True
deprecation_warnings = False
collections = ansible.posix, community.general, community.postgresql

Then create inventory file called hosts:

[vm1]
192.168.122.19 ansible_user=ubuntu

[vm2]
192.168.122.15 ansible_user=ubuntu

checking our VMs using ansible ping command:

$ ansible -m ping all

192.168.122.15 | SUCCESS => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/bin/python3"
    },
    "changed": false,
    "ping": "pong"
}
192.168.122.19 | SUCCESS => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/bin/python3"
    },
    "changed": false,
    "ping": "pong"
}

Now create playbook.yml and roles, this playbook will install and configure Docker, Flask and PostgreSQL:

---
- name: Deploy Flask
  hosts: vm1
  become: true
  remote_user: ubuntu
  roles:
    - docker
    - flask

- name: Deploy Postgresql
  hosts: vm2
  become: true
  remote_user: ubuntu
  roles:
    - docker
    - psql

Playbook to install Docker

Now create new directory called roles/docker:

$ mkdir roles
$ mkdir docker

Create a new directory in docker called tasks, then create new file main.yml. This file will install Docker & Docker Compose:

$ mkdir docker/tasks
$ vim main.yml

---
- name: Run update
  ansible.builtin.apt:
    name: aptitude
    state: latest
    update_cache: true

- name: Install dependencies
  ansible.builtin.apt:
    name:
      - net-tools
      - apt-transport-https
      - ca-certificates
      - curl
      - software-properties-common
      - python3-pip
      - virtualenv
      - python3-setuptools
      - gnupg-agent
      - autoconf
      - dpkg-dev
      - file
      - g++
      - gcc
      - libc-dev
      - make
      - pkg-config
      - re2c
      - wget
    state: present
    update_cache: true

- name: Add Docker GPG apt Key
  ansible.builtin.apt_key:
    url: https://download.docker.com/linux/ubuntu/gpg
    state: present

- name: Add repository into sources list
  ansible.builtin.apt_repository:
    repo: deb [arch=amd64] https://download.docker.com/linux/ubuntu {{ ansible_lsb.codename }} stable
    state: present
    filename: docker

- name: Install Docker
  ansible.builtin.apt:
    name:
      - docker-ce
      - docker-ce-cli
    state: present
    update_cache: true

- name: Add non-root to docker group
  user:
    name: ubuntu
    groups: [docker]
    append: true

- name: Install Docker module for Python
  ansible.builtin.pip:
    name: docker

- name: Install Docker-Compose
  ansible.builtin.get_url:
    url: https://github.com/docker/compose/releases/download/1.29.2/docker-compose-Linux-x86_64
    dest: /usr/local/bin/docker-compose
    mode: '755'

- name: Create Docker-Compose symlink
  ansible.builtin.command:
    cmd: ln -s /usr/local/bin/docker-compose /usr/bin/docker-compose
    creates: /usr/bin/docker-compose

- name: Restart Docker
  ansible.builtin.service:
    name: docker
    state: restarted
    enabled: true

Playbook to install and configure postgresql

Then create new directory called psql, create subdirectory called vars, tempalates & tasks:

$ mkdir psql
$ mkdir psql/vars
$ mkdir psql/templates
$ mkdir psql/tasks

After that, in vars, create main.yml. These are variables used to set username, passwords, etc:

---
db_port: 5433
db_user: admin
db_password: dbPassword
db_name: todo

Next, we will create jinja file called docker-compose.yml.j2. With this file we will create postgresql container:

version: '3.7'
services:
  postgres:
      image: postgres:13
      container_name: db
      restart: unless-stopped
      ports:
        - {{ db_port }}:5432
      networks:
        - flask_network
      environment:
        - POSTGRES_USER={{ db_user }}
        - POSTGRES_PASSWORD={{ db_password }}
        - POSTGRES_DB={{ db_name }}
      volumes:
        - postgres_data:/var/lib/postgresql/data
networks:
  flask_network:

volumes:
  postgres_data:

Next, create main.yml to tasks. So we will copy docker-compose.yml.j2 and run using docker compose:

---
- name: Add Postgresql Compose
  ansible.builtin.template:
    src: docker-compose.yml.j2
    dest: /home/ubuntu/docker-compose.yml
    mode: preserve

- name: Docker-compose up
  ansible.builtin.shell: docker-compose up -d --build
  args:
    chdir: /home/ubuntu

Playbook to deploy Flask App

First, you need to create directory called flask, then create sub-directory again:

$ mkdir flask
$ mkdir flask/vars
$ mkdir flask/templates
$ mkdir flask/tasks

Next, add main.yml to vars. This file refer to posgtresql variable before, with addition IP address of VM2(database VM):

---
db_port: 5433
db_user: admin
db_password: dbPassword
db_name: todo
db_host:  192.168.122.15

Next, create config.py.j2 to templates. This file will replace the old config file from Flask project:

DEV_DB = 'sqlite:///task.db'

pg_user = "{{ db_user }}"
pg_pass = "{{ db_password }}"
pg_db = "{{ db_name }}"
pg_port = {{ db_port }}
pg_host = "{{ db_host }}"

PROD_DB = f'postgresql://{pg_user}:{pg_pass}@{pg_host}:{pg_port}/{pg_db}'

Next, create docker-compose.yml.j2 to templates. With this file we will create a container using docker compose:

version: '3.7'
services:
  flask:
     build: flask
     container_name: app
     restart: unless-stopped
     ports:
       - 5000:5000
     environment:
       - DEBUG=0
     networks:
       - flask_network

networks:
  flask_network:

Next, create main.yml in tasks. With this file we will clone flask project, add compose file, replace config.py and create new container using docker compose:

---
- name: Clone html project
  changed_when: false
  ansible.builtin.git:
    repo: https://github.com/danielcristho/Flask_TODO.git
    dest: /home/ubuntu/Flask_TODO
    clone: true

- name: Add Flask Compose
  ansible.builtin.template:
    src: docker-compose.yml.j2
    dest: /home/ubuntu/Flask_TODO/docker-compose.yml
    mode: preserve

- name: Update config.py
  ansible.builtin.template:
    src: config.py.j2
    dest: /home/ubuntu/Flask_TODO/flask/app/config.py
    mode: preserve

- name: Run docker-compose up -d
  shell: docker-compose up -d --build
  args:
    chdir: /home/ubuntu/Flask_TODO

Our project structure should look like this:

├── ansible-flask-psql
│   ├── ansible.cfg
│   ├── hosts
│   ├── playbook.yml
│   └── roles
│       ├── docker
│       │   └── tasks
│       │       └── main.yml
│       ├── flask
│       │   ├── tasks
│       │   │   └── main.yml
│       │   ├── templates
│       │   │   ├── config.py.j2
│       │   │   └── docker-compose.yml.j2
│       │   └── vars
│       │       └── main.yml
│       └── psql
│           ├── tasks
│           │   └── main.yml
│           ├── templates
│           │   └── docker-compose.yml.j2
│           └── vars
│               └── main.yml
├── libvirt-kvm
│   ├── config
│   │   ├── cloud_init.yml
│   │   └── network_config.yml
│   ├── main.tf
│   ├── variables.tf

Run Playbook and testing

Finally, let's run ansible-playbook to deploy PostgreSQL and Flask:

$ ls
ansible.cfg  hosts  playbook.yml  roles

$ ansible-playbook -i host playbook.yml

 _____________________
< PLAY [Deploy Flask] >
 ---------------------
        \   ^__^
         \  (oo)\_______
            (__)\       )\/\
                ||----w |
                ||     ||

 ________________________
< TASK [Gathering Facts] >
 ------------------------
        \   ^__^
         \  (oo)\_______
            (__)\       )\/\
                ||----w |
                ||     |
...
 ____________
< PLAY RECAP >
 ------------
        \   ^__^
         \  (oo)\_______
            (__)\       )\/\
                ||----w |
                ||     ||

192.168.122.15             : ok=13   changed=3    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0
192.168.122.19             : ok=15   changed=4    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

After complete, just make sure there is no error. Then you see there are two containers created. In VM1 is Flask and VM2 is Postgresql:

$ docker ps

CONTAINER ID   IMAGE              COMMAND           CREATED              STATUS              PORTS
                   NAMES
f3978427e34c   flask_todo_flask   "python run.py"   About a minute ago   Up About a minute   0.0.0.0:5000->5000/tcp, :
::5000->5000/tcp   app

$ docker ps

fbebdff75a6e   postgres:13   "docker-entrypoint.s…"   4 minutes ago   Up 4 minutes   0.0.0.0:5433->5432/tcp, [::]:5433
->5432/tcp   db

Try to access the app using browsers, just type http://<vm1_ip>:5000:

Try to add a new task and then the data will be added to the database:

Conclusion

Finally we have created 2 VMs and deploy Flask Project with database.

Thank you for reading this article. Feel free to leave a comment if you have any questions, suggestions, or feedback.

NB: Project Repo: danielcristho/that-i-write/terrafrom-ansible-flask

Deploy Laravel API To AWS Lambda

Daniel Pepuho — Thu, 11 Apr 2024 00:15:24 +0000

About two months ago, I was given a task to deploy Laravel API to Lambda. And then I found the ways to deploy it using Serverless Framework and Bref.

What is Serverless Framework?

The Serverless Framework is an open-source framework that allows developers to build and deploy serverless applications and functions. It simplifies the process of managing serverless architectures by abstracting away the underlying infrastructure management. With the Serverless Framework, developers can focus more on writing code and less on managing servers or provisioning resources.

To install serverless, you should using npm:

npm install -g serverless

What is Bref?

To managing the function, Bref provides a command-line interface (CLI) tool for initializing, deploying, and managing serverless PHP applications. Developers can use the CLI to create new projects, deploy functions to AWS Lambda, and manage configuration settings.

You guys also, set the runtime environment for running PHP applications using Bref. This environment includes support for PHP 7.x, 8.x, extensions commonly used in PHP applications, and features like HTTP request handling, environment variables, and logging.

Install Bref using composer:

cd laravel-project

composer require bref/bref bref/laravel-bridge --update-with-dependencies

Lastly, to deploy our function or application we need to define everything, including the runtime, runtime version, stage, environment, cloud provider in a file called serverless.yml

Setup

Okay, I think that's enough explanation and installation. Now let's do some setup. :)

cd laravel-project

Create serverless.yml:

php artisan vendor:publish --tag=serverless-config

Now, you need

service: laravel

provider:
    name: aws
    region: ap-southeast-2
    stage: dev
    runtime: provided.al2
    environment:
        DB_HOST: wwww.ap-southeast-2.rds.amazonaws.com
        DB_DATABASE: xxxx
        DB_USERNAME: yyyy
        DB_PASSWORD: zzzz
   vpc:
    securityGroupIds:
    - sg-xyz
    subnetIds:
        - subnet-abcd

package:
    # Files and directories to exclude from deployment
    patterns:
        - '!node_modules/**'
        - '!resources/assets/**'
        - '!storage/**'
        - '!tests/**'
        - '!docker/'
        - '!.husky/'
        - '!.github/**'

functions:
    # This function runs the athena website/API
    athena:
        handler: public/index.php
        runtime: php-82-fpm
        timeout: 28 # in seconds (API Gateway has a timeout of 29 seconds)
        events:
            - httpApi: '*'

    # This function lets us run artisan commands in Lambda
    artisan:
        handler: artisan
        runtime: php-82-console
        timeout: 720 # in seconds
        # Uncomment to also run the scheduler every minute
        #events:
        #    - schedule:
        #          rate: rate(1 minute)
        #          input: '"schedule:run"'

plugins:
    # We need to include the Bref plugin
    - ./vendor/bref/bref

A bit of explanation

service: laravel

provider:
    name: aws
    region: ap-southeast-2
    stage: dev
    runtime: provided.al2
    environment:
        DB_HOST: wwww.ap-southeast-2.rds.amazonaws.com
        DB_DATABASE: xxxx
        DB_USERNAME: yyyy
        DB_PASSWORD: zzzz
    vpc:
    securityGroupIds:
    - sg-xyz
    subnetIds:
        - subnet-abcd

service: This specifies the name of our service. In this case, it's named "laravel".
provider:
- name: Specifies that the cloud provider. For example we using AWS.
- region: Specifies the AWS region where our resources will be deployed. In this case, it's the Asia Pacific (Sydney) region.
- stage: Specifies the deployment stage or environment. Here, it's set to "dev", indicating a development stage, Or prod indicating production.
- runtime: Specifies the Lambda runtime. provided.al2 indicates that we're using a custom runtime based on Amazon Linux 2. This is a common choice for PHP-based applications. Then you will using other runtime likes; Node Js, Python, .NET etc.
- environment: Specifies the environment variable that will be available to our Lambda function. In this case I wrote an environment for RDS Database. Anyway, later I will write about AWS CDK using Python for creating database clusters, etc.
- vpc: This indicates that we're configuring the Virtual Private Cloud settings for our Lambda functions. This is useful for scenarios where your Lambda functions need to interact with resources that are only accessible within your VPC, such as databases hosted on Amazon RDS or services running on Amazon EC2 instances.
- securityGroupIds: Specifies the security groups associated with our Lambda functions.
- subnetIds: Specifies the subnets in which our Lambda functions will be deployed. Subnets are segments of our VPC where you can place resources. subnet-abcd is the ID of the subnet where our Lambda functions will be deployed.

package:
    # Files and directories to exclude from deployment
    patterns:
        - '!node_modules/**'
        - '!resources/assets/**'
        - '!storage/**'
        - '!tests/**'
        - '!docker/'
        - '!.husky/'
        - '!.github/**'

functions:
    # This function runs the athena website/API
    athena:
        handler: public/index.php
        runtime: php-82-fpm
        timeout: 28 # in seconds (API Gateway has a timeout of 29 seconds)
        events:
            - httpApi: '*'

    # This function lets us run artisan commands in Lambda
    artisan:
        handler: artisan
        runtime: php-82-console
        timeout: 720 # in seconds
        # Uncomment to also run the scheduler every minute
        #events:
        #    - schedule:
        #          rate: rate(1 minute)
        #          input: '"schedule:run"'

plugins:
    # We need to include the Bref plugin
    - ./vendor/bref/bref

Package

patterns: Specifies the files and directories to exclude from the deployment package. These are typically directories that contain development or testing files that aren't needed in the production deployment.

Functions

functions: Defines a Lambda function named responsible for running the Athena website/API.
handler: Specifies the entry point for the function. In this case, it's public/index.php, which is typical for Laravel applications.
runtime: Specifies the runtime for the Lambda function. It's set to php-82-fpm, indicating PHP 8.2 running as FastCGI Process Manager (FPM).
timeout: Sets the timeout for the function execution in seconds.
events: Defines the event trigger for the function. In this case, it's an HTTP API event trigger.
artisan: Defines a Lambda function named "artisan" responsible for running artisan commands in Lambda.
handler: Specifies the entry point for the function. Here, it's artisan, indicating the Laravel artisan CLI.
runtime: Specifies the runtime for the Lambda function. It's set to php-82-console, indicating PHP 8.2 running in console mode.

Plugins

./vendor/bref/bref: Specifies the Bref plugin, which is necessary for deploying PHP applications to AWS Lambda. This plugin provides the necessary runtime and integration with AWS services.

Okay lastly, we'll deploy the function by running serverless command:

serverless deploy

sls deploy

Deploying laravel to stage dev (ap-southeast-2)

✔ Service deployed to stack laravel-dev (102s)

endpoint: ANY - https://xxyyzz.execute-api.ap-southeast-2.amazonaws.com
functions:
  athena: laravel-dev-athena (45 MB)
  artisan: laravel-dev-artisan (45 MB)

Want a better experience than the AWS console? Try out https://dashboard.bref.sh

Migration using bref, usually we use artisan command, like php artisan migrate but if using bref just use:

serverless bref:cli --args="migrate"

NB: You guys can use bref to run other artisan commands.

Accessing API, in this case i use api/health-check:

curl https://xxyyzz.execute-api.ap-southeast-2.amazonaws.com/api/health-check

Response:

"Hello to public API."

That's all for now, I will write about Lambda or other stuff about AWS later. Hope you enjoy it.

Build Multi Arch Docker Image

Daniel Pepuho — Fri, 06 Oct 2023 02:38:02 +0000

Case

I have several problems, when I create a Docker image on Ubuntu which in fact uses amd64 architecture then, when I share the image with my friends, sometimes they have problems. Because the architecture they use is arm64.

Goal

So in this article, I will make a short step on how to create multi-arch Docker image using Docker plugin: buildx.

What is Buildx

Before we start, what is buildx? Buildx is an open-source Docker CLI plugin that brings enhanced build capabilities based Moby BuildKit to the Docker CLI. It’s worth mentioning that the x in buildx stands for experimental, meaning that features and capabilities provided by buildx are not yet stable. Keep in mind that actual features and command line interface may change. Once a feature provided by buildx has matured and become stable, it will be integrated into regular Docker CLI commands such as docker build and others.

Let's start

First, make sure Docker is installed, then create a new builder called multi-arch using buildx command.

docker buildx create --use --name multi-arch

output:

[+] Building 4.7s (1/1) FINISHED
 => [internal] booting buildkit                                                                                                                                                           4.7s
 => => pulling image moby/buildkit:buildx-stable-1                                                                                                                                        3.7s
 => => creating container buildx_buildkit_multi-arch0                                                                                                                                     1.0s
multi-arch

Inspect costum buildx

Name:          multi-arch
Driver:        docker-container
Last Activity: 2023-09-25 18:58:30 +0000 UTC

Nodes:
Name:      multi-arch0
Endpoint:  unix:///var/run/docker.sock
Status:    running
Buildkit:  v0.12.2
Platforms: linux/amd64, linux/amd64/v2, linux/amd64/v3, linux/386
...

After create new builder then, create a Dockerfile, in this case i will create a Dockerfile using Ubuntu base image.

FROM ubuntu:18.04

LABEL maintainer="@danielcristho"

VOLUME /root

# Install packages
RUN DEBIAN_FRONTEND=noninteractive \
    apt-get update \
    && apt-get install -y net-tools build-essential tcpdump telnet traceroute curl iperf3 knot-host openssh-server mtr-tiny socat nmap iputils-ping rsyslog vim iptables iproute2 nano  \
    && rm -rf /var/lib/apt/lists/*

### Clear cache and change shell ###
RUN apt-get clean && \
    apt-get autoclean && \
    apt-get autoremove -y && \
    rm -rf /var/lib/cache/* && \
    rm -rf /var/lib/log/*

CMD ["bash"]

Finally after create the Dockerfile, let's build into multi-arch using buildx.

docker buildx build --platform linux/amd64,linux/arm64,linux/arm/v7,linux/arm/v8 .

By running the command above, we can build Docker image on multi architecture, like amd64 and arm64.

But you can push the image to DockerHub after build by using push arguments. In this case i will push the image to my repository on DockerHub.

docker buildx build --platform linux/amd64,linux/arm64,linux/arm/v7,linux/arm/v8,linux/arm64/v8 -t danielcristh0/ubuntu-bionic --push .

Verify the result in Docker Hub.

Reference:

Install Docker on Remote Server using Ansible

Daniel Pepuho — Mon, 28 Aug 2023 17:39:13 +0000

Long time no see, now i will show you how to install Docker using Ansible Playbook. Let's start!!!

Setup

First, you need to install Ansible. Just follow this link to install Ansible on your operating system installation guide.

After installation, create new directory called ansible-docker.

$ mkdir ansible-docker && cd ansible-docker

Create a new file called ansible.cfg as the Ansible configuration setting and then define the inventory file.

[defaults]
inventory = hosts
host_key_checking = True
deprecation_warnings = False
collections = ansible.posix, community.general

Then create a new file called hosts, where the name is defined on ansible.cfg.

[example-server]
0.0.0.0 ansible_user=root

NB: Don't forget to change the IP Address and host name.

After setup the Ansible configuration setting & inventory file, let's create a YAML called playbook.yml

---
- name: Setup Docker on Ubuntu Server 22.04
  hosts: all
  become: true
  remote_user: root
  roles:
    - config
    - docker

Then create roles directory:

Config, On this directory I will create a directory called tasks. After that, I should create yaml file called main.yml to run update, upgrade & install many dependencies.

---
- name: Update&Upgrade
  ansible.builtin.apt:
    name: aptitude
    state: present
    update_cache: true

- name: Install dependencies
  ansible.builtin.apt:
    name:
      - net-tools
      - apt-transport-https
      - ca-certificates
      - curl
      - software-properties-common
      - python3-pip
      - virtualenv
      - python3-setuptools
      - gnupg-agent
      - autoconf
      - dpkg-dev
      - file
      - g++
      - gcc
      - libc-dev
      - make
      - pkg-config
      - re2c
      - wget
    state: present
    update_cache: true

Docker, On this directory create 2 directories called tasks & templates.

On tasks directory create new file called main.yml. This file contains Docker installation, Docker Compose installation & private registry setup.

---
- name: Add Docker GPG apt Key
  ansible.builtin.apt_key:
    url: https://download.docker.com/linux/ubuntu/gpg
    state: present

- name: Add repository into sources list
  ansible.builtin.apt_repository:
    repo: deb [arch=amd64] https://download.docker.com/linux/ubuntu {{ ansible_lsb.codename }} stable
    state: present
    filename: docker

- name: Install Docker 23.0.1-1
  ansible.builtin.apt:
    name:
      - docker-ce=5:23.0.1-1~ubuntu.22.04~jammy
      - docker-ce-cli=5:23.0.1-1~ubuntu.22.04~jammy
      - containerd.io
    state: present
    update_cache: true

- name: Setup docker user
  ansible.builtin.user:
    name: docker
    groups: "docker"
    append: true
    sudo_user: yes

- name: Install Docker module for Python
  ansible.builtin.pip:
    name: docker

- name: Install Docker-Compose&Set Permission
  ansible.builtin.get_url:
    url: https://github.com/docker/compose/releases/download/1.29.2/docker-compose-Linux-x86_64
    dest: /usr/local/bin/docker-compose
    mode: '755'

- name: Create Docker-Compose symlink
  ansible.builtin.command:
    cmd: ln -s /usr/local/bin/docker-compose /usr/bin/docker-compose
    creates: /usr/bin/docker-compose

- name: Add private registry
  ansible.builtin.template:
    src: daemon.j2
    dest: /etc/docker/daemon.json
    mode: preserve

- name: Restart Docker
  ansible.builtin.service:
    name: docker
    state: restarted
    enabled: true

In the template, create a template file using a jinja file named daemon.j2. This file contains configuration for private registry settings (optional).

{
    "insecure-registries" : ["http://0.0.0.0:5000"]
}

NB: Field the IP using your remote server private IP

After all setup, Your project directory should look like this:

$ tree
.
├── ansible.cfg
├── config
│   └── tasks
│       └── main.yml
├── docker
│   ├── tasks
│   │   └── main.yml
│   └── templates
│       └── daemon.j2
├── hosts
└── playbook.yml

Test & Run

Okay, now test Your playbook.yml file using this command.

$ ansible-playbook --syntax-check playbook.yml

If You don't have any errors, run the playbook using this command.

$ ansible-playbook -i hosts playbook.yml

Output, wait until finish.

 ____________________________________________
< PLAY [Setup Docker on Ubuntu Server 22.04] >
 --------------------------------------------
        \   ^__^
         \  (oo)\_______
            (__)\       )\/\
                ||----w |
                ||     ||

 ________________________
< TASK [Gathering Facts] >
 ------------------------
        \   ^__^
         \  (oo)\_______
            (__)\       )\/\
                ||----w |
                ||     ||

Conclusion

In this post, I just show you how to install Docker in a specific version using Ansible Playbook when you have one or more servers.

Thank You for reading this post, If You have suggestions or questions please leave them below. Thanks

NB: In this case, I just set the user as root. I installed the Docker on Ubuntu Server 22.04. For full code follow this link ansible-docker.