The Ops Community ⚙️: kingsathurthi

Pulumi Challenge! Quickstart Guide

kingsathurthi — Fri, 14 Oct 2022 14:28:31 +0000

What is Pulumi?

Pulumi is a universal infrastructure as code platform that allows you to use familiar programming languages and tools to build, deploy, and manage cloud infrastructure. Pulumi is free, open source, and optionally pairs with the Pulumi Service to make managing infrastructure secure, reliable, and hassle-free. Currently Pulumi is running some challenge contest lets try out what it is. you can find the complete information in this official link

Quickstart

Pulumi Architecture Templates make it quick and easy to get started with a wide variety of clouds, tools, and technologies when you want to try something new but don't want to write boilerplate yourself. It's a great way to get started with Pulumi quickly, no matter what you need to build. Try it for yourself with serverless AWS templates and win some swag in the process!

Prerequisites

For this demo I'm going to create new Amazon EC2 Instance and install all the required prerequisites components. you can use own device to do this

Step 1 - Creating Pulumi Architecture Templates

you will see how to create new Pulumi Architecture Templates, specifically for serverless blueprints for AWS with the language of your choice. but here we are gonna use Python

let's create new directory named quickstart-challenge and change directory into it using below commands

`mkdir quickstart-challenge && cd quickstart-challenge`

After creating the directory let's create pulumi Architecture Templates using below command

pulumi new serverless-aws-python

Once you ran the above command it will ask for access token please login to your pulumi account and create token and save it safely.

Copy and paste the token and continue the project and stack setup like below in the screenshot. leave input blank so that it can take default value.

Note: Must use AWS region us-west-2

Step 2. Exploring Blueprints

let explore the Pulumi architectural blueprints before that configure AWS CLI by running below command and enter your aws account access key ID and secret access key and region.

aws configure

Now run below command command see the blueprint in the screen shot

pulumi preview

If you see the output for the pulumi preview command it create the IAM role, Lambda Function, S3 Bucket, API gateway under the hood.

Step 3. Creating Policy Packs

When you’re using Python or Typescript, Pulumi allows you to enforce gated deployments with Policy Packs, which can be used locally with the free tier. These are a set of rules, expressed programmatically, that are executed against the resources being deployed. Any violation of those rules will block the deployment. Usually, Policy Packs are used to enforce security or cost optimization rules, but in this case we’re going to use one to interact with pulumi's swag provider.

Within your existing Pulumi program, create a new directory for your Policy Pack and navigate into it

mkdir policy && cd policy

Creating policy pack using below command

pulumi policy new aws-python

Once you run the command it will execute and crate the respective policy like below

The default Policy Pack prevents developers from allowing public read access on an AWS S3 bucket but that isn’t what we need here. Replace the contents of the new main.py in the directory /home/ec2-user/quickstart-challenge/policy with the following

import requests
from pulumi_policy import (
    EnforcementLevel,
    PolicyPack,
    ReportViolation,
    ResourceValidationArgs,
    ResourceValidationPolicy,
    PolicyConfigSchema
)


def map_swag_to_form(swag_var):
    data_json = {
        'usp': 'pp_url'
    }
    swag = swag_var
    data_json['entry.1720843992'] = swag['name']
    data_json['entry.511943887'] = swag['email']
    data_json['entry.1289952319'] = swag['address']
    data_json['entry.1240089905'] = swag['size']
    return data_json


def pulumi_swag_not_submitted(args: ResourceValidationArgs, report_violation: ReportViolation):
    if not args.resource_type == "pulumi:pulumi:Stack":
        return
    swag = args.get_config()
    data_dict = map_swag_to_form(swag)
    print(data_dict)
    headers = {
        "Referer": "https://docs.google.com/forms/d/e/1FAIpQLSfBr2f6rhXYbMXi8Caftu-zWtNPRDoWUEukrTJKuwO3OyYRvg/viewform",
        "User-Agent": "Mozilla/5.0 (X11; Linux i686) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/28.0.1500.52 Safari/537.36"
    }
    response = requests.post(
        url="https://docs.google.com/forms/d/e/1FAIpQLSfBr2f6rhXYbMXi8Caftu-zWtNPRDoWUEukrTJKuwO3OyYRvg/formResponse",
        headers=headers,
        data=data_dict
    )
    print(response.status_code)


submit_swag = ResourceValidationPolicy(
    name="pulumi-challenge-swag",
    description="stuff",
    validate=pulumi_swag_not_submitted,
    config_schema=PolicyConfigSchema(
        properties={
            "name": {
                "type": "string",
                "minLength": 2
            },
            "email": {
                "type": "string",
                "minLength": 6,
                "format": "email"
            },
            "address": {
                "type": "string",
                "minLength": 2
            },
            "size": {
                "type": "string",
                "minLength": 1,
                "enum": [
                    "XS",
                    "S",
                    "M",
                    "L",
                    "XL"
                ]
            }
        },
        required=[
            "name",
            "email",
            "address",
            "size"
        ]
    )
)

PolicyPack(
    name="aws-python",
    enforcement_level=EnforcementLevel.MANDATORY,
    policies=[
        submit_swag
    ],
)

Step 4. Complying with Policies

The Policy Pack we’ve defined requires that your Pulumi program involve a JSON file containing specific data. In order to comply with it and get swag, we need to create this file.

Navigate one directory up, back into your main Pulumi Challenge directory. Create a new file called swag.json in the directory /home/ec2-user/quickstart-challenge/ , and add the following (replacing the values with your own information)

{
  "pulumi-challenge-swag": {
    "name": "<your name>",
    "email": "<your email>",
    "address": "<your address>",
    "size": "<one of XS, S, M, L, XL>"
  }
}

We also have new dependencies to add to requirements.txt at the root of project directory. Replace with below value.

pulumi>=3.0.0,<4.0.0
pulumi-aws>=5.10.0,<6.0.0
pulumi-aws-apigateway>=0.0.11
pulumi-awsx==1.0.0-beta.9
pulumi-policy>=1.3.0,<2.0.0
requests

Once updated the requirements.txt run the below pip install command in the virtual environment.

source venv/bin/activate
pip install -r requirements.txt

Execute your policy pack against your Pulumi Program with the following command at the root of your project to submit for your swag

use below command to preview the blueprint

pulumi preview --policy-pack policy --policy-pack-config swag.json

Once you run the preview command and get 200 http code status that everything did is correct. Now lets run the below command to deploy the application.

pulumi up --policy-pack policy --policy-pack-config swag.json

give yes and proceed to the deployment. once the deployment is completed you will see the output grab the application URL and visit

Below the application deployed using Pulumi architectural template

Congratulations! You completed the Pulumi Challenge. finally destroy the the environment and stack. using below command.

pulumi destroy -y
pulumi stack rm dev

Thanks for reading the post I hope you enjoyed the Pulumi challenge.

Docker Cheat sheet

kingsathurthi — Sat, 28 May 2022 18:04:04 +0000

Hi folks in this post we will see some important docker commands to manage docker image and containers

Process Management

Show all running docker containers

docker ps

Show all docker containers

docker ps -a

Run a container

docker run <image>:<tag>

Run a container and connect to it

docker run -it <image>:<tag>

Run a container in the background

docker run -d <image>:<tag>

Stop a container

docker stop <container>

Kill a container

docker kill <container>

Remove the stopped container

Docker rm <container>

Images/Repository

List available local images

docker image ls

Remove docker image

docker rmi <image>

Search for docker images

docker search <image>

Pull a docker image

docker pull <image>

Build an image with a dockerfile

docker build -t <image>:<tag> <run_directory> -f
<dockerfile>

Login to a remote repository

docker login <repository>

Push an image to your remote repository

docker push <image>:<tag>

Show metadata for an image

docker image inspect <image>

Remove all unused docker images

docker image prune

Docker Compose

Start your docker-compose defined resources in detached mode

docker-compose up -d -f <docker_compose_yaml>

Stop all docker-compose resources

docker-compose stop

Destroy all docker-compose resources

docker-compose down

Show docker-compose processes

docker-compose ps

Show docker-compose logs

docker-compose logs

Show docker-compose resource consumption

docker-compose top

Pull latest docker-compose images

docker-compose pull

Volumes & Ports

List volumes

docker volume ls

Create a volume

docker volume create <volume>

Delete a volume

docker volume rm <volume>

Show volume metadata

docker volume inspect <volume>

Delete all volumes not attached to a container

docker volume prune

Mount a local directory to your container

docker run -v <local_dir>:<container_dir> <image>

Copy file or folder from a docker container to host machine

docker cp <container>:<container_dir> <local_dir>

Copy file or folder from local machine onto a container

docker cp <local_dir> <container>:<container_dir>

Map a local port to a docker instance

docker run -d -p <local_port>:<docker_port> <image>

List the ports a docker container is running on

docker port <container>

Troubleshooting

Show the logs of a container

docker logs <container>

Follow/tail the logs of a container

docker logs -f <container>

Show timestamps on docker logs

docker logs -t <container>

Show details/metadata of a container

docker inspect <container>

Show a 'top' view of processes running on a container

docker top <container>

Show a 'top' view of all docker containers

docker stats

Show any files that have changed since startup

docker diff <container>

Connect to an already running container

docker attach <container>

Execute a command on a container

docker exec-it <container_id> /bin/bash

Show docker system wide information

docker system info

Show docker disk space used

docker system df

Remove build cache

docker builder prune

Deploy Angular web app in remote server using Ansible

kingsathurthi — Fri, 27 May 2022 18:23:38 +0000

Introduction

In this article we are going to see how we are going to leverage Ansible to deploy Angular web app in remote server. Ansible is an open-source IT automation tool that automates provisioning, configuration management, application deployment, orchestration, and many other manual IT processes and Angular is a platform and framework for building single-page client applications using HTML and TypeScript. let's gets into the action

Prerequisite

Ansible controller (Your PC or any other machine installed with Ansible)
Remote Server (ex.AWS EC2)
Angular github repository (use this sample repo)

Please find the Ansible installation link here. And I assume you have installed Angular and Nginx installed in your remote server and ssh should allowed from Ansible controller at least. Im using Ubuntu desktop and AWS EC2 instance.

Ansible configuration

Once you have installed Ansible in you machine. Please follow the instruction to configure the Ansible inventory.

Go to /etc/ansible directory and open hosts file using vim sudo vim hosts and add below inventory details.

<inventory_name> ansible_host=<IP or URL> ansible_ssh_user=<username> ansible_ssh_private_key_file=path_to_your_pem>

Please find below image for you reference.

That's it we configured remote host/server in the Ansible Inventory

Let's check the connectivity between Ansible controller and remote server

First Disable Host Key Checking using below mentioned command because If a host is reinstalled and has a different key in ‘known_hosts’, this will result in an error message until corrected

Go to /etc/ansible directory and open ansible.cfg file using vim sudo vim ansible.cfg and below configuration.

[defaults]
host_key_checking = False

Please check below image for your reference.

After disabling the host key checking lets check the connectivity using below command

ansible ansible-demo -m ping

You will get the response like below image which means you are able to succesfully connect to the remote server/host

Create Ansible-Playbook for angular deployment

Create Ansible-playbook yaml file in your directory for deploying angular web app in the remote server using below playbook

- hosts: ansible-demo
  gather_facts: true
  vars:
    user: "ubuntu"
    date: "{{ lookup('pipe','date +%d%m%Y') }}"
    app_name: ops-ansible-angular-demo
    home_dir: /home/ubuntu
    repo_dir: "{{home_dir}}/Libraries/{{app_name}}"
    repo_app_dir: "{{repo_dir}}"
    project_dir: "/var/www/html/"

    repo_url: https://github.com/kingsathurthi/ops-ansible-angular-demo.git
    branch: main

  tasks:
    - name: "Clone git repository - https://{{repo_url}} of {{branch}} branch/tag to {{repo_dir}}"
      git: repo={{repo_url}} dest={{ repo_dir }} version={{branch}} force=yes update=yes clone=yes
      register: repo_clone

    - name: "Install packages based on package.json"
      npm: path={{repo_app_dir}}

    - name: "Run build command"
      command: ng build
      args:
       chdir: "{{repo_app_dir}}"

    - name: "Archive compiled files - {{repo_app_dir}}/dist/{{app_name}}/* to "
      become: yes
      archive: path={{repo_app_dir}}/dist/{{app_name}}/* dest={{repo_app_dir}}/{{app_name}}.tar.gz format=gz

    - name: "Copy artifact - {{repo_app_dir}}/{{app_name}}.tar.gz to {{project_dir}} server"
      copy: src="{{repo_app_dir}}/{{app_name}}.tar.gz" dest="{{project_dir}}/{{app_name}}.tar.gz" remote_src=yes #owner=www-data group=www-data mode=0644
      become: yes
      become_user: root

    - name: "Extract archived files to {{project_dir}} path"
      become: true
      unarchive: src={{project_dir}}/{{app_name}}.tar.gz  dest={{project_dir}} remote_src=yes

    - name: "Remove unwanted archive from {{repo_app_dir}} directory"
      file: path={{repo_app_dir}}/{{app_name}}.tar.gz state=absent

    - name: "Remove unwanted archive from {{project_dir}} directory"
      file: path={{project_dir}}/{{app_name}}.tar.gz state=absent
      become: yes
      become_user: root

once above playbook is created lets run the Ansible Playbook from that directory you a have created the playbook yaml file.

ansible-playbook ansible-demo.yml

You will find the result like below image.That's it you have successfully ran you Ansible playbook

Visit your remote server URL to see the angular web app.

Thanks for reading the blog I hope you enjoyed learning something new today.

The above blog is submitted as part of 'Devtron Blogathon 2022' - https://devtron.ai/
Check out Devtron's GitHub repo - https://github.com/devtron-labs/devtron/ and give a ⭐ to show your love & support.
Follow Devtron on LinkedIn - https://www.linkedin.com/company/devtron-labs/ and Twitter - https://twitter.com/DevtronL/, to keep yourself updated on this Open Source project.

Let's Create Multi-Stage Container Image with React.js and Nginx

kingsathurthi — Thu, 26 May 2022 18:07:57 +0000

Introduction

In this post we are going to look how to build multi-stage container image with React.js and Nginx. React.js is a free and open-source front-end JavaScript library for building user interfaces based on UI components and Nginx is a web server that can also be used as a reverse proxy, load balancer, mail proxy and HTTP cache but we are going to use as web server for the react.js web app

Prerequisite

Docker Desktop installed
VS Code or your favorite editor
Node.js latest version
NPM latest version

Create React App

Now, let's create a simple React app using the create-react-app

npx create-react-app react-demo
cd react-demo
npm start

You can now view react-demo in the browser using below localhost link.

Localhost: (http://localhost:3000)

That's it you have created your first simple react app.

Note: Now we are running react app as development server in local dev environment. If you want to run the same app in production use this npm run build command instead of npm start to generate the build file react app.

Create Docker file

A Dockerfile is a text document that contains all the commands to create an container image. Using docker build . command to execute the image building process.

FROM node:slim
WORKDIR /app
COPY package.json .
RUN npm install
COPY . .
CMD ["npm", "start"]

After that create .dockerignore and add below mentioned file to avoid getting copied into to container image while building

Now you can start building container image using below command

docker build -t react-demo .

Once you run the above command image building process looks like below

That's it you have created your first container image.

let's run and see the react app using below command

docker run -p 3000:3000 react-demo

You can now view react-demo app running as container using same localhost link below.

Localhost: (http://localhost:3000)

Above react container is running as development server Now we will create production grade image using below docker file.

FROM node:slim AS builder
WORKDIR /app
COPY package.json .
RUN npm install
COPY . .
RUN npm run build

FROM nginx
EXPOSE 80
COPY --from=builder /app/build /usr/share/nginx/html

Now you can start building multi-stage container image using below command

docker build -t react-nginx-demo .

That's it you have successfully created your first multi-stage container image for production.

let's run and see the react app using below command

docker run -p 80:80 react-nginx-demo

visit to the locahost (http://localhost) Now you can see the react-nginx-demo app running as container.

That's it for now i hope this blog give you some idea to create production grade container image using multi-stage build process.