The Ops Community ⚙️: Hrittik Roy

Knative: Serverless on Kubernetes

Hrittik Roy — Wed, 25 May 2022 20:26:47 +0000

When we hear about cloud and cloud native, we often come across many technical terms, but sometimes we don’t pay any attention to them. But, keeping these terms in the back of our mind can come in handy.

So, right now, we will discuss one such term, which you have probably heard during reading about Kubernetes, called Knative.

Let’s start!

What is Knative?

We all know that Kubernetes is an open-source solution platform that helps manage workloads and services in containers present in the cloud that helps in automation and configuration. Well, Knative can be simplified as Kubernetes on steroids.

The demand around scalable and reliable services is increasing every day exponentially. The market is driven by customers demanding their favorite services to have zero downtime and companies that lose millions of dollars for every minute they’re down. If you have come across the space that is responsible for keeping the systems up, you would […]
Knative is a platform installed on top of Kubernetes that provides you with serverless capabilities. The capabilities help you to deploy, run and manage serverless and cloud-native application to Kubernetes.

Cloud-native applications are scalable applications and run in all types of cloud environments. Now, the question remains what is serverless? And what is its relationship with Knative? We will have a look at that now.

Knative and Serverless

The cloud computing execution model where the machine resources allocated by the cloud providers taking care of the infrastructure is known as serverless cloud computing. Simply, you need to worry about your code, and everything else is managed.

In recent years, serverless adoption has started, with more and more individuals depending on serverless technology to meet organizations’ specific needs. A survey conducted by Serverless Inc showed in 2018 that half of the respondents used serverless in their job, and the numbers are projected to rise further. In this post, we would get around […]
The relation between Knative and serverless cloud computing is very simple. Knative has a serverless environment. So, in Knative applications, we can be assured that machine resources of cloud vendors are managing the application servers, and that helps in faster deployment of the application.

As Knative is a serverless solution and helps in developing modern, faster applications, it saves time for a developer to build more in cloud computing. Now, let’s watch the components which hold Knative firmly.

Components of Knative

All this serverless Framework of Knative stands on the three major components, and they are the following.

Building Framework

Building Framework helps in extending Kubernetes abilities. It also helps in utilizing the Kubernetes primitives (building blocks of Kubernetes) to enable the running of the on-cluster container builds from source code, meaning helping the container containing clusters to run directly from the written code.

Eventing Framework

Eventing Framework is mainly responsible for the creation of communication between the event producers and event consumers who have zero knowledge about each other’s components to achieve the architecture which will help in running, deploying and others based on the event.

Serving Framework

Serving Framework is highly responsible for supporting the deployment of serverless applications and functions on Kubernetes and Istio. It helps in the rapid deployment of serverless containers, automatic scaling of up and down to zero, routing, networking and other things for Istio and Kubernetes.

Advantages of Knative

Knative frame and applications have several benefits and advantages that help many individuals mitigate some of the challenges. Let’s have a brief look below.

Fast Deployment
Fast iterative development is one of the big advantages that Knative offers because it helps in the rapid deployment of applications and cuts down a lot of time during container building, and, as a result, faster rollouts of container versions are possible

Code Focused
Who doesn’t likes just to focus on code? Knative applications provide an event-driven architecture simply meaning the architecture will automatically enable the application to run deploy and other things automatically, which helps developer focus on writing code and not on the infrastructure.

Serverless
Faster entry into serverless computing is possible with Knative as it is a serverless framework and helps in the quicker establishment of serverless workflows. In addition, manual configurations are not required as all the works of the servers are done behind the scene.

Knative
Usage of Kubernetes ecosystem by groups (Source: CNCF)
Disadvantages of Knative
Managing container infrastructure is the biggest and the only drawback of Knative. Knative is not aimed at the end-users, and because of which we have to manually manage the infrastructure of the containers.

To simply put, the customers have to manually manage the container infrastructure because Knative mainly facilitates developers.

Value Proposition of Knative

From all the above discussions that we have seen, we can safely say that Knative is highly helpful when it comes to application deployment and creating serverless functionalities.

Knative is fully open-source, which is a big advantage to all the companies and businesses which want to migrate to serverless cloud computing; as we all know, open-source frameworks or platforms are free of cost. This is really big when we compare such businesses with mid-market values and their aim to be serverless in cloud computing.

Companies or businesses with high market caps can hugely contribute to the Knative Framework as well as get the advantages of using it as big companies will probably prefer their developers to focus more on coding, which will help in their product development and also can get the advantage of serverless computing that automatically helps developers to focus more on code building.

Small market caps businesses get a free pass here to get the advantages of using the Knative Platform and Framework as it is completely free of cost, and you can actually get the source codes from Github, which will help a lot of businesses to grow more with the help of serverless computing.

Final Thoughts

So, we have seen all the things Knative is modernizing the cloud and serverless ecosystem and empowering Istio and Kubernetes.

Kaniko: How Users Can Make The Best Use of Docker

Hrittik Roy — Wed, 25 May 2022 20:21:51 +0000

Whether you love or hate containers, there are only a handful of ways to work with them properly that ensures proper application use with Docker. While there do exist a handful of solutions on the web and on the cloud to deal with all the needs that come with running Docker, Kaniko has something amazing to offer.

Kaniko was released as a standalone addition through the Google cloud interface and has been inducted under the Cloud Native Computing Foundation(CNCF). Kaniko as a tool helps users to build containerized images from their Docker applications through the Dockerfiles. It doesn’t require any standalone Docker run daemon’s to process it through.

Let’s take a more defined look at some of it’s more undiscussed elements, advantages and demerits.

The Problems People Run Into With Docker
Docker has become an industry standard with a multitude of uses and benefits but still has cross dependency issues due to the need for a DockerFile to rely upon interactive access to a Docker daemon. This essentially requires root access on your machine to run.

Users can thus run into problems for making containerized images in environments that don’t provide any support or can’t run the Docker daemons. Kubernetes clusters are a good example of this.

Kanniko, services these issues by creating a method to convert the container images from a Dockerfile even in the absence of any privileged root access. Users can use Kaniko to build an image from a Dockerfile and push it to a registry all in one go. Since it doesn’t require any special privileges or permissions, it can be installed and run on a typical Kubernetes cluster, Google Kubernetes Engine, or in any environment that lacks access control.

Docker vs Kaniko Source: Stackshare

Kaniko is usually run as a container itself, and needs the following information to build a docker image as per the user requirements:-

The path to Dockerfile.
The path to build a context (workspace) directory. The build context directory is a repository that contains all the necessary resources that are required while building an image.
Destination/URL of the repository where the image will be pushed after the execution completes.
How Does Kaniko Work Exactly?
Kaniko can be run as a container image that takes in the three arguments as previously seen. Once this information has been included to the main registry, the final push will contain only a static Go binary plus the configuration files needed for pushing and pulling images.

The Kaniko executor then finds and extracts the base image file system to the root, which is the image in the FROM line of the Dockerfile. It executes each command in order, and takes a snapshot of the file system after each command.

The snapshot created is then fed into a user-space by running the filesystem and comparing it to the prior state that was stored in memory. Kaniko then combines any changes made to the filesystem as a new layer to the original image. These changes are also reflected in the image metadata. After executing every command in the Dockerfile, the user is then free to push the newly built image to the output registry.

Kaniko unpacks the filesystem, executes commands and creates a copy of the filesystem completely in the user-space within the requirements of the user’s image, which is how it bypasses the needs for any privilege access needs on the machine where it is being deployed. The docker daemon or CLI is not involved in this entire process.

Image Container Creation Steps Source: Google CloudOps

The Merits and Pitfalls of Kaniko
Kaniko’s serverless all in one solution for handling container issues seems like a welcoming change from the more convoluted mess that users face when using Kubernetes or Docker. The lack of root access needs in clusters combined with a simpler interface for languages makes Kaniko an amazing tool to use, especially for software engineers who need fast acting methods to copy and deploy applications through a common Docker setup. Since there’s no dependency on the daemon process, users are free to run Kaniko in any kind of environment.

Kaniko executes each command within the Dockerfile completely in the userspace using an executor image: gcr.io/kaniko-project/executor which is often made to run inside the container. Kaniko then uses the system interface to run commands inside the Dockerfile and creates a copy of the file system after each command.

If there are changes to the file system, the executor pipeline takes a snapshot of the filesystem change and reports it as a “diff” layer. These changes are later made permanent in the image metadata. This brings another merit for using Kaniko which is a seamless mechanism to execute files and run applications.

Some backlash for the system seems to be pitched at the build for Kaniko taking up too many resources and execution time, depending on the applications which can make it unreliable when not working with a serverless platform with much needed computing. Most common build runs for Kaniko have been reported to be much slower compared to it’s cloud build. The general solution may be to simply stick to the cloud version but may not always be the best approach if cloud access isn’t always available.

Comparison between common tools Source: Slideshare

Final Thoughts and Review
If you still haven’t used Kaniko for easing the process of container image deployment and copying, it doesn’t hurt to take a small dive at community documentations and even running the application. Kaniko has a fairly simple syntax that can be run with just a few commands through the command line and has no major dependencies for it to be run.

Users shouldn’t be put off by the extreme dependence towards containers, seeing how resources can be better utilized if proper images can be built on top of them. As an application that is focused on Kubernetes and Docker, Kaniko has much to apply itself onto other common platforms and architectures.

For the novice or the expert, there are several resources available on the community pages and even on Google’s flagship website for Kaniko to get you started. Tune as we take a much needed look at other applications in the next articles.

DVC (Git For Data): A Complete Intro

Hrittik Roy — Wed, 25 May 2022 20:09:23 +0000

As a data scientist or ML engineer, have you ever faced the inconvenience of experimenting with the model? When we train the model, the model file is generated. Now, if you want to experiment with some different parameters or data, generally people rename the existing model file and train the model again and this process goes on.

The final directory looks something as shown in the below figure 😁. The same thing goes with data as well.

Fig 1: Typical models dir after experimentation

But if we talk about code, we don't have this problem there because we have Git to version the code. I mean that I can create a separate branch for changing some code to see its behaviour without altering the previous one. What if we can get a Git kind of versioning control system for data and model? Wouldn't that be amazing? Here comes the DVC (Data Version Control) which exactly does the thing which solves this problem.

Although, DVC is much more than just data and model versioning. It also helps create end-to-end pipelines, capturing metrics related to pipelines and experimenting with ML models. But in this blog, we will see the data versioning aspect of DVC.

Installing DVC

DVC can be used as a typical Python library and can be installed using package managers like pip or conda.

To install DVC using pip, you can execute below command in the python supported terminal.

$ pip install dvc

To install DVC using conda, you need to execute below commands in conda terminal.

$ conda install -c conda-forge mamba # installs much faster than conda
$ mamba install -c conda-forge dvc

Apart from these conventional installation techniques, DVC can be installed in other different ways too as described here on the official documentation website.

Getting Started

Code versioning is not really a new thing for us because we use Git in our daily life but the problem with Git is it becomes extremely slow when the tracking file size is huge as 100 GB. That is where DVC comes in. Git tracks the changes in code whereas DVC tracks the changes in data as well as models.

The foundation of DVC consists of a few commands you can run along with Git to track large files, directories, or ML model files. In short, you can call DVC "Git for data".

Let us understand data versioning by dvc using a simple demo project. Let us assume that we want to create an end-to-end pipeline to prepare, train and evaluate the MNIST dataset and we have code as well as data available and structured as shown in the below figure.

Fig 2 MNIST pipeline file structure.

By observing the file structure, we can see the MNIST dataset is stored in the data directory as a data.xml file and the code corresponding to every stage of the pipeline is stored in the src directory. Now as we understand, the code tracking (src) is the responsibility of Git and the data tracking (data) is the responsibility of DVC.

Before we version the data, we need to initialise the repository as a DVC repository. Just like we run command git init to initialise the current directory as a Git repository, we need to execute the below command which will initialise the current directory as a DVC repository.

$ dvc init

This command will create a new directory .dvc just like git init creates .git directory. Two files .gitignore and config will be generated in .dvc directory. Now, we have successfully initialized this as DVC repository.

Start Tracking

In Git, if we want it to track the changes in the code, we run a git add command which adds the code changes in the local repository. DVC has a similar command as shown below to start tracking the data files.

$ dvc add data/data.xml

Note here that a dvc add command accepts arguments which are data or model file names that you want DVC to track. All the filenames must be space-separated.

By executing this command, DVC will create two files data/data.xml.dvc and data/.gitignore.

File data/.gitignore looks something like this:

/data.xml

It means that it forces Git not to track original data.xml file which is expected because the tracking of data.xml is now responsibility of DVC and not Git.

All files ending with extension .dvc are special files which contains the information about where the actual data files will be stored and cached. If you take a look at data/data.xml.dvc, it just stores an MD5 signature as shown below.

md5: a7cd139231cc35ed63541ce3829b96db

Now looking at the complete picture, Git will not track data/data.xml because it is added in data/.gitignore but Git will track data/data.xml.dvc which will have information about where the actual data.xml is stored.

Now the Git tracked 2 files mentioned above will be stored in Git remote repository on Github, BitBucket or Gitlab and DVC tracked data file will be stored in DVC remote repository on any file system that we are going to describe in next section.

Once we do this, the directory structure for MNIST pipeline project looks something like this.

Fig 3 File structure after dvc add

Storing & Sharing

The actual data files (which are not tracked by Git) will be stored in any kind of file system which can also be called DVC repository. File system for DVC repository can be AWS S3 bucket, Google Drive, Google storage bucket, Azure storage, Object Storage Service or any custom created file system. Depending on where you will store the data, you will need to install external dependencies like dvc-s3, dvc-azure, dvc-gdrive, dvc-gs, dvc-oss, dvc-ssh. You can know more about installation here on the documentation website.

To push the data files in DVC repository, first we need to configure the remote origin of the repository where the data will be stored. We can do that using below commands.

$ dvc remote add -d storage s3://mybucket/dvcstore
$ git add .dvc/config
$ git commit -m "Configure remote storage"

dvc remote add command adds the remote origin of the DVC repository where the data will be stored. Executing this command will actually add this origin information in .dvc/config file which Git tracks. That is why we also need to commit the changes done in .dvc/config file as well.

Now, adding the remote origin doesn't automatically push the data in DVC remote repository. We can push the entire data file using below command:

$ dvc push

It pushes all the data and model files on which dvc add command is applied.

Retrieving

There is a reason behind dvc remote add stores remote origin information in .dvc/config. It does so because when next time somebody clones this Git repository, It will not have any data. It will only have MD5 signature of the data in corresponding .dvc file and it will also have DVC remote repository location information stored in .dvc/config file. So, the complete data can be pulled using below command:

$ dvc pull

It pulls all the data and model files stored in DVC remote repository.

ML Pipeline & Versioning

While working with DVC, we have to create two files dvc.yaml which contains information about the stages the ML pipeline will have and params.yaml which contains the parameters that different stages of the ML pipeline would use. You can take a look at the structure of both files on the example repository.

DVC is so flexible that you can use command dvc exp run to run the complete pipeline with just single command. The command is not only limited to this.

If you want to run your experiments with different parameters, you can mention the values of the parameters which are defined in params.yaml as shown below.

$ dvc exp run -S prepare.test_split=0.2 -S train.learning_rate=0.0001

Note here that whenever you run the experiments using the above command, DVC caches the output so well that it will only run those states which are affected by the parameters. Moreover, if you think one of the stages in the pipeline are too much time consuming and you want to checkpoint it so that in case the program crashes, it can resume from where it crashed, you can define checkpoint: true in the output of that stage in dvc.yaml file and it will keep checkpointing your results for you.

DVC keeps track of all the experiments that you perform and even you can see the history of the previously run experiments using dvc exp show. It will show you all the previously run experiments. You can filter this history as well. For example, dvc exp show --include-params=train will only show you the past experiments where any of the parameters of train stage was changed. Not just that, but you can also see the difference between two experiments' parameters and their resultant metrics as well using dvc exp diff exp-1dad0 exp-1df77. where exp-1dad0 and exp-1df77 are tags of two experiments as shown in history given by dvc exp show. Moreover, You can navigate to any previously run experiment using dvc exp apply. For example, if you run command dvc exp apply exp-1dad0, then the files will be changed as per the parameter changes specified in that particular experiment.

DVC gives us the flexibility to commit the experiment into a separate Git branch. As shown in the below command, using dvc exp branch command you can commit the experiment exp-1dad0 to separate branch my_branch.

$ dvc exp branch exp-1dad0 my_branch

Alternatively, you can also commit different experiments in different branches just by switching the branches as described below.

As we already understand switching between different versions of code by changing Git branch. Similarly, we can also change the version of the data as well.

$ git checkout experiment_v2
$ dvc checkout data/data.xml

We can observe that changing the version of data is now as simple as changing the Git branch. Note here that to switch to another version of data, we first need to switch to that particular Git branch where that version of data resides. Also, note here that you must apply dvc checkout to those files for which you want to switch the data version since git checkout only changes the code version.

How data versioning simplifies experimentation?

As described in the beginning of this blog, we don't need to keep multiple copies of model files anymore and keep renaming them. Now, we can have single model file which can be versioned using DVC during experimentation. Below image explains the same.

Fig 4 Model versioning for experimentation

Let us suppose we are working in Git branch v1 where the hyper-parameter n_estimators to train model has value 50 and we have trained a model which is stored as model.pkl. Now, we want to experiment with value 100 for the same hyper-parameter n_estimators. So, we don't need to rename the old model file anymore. We will switch to another Git branch v2 using git checkout v2 command and we will switch to different model file version using dvc checkout model.pkl. We will change the value of hyper-parameter n_estimators to 100 now in code in v2 branch and we can train the model again. The new model.pkl file generated in v2 branch is corresponding to n_estimators = 100 and model.pkl file in v1 branch is corresponding to n_estimators = 50.

Although, this is very small experiment but the same thing can be applied in complex experimentation process. And with data versioning, the experimentation process becomes very flexible. We don't need to worry about tracking of data and model files.

Large datasets versioning

When the dataset is too large, we need very efficient mechanism in terms of space and performance to share different versions of data. That is why it is suggested to store the data in shared volume or on an external system. DVC supports both of these mechanisms as described below.

A shared cache can be setup to store, version and access lot of data on a large shared volume efficiently.
As we described earlier, the more advanced approach is to store and version the data directly into remote storage (ex. S3 bucket, Google Drive, GCP storage bucket etc.). You can look here to know more about how to configure external data storage for DVC.

Conclusion

With this, I hope you were able to get a quick overview of how DVC does data versioning. DVC also has features to streamline ML reproducibility and support cloud provisioning through cloud providers. This would come in handy for the entire MLOps community as now they could interact with each other without worrying about monolithic tools.

If you like to know more about how to do it and get your hands dirty, the documentations are a great way to start.

If you prefer videos then they have a youtube channel with detailed tutorials!

Happy Versioning!

Helm: Package Manager for k8s

Hrittik Roy — Wed, 25 May 2022 20:04:41 +0000

Kubernetes was started inside Google to provide a layer of abstractions with containers for the modern infrastructure. Now, the technology is adopted by the masses and has become a de facto standard for any cloud native application. The open source system provides management, deployment, and scaling of your containers.

Kubernetes is hard to beat in orchestration, but one of the most significant drawbacks is its lack of reproducibility. Here comes Helm: A package manager for Kubernetes and a CNCF Graduate Project.

I was thrilled to speak at DoK Talks on the 114 Edition about Helm and how it tackles the reproducibility problem for Kubernetes. This post will be a summary of the beginner focused event which took place on 27th January 2022.

It's not required for you to go through the recordings as this is a very extensive summary and going through this will give you a basic understanding of Helm!

What’s the reproducibility problem?

After you have deployed your application with numerous objects: Deployments, Services, ConfigMaps, etc., how do you help your friend get to a similar state? Of course, you will share your YAML files with your friend. Correct?

Yes and No!

You can do the hard work of copy and pasting for a small application, but what if your application is a full-stack web app with 100s of configuration files. Can you still copy them? No, you can’t, as it’s prone to errors with that many large numbers.

Even if you send the object manifest to your friend, there’s one question still available. How would your friend convert site to site, or maybe change how much resource each application consumes? Will he go through the 100 manifests? No, that’s not scalable, prone to errors, and wastes a lot of time.

Now suppose there’s a security flaw on one of your dependencies. How will you update them? Find your YAML, or edit your live resources? Absolutely NO!

You need a saviour. You need Helm to your rescue.

What is Helm?

Helm is your saviour for the reproducibility problem, a package manager, and a CNCF graduate project. It was launched in 2016 and has seen massive adoption among organizations, individuals since then. Under the CNCF Umbrella, Helm has become the de facto package manager for your clusters.

What does Helm do?

Helm helps you to achieve reproducibility in the following ways:

Provides an easy way to deploy complex application
Provides easy way to update specific values for your deployments
Provides a way to version a particular package
Provides a way to share your templates across organisation, Internet
Provides an easy way to manage dependency
Provides an easy way to rollback changes

Architecture of Helm

The Helm Repository contains all the charts (packages) created by you or other people that you can use to reach the desired state. The Helm CLI pulls the package, unarchives it, and then converts the charts to a valid YAML, which is then pushed to Kubernetes API server, which creates a release.

Basic Components of Helm Charts

A basic Helm Chart has the following structure:

package-name/

charts/

templates/

Chart.yaml

values.yaml

charts/: This directory can be used to store manually maintained chart dependencies.

templates/: These contain the template files which would be used to create the final manifest after combining with values.yaml.

Chart.yaml: This file contains information about the chart, such as the name and version of the chart, the maintainer, dependencies, a related website, and search terms.

values.yaml: This contains the default configuration for your charts. You can edit this for updating values and remove the complexity of finding specific editable items in the different manifests.

The below example shows a deployent.yaml from templates being rendered with the custom values from values.yaml to produce the valid YAML.

Helm Templates Creation with values.yaml

How to edit the default values?

Manually pulling the charts and unzipping it to edit your values.yaml is not that straightforward. We have Portainer here, which does all the heavy lifting for you and helps you to get straight to editing the default configuration.

First Navigate to Helm from the Menu and then add a repository. In a standard installation of Helm, you need to add a repository, but with Portainer, you get Bitnami by default and can add more when required. Select a namespace and an application name. Then you need to select the chart you want to deploy to your cluster.

Once selected, you can navigate to a chart, and Portainer will load the values on your dashboard for you to edit. Editing values are straightforward and abstract the complexity you had to deal with manual installation and initialization via a simple GUI.

The Install button would install your chart with the specified values to your cluster. If you want to go with the default values, click on Install without editing the values.

After installation, Portainer detects and shows you Published URLs to access your applications and secrets for you to access default passwords. Forget digging through commands to get to your Services and Secrets.

Final Thoughts

Helm abstract the complexity of installing applications to your cluster. Portainer abstracts the complexity of managing your cluster. This post went through how Portainer can help you simplify your Kubernetes workflows with Helms.

Try Portainer now and learn more about the different ways to streamline managing Kubernetes with our documentation.

Portainer Documentation for Helm here: https://docs.portainer.io/v/ce-2.11/user/kubernetes/helm

Recordings are here:

The DevOps Roadmap: Containerization, Containers and why do you need them?

Hrittik Roy — Wed, 25 May 2022 20:03:40 +0000

Containers are a popular term in the industry and help developers develop and deploy apps a lot faster. While by using virtualization, you can run various operating systems on your hardware containerization to run multiple instances or deploy multiple applications using the same operating system on a single virtual machine or server.

This capacity to run multiple applications on the same resource summarizes into making our development life cycle efficient. In this post, we would summarize everything you must know about these unfamiliar words and how exactly containerization makes our life easy.

Let’s dive in!

What is Containerization?

Containerization is simply packaging all the required environment, libraries, frameworks, directories, and application code to create a container. Citrix defined it as:

Containerization is defined as a form of operating system virtualization, through which applications are run in isolated user spaces called containers, all using the same shared operating system (OS). A container is essentially a fully packaged and portable computing environment that already has all the necessary packages to run the application inside.

Simply put, the process of creating containers is called containerization.

What is a Container?

The word container has its origin form contain, and it does the same thing, i.e., it contains everything you need to run an application.

Containers packages and contains code and all its dependencies, so the application runs quickly, portably, and reliably from one computing environment to another.

How does a Container work?

Containers run via a containerization engine over a single host operating system by sharing the operating system kernel with other containers. This sharing is achieved by limiting parts of the OS to a read-only mode. This sharing makes containers extremely lightweight on the resources as you don’t need to configure a new operating system for each new container.

Docker a containerization engine running multiple containers. Image Credits: Docker

As you see, containers differ a lot from virtual environments (you’d need a separate OS per system here), and if you’d want to know more, we have put a blog discussing the core differences elaborately.

We would also discuss why VMs are more secure than containers and other drawbacks in the same blog.🤯

Why do you need Containers?

Development is a complicated task, and as a developer, you need to tackle issues and keep in mind many things while you develop. Containers help you keep your focus on the code and care less about other substances like the environment during development.

A few reasons you should use a container to make your dev life easy are:

Consistent Environment

Development is a critical task, and you must take into account a lot of factors ranging from libraries, frameworks to network configuration or directory management while deploying the application. Think about how different Linux and windows manage their directories.

Problems arise when the supporting software environment is not identical, says Docker creator Solomon Hykes. “You’re going to test using Python 2.7, and then it’s going to run on Python 3 in production and something weird will happen. Or you’ll rely on the behavior of a certain version of an SSL library and another one will be installed. You’ll run your tests on Debian and production is on Red Hat and all sorts of weird things happen.”

Containers have the opportunity for developers to build predictable environments isolated from other applications. The application’s software dependencies can also be contained in containers, such as particular versions of programming language runtimes and other software libraries.

This all turns in favor of a developer. The focus stays on quality code and not on the bugs that might creep while migrating code from his system to the server.

Less overhead

A container can be just tens of megabytes in size, while it may be several gigabytes in size for a virtual machine with its entire operating system. Because of this, far more containers than virtual machines will hold a single server, which boils down to consuming less resources for one individual container.

Just In Time

It can take several minutes for virtual machines to boot up their operating systems and begin running their host applications. At the same time, it is possible to start containerized applications almost instantly. That implies that when they are needed, containers can be instantiated in a “just in time” mode and can disappear when they are no longer required, freeing up resources on their hosts.

Modularity

Containers can split applications into modules (such as the database, the application front end, and so on) instead of running an entire complex application within a single container.

This is the method of so-called microservices. It is simpler to handle applications designed in this way because each module is relatively easy, and improvements can be made to modules without the whole application needing to be rebuilt. Since containers are so lightweight, it is possible to instantiate individual modules (or microservices) only when required and almost immediately accessible.

Run Anywhere

Containers can run almost anywhere, making development and deployment much easier: on Linux, Windows, and Mac operating systems; on virtual or bare metal machines; on the developer’s machine or on-site data centers; and, of course, in the public cloud.

Container images like that of docker help enhance portability as they are widespread and well supported.

You can make use of containers anywhere you want to run your apps.

What is Container Image?

To rebuild a container, you need some file or a template that contains the instructions of what the replication must include.

Container images are the same templates that help you to rebuild a container. The template consists of unchangeable static files that can be shared, and upon building, using the shared image yields a similar container.

Docker Image is an example for Container Image used to build containers.

An image comes in handy when you need to recreate a consistent working environment within another container without dealing with traditional long and boring manual configurations.

What are the popular Container Engines?

Docker

Docker’s open-source containerization engine is the first and still most popular container technology among various competitors. Docker works with most commercial/enterprise products, as well as many open-source tools.

You can read more about docker and docker images in this curated beginner-friendly post.

CRI-O

CRI-O, a lightweight alternative to docker, allows you to run containers without any unnecessary code or configuration, directly from Kubernetes, a container management system.

Kata Containers

Kata Containers is an open-source container runtime with lightweight virtual machines that feel and function like containers but use hardware virtualization technology as a second layer of protection to provide more robust workload isolation.

Microsoft Containers

Positioned as an alternative to Linux, Microsoft Containers supports Windows OS in very specific conditions. Typically, they run on a real virtual machine rather than a cluster manager like Kubernetes.

Final Thoughts ⭐

Containers are becoming essential as the direction of development is moving towards cloud-native. The advantages to using containers like the flexibility and agility are unmatched, and adoption is on the rise.

I hope this blog helped you understand containerization and containers in-depth, and if you want to experiment with containers, you must know more about the best practices used by various companies of all sizes while adopting containers.

Happy Containerizing!