The Ops Community ⚙️: Rutam Prita Mishra

Deploying MindsDB on Oracle Cloud

Rutam Prita Mishra — Thu, 13 Oct 2022 18:29:04 +0000

Introduction

MindsDB enables in-database machine learning capabilities. It enables the user to query the future from inside the existing database using AI tables.

Generally, MindsDB helps to build accurate predictive models by processing the existing data and feeding it into training models using standard SQL queries. It also enables to visualize the predictions using the BI tools.

So, today in this tutorial we are gonna learn how to deploy a self-hosted version of MindsDB using Oracle Cloud Compute Instance.

Deploying an Oracle Cloud Compute Instance

Oracle Cloud Infrastructure is widely used across the IT industries and provides a set of highly available and scalable cloud services that enable to build and serve a wide range of applications.

So, let's go ahead and create a Compute Instance on Oracle Cloud using the steps below.

Step 1: Log in to your Oracle Cloud Infrastructure account or signup for one if don't have one already. Oracle provides 300$ worth of credits for 30 days to try their services.

Note: It may take a while to provision your account and you will receive an email once it is ready. Your account needs to be fully provisioned and ready so that you can use the different services on the platform.

Step 2: Once your account is ready and you're signed in, you will land on a Get Started page as shown below. Simply click on the Instances from the top Pinned tab.

Step 3: On the next screen that appears, select your username from the Compartment section under List scope in the left panel and then hit the Create instance button.

Step 4: In the Create compute insatnce page, we need to fill in the configurations for the virtual machine as per our needs. But for the sake of this tutorial, we can go with the following configuration.

Name: Put a name as per your choice (I chose mindsdb-demo for now.)
Create in compartment: It will be your username(root).
Placement: Leave this section as it is at its default values.
Image and shape: Click on Edit and choose the following.
- Click on Change image.
  - Image Source: Platform Images
  - Compartment: your_username(root)
  - Image Name: Canonical Ubuntu
  - OS Version: 22.04
  - Image Build: 2022.08.10-0 (Latest when this is written)
- Now click on Select image to save these changes.
- Now click on Change shape.
  - Instance type: Virtual Machine
  - Shape Series: Speciality and previous generation
  - Shape name: VM.Standard.E2.8
- Click on Select shape to save these changes.
Networking: Leave the values as it is at its default.
Add SSH keys: Select Generate a key pair for me. Don't forget to download and save both the Private and Public keys as we will require them later to interact with the VM.
Boot Volume: Just tick the Use in-transit encryption and leave others unchecked.

Now we are all set with the specific configuration we require to run this VM. Let's click on the Create button now to provision this VM.

Step 5: After we hit the Create button, it takes us to the Work requests page for the instance. Please wait a while here, until the I image box at the left top turns green and you see the Running status below it.

Step 6: We can simply SSH into our VM now using the Private key we downloaded earlier using the CMD with the following command.

ssh -i location_of_the_private_key username_for_the_instance@public_ip_of_the_instance

If I replace the placeholders in the above command, then the command would look something like this. Type yes and hit Enter when prompted in the CMD to access the VM from your command line.

ssh -i ssh-key-2022-10-13.key ubuntu@129.154.45.47

We are now ready to proceed with the next steps to install the MindsDB instance in our VM.

Deploying MindsDB on the Cloud Instance

In this section we will deploy the MindsDB instance on the VM and then try to use its GUI using the public IP of the VM on the respective port. The following will let us do easily.

Step 1: Once we SSH into the instance, we need to update the package indexes present in it so that we can get the latest package versions when we install something. We can do that with the command below.

sudo apt-get update

Step 2: We will use Docker to install MindsDB in the later steps. So, let's install Docker first in the VM with the command below. Type Y and hit Enter when prompted to proceed with the installation.

sudo apt-get install docker.io

Once this command executes successfully, simply run the command below to check whether Docker is running fine or not. This should return a Hello from Docker! meesage.

sudo docker run hello-world

Step 3: We will now pull the latest stable release of MindsDB into our instance with the following command.

sudo docker pull mindsdb/mindsdb

You can also try working with the latest beta releases of MindsDB using the command below.

sudo docker pull mindsdb/mindsdb_beta

Step 4: We can now run the MindsDB instance and expose the specific ports that are required to access the MindsDB GUI and have successful data transactions through the MySQL APIs. The command will be as follows.

sudo docker run -p 47334:47334 -p 47335:47335 mindsdb/mindsdb

Note: Port 47334 will be used by the GUI.
Port 47335 will be used by the MySQL API.

Step 5: Now if we try accessing the GUI with the public IP of the instance on port 47334, it will simply error out. This is because the Oracle Cloud Infrastructure blocks the incoming traffic by default for the instance. So, we need to define an ingress rule for the instance now to be able to access the GUI.

Go back to the Instance details dashboard and click on the subnet name. This should take you to the Subnet Details page.

In the Security Lists table, click on the only available record of Default Security List. This will take you further to the Security List Details page.

Now click on the Add Ingress Rules button under Ingress Rules tab. This will open a dialog box where we can define the ingress rule, we want to use for the instance.

Now use the following details to define the ingress rule.
- Stateless: Leave it unchecked as it is.
- Source Type: CIDR
- Source CIDR: 0.0.0.0/0
- IP Protocol: TCP
- Source Port Range: Leave it blank. (Default is Allow All)
- Destination Port Range: 47334-47335
- Description: MindsDB Ingress Rule

Now click on Add Ingress Rule button to add this rule to the Default Security List.

Step 6: We can try to access the MindsDB GUI with the public IP and the respective port and should be successfully able to access it.

http://publicIP:47334/

MindsDB GUI has 3 major parts i.e., a Query Editor at the top, a Result Viewer at the bottom and the Learning Hub resources at the right side to make it more accessible to new users.

Note: With root user access to the instance, we will never need to use sudo before each and every command.

Exploring the Default MindsDB Tables

By this time, we already have our MindsDB deployment hosted on the public IP of the instance. So, it's time now to find out what default tables exist in MindsDB when we launch it for the first time.

Let's run the following SQL command to get the list of available Tables.

Show tables;

We can find three tables listed above, namely:

Predictors: This table stores each of the trained Predictor Models along with their specific details as a separate record.
Predictors_Versions: This table stores the versioning information of a particular Predictor Model as to how many times the model has been tweaked and newly trained by the user.
Databases: This table stores the list of all the databases connected to our MindsDB instance.

Conclusion

We have pretty much reached to the end of this tutorial now. So, before we leave, let's summarize all the things that we just did above. Initially, we started with creating our Oracle Cloud account, provisioned a compute instance, installed MindsDB using Docker inside it, created an ingress rule for the incoming traffic and finally explored the MindsDB GUI along with the default tables present.

Now, as all of you know how to get started with MindsDB on Oracle Cloud Infrastructure, it's time for all of you to deploy your own MindsDB instances and start training some interesting Predictor models with it and share about it with the community.

Lastly, I would love to know your feedback and would really appreciate it if you drop a LIKE and share this article in your circles where you believe it will add some value.

Note: All the dashboard images are respective properties of Oracle Cloud Infrastructure and MindsDB and are used here only for illustrative purposes.

Deploying MindsDB on Google Cloud Platform

Rutam Prita Mishra — Sat, 13 Aug 2022 22:05:00 +0000

Introduction

MindsDB is an AI layer that works on top of the existing databases to train, develop and deploy the finest machine learning models seamlessly. MindsDB performs the pre-processing of data from within the database and feeds it to the machine learning model using standard SQL to do precise and accurate predictions as required.

In this tutorial, we are going to deploy MindsDB on a GCP Compute Engine instance, go through its GUI and check through the default tables that are present after deployment.

Deploying a GCP Compute Engine

Google Cloud Platform is known to all and used by many in current times. So, it's time to go ahead and create a Virtual Machine on GCP to get started. You can simply follow the steps below to get it done.

Step 1: Login to your GCP account or sign up for one here. GCP provides 300$ worth of credits for 90 days to try the platform.

Step 2: Once you're signed in, click on the Go to console button. This should open up the cloud console for your account in a new tab.

Step 3: On the cloud console screen, simply hit the Create a VM button. If it is your first time using the platform, then it would ask you to enable the Compute Engine API first and then take you to the Create an instance page.

Step 4: We need to specify the specs for the compute instance on the Create an instance page. You can always specify these as per your needs, but we will be using the specs mentioned below for this tutorial.

Name: Provide a name for your instance
Labels: Leave it as it is.
Region: Select the one near to you for lower latency
Zone: Choose any zone out of the available ones
Machine Family: General Purpose
Series: E2
Machine type: e2-medium (2 vCPU, 4Gb Memory)
Display device: Leave it as it is.
Confidential VM Service: Leave it as it is.
Container: Leave it as it is.
Boot Disk: Click on Change, use the following and click on Select.
- Operating System: Ubuntu
- Version: Ubuntu 22.04 LTS x86/64
- Boot disk type: Balanced Persistent Disk
- Size: 50
Service account: Compute Engine default service account
Access Scopes: Select Allow full access to all Cloud APIs
Firewall: Select Allow HTTP traffic
Advanced Options: Leave it as it is.

Now you can click on Create to instantiate the launch of the VM.

Step 5: Once you hit the Create button, it will take you the VM Instances dashboard where you can find the instance you just created listed. Wait a bit until the Status turns to a green tick mark and the internal and external IP addresses are visible. This confirms that the instance is ready to use now.

You can now simply hit the SSH button to access the instance and proceed with the next steps.

Deploying MindsDB on the Compute Instance

As we are ready with our instance now, we will simply try to install and run MindsDB inside our instance and access its GUI. Make sure to follow the instructions below to proceed further.

Step 1: As soon as you SSH into your instance, run a apt-update command to refresh the package indexes present in the system.

sudo apt-get update

Step 2: Now it's time to check whether Docker is installed or not as we are going to use this to install MindsDB. We can do that by the following command.

sudo docker run hello-world

This should return a Hello from Docker message which confirms that Docker is present in the machine.

In case, it throws an error, you can simply install Docker using the following command. Hit Y when asked to proceed with the installation.

sudo apt-get install docker.io

Step 3: We will now need to pull the latest stable build of MindsDB into our instance. We can do that with the command below.

sudo docker pull mindsdb/mindsdb

The stable build may or may not be the latest available build. If you wish to work with the latest release, then you can always pull the Beta version with the command below.

sudo docker pull mindsdb/mindsdb_beta

Step 4: Now it's time to run MindsDB on our instance and open the specific ports so that we can access its GUI and the MySQL APIs for data transactions.

sudo docker run -p 47334:47334 -p 47335:47335 mindsdb/mindsdb

Step 5: Although we have exposed the ports above, Compute Engine blocks the traffic by default. So, we need to explicitly define a firewall rule to allow incoming traffic through these two ports. So, let's go back to the VM Instances dashboard and click on Setup firewall rules.

Once you're on the Firewall dashboard, simply hit the Create Firewall Rule at the top to add a rule for these ports.

Now use the following values to set up and create the rule.

Name: Provide a name as you wish.
Description: Provide a description if you want to.
Logs: Select Off.
Network: Select Default.
Priority: 1000
Direction of traffic: Ingress
Action on match: Allow
Targets: Select Specified Target Tags.
Target-tags: Type http-server.
Source filter: Select IPv4 ranges.
Source IPv4 ranges: 0.0.0.0/0
Second source filter: None
Protocols and ports: Select Specified protocols and ports
- Check on TCP.
- Ports: 47334,47335

We can now click on Create to use this firewall rule on our instance.

Step 6: It's time to now try and access the MindsDB GUI. We can do that by hitting the external IP of the instance on the port 47334. This should open up the MindsDB GUI for us.

As we can see, the MindsDB GUI has a SQL Editor at the top, a result viewer at the bottom and a bunch of learning hub resources on the right of the screen.

Note: We don't need to use sudo before each command if we are logged in to the instance as the root user.

Exploring the Tables in MindsDB

As we are already done deploying our MindsDB instance on the compute engine, now let us try to explore the tables that are present by default when we install MindsDB.

We can simply run this command in the GUI SQL Editor to list the available tables.

Show tables;

We can see that there are two tables namely, Predictors and Databases.

The Predictors table stores the predictor models that we train as an individual record. Each of its column contains specific details about the respective predictor model.

The Databases table contains the list of databases that are connected to the MindsDB instance.

Conclusion

We have now reached the end of this tutorial. So, it's time to recollect what we just did above sequentially. We initially created a compute engine instance, deployed MindsDB on top it, created a firewall rule to allow the incoming traffic for MindsDB and finally explored the MindsDB Web UI and tables that are present by default.

Now it's time for you to play around with your own MindsDB instance on a Compute Engine and come up with interesting Predictor models to try and test the robustness of MindsDB. MindsDB also provides its own cloud version which you can use here.

Lastly, before you leave, please drop a like if you found this tutorial helpful and don't forget to key in your feedback as well.

Note: All the dashboard images are sole properties of Google Cloud Platform and MindsDB respectively and are used here only for illustrative purposes.

Deploying MindsDB on Civo Compute

Rutam Prita Mishra — Mon, 08 Aug 2022 20:23:26 +0000

Introduction

MindsDB embeds ML capabilities into traditional databases. With the help of MindsDB, we can make our databases smarter and design several Predictor models to predict useful insights from our available datasets.

In this tutorial, we are gonna learn how we can deploy MindsDB using a Civo Compute instance easily.

Deploying a Civo Compute Instance

Civo provides powerful and reliable compute instances for a fairly decent price. These compute instances are actually high-performance virtual machines that can be launched in just 60 seconds and are powered by Kubernetes under the hood.

Now lets get started by creating a Civo Compute instance. Follow the steps as listed below.

Step 1: Login to your Civo Account or create one if you don't have one yet. You can sign up for an account here and can avail 250$ worth of free credits for a month to try different Civo products.

Step 2: Once you're logged in, click on Compute from the left navigation pane and then either hit the Launch my first instance or Create New Instance button to start configuring the compute instance.

Step 3: On the Create a new Instance page, we need to choose the specs we want our compute instance to boot up with. You can always select the specs as per your needs, but we will be using the specs as defined below for this tutorial.

- Hostname: Give a name as you want.
- How many Instances?: Leave it at 1.
- Select Size: Select Large($40/month).
- Select Image: Select Ubuntu 22.04 LTS(Jammy Jellyfish)
- Initial User: Select root as we want to have use as Admin
- Network: Leave it at Default
- Public IP Address: Select Create.
- Firewall: Select Default.
- SSH Key: Select Random Password.
- Initialisation Script: Leave it blank.
- Tags: Leave it blank.

Now simply click on the Create button to boot up the compute instance.

Step 4: Once you click on Create, the instance starts building and should be up and running in a while. As soon as the instance has started, we can simply SSH into the instance using its Public IP and the SSH information to install MindsDB and proceed with the next steps.

Installing MindsDB on the Instance

Our compute machine is booted up already and now it's time to install MindsDB on it using Docker. Let's get started with the steps below to achieve the same.

Step 1: SSH into your compute instance using the terminal of your system. Open your terminal and type the command below to get access into your instance. Type Y or Yes when prompted and hit Enter to proceed.

ssh root@PUBLIC_IP_OF_THE_INSTANCE

Copy the password from the instance dashboard under SSH Information tab and use it when prompted in the terminal to do a successful login into the system.

Step 2: Now it's time to check whether Docker is installed correctly or not. Use the command below to find this out.

docker run hello-world

This should run successfully, and you should see a message Hello from Docker.

Step 3: Let's run the following command now to pull the latest stable release of MindsDB into our instance.

docker pull mindsdb/mindsdb

We can also work with the latest Beta version images of MindDB using the command below.

docker pull mindsdb/mindsdb_beta

Note: Follow the MindsDB Getting Started documentation to find more such ways to install MindsDB other than Docker.

Step 4: We will now run a command to run MindsDB on our instance and expose specific ports so that we can access its GUI and MySQL APIs, etc. Use the command below to do it.

docker run -p 47334:47334 -p 47335:47335 mindsdb/mindsdb

Port 47334 - The MindDB GUI runs on this port. You can always use the ComputeInstancePublicIP:47334 to access it in the browser.
Port 47335 - The MySQL API runs on this port and enables to carry out database transactions over TCP/IP.

Step 5: We can now simply try to access the MindsDB GUI using the Public IP of the instance over the exposed port of 47334. If we give a closer look at the interface, we can find a query writer at the top, the result viewer at the bottom and the learning hub resources towards the right.

Exploring Tables inside MindsDB

As we have already hosted our MindsDB version on the instance, we are now ready to train the predictor models and start predicting results as we require. But let us first explore the tables that are present by default. We can simply do that with the following command.

Show tables;

As we can see, we get two tables by default i.e., Predictors and Databases.
The Predictors table stores the Predictor models that we train, and each model is stored as a record in this table as an individual row and each of its columns contain specific information about the model.
The Databases table contains the list of databases connected with the MindsDB instance.

Conclusion

Well, now we have reached the end of this tutorial and it's time to do a short recap of what we did here. So, we started with creating a Civo Compute instance, then we installed MindsDB on top of it using docker, exposed the required ports to access the MindsDB GUI and finally explored the tables that are present there by default.

Apart from the self-hosted version, MindsDB also provides a MindsDB Cloud version which you can use for free. So, do give it a try and play around with your own Predictor models.

Lastly, before you leave this page, feel free to drop a like if you liked this tutorial and don't forget to key in your feedback as well.

Note: All the Dashboard images are sole properties of Civo and MindsDB respectively and are used here for illustrative purposes.

Predicting Data Science Job Salaries using MindsDB Cloud

Rutam Prita Mishra — Sat, 06 Aug 2022 10:32:43 +0000

Introduction

MindsDB comes quite handy when we want to work with our existing databases and enable machine learning capabilities with our available datasets. With the use of MindsDB, we can easily train machine learning models and make predictions directly within the database. In short, we can say MindsDB embeds Data Science in our traditional databases to make them smart and enable us to extract meaningful insights.

MindsDB offers a cloud version which we can use completely online, or we can use self-hosted version using Docker or Pip. You can also opt for its free (Limited to 10K rows in the training dataset) or paid version as per your requirements.

Today in this tutorial, we will be using the MindsDB Cloud to train a Predictor model that can predict the salaries for different Data Science jobs based on several feature parameters.

Importing data to MindsDB Cloud

We will need a dataset initially that contains salaries of data science positions based on several candidate parameters. You can download a copy of the dataset here.

Once you have downloaded the dataset, simply extract it and save the CSV file for later use.

Now let us get started with our MindsDB Cloud account to take things further.

Step 1: Login to your existing MindsDB Cloud account or signup for a new one here.

Step 2: Once you have signed up or logged in to your MindsDB cloud account, you can find the MindsDB Cloud Editor opened up for you.

The top panel is for writing the query, the bottom panel is for displaying the results and the right panel lists out some of the learning hub resources to make things easier for the new users.

Step 3: Now simply click on Add Data from the top right and on the next screen that appears, switch over to Files instead of Databases and then click on Import File.

Step 4: Now on the Import File dashboard, browse the dataset file from your computer, set a name for the Table and then hit the Save and Continue button. This should easily upload the dataset file for you and create a Table with the name you provided.

Step 5: Now the GUI returns back to the Editor screen where you can find two generic queries mentioned to either list the tables or query the data in the table that you just uploaded. Let's run both of these queries and check through the results.

Now the second query returns the following result.

We are now ready to create a Predictor model using our Salary table that we just uploaded.

Training a Predictor Model

MindsDB makes it extremely easy to define a Predictor model and train it by using a simple SQL syntax.

Step 1: We will now use the CREATE PREDICTOR syntax to create the Predictor. Find the syntax below to learn more about it.

CREATE PREDICTOR mindsdb.predictor_name (Your Predictor Name)
FROM database_name                      (Your Database Name)
(SELECT * FROM table_name LIMIT 10000)  (Your Table Name)
PREDICT target_parameter;               (Your Target Parameter)

This query should get executed and return successful in the Result Viewer.

Note: We have used LIMIT 10000 here as we are using the free version of MindsDB and it supports upto 10K rows and will error out if we have more rows.

Step 2: The model should take a little while to get created and trained. We can check the status of the model using the syntax below. If the query returns Complete, then the model is ready to use or else wait if it returns Training status.

SELECT status
FROM mindsdb.predictors
WHERE name='name_of_the_predictor_model';

Describing the Predictor Model

MindsDB provides a DESCRIBE statement that we can use to gain some insights into the Predictor Model. We can find more details about the model in the following three ways.

By Features
By Model
By Model Ensemble

By Features

DESCRIBE mindsdb.predictor_model_name.features;

This statement is used to find out the type of encoders used on each column to train the model and the role of each of the columns for the model. A sample output for our model is posted below.

By Model

DESCRIBE mindsdb.predictor_model_name.model;

MindsDB uses several candidate models internally to train the data and then picks up the most optimized one for the model to do the predictions. This statement simply lists out all the candidate models used to train the data along with other details. The model with 1 in its selected column is the one that is the most optimized and accurate.

By Model Ensemble

DESCRIBE mindsdb.predictor_model_name.ensemble;

With this statement, we can simply query out a JSON object that lists out the multiple attributes used to select the best candidate model to do the predictions finally.

Querying the Model

Now that we are ready with our Predictor Model, we can simply execute some simple SQL query statements to predict the target value based on the feature parameters.

We will get started by predicting the Salary based on only one feature parameter and the query statement should look something like this.

SELECT salary
FROM mindsdb.salary_predictor
WHERE experience ='SE';

This should return the expected salary for the person with the experience level of SE.

Let's now try predicting the salary based on multiple feature parameters. The query should look something like this.

SELECT salary
FROM mindsdb.salary_predictor
WHERE experience ='EN' and job_title='Data Analyst';

This should return the expected salary for the person with the experience level of EN with the job title of Data Analyst.

Conclusion

With this we have reached the end of the tutorial. In this tutorial, we created our own MindsDB Cloud account, uploaded a dataset to the cloud interface, trained a predictor model with our dataset and finally predicted the salaries of different Data Science jobs.

It is really easy to go out yourself and try training your own predictor models using datasets available online through MindsDB. So, go ahead and give it a try and do all the predictions you want.

Lastly, before you leave this page, drop a LIKE if you learnt something new and interesting today and also don't forget to key in your feedback below.

Getting Started with Speedscale on Civo Managed Kubernetes

Rutam Prita Mishra — Mon, 20 Jun 2022 20:00:40 +0000

Introduction

We have been largely using APIs in our day to life to perform various operation in an application. APIs are quite crucial in establishing a connection between two intermediate services to maintain the dataflow.

But managing and monitoring large number of APIs can be very overwhelming and time-consuming, and we still might miss some deeper insights if we do it manually. In such a situation, Speedscale comes into play.

Speedscale is a framework that offers traffic replay using realistic API traffic and provides with auto-generated tests to simulate the load or chaos and infer the confidence by measuring the performance of the APIs inside the Kubernetes Clusters.

In this tutorial, we will simply spin up a Civo managed Kubernetes cluster, add Speedscale to the cluster and then patch it's sidecar with one of the application deployments to gain insights into the real-time API traffic for that deployment.

Creating a Civo Managed Kubernetes Cluster

Now we will start by creating a Kubernetes Cluster using the Civo Dashboard.

Step 1: Log in to the Civo dashboard with your credentials or you can simply sign up here if you don't have an account yet. Civo provides 250$ worth of free credits for 1 month to its new users so that they can explore the platform freely.

Step 2: Once you are logged in, make sure Kubernetes is selected in the left navigation pane and then click on Create new cluster or Launch my first cluster to open the cluster configuration dashboard.

Step 3: Once you're on the configuration page, you can select the specs you wish to have for your cluster. For the sake of this tutorial, you can use the specs mentioned below.

->Name- Give a name as you want (For e.g, speedscale-demo)
->Network- default
->Firewall- Deafult, unless you want to configure anything specific
->Select size- Large Standard (Switch to Hourly mode)
->Show advanced options- Leave it at default
->Marketplace- You can leave it at default or select any app if you wish to install (For e.g., Helm from Management Tab)

Now click on Create cluster and wait for it to get provisioned. This should take a while.

Step 4: Once the cluster is ready, simply download the kubeconfig and merge it in the config file in the .kube folder on your local system or you can simply overwrite it as well.

Note: If you've merged the kubeconfig in your existing config file, then make sure to switch the context to use the cluster you've just created. You can do that with the command below.
kubectl config use-context CONTEXTNAME

Step 5: Now we will simply try to fetch the list of nodes using kubectl to cross-check if we are able to interact with the cluster or not. Run the command below to get it done.

kubectl get nodes

If it successfully returns the list of nodes, then we are good to go with the next steps.

Note: If you need help with configuring kubectl, then you can find a detailed guide here.

Deploying Speedscale on the Cluster

Now we need to create a deployment for the Speedscale Kubernetes operator. Let's do it in the subsequent steps below.

Step 1: Sign up for a Free Trial account on Speedscale. Speedscale offers a 14-day trial and also a free community edition for all types of users.

Step 2: Once you've signed up, you should receive a mail with the available ways to install Speedscale (Either using the Helm Chart or the command line tool). Also, log in to your dashboard and stay signed in as we will come back to use this dashboard at a later stage in this tutorial.

For complete steps on using Helm chart, visit here.

For complete steps on using command line tool visit here or for using the install wizard visit here.

Step 3: For now, we will use the helm chart to install the Speedscale operator on our cluster.

-> Add the Speedscale repo using Helm first. Use the command below to do so.

helm repo add speedscale https://speedscale.github.io/operator-helm/

-> Check and update the repo

helm repo update

-> Now to install the chart, use the command below replacing the placeholders for your Speedscale API Key and the Cluster Name where you want to deploy.

helm install speedscale-operator speedscale/speedscale-operator \
    -n speedscale \
    --create-namespace \
    --set apiKey=<YOUR-SPEEDSCALE-API-KEY> \
    --set clusterName=<YOUR-CLUSTER-NAME>

Note: If you face a pre-install job failure while installing, then you can follow the troubleshooting guide here.

Step 4: Now let's check the pods to confirm whether the Speedscale charts are deployed or not.

kubectl get pods -n speedscale

This should return a list of pods in the speedscale namespace.

We are now ready with our Speedscale Kubernetes operator and can proceed with the next steps.

Deploying the Demo Podtato App

We will now deploy a demo app and stress test the API by pinging it multiple times. So, let's get started with that.

Step 1: Let's deploy the app using the Podtato yaml file. we can do that with the command below.

kubectl apply -f https://raw.githubusercontent.com/speedscale/podtato-head/main/delivery/kubectl/manifest.yaml

Step 2: Now lets check if the respective pods for the app are already deployed or not.

kubectl get pods (As it was deployed in the default namespace.)

Step 3: Now let's open this app in browser to check how it looks. The URL should look something like this.

http://[YOUR CLUSTER EXTERNAL IP]:31000/

Step 4: Now let's patch the Podtato app with the Speedscale operator sidecars to keep track of the traffic coming in/out of the app. We can achieve this with the command below.

kubectl annotate deployment --all sidecar.speedscale.com/inject="true"

We have now the Speedscale sidecars running with the Podtato app pods making it 2 deployments in Running state. Let's move on to the next steps now.

Load testing the Podtato App

It's time to put some load now on the Podtato app to trace them on the Speedscale dashboard and analyze the traffic. Follow the steps below and we should be good to go.

Step 1: Now we can simply generate some load/traffic by hitting the Podtato app URL multiple times or by simply refreshing the browser tab.

Step 2: After a while, we can see that our Podtato app services get enlisted in the Speedscale Dashboard. This ensures that the sidecars are successfully capturing the traffic logs.

Step 3: Now if we click on each of the service names, it will take us to the Services insights page where we can see specific details about the services, the inbound/outbound transfer rates and health of the service.

Step 4: Now if we click on the first item on the list which was marked in red above, then it will take us to the traffic distribution page for that specific service.

It gives a brief information about the kind of request, no of requests, the port and the service that it hits, as a part of its Service Map section.

Also, it lists out all the traffic records below at its Traffic section and clicking on each of them further provides insights like the request headers, response body, info and tags.

Well, this is pretty much how we can load test the Podtato app and visualize its traffic using Speedscale.

Conclusion

We have now reached the end of this tutorial. In this tutorial, we created a cluster on Civo, installed Speedscale operator on it, deployed a demo app, patched it with the Speedscale operator and finally visualized the traffic after load testing.

Speedscale definitely does a great job in logging the traffic for all the patched services. The way it visualizes their traffic on the dashboard and the number of insights it provides for each of them is pretty impressive. Feel free to sign up for Speedscale and give it a spin with your own use-cases.

Lastly, show some love to this article if you really found this useful and don't forget to share your feedback as well.

Note: All the dashboard images used here are sole properties of Civo and Speedscale respectively, and are used here only for illustrative purposes.

Walk-through for DigitalOcean Functions Challenge

Rutam Prita Mishra — Sat, 11 Jun 2022 21:28:32 +0000

Introduction

DigitalOcean has recently launched a new feature for its platform users and it's the most awaited Serverless Functions. With the availability of Functions, the resources are managed internally by DigitalOcean and they even take care of scaling them up or down based on demand automatically. Some of the other basic key features include the following.

More Focus on Code
This enables the developers to only work on their code and maintain its quality as DigitalOcean takes care of the provisioning and scaling of resources.
Cost-Saving
With the use of functions, we no longer need to keep any idle server up and running all the time. We are simply charged only when the functions are triggered by an event.
Autoscaling
DigitalOcean handles the infra as per the demand and one need not worry about its availability of it during its peak or idle hours.
Faster Production-ready Deployments
With functions, we only need to choose the coding language and can then deploy them easily with a few clicks. This cuts the time which used to be wasted on infra-management and hence, makes deployments faster.

The DigitalOcean Functions Challenge

DigitalOcean is always community-driven and widely wants its community to try the new features and provide suggestions for the further scope of improvements. So, they have come up with a brand-new Functions challenge where one needs to call a Serverless Function to deploy Sammy, the official DigitalOcean mascot, in the shark tank created for this challenge.

Not only one can try the new Functions on the platform, but also the first 500 valid entries can win some DigitalOcean swags as well. You can find the detailed guidelines here.

Now, let's figure out some of the ways by which you can easily complete this challenge and win some swags as well. Remember that you can do this in several other ways as well and the use of Functions is not really limited to the following ways. You are always encouraged to try as many things as possible with the all-new DigitalOcean Functions.

Creating a Namespace to Deploy Functions

Before we begin, let's get the pre-requisites set up first.

Step 1: Sign up for a new DigitalOcean account if you don't have one yet or else use your existing credentials to log in to the platform here.

Step 2: As soon as you log in, simply click on the Functions tab from the left navigation pane to open the Functions dashboard.

Now click on launch to create a namespace. This should be ready in a few seconds.

Step 3: Once you're inside the Namespace dashboard, you can clearly see that there are two ways by which we can deploy the functions; either using our local machine and code editor or directly from the browser.

We will use the second way of doing that directly through the browser for now. So, click on Go to Source button.

Step 4: Once you're in the Source tab, simply click on the New Function button.

This opens a New Function dialog box where we need to select the Runtime, Function Name and Package Name (optional).

As you would have noticed now, there are 4 available runtime options namely, Go:1.17, Nodejs:14, Php:8.0 and Python:3.9.

So, in this tutorial, we will go ahead and create Functions for each of these runtimes separately and deploy a Sammy in each case. However, deploying the function for any one of the runtimes completes the challenge for you and you're eligible to win some swag.

Walk-throughs for the Challenge

Now we are ready to deploy some Functions to create a Sammy in the Aquarium, let us first understand what exactly we need to do with the function that we create.

So, basically, we need to do a POST API request to the challenge end-point URL with the specified headers and body parameters containing the name of the user and the type of Sammy they want to deploy. You can view the detailed completion steps and the Aquarium here.

Walkthrough for Go:1.17 Runtime

Step 1: After you click on New Function, in the New Function dialog box, select Go:1.17 as the runtime, give a name as you want and then a package name if you wish to and finally hit the Create button.

Step 2: As we know what the endpoint URL, method type, payloads and the headers to be included, we can now design a function like the one below to do the API call.

package main

import (
    "fmt"
    "io/ioutil"
    "net/http"
    "strings"
)

func apiCall() []byte {
  url := "https://functionschallenge.digitalocean.com/api/sammy"
    method := "POST"

    payload := strings.NewReader(`{"name" : "YourNameHere","type" : "SammyTypeHere"}`)

    client := &http.Client{}
    req, err := http.NewRequest(method, url, payload)

    if err != nil {
        fmt.Println(err)
    }
    req.Header.Add("Accept", "application/json")
    req.Header.Add("Content-Type", "application/json")

    res, err := client.Do(req)
    if err != nil {
        fmt.Println(err)
    }
    defer res.Body.Close()

    body, err := ioutil.ReadAll(res.Body)
    if err != nil {
        fmt.Println(err)
    }
    fmt.Println(string(body))
  return body

}

func main(){
  apiCall()
}

Step 3: Simply, replace the YourNameHere and SammyTypeHere placeholders with exact values as you want, click on Save and then hit the Run button to execute the function. This will successfully create the Sammy in the aquarium with your name and type selected.

Note: You may find some developer errors when you run this code in the Output viewer, but do not worry, this doesn't fail the Sammy Creation process as you can find a success message in the Logs viewer. Feel free to go to the aquarium link above to find your Sammy swimming there.

Walkthrough for Nodejs:14 Runtime

Step 1: From the Source tab, click on New Function again and choose Nodejs:14 as the runtime this time. Make sure your Function names are unique if you are putting them inside the same package.

Step 2: Similarly, we can formulate a NodeJS function with all the given API request payloads in the following way.

var axios = require('axios');
var data = JSON.stringify({
  "name": "YourNameHere",
  "type": "SammyTypeHere"
});

var config = {
  method: 'post',
  url: 'https://functionschallenge.digitalocean.com/api/sammy',
  headers: { 
    'Accept': 'application/json', 
    'Content-Type': 'application/json'
  },
  data : data
};

exports.main = () =>
  axios(config)
.then(function (response) {
  console.log(JSON.stringify(response.data));
  return (response.data)
})
.catch(function (error) {
  console.log(error);
  return (error)
});

Step 3: Now replace the YourNameHere and SammyTypeHere here placeholders with your details, click on save and then hit the Run button. This should easily create the specified Sammy in the aquarium and you can see the response on both the Output and Logs viewers.

Walk-through for Php:8.0

Step 1: Click on the New Function button in the Source tab, select Php:8.0 as the runtime this time, fill in the name and package as you want and click on Create.

Step 2: Now in a very similar way, we can write the Php function in the following way to do the POST API call to the end-point.

<?php
function main() {
  $curl = curl_init();
  curl_setopt_array($curl, array(
  CURLOPT_URL => 'https://functionschallenge.digitalocean.com/api/sammy',
  CURLOPT_RETURNTRANSFER => true,
  CURLOPT_ENCODING => '',
  CURLOPT_MAXREDIRS => 10,
  CURLOPT_TIMEOUT => 0,
  CURLOPT_FOLLOWLOCATION => true,
  CURLOPT_HTTP_VERSION => CURL_HTTP_VERSION_1_1,
  CURLOPT_CUSTOMREQUEST => 'POST',
  CURLOPT_POSTFIELDS =>'{
    "name" : "YourNameHere",
    "type" : "SammyTypeHere"
}',
  CURLOPT_HTTPHEADER => array(
    'Accept: application/json',
    'Content-Type: application/json'
  ),
));

$response = curl_exec($curl);

curl_close($curl);
echo $response;
return json_decode($response);

}

Step 3: As we have already done before, let's replace the YourNameHere and SammyTypeHere placeholders with the values we want, click on Save and then Run the function. This creates the Sammy as we specified and we have done this using the Php:8.0 runtime.

Walk-through for Python:3.9

Step 1: One last time, let's click on the New Function button and then give the function name & package name and select Python:3.9 as the runtime. Now click on Create to proceed.

Step 2: We will now write a function similar to the one below providing all the request payloads needed to do the API call.

import http.client
import json
import requests

def main(args):
  url = "https://functionschallenge.digitalocean.com/api/sammy"

  payload = json.dumps({
    "name": "YourNameHere",
    "type": "SammyTypeHere"
  })
  headers = {
    'Accept': 'application/json',
    'Content-Type': 'application/json'
  }

  response = requests.request("POST", url, headers=headers, data=payload)

  print(response.text)
  return {'Output' : response.text}

Step 3: Now click on Save, followed by the Run button to execute the function. Make sure to replace the YourNameHere and SammyTypeHere placeholders with your desired values. This is how we can deploy Sammy in the aquarium using the Python:3.9 runtime.

Note: The name you choose for Sammy should be completely unique. You can't use a name if it's already been used to deploy a Sammy in the aquarium.

Conclusion

Well, we have reached to the end of this walk-through now where we figured out how to trigger the serverless functions for this challenge using all the 4 available runtimes. But as I mentioned earlier, this is not the only way to complete this challenge. Everyone is welcome to use their creativity and come up with more amazing ways to achieve this.

DigitalOcean is always striving to better the user experiences and is committed to it. More runtimes will be added gradually to the platform so that the developers have more options to choose from as per their preferences.

Lastly, if you found this article really helpful, then do give it a like and feel free to drop in your feedback.

Note: All the dashboard images used here are sole properties of DigitalOcean, and are used here only for illustrative purposes.

Predicting Machine Failure Rates using MindsDB

Rutam Prita Mishra — Thu, 09 Jun 2022 18:20:00 +0000

Introduction

MindsDB powers the normal databases with the Machine Learning capability. They enable users to turn their normal tables into AI tables. They are on a mission to empower the in-database machine learning with the help of simple SQL queries only. One need not worry about the underlying coding or the setup to train such machine learning models.

In this tutorial, we will be using MindsDB Cloud to train a Predictor model that can predict the machine failure rates based on the following dataset.

Importing data to the database

As a first step to complete this tutorial, we need to import the dataset from the above link into our database. So, let's start doing that.

Step 1: Download the dataset from here.

Step 2: Now open the database manager that you generally use. I will be using MySQL workbench for this tutorial. You should have MySQL installed already and should be running on the localhost so that you can test a connection.

Once the prompt returns successful, click on OK to save the connection.

Step 3: Now once you are able to connect, simply click on Create a new Schema in the connected server button and provide a name to the schema when prompted and then click on Apply twice and then followed by Finish button to create the schema.

Step 4: Now let's create a table with the dataset we have. Right-click on the schema name and click on Table Data Import Wizard. On the next dialog box. click on Browse and select the CSV file that just downloaded above.

Now click on Next. On the Select Destination dialog box, select the Create New Table option, make sure that the selected schema is same as the one we created and assign a name to the table.

Again, click on Next. On the Configure Import Settings dialog box, leave everything at default and click on Next.

Now simply click on Next again to begin the import. Wait for the execution to complete. It should take a while based on the amount of data in the dataset.

Once the execution is complete, click on Next and then click on Finish to complete the process.

Step 5: We will now just check whether the data has been imported to the table properly or not by running a simple SELECT statement.

SELECT * from machine_failure.machine_train LIMIT 10;

This should return 10 rows of data from the table as shown below.

Now, that we are ready with the table, let's move on to the next phase of connecting to the MindsDB cloud.

Connecting the Database to MindsDB Cloud

In this section, we will connect our local database to the MindsDB Cloud to train the Predictor models later in this tutorial.

Step 1: Sign up for a MindsDB cloud account if you haven't already or you can use your existing credentials to simply log in.

Step 2: Once you're in the MindsDB Dashboard, you can find a few sections on the screen like the Query Editor at the top, some Learning Hub resources on the right and a Result Viewer at the bottom. Click on Add data from the top right.

Step 3: Select MySQL from the list of available databases and this should open the Query Editor with a pre-defined query in it.

Step 4: In this pre-filled query, we need to provide certain parameters to establish a connection to our local database. In order to that, we need to expose localhost:3306 to the internet so that we can establish a tcp/ip connection to interact with it.

Step 5: We have to now use [ngrok](https://ngrok.com/) to tunnel our localhost to the internet. This command should do the job for you.

ngrok tcp 3306

Make sure you signup for a ngrok account and connect with your account using the auth token you received after signing up so that you are not time-limited for your tunnelling. You can do the following using this command.

ngrok config add-authtoken YourAuthTokenHere

Step 6: We can now fill in the parameters in the query above to connect to the database and then hit the Run(Shift + Enter) button to fire the query. This query should execute successfully. Have a look at the snip below to learn more about each of the parameters.

Step 7: Now let's run a simple SELECT query now to make sure that MindsDB can now access the data in the database.

select * from machine_failure.machine_train LIMIT 10;

This should return 10 records from the table machine_train.

Training a Predictor Model

As we are ready with all the data, we will now go ahead and create a predictor model to do the predictions for us. Let's find out how to do that in the steps below.

Step 1: MindsDB has a CREATE PREDICTOR syntax already that we need to use now to create the predictor model. Look at the snippet below to learn more about the syntax. The syntax looks as follows.

CREATE PREDICTOR mindsdb.predictor_name (Fill in the name you want)
FROM database_name                       (Fill in the Database Name)
(SELECT * FROM table_name LIMIT 10000)  (Fill in the training table name)
PREDICT target_parameter;               (Fill in the target parameter to Predict)

The query should get executed successfully.

Note: We have used LIMIT 10000 above as MindsDB allows upto 10K rows in its free edition.
Also, you may get an error as the column names have spaces in between them. So, try renaming the columns removing the spaces.
For e.g., Change the target column name from Machine failure to Machine_failure so that the query works properly.

Step 2: It takes a while to train the model. We can keep checking it's status by running the command below.

SELECT status
FROM mindsdb.predictors
WHERE name='name_of_the_predictor_model';

The status should gradually change from training to complete indicating that the model is now ready to use.

Describing the Model

The predictor model is now ready to do the predictions, but we will first try to do a deep dive and learn more about this predictor model.

We can simply use the DESCRIBE statement provided by MindsDB to gain insights into our predictor model. We can use this statement to find the details in the following three ways.

By Features
By Model
By Model Ensemble

By Features

DESCRIBE mindsdb.predictor_model_name.features;

This statement is used to display the way that the model encoded the data prior to training. You can find a sample output below.

Let's figure out what each of these columns mean.

column - List of all the columns used from the Training Table
type - Type of data inferred
encoder - Encoder used
role - Role of the column for the model (Here Machine_failure is the target as we want to predict it and all other columns are Features based on which we will try to do the prediction.)

By Model

DESCRIBE mindsdb.predictor_model_name.model;

This statement is used to find out how each of the candidate models performed that we used to train the Predictor.

Now let's try to understand the result table below.

name - Name of the candidate model
performance - Defines the accuracy of each candidate model which varies between 0-1. The value that is closer to 1 is considered to be more accurate.
training_time - Time taken by each of the candidate models to complete the training
selected - Either 0 or 1 is set for each candidate model. 1 is set for the most accurate model (Neural here) and the rest all are set at 0.

By Model Ensemble

DESCRIBE mindsdb.predictor_model_name.ensemble;

This statement shall return a JSON object that explains the different attributes used to select the best candidate model to do the prediction.

With this, we now have a clearer picture about our underlying model that we will be using for the predictions below.

Querying the Model

The predictor model machine_failure_rate_predictor is now ready to predict the machine failure rates for us. MindsDB provides simple SELECT query statements that we can fire to find the predictions based on the specified features.

Let's try predicting the failure rate based on a single feature. The SELECT statement would be as follows.

SELECT Machine_failure
FROM mindsdb.machine_failure_rate_predictor
WHERE torque =40;

This should return the Machine_failure rate which would either be 1 or 0. 1 means the machine failure is expected whereas 0 means there won't be a machine failure for the given feature.

Now let's try predicting the failure rate with more than one parameter. The query should look something like this.

SELECT Machine_failure
FROM mindsdb.machine_failure_rate_predictor
WHERE torque =100 AND Type="L";

This, again returns either 0 or 1, determining whether the machine fails or not based on the provided feature parameters.

Conclusion

We have now reached the end of this tutorial where we connected our local database to the MindsDB cloud, trained a predictor model using MindsDB by firing a few simple SQL queries and predicted the machine failure rates using the model that we created.

Now you can also go ahead and try training predictor models and doing predictions using different datasets available online.

Before you leave this page, please like the tutorial if it has really helped you to learn something new today. Feel free drop in your feedback as well.

Note: All the dashboard images used here are sole properties of MindsDB & MySQL and are used here only for illustrative purposes.

Launching your first Vultr Kubernetes Cluster

Rutam Prita Mishra — Mon, 30 May 2022 15:33:29 +0000

Introduction

Kubernetes has been a known player in the market for container orchestration systems for automating software deployment, scaling, and management. With the growing use of Kubernetes at the ORG level and also by the developers, there have been a lot of cloud service providers in the market who offer their managed Kubernetes services.

Today in this article we are gonna try and learn to create a cluster using the Vultr Kubernetes Engine (VKE). Vultr has been a key player in the cloud-hosting solution space and they aim to provide highly reliable and high-performance cloud computing environments across all geographical regions.

Creating a Cluster

We will now go ahead and deploy our first cluster using VKE.

Step 1: Create a new Vultr account if you don't have one yet or simply log in using your existing credentials if you're already a user.

Step 2: Once you're logged in, click on the Kubernetes tab from the dashboard and then hit the Add Cluster button.

Step 3: On the Create a Cluster dashboard, we need to define the specs we want to use for the cluster. Feel free to choose them as per your requirements, or you can always follow the specs mentioned below for the sake of this tutorial.

- Cluster Name- Name it as you wish (democluster for now)
- Kubernetes Version- Choose the latest one (v1.23.5+3 for now)
- Cluster Location- Choose the closest one (Mumbai for me)
- Cluster Capacity- Fill in the details mentioned below
-> Node Pool Name- Assign a name as you wish (demo-pool for now)
-> Node Pool Type- Intel High Performance
-> Plan- $24/mo (2CPUs, 4096 MB Memory, 100GB Storage)
-> Amount of Nodes- 3

Now click on Deploy Now button to provision the cluster.

Step 4: Now it should take a few moments for the cluster to complete its installation and be ready for use. Keep checking the status until it shows Running.

Once the status is set to running, click on the cluster name to navigate to the cluster dashboard page where you can find all the details about the cluster.

Connecting to the Cluster

Our cluster is now created and is running. We now need to connect to our cluster from our local system to start using it. To do that, simply follow the steps below.

Step 1: Download the cluster config by hitting the Download Configuration button on the cluster dashboard page.

Step 2: Once you've downloaded the config file to your system, you can either use it explicitly by mentioning its path in the terminal or you can simply add/overwrite your current config file in the .kube folder and start using it directly using kubectl with the following command.

kubectl get nodes

This should return you the list of nodes in our node pool.

Note: If you don't have kubectl configured in your local machine, then you follow the steps here to get it done.

Step 3: You can also go ahead and fetch the list of pods in all namespaces with the following command.

kubectl get pods -A

This should return the list of pods in all namespaces.

Now you're ready to use your cluster in any way you want using the terminal.

Conclusion
This completes the tutorial here. We have successfully deployed a cluster using Vultr Kubernetes Engine and also connected it from our local machine using the cluster configuration file.

I hope all of you had a good time reading this and do show some love to this article if you found it useful.

Note: All the dashboard images used here are sole properties of Vultr and are used here only for illustrative purposes.

Getting Started with Kubecost on Digital Ocean Managed Kuberenetes

Rutam Prita Mishra — Sun, 29 May 2022 18:27:20 +0000

Introduction

The use of Kubernetes has grown largely over a few years. But managing Kubernetes on a large Org level is never easy. Apart from security and configuration management, another crucial aspect of management is Cost. This can be a tedious task to monitor all running resources and identify the cost leaks or figure out the ways to optimize the cost consumption.

In such a scenario, Kubecost comes into play. Kubecost provides real-time cost visibility and insights by inspecting the cost allocation using Kubernetes concepts and providing unified cost monitoring and optimization suggestions.

In this tutorial, we will be creating a cluster on Digital Ocean and then deploying Kubecost on top of it to find the cost insights for the cluster.

Adding a cluster on Digital Ocean

Digital Ocean is a cloud hosting provider that offers cloud computing solutions and IaaS on the core principles of high scalability, reliability, and affordable pay-as-you-use pricing models. Now let's go ahead and create a cluster with the below-mentioned steps.

Step 1: Sign up for a Digital Ocean account if you don't have one yet or else use your existing credentials to log in. They provide 100$ credits for 60 days when you sign up.

Step 2: Once you're signed in, click on Kubernetes from the left navigation panel and then hit the Create Cluster button to choose the cluster configuration.

Step 3: In the Create a Kubernetes cluster window, you can choose the config you want for your cluster. But for the sake of this tutorial, you can use the config mentioned below to create the cluster.

Choose a datacenter region - Choose the one closest to you (BLR1 for me)
VPC network - This will be already set at Default
Select a version - Use the Recommended one (1.22.8-do.1 for now)
Choose Cluster Capacity - Select as mentioned below.
-> Node Pool Name - Give a name as you want
-> Node Plan - $24/month per node
-> Machine Type - Basic Nodes (Premium Intel)
-> Nodes - 3
Add extra reliability to critical workloads - Leave it at default
Finalize - Select as mentioned below
-> Name - Assign a name to you cluster
-> Project - Leave it at default
-> Tags - Leave it for now

Now click on Create Cluster button and wait for the cluster to get provisioned.

Step 4: Once the cluster is provisioned and is in a running state, we will have 2 ways to connect to the cluster from our local machine. We can either choose the automated way by using the [doctl](https://docs.digitalocean.com/reference/doctl/how-to/install/) or we can opt to use the manual way by downloading the config file for the cluster.

We will use the manual way, for now, to connect to our cluster. So, let's download the cluster config file.

Step 5: Once you have the config file downloaded, either use the config file explicitly with the command below to fetch the cluster nodes.

kubectl --kubeconfig=~/<pathtodirectory>/kubecost-demo-kubeconfig.yaml" get nodes

Or you can simply overwrite the config file in the .kube folder in your system to access the nodes normally using kubectl without specifying anything extra. This should return you the list of nodes in your cluster.

kubectl get nodes

This confirms that we are now ready to work with our cluster from the terminal.

Note: If you don't have kubectl installed already, please follow this guide to get it done.

Deploying Kubecost in the Cluster

We have to now deploy Kubecost in our cluster to see it in action. So, let's get started with the following steps.

Step 1: Navigate to the Kubecost install page and provide your email and click Get Started. This will take you to the instructions page and a unique kubecostToken will be generated for you.

Step 2: Now we will create a namespace for Kubecost and then run the commands below to install Prometheus, Grafana, and kube-state-metrics, along with Kubecost.

kubectl create namespace kubecost
helm repo add kubecost https://kubecost.github.io/cost-analyzer/
helm install kubecost kubecost/cost-analyzer --namespace kubecost --set kubecostToken="YourKubecostTokenHere"

Step 3: We have to now check the pods in the kubecost namespace and wait till all of them are in a running state.

kubectl get pod -n kubecost

Step 4: Once we have all the pods running, we simply need to enable the port forwarding to be able to interact with the Kubecost GUI hosted on localhost:9090.

kubectl port-forward --namespace kubecost deployment/kubecost-cost-analyzer 9090

Now, we are ready to explore the Kubecost UI and gain insights into our cost utilization.

Kubecost GUI

Kubecost GUI is generally hosted at localhost:9090. You can also expose the port 9090 as a secure endpoint on your cluster to remove the need to port forward.

We can now explore different aspects of the cost consumption as mentioned below.

Cost Allocation
This section shows the cumulative cost allocation/utilization for the last 7 days for each namespace in the cluster. The table provides a clearer picture of what configuration of resources are being consumed by that namespace and how much cost it incurs.

You can simply visit this page by clicking on Cost Allocation from the left navigation pane or by clicking on the same Cost Allocation button under the cost allocation graph on the Overview page itself.

Assets
This section lists out the assets utilized and the total cost incurred in the last 7 days filtered by daily.

You can visit this page in a similar way from the left navigation pane or from the Assets graph on the Overview page.

Savings
This is gonna be the favorite part for any ORG as it lists out the different cost-saving optimization techniques that could save them a huge amount of money in the monthly bill.

Navigating to this page is really easy and you can do that from the same navigation pane or by clicking on the View Savings on the Overview page.

Once you're on this page, you can click on each of these suggestions to find out more detailed information on how to achieve those.

Health
In addition to all those things related to cost, Kubecost also has its in-built metrics to determine the overall cluster health score based on infrastructure reliability and performance risks.

You can simply click on Health from the left navigation pane and scroll through the list of all accessed parameters for the health test.

Clicking on each of these parameters will open up a pop-up with more information about them.

Kubecost also allows you to set up customized reports and alerts as per requirement based on specific filters. For all the cost-related information that we saw above, we also have the option to download those reports in CSV formats.

Conclusion

We have now reached the end of this tutorial where we successfully created a cluster using Digital Ocean managed K8s and then deployed Kubecost in it to find out on-the-go cost-utility insights for the cluster.

We are now clear on how to configure Kubecost and how to use its GUI to gain all the information relating to Cost in our cluster. Apart from all the features, one mentionable thing is that Kubecost provides all these features even in their free plan. But, you can also have the option to upgrade to a paid Business or Enterprise version as well.

Lastly, feel free to drop in your feedback and show some love to the article if you really liked it.

Note: All the dashboard images used in this article are sole properties of Digital Ocean and Kubecost respectively and are used here only for illustrative purposes.

Deploying MindsDB on a Vultr Cloud Instance

Rutam Prita Mishra — Sat, 28 May 2022 16:49:27 +0000

Introduction

MindsDB is unique in its approach as it turns the normal tables in our databases into AI-Tables. It simply powers the In-Database machine learning as it is capable of pulling data from your tables and then feeding them into in-house machine learning models to fetch you some of the most accurate prediction tools instantly.

And what's even more awesome is its simplicity. You can simply train and use multiple prediction tools by using standard SQL queries and you don't really need to bother about the underlying programming.

So, in this tutorial, we will try to deploy MindsDB on a Vultr Cloud Instance and try to connect with it from one of our local database managers called DBeaver.

Deploying a Vultr Cloud Instance

Vultr provides cloud computing solutions globally with its multiple data centers spanning several geographical regions ensuring a standardised highly reliable and high performance cloud computing environment.

Now let's dive in and create a Cloud Instance in the Vultr Platform.

Step 1: Sign up for a Vultr account if you haven't already or simply log in using your existing credentials. Vultr offers 100$ worth of credits to try out the platform for 30 days.

Step 2: Once you have signed in, you will land on the Dashboard. Select the Instances tab and click on Deploy Server. This should take you to the Deploy New Instance screen.

Step 3: On this screen, we need to choose the specs for the compute instance that we will be creating. You are free to choose them as per your needs but for the purpose of this tutorial, you can always go ahead and use the config mentioned below.

- Choose Server: Cloud Compute
- CPU & Storage Technology: Intel High Performance
- Server Location: Choose the closest one (Mumbai for me)
- Server Image: Navigate to Marketplace Apps. Select Docker on Ubuntu 20.04 x64 Image
- Server Size: $24/month (100GB NVMe/ 2 vCPUs / 4GB Memory)
- Add Auto Backups: You can either leave it turned on or turn it off (It will cost you extra bucks.)
- Additional Features: You can check any of the features if you need them. However, we can skip it for this tutorial.
- SSH Keys: Leave it as it is. We don't need to add any SSH keys as well.
- Server Hostname & Label: Assign a Server Hostname as you wish
- Server Qty: Leave it at 1

Now we are done with selecting all the specs for the server and can simply hit the Deploy Now button to get it booted.

Step 4: We have to now wait for the installation to complete and the status of the server should be set to running on the next page.

Now once we get the status as running we can simply click on the server name and navigate to the Server dashboard where we can find all the details about the server and can then click on View Console to get started with the terminal and carry on the further steps.

Make sure to note down the username and password from the server dashboard screen as we will be needing them to log in to our system using the terminal.

Installing MindsDB on the Server

We have now set up a cloud instance that is up and running and we are ready to install MindsDB on top of it using Docker. So, let's get started with this.

Step 1: Before we start, let's check whether Docker is installed properly or not using the command below.

docker run hello-world

It should successfully run and return the message Hello from Docker. This confirms that the Docker is working properly on our instance.

Step 2: We can now run the following command using Docker to pull the latest stable MindsDB image into our server.

docker pull mindsdb/mindsdb

We can also fetch the latest Beta version image using the command below.

docker pull mindsdb/mindsdb_beta

Note: The MindsDB Getting Started documentation provides more such ways to install MindsDB other than Docker as well.

Step 3: We have to now expose the specific ports that will be needed to communicate with the MindsDB instance on the cloud from our local database manager. Use the following command to do the same.

docker run -p 47334:47334 -p 47335:47335 mindsdb/mindsdb

Port 47334 - It is used to host the MindsDB GUI. You can always use your CloudInstanceIPv4:47334 to access the GUI in the browser.
Port 47335 - It will be used as the MySQL communication port to carry out database transactions over TCP/IP.

Connecting with DBeaver (Local Database Manager)

We will now try to connect to the MindDB cloud instance using DBeaver DB manager and check the tables present inside the MindsDB instance. You can also check here to find out the steps for connecting to the cloud instance using some other database client.

Step 1: Once you've installed DBeaver on your local system, you can open it up and click on New Database Connection to setup the connection wizard.

Step 2: From the new connection wizard, select MySQL as the database and click on Next. On the next screen, we need to provide the connection parameters so that we can establish the connection. The following are required to complete the connection successfully.

- Server Host: Your Server IPv4 only
- Port: 47335 (Default Port for MySQL)
- Username: mindsdb (Default Username)
- Password: Leave it empty (Default Password is empty.)

Now hit the Test Connection button. This should return success. You can now click on Finish to save the connection.

Note: You may sometimes require to download some additional driver files in DBeaver before you can establish a connection. So, make sure to select all the recommended driver files and download those as and when prompted.
Also, with the latest release, MindsDB instance has its own SQL Editor that we can use to run the queries and train the models.

Exploring Tables inside MindsDB

As we are now connected with the MindsDB instance, we can either use the file explorer on the right side of DBeaver to explore the tables inside the instance or can otherwise use SQL queries to filter out the required data.

We can also use the following query to fetch the list of tables present.

show tables;

The Predictors table is used to store the predictor models when we train them whereas the Databases table contains the list of databases connected with our MindsDB cloud instance. Each new predictor model is stored as a new record in the table and each of its columns contain specific information about that model.

Conclusion

With this, we have successfully completed all the steps that are required to host a local instance of MindsDB on our preferred cloud partner Vultr. We have also learnt how to connect to our cloud-hosted instance using any local database manager. Additionally, we can also sign up and use the MindsDB web version for free.

Lastly, if you liked the article and found this helpful, don't forget to drop a like. Feel free to drop in your feedback as well.

Note: All the dashboard images used here are sole properties of MindsDB, DBeaver and Vultr, and are only used here for illustrative purposes.

Top 5 Lab Simulations to Learn Kubernetes

Rutam Prita Mishra — Fri, 27 May 2022 08:58:44 +0000

Introduction

Kubernetes is an open-source system of containerized apps for automating deployment, scaling, and management. We also call it K8S because it starts with K and there are 8 letters in between and ends with S

In modern times, K8s has been a part of every enterprise solution and all the cloud-native enthusiasts are very keen on learning about it. So, this article lists 5 such free resources where one can read and have hands-on experience by completing labs using K8S.

List of Lab Simulations

The labs listed here are equally good and are not really sorted from best to least. It will completely depend on the individual to choose which one suits him/her the best out of the following.

Kubernetes.io

The official Kubernetes website provides some basic modules for the K8s beginners to learn using an interactive lab environment powered by Katacoda.

Whenever you start a module, you will have a brief theory section to read through and then you can start the lab scenario to check what you have learnt and see it in action. The labs are easy to do with instructions and commands provided already. One has to just understand the objective and then run these commands to achieve the outcome.

Civo Academy

Everyone might be knowing Civo in the cloud-native space but what is even more special about them is their free course on Kubernetes available on Civo Academy.

You will find some of the great mentors in the cloud-native space teaching you with all their expertise and you can feel the ease of learning and trying hands-on using the free Civo Credits. The Civo Academy currently has 39 sessions divided into multiple courses with more sessions coming up soon. Whether you are a Beginner or a Pro, you can find these courses marked with the tags of Beginners or Intermediate or Pro which will help you decide which course you need the most at the moment.

KubeAcademy by VMware

Another great place to start your beginner's journey in K8S is with the KubeAcademy powered by VMware. They have designed a set of courses which you can choose based on your preference. If you're not sure where to start, then they have a short assessment Quiz as well after which they will start suggesting you the courses that you should take.

KubeAcademy provides 30 online courses with hands-on spanning over a duration of 39 hours tailored by 29 industry experts. You can even get a bunch of K8S Developer Resources from the VMware Tanzu Developer Centre or can even grab the Production Kubernetes e-Book written by 3 KubeAcademy instructors themselves.

Learning Kasten.io by Veeam

One can also start the K8S learning journey with Kasten Learning. They provide you with multiple Lab Modules where you can learn and carry out different live scenarios to see how things work within your K8s clusters.

Within each lab, you get materials to read followed by MCQs for knowledge check and finally the Lab execution. They even award you with badges after you complete each lab successfully that you can share with your connections to show off your learning progress and to keep you motivated to learn and complete even more labs to earn more such badges.

Play With Kubernetes

If you're someone who is well-versed with the K8s basics and just wants to try out the commands and carry out different use cases within K8s clusters, then Play with Kubernetes is the right place for you.

Play with Kubernetes is powered by Docker. You can simply log in with your Github or Docker credentials and start working with your instances. Each session in the playground stays for 4 hours. So, you have 4 hours per session to try out everything you want. But don't worry, once your session is consumed, you can always go ahead and start another new session and you will have 4 hours again to do something new.

But if you're someone who is not sure what to run and how to run things in the lab environment, then I would suggest you use Kubelabs as a learning aid and additional resource to learn and then try things out in the Play with Kubernetes sessions.

Closing Notes

Kubernetes may seem very vast and challenging initially, but you can always start with learning and growing each day with all these resources and many more available for free on YouTube and other platforms.

You can also get free credits when you sign up with so many cloud providers like Civo, Digital Ocean, Linode and so on. Use the free credits, spin up a cluster and start doing things and learning at your own pace.

And, always remember that you're not alone in this. There are plenty of resources, communities and mentors in the cloud-native space who are always there to guide you and help you to learn and succeed in your Kubernetes journey.

If you found this really helpful, don't forget to show some love to this article. Also, feel free to drop in your feedback or mention more such free resources for our Kubernetes Learners. Happy Learning!

Note: All the homepage/dashboard images are sole properties of the respective websites & platforms, and are used here only for illustrative purposes.

The Ops Community ⚙️: Rutam Prita Mishra

Deploying MindsDB on Oracle Cloud

Introduction

Deploying an Oracle Cloud Compute Instance

Deploying MindsDB on the Cloud Instance

Exploring the Default MindsDB Tables

Conclusion

Deploying MindsDB on Google Cloud Platform

Introduction

Deploying a GCP Compute Engine

Deploying MindsDB on the Compute Instance

Exploring the Tables in MindsDB

Conclusion

Deploying MindsDB on Civo Compute

Introduction

Deploying a Civo Compute Instance

Installing MindsDB on the Instance

Exploring Tables inside MindsDB

Conclusion

Predicting Data Science Job Salaries using MindsDB Cloud

Introduction

Importing data to MindsDB Cloud

Training a Predictor Model

Describing the Predictor Model

By Features

By Model

By Model Ensemble

Querying the Model

Conclusion

Getting Started with Speedscale on Civo Managed Kubernetes

Introduction

Creating a Civo Managed Kubernetes Cluster

Deploying Speedscale on the Cluster

Deploying the Demo Podtato App

Load testing the Podtato App

Conclusion

Walk-through for DigitalOcean Functions Challenge

Introduction

The DigitalOcean Functions Challenge

Creating a Namespace to Deploy Functions

Walk-throughs for the Challenge

Walkthrough for Go:1.17 Runtime

Walkthrough for Nodejs:14 Runtime

Walk-through for Php:8.0

Walk-through for Python:3.9

Conclusion

Predicting Machine Failure Rates using MindsDB

Introduction

Importing data to the database

Connecting the Database to MindsDB Cloud

Training a Predictor Model

Describing the Model

By Features

By Model

By Model Ensemble

Querying the Model

Conclusion

Top 10 Video Tutorials in Kubernetes for Beginners

Introduction

Video Tutorial Sources

Closing Notes

Launching your first Vultr Kubernetes Cluster

Introduction

Creating a Cluster

Connecting to the Cluster

Getting Started with Kubecost on Digital Ocean Managed Kuberenetes

Introduction

Adding a cluster on Digital Ocean

Deploying Kubecost in the Cluster

Kubecost GUI

Conclusion

Deploying MindsDB on a Vultr Cloud Instance

Introduction

Deploying a Vultr Cloud Instance

Installing MindsDB on the Server

Connecting with DBeaver (Local Database Manager)

Exploring Tables inside MindsDB

Conclusion

Top 5 Lab Simulations to Learn Kubernetes

Introduction