The Ops Community

Cover image for How To Scale Jenkins Build Executor and Set Up New Node
Nurul Ramadhona
Nurul Ramadhona

Posted on • Updated on

How To Scale Jenkins Build Executor and Set Up New Node

By default, Jenkins provides 2 Build Executor and 2 Build Queue. So for example if we have 4 jobs all running at the same time, 2 jobs will be executed by 2 Build Executor and 2 left in Build Queue to wait the first 2 jobs are done.

Jenkins Node 1

Jenkins Node 2

Don't worry, we can add the number of active Build Executor. There are two options we can use. Those are:

1. Scale Up
The first option, we can increase the default number of executor. But please note it doesn't mean you can increase as many as you want, it should fit with your Jenkins host specification. For example, we'll add one more executor. So, now we should have 3 in total.

Jenkins Node 3

Jenkins Node 4

2. Scale Out
The second option is by adding new node to our Jenkins Master.
Here I'll tell you the steps:

  • Download and Install Plugin

We need to install SSH Build Agents to be used later for connecting to the new node/agent.

Jenkins Node 5

  • Create New VM

This VM will act as the node. What's the difference between master and node? We don't need to install Jenkins at node. Only Git and Java will be needed.

$ aws ec2 describe-instances --query 'Reservations[].Instances[].{PublicIP:PublicIpAddress, ID:InstanceId}'
- ID: i-0bd3d9b34302fdbe0
  PublicIP: 108.137.62.173
- ID: i-002903a0767e41602
  PublicIP: 108.137.94.186

$ ssh ubuntu@108.137.94.186
Welcome to Ubuntu 22.04 LTS (GNU/Linux 5.15.0-1004-aws x86_64)

 * Documentation:  https://help.ubuntu.com
 * Management:     https://landscape.canonical.com
 * Support:        https://ubuntu.com/advantage

  System information as of Thu Jun  9 15:54:57 UTC 2022

  System load:  0.359375          Processes:             115
  Usage of /:   18.9% of 7.58GB   Users logged in:       0
  Memory usage: 22%               IPv4 address for ens5: 172.31.17.4
  Swap usage:   0%

To run a command as administrator (user "root"), use "sudo <command>".
See "man sudo_root" for details.

ubuntu@ip-172-31-17-4:~$
Enter fullscreen mode Exit fullscreen mode

Alright, because I only enable SSH key as remote authentication. Now I'll enable SSH username password to make the master is easier to connect with the node.

ubuntu@ip-172-31-17-4:~$ sudo passwd ubuntu
New password: 
Retype new password: 
passwd: password updated successfully
ubuntu@ip-172-31-17-4:~$ sudo vi /etc/ssh/sshd_config
ubuntu@ip-172-31-17-4:~$ cat /etc/ssh/sshd_config | grep PasswordAuthentication
PasswordAuthentication yes
# PasswordAuthentication.  Depending on your PAM configuration,
# PAM authentication, then enable this but set PasswordAuthentication
Enter fullscreen mode Exit fullscreen mode

Then, I'll try to SSH the node from the master and we should be asked to enter the password.

ubuntu@ip-172-31-17-9:~$ ssh ubuntu@108.137.94.186
ubuntu@108.137.94.186's password:
Enter fullscreen mode Exit fullscreen mode
  • Create New Agent

Go to Manage Jenkins => Manage Nodes and Clouds

Jenkins Node 6

Jenkins Node 7

For remote root directory:

ubuntu@ip-172-31-17-4:~$ pwd
/home/ubuntu
Enter fullscreen mode Exit fullscreen mode

Then, add the directory name you want to put the Jenkins file configuration.

Jenkins Node 8

Jenkins Node 9

As we configured above, we'll use SSH username password as authentication method.

Jenkins Node 10

Jenkins Node 11

Jenkins Node 12

Jenkins Node 13

As we can see, the number of Build Executor also increases as we add new node to the master. Now we have three from master and two from node.

Jenkins Node 14

Alright, that's it for now! Any feedback are very welcome and thank you for coming. Follow me to get notified when new post is published by me! Thank you.

Let's continue to the next part!

Discussion (0)