The Ops Community ⚙️: Jason Purdy

Automatic Checklists with Pull Request Templates

Jason Purdy — Thu, 09 Jun 2022 17:29:33 +0000

If you haven't read or heard of the Checklist Manifesto book, it's a great read about how a checklist can help get things right. There's a case study about how a hospital saw decreases in surgery deaths when they employed a checklist.

At my work, we employ a checklist when it comes to deploying code, which is a part of our pull request process. When someone has code that's ready to ship, they generate a pull request, which has some of these steps:

Get code review
Present staging server for stakeholder signoff
Have all stakeholders signoff
Pick deployment date/time and communicate w/ team that deployment is happening
Put site in maintenance mode and take backup
Deploy code
Make any database/configuration changes
Final testing on production
Remove maintenance mode
Communicate w/ stakeholders that code is live

You make make this checklist show up automatically with a pull request template. You would create a file called PULL_REQUEST_TEMPLATE.md and put it your project's .github folder. Looks something like this:

## Description
<!--
    Describe what this update does.
-->

<!--
    If this pull request closes any issues, please specify here. Otherwise, delete this section.
-->
Closes: #

## Testing
<!--
  Describe what testing you did for this.
-->

## Code Deployment Checklist
 - [ ] Get code review
 - [ ] Present staging server for stakeholder signoff
 - [ ] Have all stakeholders signoff
 - [ ] Pick deployment date/time and communicate w/ team that deployment is happening
 - [ ] Put site in maintenance mode and take backup
 - [ ] Deploy code
 - [ ] Make any database/configuration changes
 - [ ] Final testing on production
 - [ ] Remove maintenance mode
 - [ ] Communicate w/ stakeholders that code is live

Then whenever you create a pull request in GitHub, the template gets auto-populated like so:

And when you create the pull request, it looks something like this:

And as things are checked off, the number of steps are reflected in the list of pull requests:

So a checklist is a simple thing, yet I can attest to how powerful and preventative it is.

Adding code style check (+ GitHub Action)

Jason Purdy — Sun, 29 May 2022 16:29:14 +0000

Code style dictates how you write code in order to make the code readability consistent throughout your organization. Style is opinionated and there are many different approaches. With my work, we mostly develop in Drupal, which has its own style guide, and there are other PHP style guides for Zend, PSR, etc.

PHP has a code sniffer which can run through your code looking for non-compliant code, given the style you want to comply with. It also comes with a tool (phpcbf) that can automatically fix simple issues.

To get started, you want to add the sniffer to your composer.json require-dev section. If you're also developing with Drupal, you'll want to add the drupal/coder library, which will bring in the Drupal code style guides.

    "require-dev": {
        "drupal/coder": "^8.3.1",
        "squizlabs/php_codesniffer": "^3.4.0"
    }

So that will get the phpcs and phpcbf into the ./vendor/bin directory and you can add a script in your composer.json that will let you run the style check through composer (composer code-sniff). Here's what that section of the json looks like:

    "scripts": {
        "code-sniff": [
            "./vendor/bin/phpcs --standard=Drupal --extensions=php,module,inc,install,test,profile,theme,css,info,txt --ignore=node_modules,bower_components,vendor ./web/modules/custom"
        ],
    }

This will only check the code in ./web/modules/custom, checking our custom code and leaving the external code alone.

I mentioned GitHub Actions before, but you can now add this style check to your Action, by appending a composer install and composer code-sniff to your steps, like so:

jobs:
  build:
    name: Code checkout and checkup
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v1
      - name: Lint check
        uses: overtrue/phplint@7.4
        with:
          path: web
          options: --exclude=core --exclude=libraries --exclude=modules/contrib --exclude=profiles
      - name: Composer install
        run: composer install
      - name: Check coding standards
        run: composer -n code-sniff

Now this action will checkout the code, run a syntax check (excluding external code), do a composer install, and run the code-sniff command. So now with any update to GitHub, it will check for syntax errors and code style issues. The -n argument to the code-sniff command is telling composer to skip any interactions, which are impossible when running through GitHub Actions.

Local git commit hooks

Jason Purdy — Sat, 28 May 2022 21:46:01 +0000

I mentioned in a previous post about how GitHub can automatically check your code for syntax errors whenever you push an update to the server, but you can also catch those on your local development box before you commit code. You can just add a script in the code repo's .git/hooks/pre-commit path. What we do is have our script in a docs directory and then in our composer.json file, when the developer is installing/updating the php libraries, there is a script that (re)establishes the commit hook:

...
        "post-install-cmd": [
            "rm -rf .git/hooks", "ln -s ../docs/hooks .git/hooks"
        ],
        "post-update-cmd": [
            "rm -rf .git/hooks", "ln -s ../docs/hooks .git/hooks"
        ],
...

As for the script itself, I cannot take credit for it. I think it originally came from this post from Carlos Buenosvinos. It doesn't look like that post is still working, but here is the script I use:

This is for a Drupal project, which already has Symfony installed, so you may need to require that (and there may be other requirements), if you aren't already using that.

You can see by some of the commented-out areas, I'm not doing the full verification because I just haven't had the time to work on that. I kind of want to redo it using Robo, but that's a project for another day.

There are times where you want to bypass the verification. If you're in a hurry (not a good sign ;)), or if there's something wrong with the verification hook, you can skip the verification with the --no-verify argument:

git commit -m 'gotta git this code in place! ;)' --no-verify

GitHub Actions: An easy way to start DevOps

Jason Purdy — Fri, 27 May 2022 12:47:23 +0000

I'm barely scratching the surface, myself with DevOps and am enjoying reading other posts on this site to inspire digging deeper. If you're anything like me, though, you may find it a bit intimidating and I'd like to share one easy way to get started with DevOps. We use GitHub private repositories at work and mostly develop with PHP, so GitHub has GitHub Actions, which can run anytime you push an update to GitHub.

How it works is that you have a .github/workflows folder and you can create one or more action files, which are in YAML format. We have a really basic one called goodcode.yml:

name: Code checker (syntax)
on: push

jobs:
  build:
    name: Checkout code
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v2
      - name: Lint check
        uses: overtrue/phplint@7.4
        with:
          path: sites/all/modules/custom

The on in line #2 says anytime there's a push, to run this action, which goes through the jobs and steps. The first step is checking out the code that was pushed. actions/checkout is a GitHub-provided action and integrated into GitHub, so you don't need to worry about access tokens or anything like that.

Then, there's a 3rd party tool called overtrue/phplint, which will quickly and efficiently check your PHP files for syntax errors. Recently, I merged in some code on a new computer and unbeknownst to me, had injected the merge relics (<<<<<<< HEAD) and when I pushed it up, it failed and emailed me, which saved me from going further in the deployment chain.

Starting simple is a great way to start on your DevOps path and then pick up more over time as you gain experience from others' repository setups or other posts on this site. The great thing about GitHub actions is that they're not hidden and you can peek into other repo setups if you see a .github/workflows directory. I was looking for an online version of Werewolf (a great in-person social game for 6+ players with minimal setup, if you're unfamiliar) and came across this repo, which is really cool! It's an express (Node.js) app and it has a .github/workflows directory, which does a full-range of things, like syntax checks and unit tests.

What's also neat about GitHub Actions is that it provides you with a badge code you can put into your repo's README.md, which gives a dynamic status of if your repo has recently passed its tests.

Automatic EB deployment from git push

Jason Purdy — Fri, 27 May 2022 01:27:25 +0000

AWS has a service called Code Pipeline, which probably has a lot of super special skilz to do amazing things, but they also have a dead-easy yet still powerful feature that will automatically deploy updates to an Elastic Beanstalk environment. What this means is that you can set it up so that it "listens" to a GitHub repository and whenever a branch has an update, it will bundle it all together with a ZIP file and then send the ZIP file to Elastic Beanstalk to go through its deployment steps. Here's how you set it up:

First, you go to add a new Pipeline, which looks a little something like this:

After hitting next, you need to setup the Source Stage. At my work, we use GitHub and fortunately, there's a really cool integration with GitHub (Version 2):

Once it's connected, you can select your repo and branch and then hit Next. Skip the Build Stage and then for the Deploy Stage, you can select Elastic Beanstalk and then your application and environment target:

Now what's cool is that whenever I push a change up to the main branch, it automatically deploys to that Elastic Beanstalk app/env, which keeps it up-to-date with our latest work.

There are other repository options like Bitbucket or AWS's own Code Repository, if you have your code elsewhere.

In addition, I would turn on the notifications in Elastic Beanstalk so that it emails you whenever a deployment happens (or if there's a failure). This will give you confidence in that it's actually working as well as you'll know if something broke. You set this up with your environment configuration in the Notifications section:

How Elastic Beanstalk stages your code

Jason Purdy — Wed, 25 May 2022 22:45:49 +0000

Maybe it's the benefit of writing an app with over a gig of data to put in place and process when building, but I have been able to piece together something that hopefully someone else finds helpful. When you deploy your latest code, it gets extracted to /var/app/staging and then all of your .ebextensions commands run on that directory and when it's done, it moves it to /var/app/current.

So why is this important? Well, as you start to add your own .ebextensions setup scripts, you have to keep your paths relative to the app. The gig of data I was talking about was a zip file of some image files necessary for the app. The zip file was downloaded to the /tmp directory and then I was running unzip /tmp/file.zip /var/app/current/path/to/images. This would work, but it was unzipping the image files into the current app vs. what's about to be deployed. So after the deployment was done, the /var/app/staging/path/to/images was empty and then became /var/app/current/path/to/images. Then I was left wondering where my images were! So when you run commands that affect the filesystem, specify the paths relative to your project or app root. So that means it would look something like this: unzip /tmp/file.zip path/to/images.

I should also mention that if you have directories that are setup in particular ways, like permissions or ownership, you'll need to do that relative to the project or app root as well. We have a folder that we put uploaded files in from the public, so that needs to be owned by the webapp and world-writable. So those folders need to be handled the same way from your .ebextensions folder and relatively from the project root like so: chmod -R a+rw path/to/uploads.