I started a new job in July 2022. While the developer experience is great, one of the things that I missed at my new job was a CLI to rule them all (like I had at my old job). I thought of building an easily extensible CLI tool which I did with go and cobra. To handle distribution to the engineers, we decided to use both go install and homebrew.
Homebrew is a very popular package manager for macOS and Linux users. Users can easily install new packages with brew install [PACKAGE]. Homebrew handles the downloading and the installation of the packages to the appropriate directories and then creates symlinks to /usr/local (on Intel machines).
To create a Homebrew package, we create Ruby classes called Formulas. A Formula looks like this:
class Wget < Formula
homepage "https://www.gnu.org/software/wget/"
url "https://ftp.gnu.org/gnu/wget/wget-1.15.tar.gz"
sha256 "52126be8cf1bddd7536886e74c053ad7d0ed2aa89b4b630f76785bac21695fcd"
def install
system "./configure", "--prefix=#{prefix}"
system "make", "install"
end
end
While adding an open-source package is fairly straightforward, doing the same with a private package involves a bit of hacking.
First Steps
The first thing to do is create a Github release, if you use go, I highly recommend using goreleaser. I wrote a tutorial on how to do that here. It truly makes releases in go easy. If you're using another language, you should check out how to manage releases in GitHub here.
Create a Homebrew Tap
The next thing to do is to create a Homebrew tap. A Homebrew tap is a GitHub repository that houses our Formulas. To add non-Homebrew core taps, we need to add the taps locally to homebrew with brew tap username/package.
For us to use the single argument form of brew tap, we need to follow the naming convention and name our GitHub repo homebrew-tap since the GitHub repo must start with homebrew-.
Create a Formula
If you have a release already in Github, we can go ahead with creating the Formula. In our homebrew-tap repo, we'll create a folder called Formula. In the Formula directory, we'll create a file called mytool.rb.
Now, in our Formula, we cannot use the standard way of downloading the packages since our package is hosted in a private repo. To do this we simply add a new download strategy like such:
require "formula"
require_relative "../custom_download_strategy.rb"
class Wget < Formula
homepage "https://www.gnu.org/software/wget/"
url "https://ftp.gnu.org/gnu/wget/wget-1.15.tar.gz", :using => GitHubPrivateRepositoryReleaseDownloadStrategy
sha256 "52126be8cf1bddd7536886e74c053ad7d0ed2aa89b4b630f76785bac21695fcd"
def install
system "./configure", "--prefix=#{prefix}"
system "make", "install"
end
test do
system "#{bin}/wget --help"
end
end
Note the addition of , :using => GitHubPrivateRepositoryReleaseDownloadStrategy on the third line of the class.
In the root folder of homebrew-tap, we'll create a new file called custom_download_strategy.rb. Here, we'll add some code to support our download strategy.
require "download_strategy"
class GitHubPrivateRepositoryDownloadStrategy < CurlDownloadStrategy
require "utils/formatter"
require "utils/github"
def initialize(url, name, version, **meta)
super
parse_url_pattern
set_github_token
end
def parse_url_pattern
unless match = url.match(%r{https://github.com/([^/]+)/([^/]+)/(\S+)})
raise CurlDownloadStrategyError, "Invalid url pattern for GitHub Repository."
end
_, @owner, @repo, @filepath = *match
end
def download_url
"https://#{@github_token}@github.com/#{@owner}/#{@repo}/#{@filepath}"
end
private
def _fetch(url:, resolved_url:, timeout:)
curl_download download_url, to: temporary_path, timeout: timeout
end
def set_github_token
@github_token = ENV["HOMEBREW_GITHUB_API_TOKEN"]
unless @github_token
raise CurlDownloadStrategyError, "Environmental variable HOMEBREW_GITHUB_API_TOKEN is required."
end
validate_github_repository_access!
end
def validate_github_repository_access!
# Test access to the repository
GitHub.repository(@owner, @repo)
rescue GitHub::HTTPNotFoundError
# We only handle HTTPNotFoundError here,
# becase AuthenticationFailedError is handled within util/github.
message = <<~EOS
HOMEBREW_GITHUB_API_TOKEN can not access the repository: #{@owner}/#{@repo}
This token may not have permission to access the repository or the url of formula may be incorrect.
EOS
raise CurlDownloadStrategyError, message
end
end
# GitHubPrivateRepositoryReleaseDownloadStrategy downloads tarballs from GitHub
# Release assets. To use it, add
# `:using => GitHubPrivateRepositoryReleaseDownloadStrategy` to the URL section of
# your formula. This download strategy uses GitHub access tokens (in the
# environment variables HOMEBREW_GITHUB_API_TOKEN) to sign the request.
class GitHubPrivateRepositoryReleaseDownloadStrategy < GitHubPrivateRepositoryDownloadStrategy
def initialize(url, name, version, **meta)
super
end
def parse_url_pattern
url_pattern = %r{https://github.com/([^/]+)/([^/]+)/releases/download/([^/]+)/(\S+)}
unless @url =~ url_pattern
raise CurlDownloadStrategyError, "Invalid url pattern for GitHub Release."
end
_, @owner, @repo, @tag, @filename = *@url.match(url_pattern)
end
def download_url
"https://#{@github_token}@api.github.com/repos/#{@owner}/#{@repo}/releases/assets/#{asset_id}"
end
private
def _fetch(url:, resolved_url:, timeout:)
# HTTP request header `Accept: application/octet-stream` is required.
# Without this, the GitHub API will respond with metadata, not binary.
curl_download download_url, "--header", "Accept: application/octet-stream", to: temporary_path, timeout: timeout
end
def asset_id
@asset_id ||= resolve_asset_id
end
def resolve_asset_id
release_metadata = fetch_release_metadata
assets = release_metadata["assets"].select { |a| a["name"] == @filename }
raise CurlDownloadStrategyError, "Asset file not found." if assets.empty?
assets.first["id"]
end
def fetch_release_metadata
GitHub.get_release(@owner, @repo, @tag)
end
end
We should have the following folder structure:
├── homebrew-tap
│ ├── Formula
│ │ ├── mytool.rb
└── custom_download_strategy.rb
Usage
We're almost done. To install our tool via Homebrew, we need to export a Github token that provides access to our private repos. A new token can be created here. The token needs to have repo permissions. We need to export the token as HOMEBREW_GITHUB_API_TOKEN. To export the token, we run:
export HOMEBREW_GITHUB_API_TOKEN=<GITHUB_TOKEN>
# if you're using fish shell like me, run `set -x HOMEBREW_GITHUB_API_TOKEN <GITHUB_TOKEN>`
Then, we can run the command:
brew install username/homebrew-tap/mytool
This automatically adds the homebrew tap and install our tool.
Summary
- Create a GitHub release with the binaries. This can be done very easily with goreleaser in the case of go.
- Create a GitHub repo called
homebrew-tap. An example is here.- Create a
Formulafolder. - In the root of the repo, create a file called
custom_download_strategy.rb. The contents are shown above - In the
Formulafolder, create a ruby file with the name of your tool likemytool.rb
- Create a
- Create a GitHub token here and give
repopermissions to the token. - Export the token with
export HOMEBREW_GITHUB_API_TOKEN=<GITHUB_TOKEN> - Install the tool with brew using
brew install username/homebrew-tap/mytool.
Top comments (1)
Congrats on publishing your first post, @justdamilare - and what an absolute corker! Thanks for sharing this tutorial, we hope this is the first of many!