Retrieving Service Graph Relationships via the PagerDuty REST API

We recently had an interesting question on the PagerDuty Community forums regarding how to retrieve the list of services that make up a service graph using the PagerDuty REST API.

Service graphs can be made up of a mix of technical and business services, called dependencies, but the most common starting place for them is with a business service. How you organize your business service is really a matter of what makes sense for your teams.

There is no single endpoint query that will dump an entire service graph supporting a business service, so to get access to the whole graph, you’ll need to walk the graph.

Sample Graph

In my test account I have a set of services that make up my “Fabulous Shop”. Fabulous Shop is my business service. It is supported by another business service, Shipping Service. Fabulous shop has 5 technical dependencies: a frontend, a search backend, a cache, a shopping cart, and a database. The cache and the shopping cart both depend directly on the database. The service graph looks like this:

Retrieving Information from the REST API

I’m going to use a global API key in this example. You can also use Scoped OAuth, just make sure you’re watching which scopes are needed. They’ll be listed in the API docs for each endpoint.

Let’s investigate some of the endpoints before we get into how to tackle the service graph.

Business Services vs Technical Services

Business services are a separate kind of object in the REST API. They don’t contain the same information as technical services, so they don’t have the same object schema. Because of that, they also aren’t available behind the same endpoints. Business services will have their own set of endpoints for retrieving information, and traversing the dependencies in a service graph will require looking at both types of services independently.

This can definitely be confusing! Hopefully the code will help clear things up a bit.

Business Service ID

To get information about a business service, I need to have the object ID of that business service. You can find the ID of the service you want in a couple of ways; the easiest way is to find the service in the web interface and note the object ID in the URL. It will look something like:

https://mysubdomain.pagerduty.com/business-services/PXXXXXX

You can also make a request to the /business_services endpoint and find the service you want in the data that is returned.

First Level Dependencies

To find the first level dependencies of a business service, we’ll use the Service Dependencies endpoints - there are two! One endpoint for the dependencies of business services, and one endpoint for the dependencies of business services. If we send a request to a dependencies endpoint with an ID of the wrong service type, the API will return an error.

To find the dependencies of my business service, I make a request to https://api.pagerduty.com/business_services/PXXXXXX using my business service ID.

The first request to the business service dependencies endpoint returns this json data:

{
  "relationships": [
    {
      "dependent_service": {
        "id": "PFABSHOP",
        "relationships": null,
        "type": "business_service_reference"
      },
      "id": "DPXXXXXXXXXXXXXXXX",
      "supporting_service": {
        "id": "PSHIPSRV",
        "relationships": null,
        "type": "business_service_reference"
      },
      "type": "service_dependency"
    },
    {
      "dependent_service": {
        "id": "PFABSHOP",
        "relationships": null,
        "type": "business_service_reference"
      },
      "id": "DPXXXXXXXXXXXXXXXX",
      "supporting_service": {
        "id": "PFABFE",
        "relationships": null,
        "type": "technical_service_reference"
      },
      "type": "service_dependency"
    }
  ]
}

Let’s break it down:

1. The top level is an array named relationships
2. Each relationship in the array has four component objects:
   • dependent_service with id, relationships, and type
   • id of the relationship itself!
   • supporting_service with id, relationships, and type
   • The object type, in this case service_dependency

The dependent_service is the service that is relying on the other service, and the supporting_service is the service that is being depended upon. When looking at my service graph, the dependent_services will be above the supporting_services.

If a business service has no dependencies, the data returned will be an empty relationships array, not an error. As long as the ID is correct, you’ll get regular data.

Now we run up against some of the limitations of this endpoint. First, it’s only the direct dependencies of the first service, not an exhaustive list of all dependencies in the graph. Second, it returns only the object IDs for those services. That’s probably not the most helpful information if I want to present this information to humans, but it will be helpful for making more API requests.

Second Level Dependencies

My example business service has two direct dependencies: one is another business service, and the other is a technical service.

To find the dependencies of those services, I’ll need to make requests to two different endpoints.

1. Another request to /business_services/ID using the ID of the business service in the first array object - PSHIPSRV
2. A request to /technical_services/ID using the ID of the technical service in the second array object - PFABFE

Fortunately, the data returned from both of these endpoints has the same schema, so accessing the pieces of the objects will be similar. However, when we reach a layer inside the graph, there will be relationships for the requested service in both directions. For example, when I request the service dependencies for Fabulous Shop Frontend, my relationships object will include the relationship back to the Fabulous Shop business service. I already have that information, so as I build my methods to traverse the graph (this is starting to look like it needs recursion!), I want to leave out that relationship - otherwise my code will make the same requests over and over again forever.

That object looks like:

{
  "relationships": [
    {
      "dependent_service": {
        "id": "PFABSHOP",
        "type": "business_service_reference"
      },
      "id": "DPXXXXXXXXXXXXXXXX",
      "supporting_service": {
        "id": "PFABFE",
        "type": "technical_service_reference"
      },
      "type": "service_dependency"
    },

The other relationships for FABFE are

    {
      "dependent_service": {
        "id": "PFABFE",
        "type": "technical_service_reference"
      },
      "id": "DPXXXXXXXXXXXXXXXX",
      "supporting_service": {
        "id": "PFABCART",
        "type": "technical_service_reference"
      },
      "type": "service_dependency"
    },
    {
      "dependent_service": {
        "id": "PFABFE",
        "type": "technical_service_reference"
      },
      "id": "DPXXXXXXXXXXXXXXXX",
      "supporting_service": {
        "id": "PSRCHBE",
        "type": "technical_service_reference"
      },
      "type": "service_dependency"
    }
  ]
}

When I am traversing the relationships for a particular service, I will want to ignore any relationships where the current service is a supporting service - those relationships will document the service’s relationship with other services “above” it in the graph.

Other Helpful Bits

Depending on what you want the output of your requests to be for your users, these requests might be enough to get going. I want output that looks more like this:

Fabulous Shop (Business Service)
  Shipping Service (Business Service)
  Fabulous Shop Frontend (Technical Service)
    Fabulous Shop Shopping Cart Backend (Technical Service)
      Fabulous Shop Database (Technical Service)
    Fabulous Shop Search Service Backend (Technical Service)
      Fabulous Shop Search Caching (Technical Service)
        Fabulous Shop Database (Technical Service)

To get the service names of each service, I’ll have to make more requests, one for each service.

HOWEVER.

Getting information about a service, like the service name, will require a different endpoint for business services and technical services. Unfortunately, these requests won’t return the same object schema.

To find the name of a technical service, use the endpoint /services/ID with the service ID. The service name will be returned in the service.name subkey.

For business services, the endpoint is /business_services/ID using the service ID. The name of the service will be in the business_service.name subkey.

Put It Together

To traverse my graph, I’m going to use what is basically a Depth-First Search, except that I am interested in the arms of the graphs, not necessarily the nodes. I want to know if a node is included more than once - for example, the Fabulous Shop Database is a dependency of both the Shopping Cart and Caching services. So I don’t have to note that I’ve already seen a particular node the way you might in regular DFS.

I need to be able to make four kinds of requests:

• Get relationships for a business service
• Get relationships for a technical service
• Get the service name for a business service
• Get the service name for a technical service

I’m going to work in Python, using the requests package to make dealing with the JSON a little more straightforward.

You can find the full source code in my Github at https://github.com/lnxchk/pdgarage-samples/blob/main/python/find_dependent_services.py

Initialization

Set up. I’m importing os to access my API key from the environment, sys to read the business service ID from the command line, and requests to make the API requests.

import os
import sys
import requests

# auth
# find the api tokens in your account /api-keys
# to create a new key, you'll need to be a "manager" or "owner"
api_token = os.environ['PD_API_KEY']

Globals. To print the services out with indentation, track the indent_level. I’m also creating a dictionary to store service names so I won’t have to request a name I’ve already seen.

indent_level = 0
names_dict = {}

def print_indent():
    print("  " * indent_level, end="")

Functions

Set up a function for making requests. See the API documentation for more info on these headers.

def make_req(endpoint):
    url = "https://api.pagerduty.com/{}".format(endpoint)
    headers = {"Accept": "application/vnd.pagerduty+json;version=2",
           "Authorization": "Token token={}".format(api_token),
           "Content-Type": "application/json"}
    return requests.get(url, headers=headers)

This function will make a request to find the dependencies of a business service. It reads the indentation level, then looks for the service name - this is all for output, so can be changed for the output you prefer.

Output the service name and note it is a business service, then get the dependencies of this service.

def get_biz_deps(serv_id):
    global indent_level
    if serv_id in names_dict.keys():
        my_name = names_dict[serv_id]
    else:
        my_name = get_biz_serv_name(serv_id)
    print_indent()
    print("{} (Business Service)".format(my_name))
    indent_level += 1
    endpoint = "service_dependencies/business_services/{}".format(serv_id)
    my_deps = make_req(endpoint)

Read the data from the dependencies request. This will be a relationships array, and we want to know about only the relationships where the current service is a dependent_service. We’ll skip the relationships where the current service is a supporting_service; we’ve already seen those. They’re “above” this service in the graph.

For each dependent service, call the appropriate function to find that service’s dependencies.

    data = my_deps.json()
    for relation in data['relationships']:
        if relation['supporting_service']['id'] == serv_id:
            continue
        if relation['supporting_service']['type'] == "business_service_reference":
            get_biz_deps(relation['supporting_service']['id'])
        elif relation['supporting_service']['type'] == "technical_service_reference":
            get_tech_deps(relation['supporting_service']['id'])
    indent_level -= 1

This function works the same way as get_biz_deps, but it requires different endpoints.

def get_tech_deps(serv_id):
    global indent_level
    if serv_id in names_dict.keys():
        my_name = names_dict[serv_id]
    else:
        my_name = get_tech_serv_name(serv_id)
    print_indent()
    print("{} (Technical Service)".format(my_name))
    indent_level += 1
    endpoint = "service_dependencies/technical_services/{}".format(serv_id)
    my_deps = make_req(endpoint)
    data = my_deps.json()
    for relation in data['relationships']:
        if relation['supporting_service']['id'] == serv_id:
            continue
        if relation['supporting_service']['type'] == "business_service_reference":
            get_biz_deps(relation['supporting_service']['id'])
        elif relation['supporting_service']['type'] == "technical_service_reference":
            get_tech_deps(relation['supporting_service']['id'])
    indent_level -= 1

Functions to retrieve the names of the services. Populate the dictionary.

def get_tech_serv_name(id):
    endpoint = "services/{}".format(id)
    this_service_resp = make_req(endpoint)
    this_service = this_service_resp.json()
    names_dict[id] = this_service['service']['name']
    return(this_service['service']['name'])


def get_biz_serv_name(id):
    endpoint = "business_services/{}".format(id)
    this_service_resp = make_req(endpoint)
    this_service = this_service_resp.json()
    names_dict[id] = this_service['business_service']['name']
    return(this_service['business_service']['name'])

Main loop. Start the process by calling get_biz_deps on the service you want to explore.

if __name__ == '__main__':
    # you can pass the service ID on the command line or 
    # enter it at the prompt
    if len(sys.argv) < 2:
        this_service = input("Which service? ")
    else:
        this_service = str(sys.argv[1])
    get_biz_deps(this_service)

Summary

Thanks to our community member Olivia Mo for this question! If you have questions about PagerDuty or the PagerDuty API, join our community forums and we’ll do our best to help.