Building CI checks with a GitHub App

Introduction

This tutorial demonstrates how to build a continuous integration (CI) server that runs tests on new code that's pushed to a repository. The tutorial shows how to build and configure a GitHub App to act as a server that receives and responds to check_run and check_suite webhook events using GitHub's REST API.

In this tutorial, you will use your computer or codespace as a server while you develop your app. Once the app is ready for production use, you should deploy your app to a dedicated server.

This tutorial uses Ruby, but you can use any programming language that you can run on your server.

This tutorial is broken into two parts:

In part one, you'll learn how to set up the framework for a CI server using GitHub's REST API, create new check runs for CI tests when a repository receives newly pushed commits, and re-run check runs when a user requests that action on GitHub.
In part two, you'll add functionality to your CI test, by adding a linter test to your CI server. You'll also create annotations that are displayed in the Checks and Files Changed tab of a pull request, and automatically fix linter recommendations by exposing a "Fix this" button in the Checks tab of the pull request.

About continuous integration (CI)

CI is a software practice that requires frequently committing code to a shared repository. Committing code more often raises errors sooner and reduces the amount of code a developer needs to debug when finding the source of an error. Frequent code updates also make it easier to merge changes from different members of a software development team. This is great for developers, who can spend more time writing code and less time debugging errors or resolving merge conflicts.

A CI server hosts code that runs CI tests such as code linters (which check style formatting), security checks, code coverage, and other checks against new code commits in a repository. CI servers can even build and deploy code to staging or production servers. For examples of the types of CI tests you can create with a GitHub App, see the continuous integration apps that are available in GitHub Marketplace.

About checks

GitHub's REST API allows you to set up CI tests (checks) that are automatically run against each code commit in a repository. The API reports detailed information about each check in the pull request's Checks tab on GitHub. You can use checks in a repository to determine when a code commit introduces errors.

Checks include check runs, check suites, and commit statuses.

A check run is an individual CI test that runs on a commit.
A check suite is a group of check runs.
A commit status marks the state of a commit, for example error, failure, pending, or success, and is visible in a pull request on GitHub. Both check suites and check runs contain commit statuses.

GitHub automatically creates check_suite events for new code commits in a repository using the default flow, although you can change the default settings. For more information, see REST API endpoints for check suites. Here's how the default flow works:

When someone pushes code to the repository, GitHub automatically sends the check_suite event with an action of requested to all GitHub Apps installed on the repository that have the checks:write permission. This event lets the apps know that code was pushed to the repository, and that GitHub has automatically created a new check suite.
When your app receives this event, it can add check runs to that suite.
Your check runs can include annotations that are displayed on specific lines of code. Annotations are visible in the Checks tab. When you create an annotation for a file that is part of the pull request, the annotations are also shown in the Files changed tab. For more information, see the annotations object in REST API endpoints for check runs.

For more information about checks, see REST API endpoints for checks and Using the REST API to interact with checks.

Prerequisites

This tutorial assumes you have a basic understanding of the Ruby programming language.

Before you get started, you may want to familiarize yourself with the following concepts:

Checks are also available to use with the GraphQL API, but this tutorial focuses on the REST API. For more information about the GraphQL objects, see Check Suite and Check Run in the GraphQL documentation.

Setup

The following sections will lead you through setting up the following components:

A repository to store the code for your app.
A way to receive webhooks locally.
A GitHub App that is subscribed to "Check suite" and "Check run" webhook events, has write permission for checks, and uses a webhook URL that you can receive locally.

Create a repository to store code for your GitHub App

Create a repository to store the code for your app. For more information, see Creating a new repository.
Clone your repository from the previous step. For more information, see Cloning a repository. You may use a local clone or GitHub Codespaces.
In a terminal, navigate to the directory where your clone is stored.
Create a Ruby file named server.rb. This file will contain all the code for your app. You will add content to this file later.
If the directory doesn't already include a .gitignore file, add a .gitignore file. You will add content to this file later. For more information about .gitignore files, see Ignoring files.

Create a file named Gemfile. This file will describe the gem dependencies that your Ruby code needs. Add the following contents to your Gemfile:

Ruby

source 'http://rubygems.org'

gem 'sinatra', '~> 2.0'
gem 'jwt', '~> 2.1'
gem 'octokit', '~> 4.0'
gem 'puma'
gem 'rubocop'
gem 'dotenv'
gem 'git'

source 'http://rubygems.org'

gem 'sinatra', '~> 2.0'
gem 'jwt', '~> 2.1'
gem 'octokit', '~> 4.0'
gem 'puma'
gem 'rubocop'
gem 'dotenv'
gem 'git'

Create a file named config.ru. This file will configure your Sinatra server to run. Add the following contents to your config.ru file:
Ruby
```
require './server'
run GHAapp
```
```
require './server'
run GHAapp
```

Get a webhook proxy URL

In order to develop your app locally, you can use a webhook proxy URL to forward webhook events from GitHub to your computer or codespace. This tutorial uses Smee.io to provide a webhook proxy URL and forward events.

In a terminal, run the following command to install the Smee client:
Shell
```
npm install --global smee-client
```
```
npm install --global smee-client
```
In your browser, navigate to https://smee.io/.
Click Start a new channel.
Copy the full URL under "Webhook Proxy URL".

In the terminal, run the following command to start the Smee client. Replace YOUR_DOMAIN with the Webhook Proxy URL you copied in the previous step.

Shell

smee --url YOUR_DOMAIN --path /event_handler --port 3000

smee --url YOUR_DOMAIN --path /event_handler --port 3000

You should see output like the following:

Forwarding https://smee.io/YOUR_DOMAIN to http://127.0.0.1:3000/event_handler
Connected https://smee.io/YOUR_DOMAIN

The smee --url https://smee.io/YOUR_DOMAIN command tells Smee to forward all webhook events received by the Smee channel to the Smee client running on your computer. The --path /event_handler option forwards events to the /event_handler route. The --port 3000 option specifies port 3000, which is the port you will tell your server to listen to, when you add more code later in the tutorial. Using Smee, your machine does not need to be open to the public internet to receive webhooks from GitHub. You can also open that Smee URL in your browser to inspect webhook payloads as they come in.

We recommend leaving this terminal window open and keeping Smee connected while you complete the rest of the steps in this guide. Although you can disconnect and reconnect the Smee client without losing your unique domain, you may find it easier to leave it connected and do other command-line tasks in a different terminal window.

Register a GitHub App

For this tutorial, you must register a GitHub App that:

Has webhooks active
Uses a webhook URL that you can receive locally
Has the "Checks" repository permission
Subscribes to the "Check suite" and "Check run" webhook events

The following steps will guide you through configuring a GitHub App with these settings. For more information about GitHub App settings, see Registering a GitHub App.

In the upper-right corner of any page on GitHub, click your profile picture.
Navigate to your account settings.
- For an app owned by a personal account, click Settings.
- For an app owned by an organization:
  1. Click Your organizations.
  2. To the right of the organization, click Settings.
- For an app owned by an enterprise:
  1. Click Enterprise settings.
Navigate to the GitHub App settings.
- For an app owned by a personal account or organization:
  1. In the left sidebar, click Developer settings, then click GitHub Apps.
- For an app owned by an enterprise:
  1. In the left sidebar, click Settings, then click GitHub Apps.
Click New GitHub App.
Under "GitHub App name", enter a name for your app. For example, USERNAME-ci-test-app where USERNAME is your GitHub username.
Under "Homepage URL", enter a URL for your app. For example, you can use the URL of the repository that you created to store the code for your app.
Skip the "Identifying and authorizing users" and "Post installation" sections for this tutorial.
Make sure that Active is selected under "Webhooks".
Under "Webhook URL", enter your webhook proxy URL from earlier. For more information, see

In this article