Ad-hoc reports from GitHub in F#

We can process data the way we did with ClickUp, using JSON and HTTP type providers or we can opt for the official GitHub library, Octokit to communicate with the API. With Octokit, once authenticated you can easily build your reports in Ruby, Node.js and .NET languages but there are third party libraries for other frameworks too.

The next few paragraphs will introduce methods for processing data from GitHub API.

Octokit in F#

The .NET documentation encourages us to write code in C# but a lazy coder like me prefers fewer lines. Also, we do not need to compile the code, our beloved F# Interactive will do the job instead.

We need a couple of packages:

• Octokit to get the data
• optionally, Deedle for data frames and statistics
• Xplot to present our reports.

open Octokit
open Deedle
open Xplot.Googlecharts

Connect

You can try the API without authentication but that will limit you to 60 requests an hour. Authentication is easy once you registered your application at GitHub. GitHub requires a valid User-Agent value, so use the name of your application. Use your token the following way to establish a connection.

let appName = "Bitrise-GitHub-App"

let client = GitHubClient(new ProductHeaderValue(appName));

let token = 'abc123...'
let tokenAuth = Credentials(token)
let client.Credentials = tokenAuth

Get that data

client.Repository.GetAllForOrg() method needs only the name of the organization as a parameter. The signature of Task<IReadOnlyList<Repository>> tells us this is a task that is generating a list of repositories. The following lines count the repositories of bitrise-steplib where you can find the code of our workflow steps.

let org = "bitrise-steplib"

let repos  = client.Repository.GetAllForOrg(org)

// execute task
let repoResults = repos.Result

// number of repositories
repoResults.Count

‍

Process the data

For F# Deedle is what pandas are for Python: it has all the functions to process data in a tabular format. Let's draw a table of the number of forks, open issues and stargazers to understand which step has the most active community.

repoResults

// filter attribute
|> Seq.map (fun r -> (r.FullName, r.ForksCount, r.OpenIssuesCount, r.StargazersCount))

// build a data frame from F# record
|> Frame.ofRecords

// unfortunately it will not inherit the keys
|> Frame.indexColsWith ["repository"; "forks"; "open issues"; "stargazers"]

// descending order
|> Frame.sortRowsBy "stargazers" (fun v -> -v)

// Google Charts table
|> Chart.Table

We can easily collect high-level figures with Deedle and its Stats namespace. This last contains members for several statistical indicators like mean, standard deviation or skewness etc.

let options = Options(colors = [|"#6B25BC"; "#60D0C4"|])

repoResults
|> Seq.map (fun r -> (r.Owner.Login, r.ForksCount, r.OpenIssuesCount))
|> Frame.ofRecords
|> Frame.indexColsWith ["organization"; "forks"; "open issues"]

// total numbers by organization
|> Frame.aggregateRowsBy ["organization"] ["forks"; "open issues"] Stats.sum
// we need only the numerical columns for the chart
|> Frame.dropCol "organization"
|> Chart.Column 
|> Chart.WithTitle "Total number of forks and open issues on repositories of bitrise-steplib"
|> Chart.WithOptions options
|> Chart.WithLegend true

Stay tuned for the next article on data wizardry titled Understanding Variance in User Attributes coming soon.