Ad-hoc reports from GitHub in F#

At Bitrise we have multiple organizations registered on GitHub with several open source repositories. Our developers and managers often need to access a quick overview of the current status of repositories and/or issues. Read about our methods for processing data from GitHub API.

We can process data the way we did with ClickUp, using JSON and HTTP type providers or we can opt for the official GitHub library, Octokit to communicate with the API. With Octokit, once authenticated you can easily build your reports in Ruby, Node.js and .NET languages but there are third party libraries for other frameworks too.

The next few paragraphs will introduce methods for processing data from GitHub API.

Octokit in F#

The .NET documentation encourages us to write code in C# but a lazy coder like me prefers fewer lines. Also, we do not need to compile the code, our beloved F# Interactive will do the job instead.

We need a couple of packages:


open Octokit
open Deedle
open Xplot.Googlecharts
Copy code

Connect

You can try the API without authentication but that will limit you to 60 requests an hour. Authentication is easy once you registered your application at GitHub. GitHub requires a valid User-Agent value, so use the name of your application. Use your token the following way to establish a connection.


let appName = "Bitrise-GitHub-App"

let client = GitHubClient(new ProductHeaderValue(appName));

let token = 'abc123...'
let tokenAuth = Credentials(token)
let client.Credentials = tokenAuth
Copy code

Get that data

client.Repository.GetAllForOrg() method needs only the name of the organization as a parameter. The signature of Task<IReadOnlyList<Repository>> tells us this is a task that is generating a list of repositories. The following lines count the repositories of bitrise-steplib where you can find the code of our workflow steps.


let org = "bitrise-steplib"

let repos  = client.Repository.GetAllForOrg(org)

// execute task
let repoResults = repos.Result

// number of repositories
repoResults.Count
Copy code

Process the data

For F# Deedle is what pandas are for Python: it has all the functions to process data in a tabular format. Let's draw a table of the number of forks, open issues and stargazers to understand which step has the most active community.


repoResults

// filter attribute
|> Seq.map (fun r -> (r.FullName, r.ForksCount, r.OpenIssuesCount, r.StargazersCount))

// build a data frame from F# record
|> Frame.ofRecords

// unfortunately it will not inherit the keys
|> Frame.indexColsWith ["repository"; "forks"; "open issues"; "stargazers"]

// descending order
|> Frame.sortRowsBy "stargazers" (fun v -> -v)

// Google Charts table
|> Chart.Table
Copy code

We can easily collect high-level figures with Deedle and its Stats namespace. This last contains members for several statistical indicators like mean, standard deviation or skewness etc.


let options = Options(colors = [|"#6B25BC"; "#60D0C4"|])

repoResults
|> Seq.map (fun r -> (r.Owner.Login, r.ForksCount, r.OpenIssuesCount))
|> Frame.ofRecords
|> Frame.indexColsWith ["organization"; "forks"; "open issues"]

// total numbers by organization
|> Frame.aggregateRowsBy ["organization"] ["forks"; "open issues"] Stats.sum
// we need only the numerical columns for the chart
|> Frame.dropCol "organization"
|> Chart.Column 
|> Chart.WithTitle "Total number of forks and open issues on repositories of bitrise-steplib"
|> Chart.WithOptions options
|> Chart.WithLegend true
Copy code

Stay tuned for the next article on data wizardry titled Understanding Variance in User Attributes coming soon.

No items found.
The Mobile DevOps Newsletter

Explore more topics

App Development

Learn how to optimize your mobile app deployment processes for iOS, Android, Flutter, ReactNative, and more

Bitrise & Community

Check out the latest from Bitrise and the community. Learn about the upcoming mobile events, employee spotlights, women in tech, and more

Mobile App Releases

Learn how to release faster, better apps on the App Store, Google Play Store, Huawei AppGallery, and other app stores

Mobile DevOps

Learn Mobile DevOps best practices such as DevOps for iOS, Android, and industry-specific DevOps tips for mobile engineers

Mobile Testing & Security

Learn how to optimize mobile testing and security — from automated security checks to robust mobile testing and more.

Product Updates

Check out the latest product updates from Bitrise — Build Insights updates, product news, and more.

The Mobile DevOps Newsletter

Join 1000s of your peers. Sign up to receive Mobile DevOps tips, news, and best practice guides once every two weeks.