A brief history of mobile in an automation world.
Guest post by Jessy Hanzo. The original post appeared on Medium.
Jessy lives in Aix en Provence, France and leads a mobile team in a travel company called Voyage Privé. He likes to develop mobile apps both as an employee and a freelancer and tries to improve himself by sharing, discussing and learning. His favorite topics are: iOS, Bots, Internet of Things and mathematical AI projects.
Like any other developer, I love to develop and spend a lot of time being up-to-date, learning new languages, discovering new things, sharing with people and trying to improve myself. But in my day-to-day work, there are many other things to do: daily-weekly-monthly meetings, daily-weekly-monthly reports, monitoring app quality (bugs/ code reviews/ code coverage/ performances…), writing code, managing the developer team, calling providers, understanding other services’ needs and so on.
But among all of these, technical points are the most exciting for me.
It would be great if we would have a solution to replace my time-consuming tasks…
My work life before automation
In our App team at work, we are expected to use a lot of tools. But my human brain loses a lot of time to switch from one subject to another one, that’s why my productivity exponentially decreases with the growth of the number of tools used. Reducing their number is not an option as a lot of them are used by other departments, and achieving this would be tricky to manage: more meetings, more reports and finally all that we want to avoid.
An app builder’s life
Among my time-consuming tasks, one of the worst is definitely archiving apps.
Usually, customers’ and departments’ needs are shared in weekly meetings. Once validated by teams, we write tickets and assign them to developers according to priorities. No problem.
But what about app builds? For example in a 3-week sprint, we have to create one build for each app, both on iOS and Android. Obviously, internal processes guarantee quality but increase the theoretical number of builds.
- Alpha: ~2 builds / ticket
- Beta: ~5 builds
- Release Candidate: ~3 builds
- Release: 2 builds
Usually, there are about 5 tickets for each developer in a 3-week sprint and therefore, about 40 builds would be generated. Forty apps built (compiled, archived and sent to testers) within 30 minutes each and generating ready-to-use add up to about 20 hours (almost 3 days’ worth of working hours!) per sprint.
This has to be fixed!
Hiring a bot 🤖
First I thought about recruiting a developer to reduce the time needed, but finding and training someone costs quite some money and time and in the end I decided I don’t have this much time for this. So we have decided to hire a bot.
Today, a lot of solutions exist for creating bots very easily. One solution I am very keen on is Hubot, an open-source solution developed by Github which has a very convenient way to connect with chat tools through many adapters (Skype, Slack, Telegram, Discord…).
Among all of these, which is the best one for creating a bot very quickly and cleanly? Definitely, Slack. 5 minutes required. A Slack doc is available here.
Then, a smart approach to develop bots is splitting them into unitary jobs. This way, we can easily point out what our bots would have to do:
- a Manager bot: for centralizing all requests from users and bots
- a Builder bot: for requesting app builds
- a Ticket bot: for updating Redmine tickets
Ok, now we have three bots available in Slack with just a few lines of code (a builder bot example available here).
Requesting providers by bots is now really easy and fun. Everybody now enjoys using Slack and bringing new great ideas to be developed.
But under the hood, how do these bots work? Our bots are built around API and webhooks…
🆘 Webhooks to the rescue
For creating automatic builds, bots need to consume API and listen to webhooks. In Github, webhooks can be easily configured in a couple of minutes in repo settings. Once defined, Github will send events to a designated server.
Now, for example, when a ticket has been reviewed twice, an HTTP request is sent from Github to the server thanks to webhooks. Then, the server creates a comprehensive message to send to the Slack bot.
Some of our webhook rules:
- when a PR has been labeled with a status, Redmine is updated with this status
- when a PR has been commented with “build” keyword, a build is created
- when a PR has been merged, a build on “master” branch is created
- During each sprint, we create the next three milestones, thus each ticket is linked with a version in Redmine and a milestone in Github
- each tag on “master” triggers build creation
- and so on…
Finally, people can interact easily with bots in the same way a webhook does and we can keep a trace on it.
Parsing webhooks is not very complicated but it could be tricky and time consuming, what about automating them?
🏀 Node-RED: a webhook playground
An important need in automating processes was to let everybody bring their ideas without coding. A few years ago, the same need has appeared in Internet Of Things. Indeed, a lot of people would have really loved to automate their home without learning programming languages. That’s why in 2012, Raspberry was born and a revolution in IoT world has been started.
A few years later, IBM released a new tool named Node-Red for managing our own IoT environment very easily. By drag and drop any user can link objects with actions. More advanced users can also add custom code in Node-Red thanks to block functions in UI. Since this tool runs on NodeJS on any platform, there are a plenty of flows and nodes to install through npm: Slack, Push Notifications, SQLite…
In our company, about three quarters of people are non-technical. So it is critical to bring them a similar solution. For example, Jean Barthe one of our technical testers has no development skills but has an important will to improve processes. And we are so lucky as that he is really in love with home automation: he has connected blinds, a living room full of Philips Hue lights, a fresh version of Google Home, some Fibaro sensors and triggers... He is the perfect profile for developing our automation tools. He has quickly succeeded in drawing a very nice operational process workflow in Node-Red. It is very interesting to note that the most important time-consuming task with Node-Red is finding what kind of jobs could be automated.
Now everyone can add processes in Node-RED. And it is really easy to check what is done. Today, thanks to this tool, we are managing in UI about 300 nodes: Github, Cron, Store reviews, app crashes, app statistics, build automation, Slack/Skype/Facebook messages, Shorten URL, daily/weekly/sprint reports…
Really nice! … But finally, despite of bots and webhook worfklows, builds remained sluggish. And worst of all, we have to manage builds with an Xcode server…
Bitrise with ❤️
For managing an Xcode Server, Apple has released in Xcode 9 a new feature for managing bots in Xcode. It means you don’t have to download Mac App Server like it was required before. The main configuration is done for you. So what’s wrong?
Maintaining a Xcode Server is a little bit painful. Mainly because you have to manage OS and tools versions. Even if I love to learn and discover new things I am NOT an iOS Devops Engineer and I cannot consider spending time on server maintenance.
In 2017, automation has provided a great opportunity for teams in our company to spend more time on other projects: opening minds, discovering people and sharing knowledge. In a meeting discussion with Jeffrey Macko from @PagesJaunes iOS team, my curiosity was stirred up about a tool I have only heard before: Bitrise.io.
In Mobile Continuous Integration there are plenty of tools and it can be hard to find the best one. What surprised me the most with Bitrise was the ability to manage builds. As long as you have your own server you can install and use Bitrise with a very nice interface without any cost. Indeed, this platform can be used completely by self-hosting it. More info can be found on Bitrise.io CLI website. But to avoid having to manage server, we have decided to pay for hosting a CI server at Bitrise (pricing here). According to our sizing needs, pricing is very affordable compared with other tools like BuddyBuild: 100$/month for 2 concurrent thread (EDIT jan-18: BuddyBuild is definitely out of game with Apple acquisition).
But instead of only talking about Bitrise, it is more efficient to test it and see it for yourselves.
Bitrise’s workflow management is a very relevant topic. Just like Node-RED, Bitrise lets you manage workflows in a user-friendly interface where you can add a lot of steps.
Based on Bitrise steps, we have decided to split workflows into a number of Xcode schemes. Thus each of them (DEV, RECETTE, PREPROD, PROD, APPSTORE) has its own workflow and steps.
As you can see, Bitrise has a cache process for improving build time. The first time we launched a successful workflow, the process lasted 45 minutes (not so good). But the first working version of build automation was complete so we could spend more time on decreasing build time. Today our entire process is about 20 minutes long. We don’t mind this (Bitrise’s limit is 60 minutes), this is only useful for having an app available for testers very quickly.
Check out our Bitrise workflow above, a Curl Step sends a message to our Node-RED server. Once received, Node-RED parses the response and try to send a message to chat tools. Bitrise has recently released a very nice feature for sending webhooks and replacing our Curl Step, just like Github does.
. . .
Conclusion
During the year of using automation processes, new workflows have succeeded in decreasing all time-consuming (and boring) tasks in a very profitable way.
What automation has brought us:
① Our single-repo iOS app was divided in six different repositories in Github
② Non-technical people have developed the third of Node-RED nodes
③ Build time has decreased from 15 minutes to 5 minutes
④ In Dec 2017 we created an higher number of apps than in whole year
⑤ Since Dec 2017, there weren’t any applications manually created by a developer
As next steps, we will bring in more automation: for others teams, other projects and other departments. All the people have so many things to share and improve, our technical skills should help anyone bring their ideas to life.
