Recently, I had the pleasure of giving a talk with Dima, the co-author of Maestro at mobile.dev, about how to set up UI tests in minutes with Bitrise and Maestro. Meeting different mobile & test automation engineers and other attendees worldwide was an excellent opportunity.
❗️If you missed the session, you can check out the on-demand recording here❗️
During the session, we got various questions about mobile testing with Maestro and Bitrise for CI/CD, so I wrote this blog to answer them all.
Let’s get started!
Maestro-related questions
- Is there something like a cucumber life "background" that can be used before each scenario in a flow? Are there methods like a cucumber "BeforeAll" / "Before" / "BeforeStep"?
Answer: You can utilize the [runFlow] command to achieve a similar result. For example, you can extract commands that should run before or after each flow and refer to them from your main flow.
- Is there tagging/grouping of tests in Maestro?
Answer: Yes! Maestro supports tags.
- Can you add scroll elements into the middle of the screen out-of-the-box, or is coding needed to achieve this?
Answer: That is, unfortunately, not possible at the moment. We don’t think aligning elements perfectly on the screen is a common user behavior.
- Does Maestro support the custom functions which are used in the input text like, for example, “Random data”?
Answer: Yes. There are some pre-built functions:
- inputRandomText
- inputRandomNumber
- inputRandomEmail
- inputRandomPersonName
Furthermore, you can get custom functionality by writing your own JavaScript snippets and using runScript or evalScript commands.
- What is the cost of Maestro Cloud? Is it advisable to just use it for Regression of Production Builds vs. Dev Merges?
Answer: The cost is $0.10 per flow-run. The price is highly flexible based on your needs and can be very cheap if you only run tests before releases or more expensive if you run them on each pull request. It is entirely within your control.
- Can I trigger a custom code from the maestro test in the native Android app (that would, for example, simplify the app behavior to avoid flakiness), and how?
Answer: Not at the moment, but we are considering adding such support with Maestro SDK.
- I have a Zoom-like app where multiple people can “Join a Room”, or can publish audio, video, screen share, PIP, etc. Apart from App UI Navigation, can Maestro detect that Audio & Video is publishing as expected? Has anyone tested a similar scenario using Maestro?
Answer: Not at the moment, but we will be able to achieve something in that area in the future.
- How can we test multiple flows? For example, Onboarding + Login, Onboarding + Forget Pass, Onboarding + Forget Pass + Login?
Answer: Our general advice is to think about flows as end-to-end tests, a last step in your testing pipeline. As such, try to package your tests in flows that would be meaningful to the user without getting too granular. In your example, I would test 2 flows: Happy Path (Onboarding + Login), and Forgot Password Path (Onboarding + Forgot Password).
- I would love to use Maestro, but I build for iPad fixed in landscape mode and, unfortunately, UI elements are not getting hit. I was hoping Maestro Studio and tapping using coordinates would solve this, but it hasn’t. Is there a temporary hack for this? And a permanent fix on the way?
Answer: We hear you and hope to address iPad issues sometime in the future once we feel that iPhone support is impeccable as we would want it to be. Stay tuned!
Bitrise-related questions
- Is there a recording of the presentation that you can share?
Answer: Yes, you can find the on-demand recording here.
- Where can I find an example of a full workflow in Bitrise running Maestro for a native Android app and showing the results/errors that Maestro found?
Answer: Mobile.dev GitHub repository contains a number of samples that you can try on your Bitrise account.
- How does the cost of Bitrise rise with Maestro tests?
Answer: Bitrise is using a pay-as-you-go pricing model where you have credit, and each build consumes from this credit, so it depends on the configuration of the Maestro Step. For example, you have two options; if you don't want the action to wait until the Upload has been completed to Maestro Cloud as is the default behavior, set the async argument to true — which will take seconds only in your Bitrise build. But, you need to check the results manually in the Maestro Cloud Console. Otherwise, if it’s false, the build will wait till the Maestro Step finishes the action and continue with the remaining steps.
- While tests run on Maestro Cloud, the Bitrise machine is idling and thus is using credits, right? IMHO, this idling/waiting should happen on the smallest machine stack (to save credits). Will the integration (Maestro <> Bitrise) lower the machine stack to the smallest one, and will I need to do that manually?
Answer: That’s true, and you have two options here; if you don’t want the build to be idle, you can set the async argument to be accurate. In this case, the build will not wait for the Maestro Step to be finished, but you need to check the results manually in the Maestro Cloud Console. The second option is that if you have a dedicated Bitrise Workflow for Maestro, you can change the machine type to save your credits.
Find out more about Maestro at https://maestro.mobile.dev/
Or, contact Dima at:
Dima Zaytsev, the Co-author of Maestro
[email protected]
