Halo Insights
Your job as a Tester or Quality Assurance Professional is to provide the fastest application quality feedback possible to your wider delivery team.
Halo Insights with in-built test failure screen capture makes it easy to accurately diagnose functional regressions vs troublesome tests.
Test Evolve Insights has been designed to allow you to rapidly diagnose your failing tests.
Review at a ‘Run’ and ‘Scenario’ Level.
Compare run times, run pass/fail rates, changes in run properties and persistently failing tests with full traceability and root cause analysis.
Understand at-a-glance if your Test Run has ‘improved’ or ‘regressed’.
Test Evolve Halo run level insights show you immediately how your latest completed run compares to its immediate predecessor or to an average of the last 100 runs.
If there are no new failures and you ‘signed-off’ on the run before it, it’s safe to say that your latest build hasn’t regressed.
And how are we so sure? Well, we store over a year’s worth of continuous run history and we’ve built an analytics model that looks at your test failures down to the exact line of code at failure point.
Review the latest makeup of your failing tests in a run, compared to the last run or the average run.
New failures will be key and will tell you quickly if you need to dig a little deeper in your investigation.
Recurring failures are a sign that you have as-yet unraised bugs or a test that needs to be made a little more resilient. Either way, they are starting to appear frequently and need addressing.
Flaky tests are tests that are passing but only when you allow them at least 1 retry after a first failure. They’re not the worst thing in the world but they are a sign that a particular test can’t always be relied upon and should be checked or refactored until it’s a consistent performer.
Observe your % increases/decreases across overall pass rate and scenario runtimes.
And when things shift unexpectedly, consider whether any noted change in test run properties could be the cause.
Enjoy full traceability from Feature to Scenario to Step to Stacktrace to screenshot.
Embedded failure screenshots provide you with a clear view of your application state at the point of test failure.
Coupled with your Scenario, Step and full stacktrace, you should be able to easily diagnose whether you have a true feature regression or a problematic test to deal with.
Get a full test case failure breakdown across the last 100 executions.
At its core, Halo Insights has the ability to tell you whether your failing test is continuously failng for exactly the same reason every time, down to the lowest line of automation code.
We built this to enable testers to feedback as quickly as possible and to allow them to focus on the failing tests that are likely the quickest to resolve. Either because there is a new bug to raise or there is a persistently troublesome test that requires intervention.
Export your failing test data with insights for external share or BI reporting tool import.
Whilst we’ve developed Halo to be a one-stop complete test reporting solution, we appreciate there may be times when you want to extract your data and incorporate it within your other reporting suites.
No problem. Simply use our handy Test Run Failures export feature to grab a csv for onward use.
Frequently Asked Questions
-
Simply put, Test Run Insights make a comparison between your latest run as a whole and either the previous run or the 100 run average, letting you know quickly whether things are getting better or worse.
Test Scenario Insights tell you about individual test scenario failures and any obvious and repeating patterns of failure that should be investigated.
-
They allow you to ‘approve’ a build in an accelerated timeframe and pinpoint failing tests that should be focussed on as a priority.
-
Following any given test cycle, you now stand your best chance of feeding back to your delivery team quickly and accurately when raising bugs.
When maintaining your automated test suite, you are now able to focus on priority test cases above lower risk failures.