Hi,
I’m hitting a persistent issue with the Automated Behavior Analysis (ABA) certification tests that is blocking my app update submission.
The test suite appears to be executing tests 2–6 (i.e. Deep Linking and Playback) before running the On-Screen Sign-In script (Test 1). Because our app requires an active login to initiate playback and landing on deep links defaults unauthenticated users to our WelcomeAuthScreen, tests 2–6 continuously fail with:
“While watching media player events, nothing was found. The active application Nitro.Film (dev) shows “WelcomeAuthScreen” as the active element. Suggest looking if not logged in, channel UI has changed or maximum streams limit was reached”
It is expect for a subscription service that you need to be signed in with an account for playback to occur whether that happens from a deeplink or inside the app. Because it’s not running the sign in script first as expected, it’s hitting the sign in screen and thinks it failed. If you sign in at that point, playback will occur. If you’re already signed in, the deeplink works. And when the Sign-In and Sign-Out scripts do eventually run at the very end of the test execution, they PASS.
Per the Roku Authenticated Cert Testing documentation, testing devices are supposed to sign in first, run the deep linking/performance tests while authenticated, and sign out at the end. This is not happening.
Has anyone encountered this sequence issue where ABA runs deep linking before executing the sign-in script, or found a fix/workaround to ensure authentication completes first? I’ve had this issue on every prior submission and the workaround was to just, submit it again and eventually it tests in the right order and passes. I’ve had 4 prior releases this way but this time the workaround is not working. I opened a ticket a few days ago but no response yet and I’m still blocked.
Thanks!
