App Behavior Analysis: Sign In/Out pass, but Deep Linking (5.1) and Content Play Performance (3.6) fail because the app is still on the sign-in keyboard (VKBKey)

Roku Developer Program
1

Setup

  • Non-TVE subscription channel (SceneGraph). Sign-in happens entirely on-device: a two-step email/password flow using DynamicKeyboard. The app POSTs the credentials to our own backend, stores the returned token in roRegistry, and shows the content grid.
  • The channel passed certification with zero errors. It also has a Sign Out button that clears the token and returns to the login screen.
  • Real hardware: sign-in, playback and deep links all work end to end.

The problem
In App Behavior Analysis:

  • Sign In Flow and Sign Out Flow are reported as passed.
  • Channel Deep Linking Basic (5.1) and Channel Content Play Performance (3.6) error. The detail text says nothing was found in the media player events, and that the active application shows VKBKey as the active element. The attached screenshot is our sign-in keyboard.
  • Screensaver Policy (4.5) gives a warning, which we assume follows from playback never starting.

So the harness reaches our sign-in screen for the deep-link and playback tests, even though the Sign In script is reported as passing. I’m concerned that “passed” only means the script executed, not that the app ended up signed in.

Sign-in RASP script (Test Credentials are configured in the Dashboard)

  • launch: The Corpse Collective Premium
  • pause: 8
  • text: script-login
  • press: down (x4)
  • press: ok
  • pause: 2
  • text: script-password
  • press: down (x4)
  • press: ok
  • pause: 10

What we verified on a real Roku

  • ECP Lit_ keypresses (what text: uses) enter text into DynamicKeyboard correctly, including @, - and ..
  • The four down presses are needed because the key grid consumes three before focus reaches the Next/Sign In button. We measured this on the device.
  • Deep links via ECP /launch/dev?contentId=…&mediaType=… and /input play directly (Direct to Play). AppLaunchComplete fires when playback starts. VODStartComplete comes about 2.6 s after a cold launch (about 5.6 s in a separate measurement), inside the 8 s limit. An unknown contentId leaves the app on the grid with no crash.
  • The test account is verified and entitled.

What we’ve tried

  • Fixed a real LoginScreen bug where the keyboard didn’t hold focus.
  • Switched from opening a Details screen to Direct to Play.
  • Added an AppLaunchComplete beacon tied to playback start.
  • Lengthened the sign-in script waits, with a 10 s pause after the final Sign In, in case the login network call hadn’t returned before the next test relaunched the app.

Questions

  1. Between the sign-in script and the deep-link and playback tests, is the app relaunched, and is the device’s registry preserved? Or is each test run from a fresh state?
  2. Is a token saved to roRegistry by the sign-in script expected to persist into the later tests?
  3. Does “Sign In Flow: passed” only mean the script ran without error, or does it check that the app left the login screen?
  4. Is there any guidance on scripting DynamicKeyboard text entry, or on the timing to allow for a network sign-in call?

Thanks for any pointers.