Hi, we got bored from tedious unit testing on Roku, particularly on large projects. That is why we developed this tool MAMUT ( mamuttool.com ).
What do you think of it? Do you have experience with any other unit testing system for Roku?
This looks cool. I’m not thrilled about running a node.js server, but I guess that’s par for the course these days. We’ve been kicking around the idea of writing a test framework here, so I’d love to try this out and see if it meets our needs.
Hi Tim, thanks! Most important for us is to know what do you think of it. Now we are working on a new version which is going to include global server.
Roku has a testing frameworking here that I’m thinking about building into my tool http://www.stegmancompany.com/#RokuDashboard
has anyone actually used the roku framework? Curious about it.
“tim_beynart” wrote:
has anyone actually used the roku framework? Curious about it.
Yes. It works fairly well. Comparable in features to the older brstest but a little better set up to manage test suites for related tests. Also supports SG component testing though I haven’t used it for that.
Had any one tried some testing system which supports continuous integration? We are considering it in Mamut right now.
How would you do automation with Roku? There’s no virtualization for Roku OS, so you have to test on a physical device. We use precommit hooks to run tests on the developer’s machine before they commit to git, but that is not ideal.
“tim_beynart” wrote:
How would you do automation with Roku? There’s no virtualization for Roku OS, so you have to test on a physical device. We use precommit hooks to run tests on the developer’s machine before they commit to git, but that is not ideal.
Roku devices are always necessary, but the server allows to distribute packages with tests, run it and collect the results from those devices. Tim, you are probably not using Unit tests in this way right?
Has anyone tested Roku tasks with the unit test framework? I haven’t been able to figure out how to fit the scenario of starting a task where the result comes back later in an observed event callback into the unit test framework.
Thanks in advance.
I have setup CI server for our roku application, but it requires an external server to handle the build webhook, and parsing the test results into something meaningful.
I am using https://github.com/rokudev/unit-testing-framework to accomplish this.
We built out some functionality for this on ukor. It includes the ability to generate test reports. But unfortunately you still require a physical device to run your tests. 
I’ve (imho) vastly improved the framework, with the use of tags
https://github.com/georgejecook/unit-testing-framework
###New tags
'@TestSuite [NAME OF TESTSUITE] - denotes that a file is a testsuite. Must be at the top of the file (must be within the first 100 lines). e.g. '@TestSuite My super duper test suite
'@It [NAME OF TEST GROUP] - describes a block of tests - e.g.tests the constructor or '@It tests the generator methods. All following '@Test declarations will belong to thisgroup, until the nextgroup or end of file. These groups can accept the tags '@Only and '@Ignore
'@Test [NAME OF TEST] - Indicates the following method is a test case. Must be the line before a method - if name is provided, it will be used in reporting; otherwise the function name is used
'@Setup - Indicates the following method will Setup the '@It group. Must be the line before a method. This property will be applied to all following '@It groups. This allows you to specify the setup for every group in the file, or override it for specific groups.
'@Teardown - Indicates the following method will teardown the test suite. Must be the line before a method. This property will be applied to all following '@It groups. This allows you to specify the setup for every group in the file, or override it for specific groups.
'@BeforeEach - Indicates the following method will Run before EVERY test in the suite. Must be the line before a method. This property will be applied to all following '@It groups. This allows you to specify the setup for every group in the file, or override it for specific groups.
'@AfterEach - Indicates the following method will Run after EVERY test in the suite. Must be the line before a method. This property will be applied to all following '@It groups. This allows you to specify the setup for every group in the file, or override it for specific groups.
'@Ignore - Indicates that the next '@It, '@Test, '@TestSuite is to be ignored when the tests run. Must be the line before a ‘@Test/’@TestSuite tag
'@Only - Indicates that the ONLY this test/testsuite will run. Affects the next '@It, '@Test, '@TestSuite tag . This tag is accumluative. i.e. if you have several '@Only tags, then all of the '@Solo tagged tests will run.
Tag case is importan. ####Example
'@TestSuite [ETC] ExampleTestComponent test suite
'Note the ETC__ name space, and me putting [ETC] in the suite name, is not require. I just do it coz it works for me for namepsacing/readability
'@Setup
function ETC__SetUp()
'This setup method will apply to all it groups in the test
? "setting up test suite for " ; m.Name
end function
'@BeforeEach
function ETC__BeforeEach()
'This beforeEach method will apply to all it groups in the test
end function
'+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
'@It tests basic functionality
'+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
'@Test
function ETC__basicTest() As String
return m.AssertEqual("true", "true")
end function
'@Ignore
'@Test
function ETC__basicTest() As String
'This test is ignored
return m.AssertEqual("true", "true")
end function
'@Only
'+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
'@It tests another method
'+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
'@Test
function ETC__AnotherMethod1() As String
return m.AssertEqual("true", "true")
end function
'@Test
function ETC__AnotherMethod2() As String
'This test is ignored
return m.AssertEqual("true", "true")
end function
'+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
'@It tests some connectivity stuff
'+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
'@Test
function ETC__Connect1() As String
return m.AssertEqual("true", "true")
end function
'@Test
function ETC__Connect2() As String
'This test is ignored
return m.AssertEqual("true", "true")
end function
