AutomationSOP

(Under Construction)
Standard Operating Procedures for Automated Tests

Research and Development Phase: 1. Proof of Concept:

2. Writing of Requirements/Spec Document: Once the concept of design is proven to be viable, a spec needs to be written. Beside covering the usual topics, this spec must specify the performance of the tests system, such as the up time each day.

3. Development:

4. Maintanence: From time to time, improvement and change are needed for the Auto Tests System. After any change to the code, a sub set of the affected tests must be run to make sure the entire system is not broken before deploying. It is best to test both the pass and fail conditions.

Format of Log Files: In ordor for the Test Manager PC to parse and summarize the tests results, the log files must be in the follwoing format:

--TESTCASE-- Description of the test case.

--PASS-- The expected and actual value are (album name) and (album name) respectively.

Or --FAIL-- The expected and actual value are (something) and (something else) respectively.

Or --ERROR-- Error has occurred during test. Test cannot be completed. (Some note goes here.)

Or --INFO-- Some information about the test result.

Or --WARNING-- Something that is neither pass nor failure, but a little more serious than info.

Example: --TESTCASE-- Testing the CLI command 00:04:20:16:02:5e mixer bass ? context.

--FAIL-- $sCLIReturnActual is 00:04:20:16:02:5e mixer bass %3F context, and $sCLIReturnExpected is 00:04:20:16:02:5e mixer bass context 50.

--TESTCASE-- Testing the CLI command player 0 2 context:1.

--PASS-- $sCLIReturnActual is player 0 2 context:1, and $sCLIReturnExpected is player 0 2 context:1.

If the unit under tests has its own log file, a parser needs to be written to generate a log file in this format.