MIPSUnit::MUnit

Current Version: 1.4.4 released December 2020

MIPSUnit::MUnit is a JUnit-based unit testing framework for MIPS assembly. In particular, it is a Java interface to the MARS MIPS simulator that helps users write JUnit tests for their assembly code.

MUnit provides methods that

Programmers use JUnit's Assert methods to verify that the function under test behaved as expected. Errors are reported using JUnit's built-in error reporting mechanism.

Example Workflow

Suppose you have assigned the following to your assembly language class:

Write a function named in_range that takes three parameters — value, min, and max — and returns 1 (for true) if min ≤ value ≤ max and 0 (for false) otherwise.

Be sure that your function is named in_range (spelled correctly) and declared in your code's .globl section.

Begin by writing a JUnit test class that describes the expected behavior of in_range:


Example 1: Simple MUnit example. (InRangeTest.java)

Next, compile InRangeTest.java.

javac -cp munit.jar InRangeTest.java

Finally, run the tests. Pass the name of the assembly file and the name of the class file on the command line. (Passing the name of the class file is unusual, but provides a way to run the tests without requiring the user to explicitly set up the runtime classpath.)

java -jar munit.jar in_range_correct.asm InRangeTest.class

Download these files and try it yourself:

Basics

Users use the Registers enumeration to specify registers. Including this line:

import static edu.gvsu.mipsunit.munit.MUnit.Register.*;

allows users to refer to registers without further qualification. Similarly, all methods are static members of MUnit; thus, including

import static edu.gvsu.mipsunit.munit.MUnit.*;

allows users to call the key methods without further qualification. Our goal in using the enumeration and static methods was to minimize the amount of typing needed while still providing a reasonable level of compile-time error checking.

We expect that the vast majority of users will primarily use three groups of methods from the MUnit class:

...data
Create a data label (i.e., a line in the .data section) with the given values:
  • Label d1 = wordData(1, 2, 4, 9, 16, 22, 18, 14);
  • Label c1 = byteData('a', 'b', 'c', 'd');
  • Label s1 = asciiData("Go, Jackets!");
set
Set a register to a specific value:
  • set(t1, 16); // equivalent to li $t1, 16
  • set(t1, d1); // equivalent to la $t1, d1
run
Run the function under test.
  • run("in_range", 10, 20, 30); // extra parameters placed into $a0, $a1, $a2
  • run("index_of", d1,  5, 10); // d1 is a previously-created Label
get
Get the value of the register:
  • get(v0);
get...
Get values from memory:
  • getWord(d1, 3) // get the word beginning 3 bytes past Label d1
  • getByte(c1, 3) // get the 3rd byte past Label c1
  • getWords(d1, 0, 10) // get the 10 words beginning at Label d1
  • getString(c1) // get the String beginning at Label c1 and ending with the next null character.
noOther...Modifications
Verifies that every memory address modified was retrieved by some "get..." method (thereby verifying that the program didn't accidentally write to unexpected memory addresses).

Advanced Techniques

Other JUnit features

Users can take advantage of all JUnit features. For example, adding the @Before method below sets register v0 to a known value before each test. Setting v0 to a value other than 0 or 1 allows us to verify that the user is explicitly setting v0 to 0 (as opposed to incorrectly assuming that v0 always contains 0 at the beginning of a function). 

  @Before
  public void init_v0() {
   set(v0, 3343);
  }

Testing Memory

When testing a program that modifies memory, not only must we verify that the memory locations we expect to modify contain the correct values, but we must also verify that the program didn't incorrectly modify other locations. Explicitly checking every possibly memory location would take a long time. Therefore MUnit tracks which memory locations are accessed by get... methods and provides the noOther...Modifications methods to verify that all modified memory locations were explicitly retrieved. (The choice of the word "retrieved" is intentional. We can verify that the test writer requested the value of each modified memory location; but, we cannot verify that the test writer actually verified the correctness of the returned value.)

The sample below demonstrates how to test an assembly function that reverses the contents of an array.


Example 2:Using MUnit to verify memory. (ReverseTest.java)

There are three methods that check different areas of data memory:

Download these files and try it yourself:

Why MUnit?

MIPSUnit::MUnit is one of two related testing frameworks. The other is the RSpec-inspired MSpec.

Whereas MUnit interacts directly with MARS, MSpec produces an assembly file that contains all the tests. This assembly file can run independently from MSpec.

MUnit has two main advantages over MSpec:

  1. More students know Java and JUnit than Ruby and RSpec; therefore, most students are likely to find MUnit easier to use.
  2. MUnit tests are better isolated from each other. Because MSpec generates a single assembly language test driver, the tests share the same low-level runtime environment as the driver. This means:
    • It is not possible to completely re-set the state of the CPU between tests: For example, a program could conceivably modify the part of the .data section that will be input for another test.
    • The driver halts after reporting the first error. Storing the results of multiple tests would require keeping some registers and memory "off limits" to programs under test. We wanted to avoid limiting the programmer for sake of the testing environment; so we report the first error only.

MSpec's main advantage over MUnit is expressiveness and flexibility. We like to use very thorough, exhaustive tests when evaluating students' code. Preparing such thorough tests in Java/JUnit requires that we either

In contrast, compare how concisely we can write an exhaustive set of tests in MSpec for a function to compute n choose k:

Using MSpec to exhaustively test a function to compute n choose k.
Also, because it generates an assembly language test file, students can run MSpec-generated tests without knowing anything about MSpec. They don't even need to know that such a program exists! In contrast, at minimum, students need to be taught how to use munit.jar to run MUnit tests.

See the MSpec page for more advantages of MSpec.

FAQ

What questions do you have?

Contact Info

You can e-mail me at last-name first-initial@gvsu.edu, or see my complete contact info here.
(C) 2014 Zachary Kurmas
Last updated: 7 December 2018