`DLUnit`

DLUnit is a JUnit-based unit testing framework for simulated digital logic circuits. Specifically, it allows students to write JUnit tests for circuits they build using either the JLS or Logisim digital logic simulators.

DLUnit provides methods that

set the initial circuit state (currently input pins and registers --- memory coming soon),
instruct JLS or Logisim to simulate the circuit under test, and
query the final state of the circuit (currently, read the output pins and registers).

Programmers use JUnit's Assert methods to verify that the circuits under test behaved as expected. Errors are reported using JUnit's built-in error reporting mechanism.

Example Workflow

Suppose the assignment is to build a half adder. Begin by writing a JUnit test class that describes the expected behavior of the circuit:

Example 1: SimpleDLUnit example.(HalfAdderTest.java)

Next, compile HalfAdderTest.java.


            javac -cp dlunit.jar HalfAdderTest.java

Now, implement a half adder in your favorite digital logic simulator. (You did write the test first, right? 😉)

Finally, run the tests. Pass the name of the circuit and the name of the class file on the command line. (Passing the name of the class file is unusual, but provides a way to run the tests without requiring the user to explicitly configure the runtime classpath.)


            java -jar dlunit.jar halfAdder.circ HalfAdderTest.class

Download these files and try it yourself:

JLS:
Logisim:

Basics

The methods in DLUnit allow the unit tests to interact with the underlying simulator. Adding this line to the beginning of the test class

import static edu.gvsu.dlunit.DLUnit.*;

allows programmers to call DLUnit methods without further qualification. Our goal in using static methods is to minimize the amount of typing needed while still providing a reasonable level of compile-time error checking.

setPin...

Set an input pin to a specific value:

setPin("FeatureEnabled", true);
setPinSigned("InputA", -234);
setPinUnsigned("InputB", 0xABCD);

setRegister...

Set a register to a specific value:

setRegister("LightOn", true);
setRegisterSigned("RunningTotal", -234);
setRegisterUnsigned("CurrentState", 0xABCD);

setMemory...

Set a segment of memory to a values:

setMemorySigned("MyRAM", 0x12C, -17);
setMemorySigned("MyRAM", 0xAB00, new int[]{10, -11, 12, -13});
setMemoryUnsigned("MyRAM", 0x1240, 23);
setMemoryUnsigned("MyRAM", 0x0, new int[]{10, 11, 12, 13});

Note: Logisim does not name its RAM elements. This means (1) The name parameter is ignored when testing logisim circuits, and (2) DLUnit does not support the testing of Logisim circuits with more than one RAM element per subcircuit.

run

Run the simulator:

run()

readPin...

Get the value of an output pin:

readPin("WasErrorDetected"); // returns boolean
readPinSigned("InputA"); // returns long; interprets most significant bit as a sign bit
readPinUnsigned("InputB"); // returns long

readRegister...

Get the value of a register:

readRegister("LightOn"); // returns boolean
readRegisterSigned("CurrentSum"); // returns long; interprets most significant bit as a sign bit
readRegisterUnsigned("CurrentState"); // returns long

readMemory...

Get the values in a segment of memory:

readMemorySigned("MyRAM", 0x12C); // returns single word; most significant bit is sign bit.
readMemorySigned("MyRAM", 0xAB00, 4); // returns the four words beginning with 0xAB00
readMemoryUnsigned("MyRAM", 0x12C); // returns a single word as a positive integer
readMemoryUnsigned("MyRAM", 0xAB00, 4); // returns the four words beginning with 0xAB00

CPU Testing

DLUnit includes classes to test the single-cycle and multi-cycle MIPS CPUs presented in the Patterson and Hennessy text (and also in the Harris and Harris text). These test classes take a MIPS assembly language file as input and

Use the MARS MIPS simulator to generate machine code
Load that machine code into the CPUs instruction memory
Simulate the CPU
Simulate the assembly code using MARS
Compare the registers and memory in the simulated CPU to the expected values (as determined by MARS)

To test a single-cycle CPU, run the following command:


            java -jar dlunit.jar myCPU.jls builtin.SingleCycleCPUTest.java --param myTestFile.a

The test file assumes the following names for registers and memory:

The Registers must be contained in a subcircuit named "RegisterFile" and named "R1", "R2", "R3", ..., "R31".
The instruction memory must be contained in a subcircuit named "InstMemory and named "InstructionMemory"
The data memory must be in the main circuit (i.e., not in a subcircuit) and named "MainMemory".

(These values are relatively easy to find and change in the source code, if desired.)

Other Use Cases

Here are some examples of how I use DLUnit in my classroom:

Thorough/Exhaustive Testing

Experienced programmers tend to write unit tests that focus on corner cases and other key examples. However, my students are novices: Their bugs don't always occur in the expected places. Therefore, I prefer to test my students' circuits very thoroughly. Writing a large number of individual tests would be rather tedious. Fortunately, it is rather easy to write thorough/exhaustive DLUnit tests.

RippleCarryAdderTest.java is a test class I use to evaluate my students' ripple carry adders. Notice that:

Because all the tests have the same basic structure, I wrote a helper method (verifyOutputAndCarryOut) that is called by all tests.
This helper method calculates the expected outputs based on the inputs. By calculating the output, (1) I avoid making silly mistakes, and (2) it is easier to change/add inputs.
Rather than writing a separate method for each set of inputs, I write a several methods that iterate through a list of inputs. This is not a standard unit test technique; but, it serves me well for grading purposes. (Notice the list of input values at the beginning of the file. I can quickly and easily add or remove tests by modifying these lists.)
By using the @FixMethodOrder annotation to specify the order in which JUnit runs the tests, I can quickly evaluate where a submitted circuit falls within a rubric. Circuits that fail one of the "a" tests have a fundamental error in the addition circuit. In this case, circuits that pass all the "a" tests but fail a"b" test have used the CarryIn input incorrectly. Circuits that fail only "c" tests have a bug in the overflow detection. (This rubric is not the only factor in the final grade: I also look at each circuit and comment on style and technique.)

Supported Simulators

JLS: JLS was written by David Poplawski at Michigan Tech. I have used this simulator for 10 years. I find it intuitive and easy to use. It has a number of built-in components that are very helpful when building ALUs and Patterson and Hennessy's example CPU from their textbook.
JLS has two main limitations: (1) It has a couple of minor UI bugs that can be annoying; and (2) there is no quick way to fix a bug in a subcircuit that been included multiple times into a main circuit. For example, if a student finds a bug in a full adder after she has imported it 16 times into a ripple carry adder, she must delete and re-import each full adder. It is not possible to fix the sub-circuit in one place only.
Logisim: Logisim was written by Carl Burch at Hendrix College (now at Google). This appears to be the most popular simulator in use today in CS classrooms. It has more features than JLS; consequently, I find it slightly more difficult to learn and to use. This additional difficulty is well worth it if you make use of Logisim's additional features. Those who are building only a few, simple circuits are (in my opinion) better off using JLS.

To add support for a new simulator, one need only (1) write a new class implementing the ISimulator interface, and (2) make some minor changes to DLUnitCore so it recognizes the simulator's file extension.

Notes / Quirks

When testing JLS, calls to setMemory* don't take effect until the simulator is run. (I discovered this when writing end-to-end tests. I'm not sure why else it would be helpful to call readMemory* before calling run.)
Note: Logisim does not name its RAM elements. This means (1) The name parameter is ignored when testing logisim circuits, and (2) DLUnit does not support the testing of Logisim circuits with more than one RAM element per subcircuit.

What questions do you have?

FAQ

What questions do you have?

Contact Info

You can e-mail me at last-name first-initial@gvsu.edu, or see my complete contact info here.

(C) 2017 Zachary Kurmas
Last updated: 20 October 2017