Testing and JUnit

• Suppose I handed you an Alexa and told you that I taught her to determine leap years.
  Your job is to verify that the skill works correctly.
  • What years would you ask her and why?
  • How do you decide which years to ask since you can’t ask them all?
• These are the skills you need to apply when testing your own software.
• The basic question is “How do I know when my software is correct?”
• You should ask these questions and lay out your test cases before you write any code. Why?
  • It’s foolish to start a project if you have no idea how you’ll know when it’s done. (How can you even estimate how long it will take?)
  • Knowing the test cases in advance reduces the number of times you’ll run into a bug later and have to awkwardly “bolt” it on the side of your existing code.
  • The test you write after you write code are often flawed: When you write tests while looking at completed code, you often incorporate the flaws in the code into your tests, thereby rendering the tests useless.
    • In other words, you might unconsciously write tests that will pass the code you wrote, rather than pass the code you should have written.
    • For example, if you are looking at code that says for(int x = 0; x < 100; x++), you will tend to automatically choose test cases of 0, 99 and 100, even though the code should have been x <= 100. If you had written the tests first, you will be more likely to choose the correct cases.
• This process of writing tests first is called “Test Driven Development” (TDD)
  • It’s a good theory, but hard to do in practice.
  • It requires discipline and a commitment to code quality.

Types of tests

• Two general types of tests: “White box” and “black box”
• “black box” tests treat the program as a “black box” — one you can’t see into
  • You write the tests based solely on the program’s specs.
  • In the case of isLeapYear, you would write the tests based solely on the rules for a leap year.
  • Key things to consider:
    • Normal / typical behavior
    • “corner cases” (places where the behavior changes)
    • Common error conditions
    • Inputs that should never happen but aren’t technically prevented (e.g., negative numbers in the case of isLeapYear)
• “White box” tests are written with the code in mind – you can see into the box.
  • Key things to look for
    • Values that begin and end a loop.
    • Values that trigger either part of an if statement.
    • 0 values
    • null values
    • make sure both parts of an if statement are taken
    • make sure all the parts of a compound condition (a && b || c) are tested.
• As mentioned above: Start with black box tests. White box tests are important; but, bugs in your code can lead to oversights.
• Both tests are important. Black box tests focus on the rules. But, when you write the tests, you are implicitly making assumptions about how the code is written. For example, no (reasonable) amount of black box testing is going to catch a bug like this: if (year == 1942) {return true} The white box test is what tells us that we should use 1942 as a test case.

JUnit

• In order to make testing practical, it needs to be automated.
  • You need to be able to run tests at a push of a button every time you modify the code, otherwise it takes too long, is too boring and doesn’t get done.
  • In theory, you should run all your tests after every change. Even if you “only” touched method x and there is “no way” method y could be affected. You will be surprised. Often. Trust me.
  • JUnit https://junit.org and TestNG https://testng.org/ are the two most popular
    • We’ll use JUnit.
• Basic idea
  • Write a test method that
    1. Calls the method you are testing

    2. Uses an assert statement to verify you see the expected result.

       @Test
       public void isLeapYear() {
         assertTrue(SimpleDateKurmas.isLeapYear(1956));
       }
       

  • Key points
    • method annotated with @Test (allows the magic of “reflection” to identify and run the tests wihtout you having to write code to explicitly call each test)
    • method is public void
    • method takes no parameters (applies to methods annotated with @Test)
    • assertTrue method is a static provided by JUnit. The framework keeps track of failures and reports them at the end.
    • There are many assert* methods. assertEquals is probably the most popular. https://junit.org/junit5/docs/5.0.1/api/org/junit/jupiter/api/Assertions.html
• Suppose you wanted to test the instance method version of isLeapYear. Write a test for such a method
• Now write a test method for the constructor.
  • The constructor doesn’t return a value, so what do you assert?

• Shortcut: Write one method that can run a test with different values

   @ParameterizedTest
   @DisplayName(".isLeapYear recognizes 'regular' years")
   @ValueSource(ints = { 1997, 1998, 1999 })
   public void regularYearStatic(int year) {
     assertFalse(SimpleDateKurmas.isLeapYear(year), year + " should be a regular year");
   }
  

• Have everybody open the project in an IDE and make sure the tests compile and run.

• The JUnit code (@Test, the assert* methods, etc., etc.) lives in .jar files.
  • Your IDE may provide them automatically.
  • If not, you can download them from http://junit.org and configure them locally.
  • For this project, I provide a local copy of junit-platform-console-standalone.jar
    • To compile from command line: java -cp lib/junit-platform-console-standalone.jar java_files
    • To run from command line:
      java -jar -cp lib/junit-platform-console-standalone.jar -cp . -c TestClassName

Debugging

• Demonstrate debugging on P1 starter code
  • Run the tests. Notice that constructorWorks fails.
  • Set breakpoint on first line of constructorWorks test.
  • Add “dummy” new Date163(1, 2, 2003) to avoid class loader.
  • Step over the first few lines of code.
    • Notice that new variables appear under “Local”
    • Notice that you can look inside object.
  • After stepping over d = new Date163, notice that the instance data are all 0!
  • Stop the test. Re-run. This time, step into.
    • Hey: There’s no code here!
  • Step into split.
  • Show how to step out of.
  • Step into constructor. Show the difference between locals and instance variables
  • Change params to lmonth, lday, lyear
    • Watch instance variables change.
  • Show “rename symbol”. Notice how it changes everywhere.
  • Notice test fails now.
  • Step through the code to see why
    • Instance variables don’t change.
    • Local variable is “shadowing” the instance variable.
    • Fix: Use “full name” of instance variable (i.e., prefix with this)
      • Show block edit.
  • Show that you can look back through the stack trace to variables in other frames.