blackbox_testing.md

Blackbox Testing

Due: June 18, 2020 at 11:00am (CDT, UTC-5)

Overview

In this assignment you will test a simple calculator program based on its functional specification. The functional specification follows the assignment instructions.

You will:

• Demonstrate equivalence partitioning, aka category partitioning
• Develop blackbox unit test cases for those partitions
• Create JUnit tests for those test cases

Instructions

This assignment is to be completed individually; no collaboration is permitted.

Setup

1. Navigate to the starter code on GitLab and click on the "Fork" button to copy it to your account.

2. Set the copied project to Private and add the professor and TAs with "Maintainer" access.

   • On your repository's side-menu, click on Settings->Members. On the resulting page, find "Visibility, project features, permissions" and click on its Expand button. Using the dropdown, set the project visibility to Private, and click on the Save changes button.

   • Add the professor and TAs in the same manner you did for your homework repository.

3. Verify that GitLab shows a lock icon on the fork. Note that, per the course’s collaboration policy, you are responsible for safeguarding your own work, and it is an academic integrity violation to have your work posted publicly.

4. Clone your copied project: git clone *URL* where *URL* is your calculator repository's "Clone with SSH" URL. (The URL probably is git@git.unl.edu:your_username/calculator.git)

   • Do NOT place your calculator repository inside your csce361-homework repository!

5. Import your cloned repository into your IDE as a Maven project.

   • Instructions for importing Maven projects into some IDEs are available in the "Importing Maven Projects" page on Canvas.

     • For other IDEs, after you find instructions to import Maven projects, please share them on Piazza for the benefit of others

   • If you are using an editor that does not integrate Maven, you can still use Maven from the command line

     • Check wither Maven is already installed:

       mvn --version

       • If Maven is not installed, you should be able to install it if you have a package manger (such as apt or brew) installed. Otherwise, download and install it manually.
     • To build the project:

       mvn compile

     • To run the JUnit tests:

       mvn test

     • To run the program:

       mvn exec:java -Dexec.mainClass=edu.unl.cse.csce361.calculator.Calculator

What is a Postfix Calculator?

If you already know what a postfix calculator is, you can safely move on to the Assignment section. This description is to help students understand the behavior of Calculator.

The calculators you are probably accustomed to using use infix (or algebraic) notation, in which a mathematical operator is located between its operands. In the early 20th Century, Polish logician Jan Łukasiewicz invented prefix notation (which came to be known as Polish Notation) in which a mathematical operator is located in front of both of its operands.

Postfix notation (also known as Reverse Polish Notation, or RPN) places the mathematical operator after both of its operands. Postfix notation is simpler to parse and cannot produce an ambiguous expression. Consider the infamously ambiguous viral expression 8÷2(2×2). In RPN, this would be expressed either as 8 2 2 × 2 × ÷, equalling 1, or 8 2 ÷ 2 2 × ×, equalling 16. The reason RPN expressions are so easy to parse is because each token is pushed onto a stack. If the token is an operator then the previous two tokens at the top are popped off the stack to be used as operands. The result of the computation is then pushed onto the stack.

The first handheld electronic calculator was an RPN calculator, and engineers continued to use RPN calculators for decades. The computation is faster than algebraic calculators (though on human scale, the difference is unnoticeable). Data entry, requiring fewer keystrokes, is faster. And studies show that engineers make fewer errors when using postfix notation than infix notation.

The postfix Calculator program differs from RPN calculators in one regard: real postfix calculators do not have an equals sign. The Calculator uses the equals sign to instruct the program to display the value at the top of the stack. For example:

8 2 2 * 2 * / =
	1.0
8 2 / 2 2 * * =
	16.0
+ =
	17.0

Assignment

1. Run Calculator (not CalculatorTest) a few times to get a feel for its behavior. The functional specification for the full program follows the assignment instructions.

2. Complete your test plan for the Calculator.calculate() method.

   • You do not need to look at Calculator's source code to develop your tests. That's the point of blackbox testing. If you do look at Calculator's source code, be warned that it has some bad design in it; this is intentional for discussion purposes later. Bear in mind that these bad design aspects would make it more error-prone and harder to maintain.

   1. Complete developing the testing requirements. The testing requirements should be traceable to Calculator.calculate()'s functional requirements (listed after the assignment instructions).

      • Do not use the "Postfix Calculator Functional Specification." This system-level specification is included for context. Your testing requirements need to trace to the "Calculator.calculate() Functional Specification."

   2. Develop the rest of the test cases in your test suite. The test cases should be traceable to the testing requirements.

   3. There is a file called TESTING.md in the top-level directory of your calculator repository. Record your test plan in TESTING.md.

      • The .md suffix indicates that this is a "markdown" file. You can use markdown formatting annotations (see the Markdown Cheatsheet on Canvas).

      • Clearly identify the equivalence partitions.

      • Specify your test cases. Each test case should have a short description (such as "Pop an Empty Stack") the number(s) of the functional requirement(s) the test case addresses, the inputs, and the expected output (the oracle).

      • Some of your test cases may require multiple calls to Calculator.calculate().

3. Edit CalculatorTest.java. Create a JUnit test for each test case in TESTING.md. The name of each JUnit test should be in camelCase, beginning with test and then the name of the test case (omitting definite & indefinite articles), such as testPopEmptyStack().

   • Remember that you should use the @Test annotation and that the method must be public void and with no parameters.

4. Run CalculatorTest.

   1. For any test cases that fail, look at the details and compare the expected output to the actual output.

   2. Do not attempt to fix the faults revealed by your tests.

   • There are bugs that are intentionally included in Calculator. If a test fails, do the due diligence to make sure that your test isn't faulty, but otherwise it's okay that you found a bug. On a real project, finding a bug is a good thing because it means that you can fix it and that bug won't get shipped. (Again, do not attempt to fix the bugs that you find in this assignment.)

5. Produce a test report. Normally you wouldn't commit an auto-generated test report to the repository; however, for grading purposes we will require that you do so.

   • If your IDE will generate an html test report, then you may use that for your test report. Give it a meaningful name and store it in the top- level directory of your calculator repository.

     • Your IDE might try to place the html test report in the target/site directory. If it does, or if it places the test report in any location other than the top-level directory, move the test report from wherever the IDE stores it to the top-level directory.

   • Alternatively, you can take a screenshot of your IDE's JUnit "stoplight" results and save it in the top-level directory of your calculator repository as a jpg, gif, png, or pdf file. Give it a meaningful name.

     • If you're running mvn test from the command line, copy the output and save it in the top-level directory of your calculator repository as a txt file. Give it a meaningful name.

6. Upload all of the deliverables to your copied project using the standard add/commit/push pattern:

   git add .
   git commit
   git push

   (or you can use your IDE to add/commit/push)

   Go to your copy of the project on git.unl.edu to confirm that the required deliverables are in place.

Deliverables