Decide up front what level of testing the project requires. Engineers under deadline pressure frequently give less attention to testing, devoting more time to other development. With a certain level of testing, you are guaranteed to save time.
Developers must understand clearly the degree to which you expect testing. Also, you must standardize testing methodologies and track the results of tests. As you develop the requirements and design specifications, also develop a test plan to help you verify that the system and all its components work. Testing reflects the quality goals you want to achieve. For example, if performance is more critical than robustness, develop more tests for performance and fewer that attempt incorrect input or low-memory situations.
Testing is not an afterthought. Consider testing as part of the initial design phases and test throughout development to find and fix problems as soon as possible.
There are a variety of testing methodologies. You can use the following testing methodologies to help increase the quality of VI projects.
Static code analysis refers to a technique or tool you use to compare the source code to pre-established criteria for style, organization, and quality. You do not need to compile or execute the application to perform static code analysis.
|
Note��You can use the LabVIEW VI Analyzer Toolkit to perform static code analysis. |
Dynamic code analysis refers to a tool you use to understand how the software works during execution, which involves testing the code while it runs. You can use black box and white box testing when you perform dynamic code analysis. Perform dynamic code analysis to detect memory leaks and areas for performance improvement, identify the source of an undesired behavior or error, and ensure an application performs the same on different targets. Use the Profile Performance and Memory window to perform dynamic code analysis.
|
Note��You also can use the Desktop Execution Trace Toolkit to perform dynamic code analysis. |
The method of black box testing is based on the expected functionality of software, without knowledge of how it works. It is called black box testing because you cannot see the internal workings. You can perform black box testing based largely on knowledge of the requirements and the interface of a module. For a subVI, you can perform black box tests on the interface of a subVI to evaluate results for various input values. If robustness is a quality goal, include erroneous input data to see if the subVI successfully deals with it. For example, for numeric inputs, see how the subVI deals with Inf (infinity), NaN (not a number), and other out-of-range values.
The method of white box testing is designed with knowledge of the internal workings of the software. Use white box testing to check that all the major paths of execution work. By examining a block diagram and looking at the conditions of Case structures and the values controlling loops, you can design tests that check those paths. White box testing on a large scale is impractical because testing all possible paths is difficult.
Although white box testing is difficult to fully implement for large programs, you can choose to test the most important or complex paths. You can combine white box testing with black box testing for more thorough software testing.
Use black box and white box testing to test any component of software, regardless of whether it is an individual VI or the complete application. Unit, integration, and system testing are phases of the project at which you can apply black box and white box tests.
You can use unit testing to concentrate on testing individual software components. For example, you can test an individual VI to see that it works correctly, deals with out-of-range data, has acceptable performance, and that all major execution paths on the block diagram are executed and performed correctly. Individual developers can perform unit tests as they work on the modules.
Some examples of common problems unit tests might account for include the following:
Define various sets of inputs that thoroughly test the VI, write a test VI that calls the VI with each combination of inputs, and checks the results. Use interactive data logging to create input sets, or test vectors, and retrieve them interactively to re-test the VI. You also can retrieve data automatically from a test VI that performs data logging programmatically.
To perform unit testing, you might need to create a stub of some components that you have not implemented yet or that you are developing still. For example, if you are developing a VI that communicates with an instrument and writes information to a file, another developer can work on a file I/O driver that writes the information in a specific format. To test the components early, you can create a stub of the file I/O driver by creating a VI with the same interface. This stub VI can write the data in a format that is easy for you to check. You can test the driver with the real file I/O driver later during the integration phase.
You also can use the LabVIEW VI Analyzer Toolkit to test VIs interactively or programmatically for style, efficiency, and other aspects of LabVIEW programming.
Regardless of how you test VIs, record exactly how, when, and what you tested and keep any test VIs you created. This test documentation is especially important if you create VIs for paying customers and for internal use. When you revise the VIs, run the existing tests to ensure you have not broken anything. Also update the tests for any new functionality you added.
You perform integration testing on a combination of units. Unit testing usually finds most bugs, but integration testing can reveal unanticipated problems. Modules might not work together as expected. They can interact in unexpected ways because of the way they manipulate shared data.
|
Note��You can perform integration testing only in the LabVIEW Full and Professional Development Systems. |
You also can perform integration testing in earlier stages before you put the whole system together. For example, if a developer creates a set of VIs that communicates with an instrument, he can develop unit tests to verify that each subVI correctly sends the appropriate commands. He also can develop integration tests that use several of the subVIs in conjunction with each other to verify that there are no unexpected interactions.
Do not perform integration testing as a comprehensive test in which you combine all the components and try to test the top-level program. This method can be expensive because it is difficult to determine the specific source of problems within a large set of VIs. Instead, consider testing incrementally with a top-down or bottom-up testing approach.
With a top-down approach, you gradually integrate major components, testing the system with the lower level components of the system implemented as stubs. Once you verify that the existing components work together within the existing framework, you can enable additional components.
With a bottom-up approach, you test low-level modules first and gradually work up toward the high-level modules. Begin by testing a small number of components combined into a simple system, such as the driver test. After you combine a set of modules and verify that they work together, add components and test them with the already-debugged subsystem.
The bottom-up approach consists of tests that gradually increase in scope, while the top-down approach consists of tests that are gradually refined as new components are added.
Regardless of the approach you take, you must perform regression testing at each step to verify that the previously tested features still work. Regression testing consists of repeating some or all previous tests. If you need to perform the same tests numerous times, consider developing representative subsets of tests to use for frequent regression tests. You can run these subsets of tests at each stage. You can run the more detailed tests to test an individual set of modules if problems arise or as part of a more detailed regression test that periodically occurs during development.
System testing happens after integration to determine if the product meets customer expectations and to make sure the software works as expected within the hardware system. You can test the system using a set of black box tests to verify that you have met the requirements. With system testing, you test the software to make sure it fits into the overall system as expected. Most LabVIEW applications perform some kind of I/O and also communicate with other applications. Make sure you test how the application communicates with other applications.
When testing the system, consider the following questions:
You can complete this testing with alpha and beta testing. Alpha and beta testing serve to catch test cases that developers did not consider or complete. During alpha testing, test a functionally complete product in-house to see if any problems arise. Beta testing occurs after alpha testing is complete. During beta testing, the customers in the field test the product.
Alpha and beta testing are the only testing mechanisms for some companies. Unfortunately, alpha and beta testing actually can be inexact and are not a substitute for other forms of testing that rigorously test each component to verify that the component meets stated objectives. When this type of testing is done late in the development process, it is difficult and costly to incorporate changes suggested as a result.
Some testing methodologies attempt to find problems by exploration, but many software engineers are proponents of formal verification of software. Formal methods of verification attempt to prove the correctness of software mathematically.
The principal idea is to analyze each function of a program to determine if the function does what you expect. You mathematically state the list of preconditions before the function and the postconditions that are present as a result of the function. You can perform this process either by starting at the beginning of the program and adding conditions as you work through each function or by starting at the end and working backward, developing a set of weakest preconditions for each function. Several third-party resources include more information about the verification process.
This type of testing becomes more complex as more possible paths of execution are added to a program through the use of loops and conditions. Many people believe that formal testing presents interesting ideas for looking at software that can help in small cases, but that verification testing is impractical for most programs.