System Testing

Testing is an integral part of the development of any component of a software product and system testing is the final phase of this product’s development life cycle. Software testing activities are shared between software developers and the system testing team. Software developers perform unit testing, integration, and software performance testing. System testing is performed in a separate group, which is called a system testing group.

What is system testing?

The system testing group does not participate in the development of software code and user documentation, but is only engaged in system testing of the finished software version. The main purpose of system testing is to detect defects in the finished software version and identify discrepancies between customer requirements for software and real software behavior during its operation (confirmation of compliance of real software behavior with customer requirements). To do this, a system test complex (test suite) is developed in the system requirements group.

For each software development project, a responsible system testing engineer is assigned, who begins system testing work immediately after the approval of the “Work Regulations” for this software project. He gets access to any information on the project of the software being tested, including the source codes of the software in the future. This leads to more efficient and targeted system testing. According to the results of familiarization with the customer’s requirements for software, the system tester checks each requirement for the possibility of its testability.

The system begins its work on creating a test suite for software with the preparation of a system test plan immediately after the approval of the customer’s requirements for it. Here is a list of the main materials created during the development of the test suite and the execution of the system testing cycles, which is accepted as mandatory:

  • A system testing plan, which is based on approved customer requirements for software and a software development project plan;
  • System testing jobs – documents describing the procedure for performing tests, ensuring the required sequence of system tests and the criteria for their successful completion (in the system testing procedures, it is also necessary to provide detailed descriptions of the manual tests run);
  • Requirements coverage matrix, in which one or several tests from the test suite are assigned to each customer’s requirement;
  • Automated test suite – a set of text (in the accepted programming language), boot files and other files that support the execution of automated software system testing operations (there may be manual tests in the test suite);
  • A report on the results of system testing performed on the basis of the use of an automated test suite in accordance with the system test procedures (including manual tests).

Test suite development for system testing of software

The system is developing an automated test suite for software in accordance with the system test plan for this software. The test suite should cover all customer requirements for software. For this, the responsible system tester should work together with the developers to remove all non-tested requirements and ensure that 100% of all customer requirements for software can be covered (example of non-test requirements: the interface should be friendly). Most of the tests included in the test suite should be performed in automatic mode, with the exception of tests to verify those requirements that cannot be verified in automatic mode or impractical.

The system test plan and the software development project plan must be consistent. This means that the major milestones of the development of the test suite are consistent in terms with milestones of software development. In this case, it is advisable that the completion of the development of the test suite be planned at the same time as the completion of debugging the current software version.

System testing services are the activities of system testers in checking software between the time when developers provide a version of software for system testing and the moment they submit a report on system testing to developers. Testing cycles must be planned in accordance with the milestones of software development specified in the software project plan. The last cycle of system testing is often referred to as output testing. The input data for each cycle of system testing in accordance with the order are: a set of software installation files (diskettes); reports on unit and integration testing, on performance testing; project manager’s report on software version build; test log.

The list of system test cycle output includes: a system test report, including a description of the test suite, the test suite itself, and test results; a test log containing records of actions performed in chronological order when performing test cycles; report on system testing incidents. An incident of system testing is understood to be a deviation of the progress of a system test from an expected one fixed in the database of defects by a system tester.