A Comprehensive Guide on Fundamentals of Software Testing –
What is Software ?
Software is a collection of computer programs (written in a coded language) that help us perform a task.
Types of Software :
- System Software
For example: device drivers, operating systems, servers, utilities, etc. - Programming software
For example: compilers, debuggers, interpreters, etc. - Application Software
For example: web applications, mobile apps, desktop applications etc.
What is Software Testing ?
Software testing is a part of the software development life cycle process.
Software testing is an activity of detecting and identifying defects in software.
The purpose of testing is to release a quality product to the client.
Testing is a process used to determine whether computer software is accurate, comprehensive, and of high quality.
Definition as per ISTQB standard glossary :
This process encompasses all life-cycle activities, both static and dynamic, related to the planning, development, and evaluation of software products and related work products to determine that they meet specified requirements. to ensure they are fit for purpose and to detect defects.
Objectives of software testing ?
# Finding defects early in the SDLC
# Gaining confidence about the level of quality and providing information
# Preventing defects
What is Software Quality ?
Bug-free
Delivered on time
Within budget
Meets requirement and /or expectations
Maintainable
Project vs Product
Project –
If software application is developed for specific customer based on the requirements, then it is called project. Temporary, Unique, Requirement based, Closed after completion.
Product –
If software application is developed for multiple customers based on market requirements, then it is called Product, Ongoing, Similar items or Services, Needs based, Permanent and Continuous improvements, Closed when no demand.
Why do we need testing ?
Software is not defect free
Defects cause failures
Unreliable software can cause failures
Failures have associated costs like loss of business
Testing helps to find defects and learn about reliability of software
Why is testing necessary ?
Is it possible to test everything ?
Exhaustive testing is testing with all combinations of inputs and pre-conditions
“ Though not impossible, certainly impractical “
Testing improves quality
Finding defects and measuring quality in terms of defects
Building confidence
Preventing defects
Reducing risk
Error, Bug/Defect and Failure
Causes of software defects –
A human being can make an error(mistake), which produces a defect ( bug ) in the code in software or a system or in a document. If defect in the code is executed, the system will fail to do what it should do ( or do something it shouldn’t), causing a failure.
Errors or mistakes
The defects in the human thought process, made while trying to understand given information to solve problem or to use methods and tools.
Bug or Defects or Fault
The concrete manifestations of errors within the software. One error may cause several defects and various errors may cause identical defects.
Failure
The departures of the operational software system behavior from the user requirements.
Cost of Bug
If a bug is found in the requirements-gathering phase, the cost could be $10.
If a bug is found in the coding phase, the cost could be $50.
If a bug is found in the integration testing phase, the cost could be $100.
If a bug is found in the system or acceptance testing phase, the cost could be $1000.
If a bug is found in the production, the cost could be $10000.
How much testing is enough ?
Below Factors are deciding how much to test :
Level of Risk –
Technical Risk
Business product risk
Project Risk
Project Constraints –
Time
Budget
Testing –
Decide what to test first
Decide what to test most
Decide what not to test
Why the software has bug ?
Miscommunication or no communication
Software complexity
Programming errors
Changing requirements
Lack of skilled testers
General testing principles
Principle 1 – Testing shows presence of defects
Testing can show that defects are present, but cannot prove that there are no defects
Testing reduces the probability of undiscovered defects remaining in the software but, even if no defects are found, it is not a proof of correctness.
Principle 2 – Exhaustive testing is impossible
Testing everything ( all combinations of inputs and preconditions ) is not feasible except for trivial cases. Instead of exhaustive testing, we use risk and priorities to focus testing efforts
Principle 3 – Early testing
Testing activities should start as early as possible in the software or system development life cycle, and should be focused on defined objectives.
Principle 4 – Defect clustering
A small number of modules contain most of the defects discovered during pre-release testing, or show the most operational failures.
Principle 5 – Pesticide paradox
If the same tests are repeated over and over again, eventually the same set of test causes will no longer find any new bugs.
To overcome this “pesticide paradox” the test cases need to be regularly reviewed and revised and new and different test need to be written to exercise different parts of the software or system to potentially find more defects.
Principle 6 – Testing is context dependent
Testing is done differently in different contexts.
For example, safety-critical software is tested differently from and e-commerce site
Principle 7 – Absence-of-errors fallacy
Finding and fixing defects does not help if the system built is unusable and does not fulfill the user’s needs and expectations.
Fundamental test process
Test planning and Control
Test analysis and design
Test implementation and execution
Evaluating exit criteria and reporting
Test closure activities
Test planning and Control
Defining testing objective and scope
Identifying risk and constraints
Defining test policy and/or the test strategy
Deciding test approach ( techniques, test items, coverage, identifying and interfacing)
Defining entry and exit criteria for testing
Identifying teams and skills involved in testing
Identifying test resources ( e.g. people, test environment, hardware, software etc.)
Scheduling testing activities/tasks
Test control
Measuring and analyzing results
Monitoring and documenting progress
Test coverage and exit criteria
Initiation of corrective actions
Making decisions
Test analysis and design
Reviewing the test basis
Identifying test conditions or test requirements
Designing test conditions
Evaluating testability
Designing the test environment set-up
Test implementation and execution
Developing and prioritizing test cases
Creating test suites from the test cases
Verifying the test environment
Executing test cases
Logging the outcome of test execution
Comparing actual results with expected results
Reporting discrepancies as incidents and analyzing them
Repeating test activities as result of action taken ( confirmation/retesting) and regression
Evaluating exit / completion criteria and reporting
Check the test logs against the exit/completion criteria defined in the test plan
Assess if more tests are needed or if the exit criteria specified should be changed
Writing a test summary report for stakeholders
Test closure activities
Checking which planned deliverables have been delivered
Finalizing and archiving testware
Handover of testware
Analyzing lesion learned
Psychology of testing
Why test ?
Build confidence in software under test
To find defects
Prove that the software confirms to user requirements/functional specifications
Reduce failure costs
Assess quality of software
Can testing prove software is correct ?
Not possible to prove system has no defects
Only possible to prove system has defects
The purpose of testing is to build confidence that the system is working
But purpose of testing is also to find defects
Finding defects destroys confidence
So, purpose of testing is to destroy confidence ?
Paradox of testing
The best way to build confidence is to try and destroy it
Should programmers test their own code ?
Contribute to specification of test cases due to their technical understanding of software.
Execute tests on their own code since it is cheaper and defect fixing is faster with less effort.
Levels of independence
“Independent testing is more effective”
Test designed by the person(s) who wrote the software under test(low level of independence)
Tests designed by another person(s) (e.g. From the development team)
Tests designed by a person(s) from a different organizational group (e.g An independent test team)
Tests designed by a person(s) from a different organization or company (i.e. Outsourcing or certification by an external body)
Communication
Testing perceived as a destructive activity
Testers generally bring “bad/unwanted news” or “criticism”
Need to communicate defect information to developers carefully
Good interpersonal skills are needed to communicate factual information about defects progress and risks in a constructive way
Testers should maintain a good relationship with developers