How to advertise
on Softwareqatest.com

Software QA and Testing Frequently-Asked-Questions, Part 1


What is 'Software Quality Assurance'?
Software QA involves the entire software development PROCESS - monitoring and improving the process, making sure that any agreed-upon processes, standards and procedures are followed, and ensuring that problems are found and dealt with. It is oriented to 'prevention'. (See the Bookstore section's 'Software QA' category for a list of useful books on Software Quality Assurance.)

Return to top of this page's FAQ list

What is 'Software Testing'?
Testing involves operation of a system or application under controlled conditions and evaluating the results (eg, 'if the user is in interface A of the application while using hardware B, and does C, then D should happen'). The controlled conditions should include both normal and abnormal conditions. Testing should intentionally attempt to make things go wrong to determine if things happen when they shouldn't or things don't happen when they should. It is oriented to 'detection'. (See the Bookstore section's 'Software Testing' category for a list of useful books on Software Testing.)

Return to top of this page's FAQ list

What are some recent major computer system failures caused by software bugs?

For more lists of software bugs see 'Collection of Software Bugs', a large collection of bugs and links to other bug lists maintained by Prof. Thomas Huckle at the Institut f?r Informatik in Germany, and a 'List of software bugs' in various categories maintained on Wikipedia.

Return to top of this page's FAQ list

Does every software project need testers?
While all projects will benefit from testing, some projects may not require independent test staff to succeed.

Which projects may not need independent test staff? The answer depends on the size and context of the project, the risks, the development methodology, the skill and experience of the developers, and other factors. For instance, if the project is a short-term, small, low risk project, with highly experienced programmers utilizing thorough unit testing or test-first development, then test engineers may not be required for the project to succeed.

In some cases an IT organization may be too small or new to have a testing staff even if the situation calls for it. In these circumstances it may be appropriate to instead use contractors or outsourcing, or adjust the project management and development approach (by switching to more senior developers and agile test-first development, for example). Inexperienced managers sometimes gamble on the success of a project by skipping thorough testing or having programmers do post-development functional testing of their own work, a decidedly high risk gamble.

For non-trivial-size projects or projects with non-trivial risks, a testing staff is usually necessary. As in any business, the use of personnel with specialized skills enhances an organization's ability to be successful in large, complex, or difficult tasks. It allows for both a) deeper and stronger skills and b) the contribution of differing perspectives. For example, programmers typically have the perspective of 'what are the technical issues in making this functionality work?'. A test engineer typically has the perspective of 'what might go wrong with this functionality, and how can we ensure it meets expectations?'. A technical person who can be highly effective in approaching tasks from both of those perspectives is rare, which is why, sooner or later, organizations bring in test specialists.

Return to top of this page's FAQ list

Why does software have bugs?

Return to top of this page's FAQ list

How can new Software QA processes be introduced in an existing organization?

Also see 'How can QA processes be implemented without reducing productivity?' in the LFAQ section.

(See the Softwareqatest.com Bookstore section's 'Software QA', 'Software Engineering', and 'Project Management' categories for useful books with more information.)

Return to top of this page's FAQ list

What is verification? validation?
Verification typically involves reviews and meetings to evaluate documents, plans, code, requirements, and specifications. This can be done with checklists, issues lists, walkthroughs, and inspection meetings. Validation typically involves actual testing and takes place after verifications are completed. The term 'IV & V' refers to Independent Verification and Validation.

Return to top of this page's FAQ list

What is a 'walkthrough'?
A 'walkthrough' is an informal meeting for evaluation or informational purposes. Little or no preparation is usually required.

Return to top of this page's FAQ list

What's an 'inspection'?
An inspection is more formalized than a 'walkthrough', typically with 3-8 people including a moderator, reader, and a recorder to take notes. The subject of the inspection is typically a document such as a requirements spec or a test plan, and the purpose is to find problems and see what's missing, not to fix anything. Attendees should prepare for this type of meeting by reading thru the document; most problems will be found during this preparation. The result of the inspection meeting should be a written report. Thorough preparation for inspections is difficult, painstaking work, but is one of the most cost effective methods of ensuring quality. Employees who are most skilled at inspections are like the 'eldest brother' in the parable in 'Why is it often hard for organizations to get serious about quality assurance?'. Their skill may have low visibility but they are extremely valuable to any software development organization, since bug prevention is far more cost-effective than bug detection.

Return to top of this page's FAQ list

What kinds of testing should be considered?

(See the Bookstore section's 'Software Testing' category for useful books on Software Testing.)

Return to top of this page's FAQ list

What are 5 common problems in the software development process?

In agile projects, problems often occur when the project diverges from agile principles (such as forgetting that 'Business people and developers must work together daily throughout the project.' - see the Manifesto for Agile Software Development.)

(See the Softwareqatest.com Bookstore section's 'Software QA', 'Software Engineering', and 'Project Management' categories for useful books with more information.)

Return to top of this page's FAQ list

What are 5 common solutions to software development problems?

(See the Softwareqatest.com Bookstore section's 'Software QA', 'Software Engineering', and 'Project Management' categories for useful books with more information.)

Return to top of this page's FAQ list

What is software 'quality'?
Quality software is reasonably bug-free, delivered on time and within budget, meets requirements and/or expectations, and is maintainable. However, quality is obviously a subjective term. It will depend on who the 'customer' is and their overall influence in the scheme of things. A wide-angle view of the 'customers' of a software development project might include end-users, customer acceptance testers, customer contract officers, customer management, the development organization's management/accountants/testers/salespeople, future software maintenance engineers, stockholders, magazine columnists, etc. Each type of 'customer' will have their own slant on 'quality' - the accounting department might define quality in terms of profits while an end-user might define quality as user-friendly and bug-free. (See the Softwareqatest.com Bookstore section's 'Software QA' category for useful books with more information.)

Return to top of this page's FAQ list

What is 'good code'?
'Good code' is code that works, is reasonably bug free, secure, and is readable and maintainable. Some organizations have coding 'standards' that all developers are supposed to adhere to, but everyone has different ideas about what's best, or what is too many or too few rules. There are also various theories and metrics, such as McCabe Complexity metrics. It should be kept in mind that excessive use of standards and rules can stifle productivity and creativity. 'Peer reviews', 'buddy checks' pair programming, code analysis tools, etc. can be used to check for problems and enforce standards.
For example, in C/C++ coding, here are some typical ideas to consider in setting rules/standards; these may or may not apply to a particular situation:

Return to top of this page's FAQ list

What is 'good design'?
'Design' could refer to many things, but often refers to 'functional design' or 'internal design'. Good internal design is indicated by software code whose overall structure is clear, understandable, easily modifiable, and maintainable; is robust with sufficient error-handling and status logging capability; and works as expected when implemented. Good functional design is indicated by an application whose functionality can be traced back to customer and end-user requirements or user stories. (See further discussion of functional and internal design in 'What's the big deal about requirements?' in FAQ #2.) For programs that have a user interface, it's often a good idea to assume that the end user will have little computer knowledge and may not read a user manual or even the on-line help; some common rules-of-thumb include:

Return to top of this page's FAQ list

What is SEI? CMM? CMMI? ISO? IEEE? ANSI? Will it help?

       Level 1 - 'Initial': characterized by chaos, periodic panics, and heroic efforts 
                 required by individuals to successfully complete projects.  Few if any 
                 processes in place; successes may not be repeatable.
       Level 2 - 'Managed': projects carried out in accordance with policies and employ 
                 skilled personnel with sufficient resources.   Project tracking and reporting 
                 is in place. Schedules and budgets are set and revised as needed.  Work 
                 products are appropriately controlled. 
       Level 3 - 'Defined': standard development and maintenance processes are established, 
                 integrated consistently throughout an organization, 
       Level 4 - 'Quantitatively Managed': metrics are used to track process performance.  
                 Project performance is controlled and predictable.
       Level 5 - 'Optimizing': the focus is on continuous process improvement. The impact of 
                 new processes and technologies can be predicted and effectively implemented 
                 when required.  Quality and process objectives are established and regularly 
                 revised to reflect changing objectives and organizational performance, and 
                 used as criteria in managing process improvement.


      Perspective on CMMI ratings: during 2002-2005, of 782 organizations assessed, 4% were 
      rated at Level 1, 34% at 2, 30% at 3, 4% at 4, and 19% at Level 5.  37% were goververnment 
      contractors or agencies, and 68% had more than 100 employees; 88% were software-related 
      organizations. (The CMMI applies to a wide variety of organizations, not just software
      organizations.)  The majority of organizations were NOT U.S.-based. 

      Perspective on the old CMM ratings:  During 1997-2001, 1018 organizations were assessed.  
      Of those, 27% were rated at Level 1, 39% at 2, 23% at 3, 6% at 4, and  5% at 5.  (For 
      ratings during the period 1992-96, 62% were at Level 1, 23% at 2, 13% at 3, 2% at 4, 
      and 0.4% at 5.)  The median size of organizations was 100 software engineering/maintenance 
      personnel; 32% of organizations were U.S. federal contractors or agencies.  For those 
      rated at Level 1, the most problematical key process area was in Software Quality Assurance.

Return to top of this page's FAQ list

What is the 'software life cycle'?
The life cycle begins when an application is first conceived and ends when it is no longer in use. It includes aspects such as initial concept, requirements analysis, functional design, internal design, documentation planning, test planning, coding, document preparation, integration, testing, maintenance, updates, retesting, phase-out, and other aspects. (See the Softwareqatest.com Bookstore section's 'Software QA', 'Software Engineering', and 'Project Management' categories for useful books with more information.)

Return to top of this page's FAQ list