Chapter 8 – Quality Assurance

  • The SQA role is to monitor the methods and standards the software experts use to verify that they have properly applied their expertise
  • It includes knowledge of statistical methods, quality control principles, the software process and an ability to deal effectively with people in contentious situations
  • Quality Management
    • Before establishing an SQA organization, it is essential to first decide how important software quality is to the organization
    • If management’s commitment to software quality does not come down to day-to-day actions, no consistent improvement is likely
    • If management views the mere act of setting up an SQA organization as satisfying its quality responsibilities, it will have wasted a lot of money, established another bureaucratic bottleneck , and probably damaged product quality
    • SQA is a management tool that must be properly used to be effective
    • SQA ensures
      • An appropriate development methodology is in place
      • The projects use standards and procedures in their work
      • Independent reviews and audits are conducted
      • Documentation is produced to support maintenance and enhancement
      • The documentation is produced during and not after development
      • Mechanisms are in place and used to control changes
      • Testing emphasizes all the high-risk product areas
      • Each software task is satisfactorily completed before the succeeding one is begun
      • Deviations from standards and procedures are exposed as soon as possible
      • The project is auditable by external professionals
      • The quality control work is itself performed against established standards
      • The SQA plan and the software development plan are compatible
    • Software has an enormous impact on everything we do, and this impact will only increase in the future
    • It is hard for anyone to be objective about auditors. We generally do our own jobs pretty carefully and resent any contrary implication
    • When quality is critical, some independent checks are necessary, not because people are untrustworthy, but because they are human
    • The issues with software are not whether checks are needed, but who does them and how
    • Goals of SQA
      • To improve software quality by appropriately monitoring both the software and the development process that produces it
      • To ensure full compliance with the established standards and procedures for the software and the software process
      • To ensure that any inadequacies in the product, process, or the standards are brought to management’s attention so these inadequacies can be fixed
    • The SQA organization is not responsible for producing quality products or for making quality plans; these are development jobs
    • SQA is responsible for auditing the quality actions of the line organization and for alerting management to any deviations
    • To be effective, SQA needs to work closely with development
    • If the development people view SQA as the enemy, it will be hard for them to be effective
  • The Role of SQA
    • The people responsible for the software projects are the only ones who can be responsible for quality. The role of SQA is to monitor the way these groups perform their responsibilities
    • Pitfalls in monitoring
      • It is a mistake to assume that the SQA people themselves can to anything about quality
      • The existence of an SQA function does not ensure that the standards and procedures are followed
      • Unless management periodically demonstrates its support for SQA by following their recommendations, SQA will be ineffective
      • Unless line management requires that SQA try to resolve their issues with project management before escalation, SQA and development will not work together effectively
    • Management must insist that the quality problems be fixes before the product is shipped; otherwise SQA becomes an expensive bureaucratic exercise
    • SQA can be effective when the report through an independent management change, when they are properly staffed with competent professionals, and when they see their role as supporting the development and maintenance personnel in improving product quality
    • If SQA fulfills its responsibilities and it senior management refuses to allow line management to commit or to ship products until the SQA issues have been addressed, then SQA can help management improve product quality
    • SQA functions
      • Quality Assurance practices
      • Software project planning evaluation
      • Requirements evaluation
      • Evaluation of the design process
      • Evaluation of coding practices
      • Evaluating the software integration and test process
      • In-process evaluation of the management and project control process
      • Tailoring of Quality Assurance procedures
    • The one simple rule on SQA reporting is that it not be under the software development manager
    • SQA should report to a high-enough management level to have some change of influencing priorities and obtaining the resources and time to fix the key problems
    • Reporting level guidelines
      • SQA should not report to the project manager
      • SQA should report somewhere within the local lab or plant organization
      • There should typically be no more than one management position between SQA and the senior location manager
      • SQA should always have ‘dotted-line’ relationship to a senior corporate quality executive
      • Whenever possible, SQA should report to someone who has a vested interest in software quality
    • It helps to have SQA report to someone who is actually affected by poor software quality
  • Launching the SQA Program
    • The eight steps for launching an SQA program
      1. Initiate the SQA program
      2. Identify SQA issues
      3. Write the SQA plan
        • Possible sampling methods
          • Ensure that all required design and code inspections are performed and participate in a selected set
          • Review all inspection reports and analyze those outside of established control limits
          • Ensure that all required tests are performed and test reported produced
          • Examine a selected set of test reports for accuracy and completeness
          • Review all module test results and further study the data on those modules with test histories that are outside of established control limits
      4. Establish standards
      5. Establish the SQA function
      6. Conduct training and promote the SQA program
      7. Implement the SQA plan
      8. Evaluation the SQA program
        • It is important to establish means to evaluate SQA effectiveness
        • One way is to gather data on post-shipment product quality and relate it to the prior SQA evaluations
  • The SQA Plan
    • Each development and maintenance project should have a SQAP (Software Quality Assessment Plan) that specified its goals, the SQA tasks to be performed, the standards against which the development work is to be measured, and the procedures and organizational structure
  • SQA Considerations
    • Common reasons why SQA organizations fail to make an impact
      • SQA organization are rarely staffed with sufficiently experienced or knowledgeable people
      • The SQA management team is often not capable of negotiating with development
      • Senior management often backs development over SQA on a large percentage of issues
      • Many SQA organization operate without suitably documented and approved development standards and procedures
      • Software development groups rarely produce verifiable quality plans
    • Debates around which critical bugs to fix and which to defer miss the point. If the product quality is poor, fixing those bugs will not address the larger issue. SQA should focus on whether proper process was followed
    • The most effective SQA arguments are based on established quality standards and plans that spell out the required steps for people development work
    • When these standards are not followed, SQA should promptly non-concur rather than wait until the last minute
  • SQA People
    • Getting good people into SQA is a serious issue
    • For SQA to be effective, they must have good people and full management backing
  • Independent Verification and Validation
    • Development management uses SQA to monitor its own organization and to ensure that established standards and procedures are followed. IV&V does essentially the same thing for the customer
    • customer roles
      • Highlight any shortcomings in the SQA organization
      • Ensure customer needs are adequately reflected in the work
      • Ensure that the right skills and attitudes are in place
    • Focus on the way the contractor’s work is done rather than try to duplicate it