Get Certified: Proven Methods to Pass the Software Certifications CSQA Exam

Level 1 – Validation

Validation is testing the software in an operational state. While there are several iterations of testing, they do not occur until the latter part of the system development life cycle. Thus, validation does not take advantage of the control theory stating that the control should be placed as close as possible to where the defect occurs. The most common types of validation are unit testing, integration testing, and system testing. Unit testing tests the smallest operating module of a system; integration testing tests the connecting logic between the modules/units; and the system testing validates that the system executes in the organization's operating environment.

Level 2 – Verification

Verification is the static testing of the system in a nonoperational status. It primarily tests the documentation produced by the project team. The documentation includes requirements, design,code, test, and operating documentation. Verification is performed through a variety of processes, including code analyzers, walkthroughs, reviews, and inspections. These verification processes also interact with the project team, end users, and other interested parties. Level 2 also incorporates acceptance testing, in which the end users define the criteria, which the system must meet in order to, be acceptable, and then test against those criteria.

Level 3 -- Defect Management

This level involves naming defects, counting defects, recording defects in a database, analyzing them, and then managing them throughout the entire life cycle of software. There are two types of defects, just as there are two types of quality. One defect is a variance from requirements/specifications, and the second defect is anything that dissatisfies the end user. For example, if the system was to add A plus B to get C, but in fact added A to X that would be a variance from specification defect. If the output was correct, but hard to use from an end user viewpoint, it would be an ease of use defect. Defects are generally only considered to be defects when they are uncovered in a task beyond the task in which the defect occurred. If the individual who makes the defect finds it during the same task, it is generally not considered to be a defect.

Level 4 -- Statistical Process Control

To be effective, statistical process control needs stabilized processes and defect recording. Some statistical process control techniques will be introduced at Level 3, but their use will be confined more to process improvement than for process management. Good statistical process control techniques require the type of measurement data that is only produced at Level 4. The statistical process control techniques will expand on root cause analysis, use sophisticated statistical tools and analysis, and create dashboards to be used in managing processes.

Level 5 - Preventive Management

This level is focused on preventing defects. It uses the lessons learned from the previous four levels of control/test to address the problems before they can become defects. During the previous four levels, the causes of defects are available and some addressed through process improvement. This level accelerates that trend and in addition provides the professional information staff with methods to modify the way work is performed so that the root causes of defects will not be built into software. This involves building risk models, developing defect expectation charts and/or defect profiles, and processes to customize processes so that the do component of processes can build software that is almost defect free.

In business and accounting literature, the environment is referred to in many different ways. In accounting literature it is sometimes called the “control environment,” other times the “management environment,” and sometimes just the environment in which work is performed.

For the purpose of this skill category, we will refer to it as the “quality environment.” What is important to understand is that the environment significantly impacts the employee’s attitude about complying with policies and procedures. For example, if IT management conveys that they do not believe that following the system development methodology is important, project personnel most likely will not follow that system development methodology. Likewise, if IT management conveys a lack of concern over security, employees will be lax in protecting their passwords and securing confidential information.

In order to participate in the various aspects of contracting for software development, it is necessary to establish an acquisition life cycle for contracted software. This life cycle contains the following three activities:

Selecting an Outside Organization

Selecting a contractor is similar to systems design. It is a time of studying alternative contractors, costs, schedules, detailed implementation design specifications, and the specification of all the deliverables, such as documentation. The selection of an outside organization may involve the following individuals in addition to the quality control reviewer:

Systems analysts

User personnel

Internal auditor

Purchasing agent

Legal counsel

Contract Negotiations

In some organizations, the purchasing agent conducts all the negotiations with the contractor; the other parties are involved only in the needs specification. In other organizations, there is no purchasing agent, so the data processing department deals directly with contractors for the acquisition of application systems.

Operations and Maintenance

The maintenance and operations of purchased applications may be subject to contractual constraints. For example, some contracts limit the frequency with which an application can be run without paying extra charges, and limit an organization’s ability to duplicate the application system in order to run it in a second location. In-house personnel can maintain some purchased applications, but others do not come with source code and thus are not maintainable by in-house personnel. There may also be problems in connecting a purchased application from one contractor with purchased or rented software from another contractor. It is important that software from multi contractors be evaluated as to its capability to work in the same operating environment.

Verification requires several types of reviews, including requirements reviews, design reviews, code walkthroughs, code inspections, and test reviews. The system user should be involved in these reviews to find defects before they are built into the system. In the case of purchased systems, user input is needed to assure that the supplier makes the appropriate tests to eliminate defects. The following table shows examples of verification. The list is not exhaustive, but it does show who performs the task and what the deliverables are. For purchased systems, the term “developers” applies to the supplier’s development staff.

Rahail HDD:Users:iMac:Desktop:Screen Shot 2015-01-11 at 6.18.47 PM.png

Validation is accomplished simply by executing a real-life function (if you wanted to check to see if your mechanic had fixed the starter on your car, you’d try to start the car). Examples of validation are shown in the following table. As in the table above, the list is not exhaustive.

Determining when to perform verification and validation relates to the development, acquisition, and maintenance of software. For software testing, this relationship is especially critical because:

The corrections will probably be made using the same process for developing the software. If the software was developed internally using a waterfall methodology, that methodology will probably be followed in making the corrections; on the other hand, if the software was purchased or contracted, the supplier will likely make the correction. You’ll need to prepare tests for either eventuality.

Testers can probably use the same test plans and test data prepared for testing the original software. If testers prepared effective test plans and created extensive test data, those plans and test data can probably be used in the testing effort, thereby reducing the time and cost of testing.

Rahail HDD:Users:iMac:Desktop:Screen Shot 2015-01-11 at 6.20.18 PM.png

1 – The major advantages are”

Relinquishment of control -The software is developed by individuals who are not employees of the organization,and thus it is difficult to oversee the development process. The contractingorganization cannot direct the employees of the other organization, nor have controlover the many day-to-day decisions that are made in developing software.

Loss of control over reallocation of resources -If work needs to be done to correct problems and/or speed up development, thecontractor cannot take workers off one project and assign them to another project.

2 – The major disadvantages are:

Cultural differences -There may be a significant difference in the culture and values between the contractorand the offshore organization.

Communication barriers -The language of the offshore organization may be different or difficult to comprehendwhich causes difficulty in communicating the needs and desires of the contractor.

The most important step of the acquisition life cycle is contractor selection. This step has the same importance in the acquisition life cycle as the systems design step has in the system development life cycle. It is determined during this step whether or not the purchased application meets the needs of the organization. During the systems design phase, the needs are translatedinto systems design specifications. During selection those needs are translated into contractual requirements.

Selecting the right organization to develop software is critical to successfully outsourcing development. The selection process must find a competent and compatible organization to develop all or part of the software development. What cannot be outsourced is the responsibility for software quality. The following concerns need to be addressed when selecting an outsourced organization.

Assure That Requirements and Contract Criteria are Testable

A common term used for contracting today is performance-based contracting. This means that the performance criteria of the contractor will be defined and once defined, can be monitored and measured.

It is important that all the requirements that will be given to the contractor are testable. That means as much as possible, an objective measurement can be made as to whether or not that requirement or criteria has or has not been met. For example, easy-to-use criteria might specify the type and number of help screens included within the software.

As a general rule, if a requirement or criteria is not testable the contractor has significant discretion on how that requirement or criteria is implemented. It also limits the contractee’s ability to have corrections made without additional cost for a requirement or criteria that does not meet the customer’s need. If it is a testable requirement, it is easy to demonstrate that the contractor has failed to meet the component of the contract.

Assure That the Contractor Has an Adequate Software Development Process

The contractor should provide a certificate indicating what SEI CMMI level they have achieved or equivalent; for example, ISO certified.

Assure That the Contractor Has an Effective Test Process

This responsibility is primarily for contracted developed software. It is important for the quality professional to develop an opinion on the contractor’s ability to deliver software that meets the requirements or criteria in the contract. It can also provide insight on the ability of the contractor to develop software at a reasonable cost. For example, if the test plan indicates that the defect removal efficiency at the requirements phase is 95%, then the quality professional knows that only 5% of the requirement defects will move to the design phase. On the other hand, if the contractor does not begin testing until after the unit is compiled, then there may be extensive rework and potential delays in getting the software on the scheduled date.

Define Acceptance Testing Criteria

The quality professional should not allow COTS or contracted software to be placed into operation without some acceptance testing. If the COTS software is acquired by the Internet orfrom a business college its reliability is questionable, and it may contain viruses. Many organizations do not permit their employees to acquire software from unapproved sources such as the Internet.

The quality professional may or may not be involved in the actual acceptance testing. However, the quality professional must determine that acceptance criteria are developed. It has been demonstrated extensively that the cost of not acceptance testing is normally much greater than the cost of acceptance testing. It only takes a few problems in acquired software to far exceed the cost of acceptance testing.

In the process of creating desirable standards for business performance, there are three fundamental mistakes a would-be IT leader can make:

Isolation

Leaders must maintain regular, if not frequent, contact with a significant percentage of the people they manage. Typical criticism centers on their preoccupation with technology, their inability to see the big picture, and their fondness for tasks rather than a genuine interest in the organization.

Inability to reward

Managers who lack the ability or do not take the time to reward are never able to build employee commitment. When performance is measured against possibility, extraordinary rewards are warranted.

Lack of business perspective

Business perspective is the ability to take advantage of opportunities, articulate goals, effectively deploy resources, take risks and accept responsibility for the outcome of actions. Business perspective also involves strict attention to cost.

The steps below describe a sample defect tracking process. Depending on the size of the project or project team, this process may be substantially more complex.

1. Execute test and log any discrepancies.

The tester executes the test and compares the actual results to the documented expected results. If a discrepancy exists, the discrepancy is logged as a “defect” with a status of “open.” Supplementary documentation, such as screen prints or program traces, is attached if available.

2. Determine if discrepancy is a defect.

The Test Manager or tester reviews the defect log with an appropriate member of the development team to determine if the discrepancy is truly a defect, and is repeatable. If it is not adefect, or repeatable, the log should be closed with an explanatory comment.

3. Assign defect to developer.

If a defect exists it is assigned to a developer for correction. This may be handled automatically by the tool, or may be determined as a result of the discussion in step 2.

4. Defect resolution process.

When the developer has acknowledged the defect is valid, the resolution process begins.

The maturation of the quality management system can be divided into three phases:

Initial Phase

The initiation point of the quality management system can be considered the formalization of quality control activities. An organization in this phase is results-driven, focusing on defining and controlling product quality. It normally takes at least two years in this phase of maturity for management and staff to recognize that quality cannot be tested into a product; it must happen through process maturity.

Intermediate Phase

In this phase an organization's objectives move from control to assurance. The emphasis is on defining, stabilizing, measuring, and improving work processes. While process improvement occurs before and after this phase, it is during this phase that resources are allocated to make process maturity happen. It takes between two and four years for significant process maturity. At this level products and services achieve consistency in product quality (consistency being the prerequisite to improved quality and productivity).

Final Phase

During this phase, objectives such as consulting, motivating, and benchmarking move the organization toward optimization. The MBNQA program defines such an organization as a “world-class” organization. World-class begins when work and management processes are well defined, are being continually improved, and are producing high-quality results that yield high customer satisfaction. These processes are integrated to obtain maximum customer satisfaction at minimum cost.

These six steps are commonly used to build a security baseline:

1. Establish baseline team.

2. Set baseline requirements and objectives.

3. Design baseline data collection methods.

4. Train baseline participants.

5. Collect baseline data.

6. Analyze and report computer security status.

Measure defect rate in defects per thousand function points or defects per thousand lines of code. As processes become better defined, controls become more effective, and as people become more motivated, the defect rate drops significantly.

Rahail HDD:Users:iMac:Desktop:Screen Shot 2015-01-12 at 3.24.25 PM.png

A post-implementation audit is conducted to determine any, or all, of the following:

The system objectives were met

The system specifications were implemented as specified

The developmental standards were followed

The IT quality objectives (e.g., maintainability) were achieved

Post-implementation audits are a quality assurance activity. The post-implementation audit is used to assess the ability of the IT organization to perform effective and efficient work. The results of the audit will be used to both improve the software system and to improve the process that builds software systems.

Industry accepted definitions of quality are “conformance to requirements” (from Philip Crosby) and “fit for use” (from Dr. Joseph Juran and Dr. W. Edwards Deming). These two definitions are not inconsistent.

Meeting requirements is a producer’s view of quality. This is the view of the organization responsible for the project and processes, and the products and services acquired, developed, and maintained by those processes. Meeting requirements means that the person building the product does so in accordance with the requirements. Requirements can be very complete or they can be simple, but they must be defined in a measurable format, so it can be determined whether they have been met. The producer’s view of quality has these four characteristics:

Doing the right thing

Doing it the right way

Doing it right the first time

Doing it on time without exceeding cost

Being fit for use is the customer’s definition. The customer is the end user of the products or services. Fit for use means that the product or service meets the customer’s needs regardless ofthe product requirements. Of the two definitions of quality, fit for use, is the more important. The customer’s view of quality has these characteristics:

Receiving the right product for their use

Being satisfied that their needs have been met

Meeting their expectations

Being treated with integrity, courtesy and respect

The quality professional should assure that reports will be issued regularly by the contractor on the software being developed.

Level 1 -- Product Focus

In this process category, management's focus is on products. The product focus emphasizes management by objective; specifically meeting schedules, managing people and jobs, and using test and control to determine whether those objectives have been met. At this level, status reports, budgets, checkpoints, and meetings are an important component of how management executes their management function.

Level 2 -- Process Focus

Management's emphasis is to bring discipline to their organization through definition and compliance to processes. The processes are usually an accumulation of the best practices currently in use within the information activity. Management creates an environment for discipline by establishing a quality infrastructure, usually a quality council or committee with several lower-level committees such as a standards or process management committee. At this level, management begins quality planning, conducts end user surveys, encourages employees to make suggestions and recommendations for improvement, and changes the methods by which projects are managed to a more scientific method using process management tools with emphasis on managing the process.

Level 3 -- End User Focus

At this level, management is truly able to focus on the end user. This level is achieved by building a competency-based organization. Management has focused the resources of the organization to meeting its mission/objectives. The practices common at this phase are process management practices based on competency, quality incentives based on competency, and end user feedback systems that impact the establishment of competencies. This level formalizes the use of statistical tools in evaluating processes, and formalizes staff’s process improvement teams.

Level 4 -- Team Focus

The emphasis at this level is moving from committees directed and controlled by management to teams empowered to take action on their own. Turning over the day-to-day operation of the team is possible because the quantitative data initiated at Level 4 enables management to manage byfact as opposed to managing people and jobs. The management practices common at this level include team building, team rewards, cross-functional teams, employee surveys, and the widespread use of management tools by the teams.

Level 5 – World-Class Focus

The teams become truly self-directed and motivated through sharing in the gains made by the organization. Management practices now include benchmarking and advanced statistical tools such as quality function deployment.