How to set goals effectively for software quality
The International Software Testing Qualifications Board is one of the most respected software certification bodies in the world. It defines software quality “The sum of all functionality and features of a product that bears on its ability to fulfill stated or implied need”, while the IEEE describes software quality “The degree to what a system, component or process meets user or customer needs or expectations.”
These Case Studies will help you understand the impact of quality on major organizations over the years.
1962, Mariner 1 Spacecraft. NASA – NASA launched an unmanned space mission in 1962 to gather data and fly past Venus. The spacecraft barely reached Cape Canaveral before it veered off course. NASA activated its self-destruct button after a potentially dangerous crash landing. After further investigation, it was discovered that the self-destruct button had been activated by NASA because of a simple software error. A missing hyphen in a code line led to an estimated loss 18 million USD (as per 1962).
EDS Child Support System (2004), Government of the United Kingdom – In 2004, the UK introduced a new system for managing the Child Support Agency’s operations. The contract was awarded to Electronic Data Systems (EDS), a Texas-based company. It was called CS2. It was plagued by a multitude of issues from the initial go live. There were over 1000 reported problems of which 400 had no workaround. This resulted in approximately 3,000 IT incidents per week. An internal memo leaked later described the system as badly designed-tested-implemented. It led to the UK government spending 768million GBP instead the estimated 450million GBP. The contractor also declared a fiscal deficit of USD 153million for the following year.
Bitcoin Hack, Mt. Gox, 2011- Bacon in Mt. Gox in 2010 was the biggest bitcoin exchange until a software bug led to unredeemable transactions that cost them USD 1.5 million in lost bitcoins. The company declared bankruptcy several years later due to this severe loss in a volatile market.
The above cases show that the software did not satisfy either stated or implied needs (as determined by ISTQB), or meet user or customer needs (as determined by IEEE). This essentially violates both the accepted definitions and principles of Software Quality.
Despite spending large amounts of money, time, and resources on QA, companies still fail to meet their goals. This is evidently due to a lack in Software Quality.
The 12th Edition of the World Quality Report 2020-21 mentions “QA is a key priority for organizations to achieve Digital Transformation”, but the Consortium for Information & Software Quality records in its 2020 Report that poor-quality software costs US companies more than USD 2 trillion.
To ensure that software quality is maintained within their organizations, Diligent Quality Assurance Leaders will always strive to achieve this goal. This article will help them to set SMART goals for software quality.
Software Testing Objectives: Overview
Below is a diagram that shows the main objectives of software testing. These objectives will be important when setting quality goals for software development.
- Preventing Defects: The most important goal of any Software Quality process, is to avoid bugs in the software development process. The cost and labor involved in fixing these defects can be reduced by early defect detection. We can future-proof our solution by conducting a root cause analysis on any previous defects.
- Assess Work Product: To achieve software quality assurance, artifacts must be created that are relevant to the product being built. In order to adhere to agile development principles, there must be a shift in terms of QA. It is important to identify any contradictions or ambiguities in the requirements document, design, or user stories before you begin development. This will save you time and money in the long-term.
- Verify Requirements The primary goal of software development is to meet the client’s needs. Software testing goals all revolve around ensuring that standards are met. QA techniques are designed to meet both industry and user expectations.
- Validate test object: To ensure they meet project business requirements, all test objects must be validated. To ensure that test objects are valid, both manual and automated testing can be used.
- Build confidence: Client confidence in a project rests on the software quality assurance process. A low defect rate during user acceptance testing is a good way to build trust in your product. Clients should be presented with detailed test reports.
- Reduce risk Software quality processes should be able to test as many scenarios as possible while testing the product in the shortest time possible to find and eliminate any potential risks during deployment.
- Tip: BrowserStack’s cloud Selenium grid contains 3000+ real browsers, devices and browsers to perform all required tests in real user circumstances. This will reduce risk. Manual testing can also be done on the BrowserStack cloud. Register for and choose the required device-browser combinations to start testing.
- Share information with stakeholders Software quality processes must provide objective and accurate information that allows stakeholders to make informed decisions about the product’s quality. Reports on Test Coverage and Other Software testing metrics These bases can be covered.
- Tip: Use BrowserStack to test real devices. To test apps and websites, you can access 3000+ browsers. BrowserStack’s extensive debugging options allow you to report, track, and fix bugs.
- Find Failures and Defects: Software quality assurance (QA) is designed to find and fix any defects in software.
The software quality objectives serve as a helpful yardstick for the QA team, allowing them to assess and adjust any software testing goals based upon their progress. They also allow us to see the activities that lead to our software quality goals being met.
Setting Goals for Software Quality
1. Get buy-in from stakeholders and understand what is most important
It is crucial to know what the primary goals of the software testing process are for the customer before we can set any goals regarding software quality. It is important to ask several questions depending on the needs of the user.
- What is the time frame for testing?
- What amount of money is required for the test?
- What percentage of software can be automated?
- Is the effort sufficient? Could there have been more tests done in the same time?
Software quality management in software engineering should have the primary goal of measuring software performance throughout the user journey. This means that you should consider the entire user experience, from login to checkout. Users should have the opportunity to enjoy the desired experience at all stages of their user journey.
This stage helps to verify the boxes for Building Confidence, Sharing Information and other objectives.
2. Define a Smart Goal
The QA Leader should have a plan for setting software quality goals. They should also set SMART goals to gain clarity and motivate them to accomplish their tasks. They are more likely than others to succeed in the important things and not just in the low-hanging ones.
SMART stands for –
- Specific: A very specific goal. It is better to have one objective goal for each action item.
- Measurable It’s important to ensure that the goal is objectively measurable to determine if it has been met. This can be expressed in test coverage. Software testing metrics can also be used to quantify the process.
- Attainable Goals should be achievable. It shouldn’t have any limits on how large or small it can become, but it should be accomplished within the given timeframe.
- Relevant Both the QA team and the other stakeholders should understand the importance of this goal. The previous step is therefore of paramount importance.
- Time-Bound – It’s important to set a deadline for the goal to avoid it getting lost in the shuffle of daily activities. An agile testing method helps to organize the work into manageable sprints that help you achieve these goals.
Continuous Testing is a great way to ensure that software is both customer-centric and in compliance with good development practices. The accuracy of the code can only be ensured if it is run on real devices. BrowserStack’s cloud Selenium grid provides QAs with access to 3000+ real browsers for automated testing. This allows every feature to be monitored under real-world conditions.
These are some examples of quality objectives:
- Reduce the number defects by 20% in the next quarter – Assuming that the number bugs reported during the previous quarter is known.
- ” Keep re-work time under 25% of total development time in each sprint“
- ” Increase Internal Testing Effective” – This is a strategy to increase defect detection rates (before and after delivery) beyond, say, 80% per release, or any major milestone.
A bad quality goal could be “Reduce bugs in every sprint”, which isn’t achievable and time-bound.
The definition of SMART goals is helpful in Evaluating Work Products and Building Confidence. It also helps with sharing information with stakeholders based on the above objectives.
3. Describe the steps required to achieve the goal
There are three ways to define the activities necessary for achieving the goals of software quality:
Preventive Actions
These include:
- Plans to train your staff in the use of automation suites and other processes that will improve overall QA practice.
- R&D activities to ensure that the best automated tool is selected and that best practices are adopted for your QA process.
- If possible, set up a Continuous Test process. This ensures that every code runs through a series test from the smallest unit to complete components before it is merged into the main repository.
Appraisal Activities
An appraisal is an objective assessment of an artifact in relation to a set of standards. It includes:
Reviews
These include periodic reviews of the test process and the progress of test execution. Also, documents created during the software-development lifecycle are used to verify that daily activities are aligned with customer goals and user requirements.
Tests
This is the core of any software quality management system. It is important to perform tests as often as possible and as soon as possible. Test automation or parallel testing can speed up the test procedure and allow for faster coverage. All aspects of QA are dependent on the availability of a device cloud. It is impossible to find every bug that a user might encounter without real device testing. Undiscovered bugs cannot be tracked, monitored or solved. QA metrics can’t be used to establish baselines or measure success without accurate information about bugs. This holds true for manual testing as well as automated testing .
Audits
An independent auditor is usually hired to audit the work of the QA team. They assess the work against industry standards such as ISO standards. Audits should be carefully planned, documented using a standard model such as CMMI and shared with stakeholders. To ensure compliance with industry standards, any discrepancies must be transformed into a set SMART goals.
NOTE: BrowserStack is SOC2 Type 2 compliant, uses HTTPS as a default, stores credentials securely and does more to protect our users.
Fix/Repair Activities
This includes fixing bugs discovered during testing. This activity can also include a variety of tasks such as changing the team structure, test tool or test methodology.
This step is necessary to prevent defects, validate test objects, reduce risk, share information with stakeholders, and find failures and defects that are not in line with the initial objectives.
4. To determine if your resources are adequate to meet your goals, you should assess them.
Three “actors” are members of a team that interact and have an influence on the team’s dynamic. These three “actors” are the people, their availability and knowledge, the technology that supports the QA process, and the work processes (waterfall and agile, DevOps and so forth). To achieve the highest quality software, there must be a balance among these three. There is however a cost to this balance. This can be summarized using the following formula:
CoQ = CoP + CON + coA ( source – American Society for Quality).
- COP refers to the cost of prevention activities and resources;
- CoN refers to the cost of non-conformities. This includes efforts required to fix defects, retest and rework / refactor your product.
- COA refers to the costs of appraisals. They include review time, testing new or modified functionalities, independent verification and validation sessions, or auditing sessions.
A quality assurance plan that is successful will ensure that the CoQ remains within a budget acceptable range.
5. Your activities should be timed to meet your goals and objectives
Some of these activities are one-off events, such as Audits or the definition of SMART goals. Others are more ongoing. For example, any processes related to validation, verification, or review will be repeated across sprints in an agile process.
6. Continuously monitor Quality Objectives
It is crucial to keep track of the goals once they have been established. This will ensure that they remain relevant to the project as well as ensuring that they meet the overall need to conform to quality standards and satisfy user expectations.
Conclusion
This article will explain what software quality is, why it is important for software engineering teams, as well as how to define goals to achieve it. Software quality can be managed within a budget and timeframe that is agreed upon. This helps the team focus on the most important aspects for the project’s success.
If done right, the benefits of setting Software Quality Goals in terms customer satisfaction, defect reduction, product quality, and team satisfaction are undisputed. You can also be seen as working towards a goal and not reacting to the SDLC process. This builds customer confidence and results in a product that does the job well.