Sqm Assignment.docx

Software Testing - ISO Standards Many organizations around the globe develop and implement different standards to improve the quality needs of their software. This chapter briefly describes some of the widely used standards related to Quality Assurance and Testing.

ISO/IEC 9126 This standard deals with the following aspects to determine the quality of a software application − 

Quality model



External metrics



Internal metrics



Quality in use metrics

This standard presents some set of quality attributes for any software such as − 

Functionality



Reliability



Usability



Efficiency



Maintainability



Portability

The above-mentioned quality attributes are further divided into sub-factors, which you can study when you study the standard in detail.

ISO/IEC 9241-11 Part 11 of this standard deals with the extent to which a product can be used by specified users to achieve specified goals with Effectiveness, Efficiency and Satisfaction in a specified context of use. This standard proposed a framework that describes the usability components and the relationship between them. In this standard, the usability is considered in terms of user performance and satisfaction. According to ISO 9241-11, usability depends on context of use and the level of usability will change as the context changes.

/IEC 25000:2005 ISO/IEC 25000:2005 is commonly known as the standard that provides the guidelines for Software Quality Requirements and Evaluation SQuaRESQuaRE. This standard helps in organizing and enhancing the process related to software quality requirements and their evaluations. In reality, ISO-25000 replaces the two old ISO standards, i.e. ISO-9126 and ISO14598. SQUARE is divided into sub-parts such as − 

ISO 2500n − Quality Management Division



ISO 2501n − Quality Model Division



ISO 2502n − Quality Measurement Division



ISO 2503n − Quality Requirements Division



ISO 2504n − Quality Evaluation Division

The main contents of SQuaRE are − 

Terms and definitions



Reference Models



General guide



Individual division guides



Standard related to Requirement Engineering i.e.specification,planning,measurementandevaluationprocessi.e.specificatio n,planning,measurementandevaluationprocess

ISO/IEC 12119 This standard deals with software packages delivered to the client. It does not focus or deal with the clients’ production process. The main contents are related to the following items − 

Set of requirements for software packages.



Instructions for testing a delivered software package against the specified requirements.

Miscellaneous Some of the other standards related to QA and Testing processes are mentioned below −

Sr.No

Standard & Description

IEEE 829 1

A standard for the format of documents used in different stages of software testing.

2

IEEE 1061 A methodology for establishing quality requirements, identifying, implementing, analyzing, and validating the process, and product of software quality metrics.

3

IEEE 1059 Guide for Software Verification and Validation Plans.

4

IEEE 1008 A standard for unit testing.

5

IEEE 1012 A standard for Software Verification and Validation.

6

IEEE 1028 A standard for software inspections.

7

IEEE 1044 A standard for the classification of software anomalies.

8

IEEE 1044-1 A guide for the classification of software anomalies.

9

IEEE 830 A guide for developing system requirements specifications.

10

IEEE 730 A standard for software quality assurance plans.

11

IEEE 1061 A standard for software quality metrics and methodology.

12

IEEE 12207 A standard for software life cycle processes and life cycle data.

13

BS 7925-1 A vocabulary of terms used in software testing.

14

BS 7925-2 A standard for software component testing.

Capability Maturity Model The Capability Maturity Model (CMM) specifies an increasing series of levels of a software development organization. The higher the level, the better the software development process, hence reaching each level is an expensive and time-consuming process.

Levels of CMM



Level One: Initial - The software process is characterized as inconsistent, and occasionally even chaotic. Defined processes and standard practices that exist are abandoned during a crisis. Success of the organization majorly depends on an individual effort, talent, and heroics. The heroes eventually move on to other organizations taking their wealth of knowledge or lessons learnt with them.



Level Two: Repeatable - This level of Software Development Organization has a basic and consistent project management processes to track cost, schedule, and functionality. The process is in place to repeat the earlier successes on projects with similar applications. Program management is a key characteristic of a level two organization.



Level Three: Defined - The software process for both management and engineering activities are documented, standardized, and integrated into a standard software process for the entire organization and all projects across the organization use an

approved, tailored version of the organization's standard software process for developing, testing and maintaining the application. 

Level Four: Managed - Management can effectively control the software development effort using precise measurements. At this level, organization set a quantitative quality goal for both software process and software maintenance. At this maturity level, the performance of processes is controlled using statistical and other quantitative techniques, and is quantitatively predictable.



Level Five: Optimizing - The Key characteristic of this level is focusing on continually improving process performance through both incremental and innovative technological improvements. At this level, changes to the process are to improve the process performance and at the same time maintaining statistical probability to achieve the established quantitative process-improvement objectives.

Software Reliability A test is considered reliable if we get the same result repeatedly. Software Reliability is the probability of failure-free software operation for a specified period of time in a specified environment. Software Reliability is also an important factor affecting system reliability. Reliability Testing It is about exercising an application so that failures are discovered and removed before the system is deployed. The purpose of reliability testing is to determine product reliability, and to determine whether the software meets the customer’s reliability requirements. 

 

Reliability refers to the consistency of a measure. A test is considered reliable if we get the same result repeatedly. Software Reliability is the probability of failure-free software operation for a specified period of time in a specified environment. Software Reliability is also an important factor affecting system reliability. Reliability testing will tend to uncover earlier those failures that are most likely in actual operation, thus directing efforts at fixing the most important faults. Reliability testing may be performed at several levels. Complex systems may be tested at component, circuit board, unit, assembly, subsystem and system levels.

Software reliability is a key part in software quality. The study of software reliability can be categorized into three parts: 1.Modeling 2.Measurement 3.Improvement 1. Modeling: Software reliability modeling has matured to the point that meaningful results can be obtained by applying suitable models to the problem. There are many models exist, but no single model can capture a necessary amount of the software characteristics. Assumptions and abstractions must be made to simplify the problem. There is no single model that is universal to all the situations. 2. Measurement: Software reliability measurement is naive. Measurement is far from commonplace in software, as in other engineering field. “How good is the software, quantitatively?” As simple as the question is, there is still no good answer. Software reliability cannot be directly measured, so other related factors are measured to estimate software reliability and compare it among products. Development process, faults and failures found are all factors related to software reliability. 3. Improvement: Software reliability improvement is hard. The difficulty of the problem stems from insufficient understanding of software reliability and in general, the characteristics of software. Until now there is no good way to conquer the complexity problem of software. Complete testing of a moderately complex software module is infeasible. Defect-free software product cannot be assured. Realistic constraints of time and budget severely limit the effort put into software reliability improvement.

Project Quality Plan To address all the quality requirements, standards and quality assurance mechanisms in a project, a document called 'project quality plan' is developed by the project team. This plan acts as the quality bible for the project and all the stakeholders of the project should adhere to the project quality plan.

The Components of a Project Quality Plan Depending on the nature of the industry and the nature of the project, the components or the areas addressed by a quality plan may vary. However, there are some components that can be found in any type of quality plan. Let's have a look at the most essential attributes of a project quality plan. Responsibility of Management This describes how the management is responsible for achieving the project quality. Since management is the controlling and monitoring function for the project, project quality is mainly a management responsibility. Document Management and Control Documents are the main method of communication in project management. Documents are used for communication between the team members, project management, senior management and the client. Therefore, the project quality plan should describe a way to manage and control the documents used in the project. Usually, there can be a common documentation repository with controlled access in order to store and retrieve the documents. Requirements Scope The correct requirements to be implemented are listed here. This is an abstraction of the requirements sign-off document. Having requirements noted in the project quality plan helps the quality assurance team to correctly validate them. This way, quality assurance function knows what exactly to test and what exactly to leave out from the scope. Testing the requirements that are not in the scope may be a waste for the service provider. Design Control This specifies the controls and procedures used for the design phase of the project. Usually, there should be design reviews in order to analyse the correctness of the proposed technical design. For fruitful design reviews, senior designers or the architects of the respective domain should get involved. Once the designs are reviewed and agreed, they are signed-off with the client.

With the time, the client may come up with changes to the requirements or new requirements. In such cases, design may be changed. Every time the design changes, the changes should be reviewed and signed-off. Development Control and Rigor Once the construction of the project starts, all the processes, procedures and activities should be closely monitored and measured. By this type of control, the project management can make sure that the project is progressing in the correct path. Testing and Quality Assurance This component of the project quality plan takes precedence over other components. This is the element, which describes the main quality assurance functions of the project. This section should clearly identify the quality objectives for the project and the approach to achieve them. Risks & Mitigation This section identifies the project quality risks. Then, the project management team should come up with appropriate mitigation plans in order to address each quality risk. Quality Audits For every project, regardless of its size or the nature, there should be periodic quality audits to measure the adherence to the quality standards. These audits can be done by an internal team or an external team. Sometimes, the client may employ external audit teams to measure the compliance to standards and procedures of the project processes and activities. Defect Management During testing and quality assurance, defects are usually caught. This is quite common when it comes to software development projects. The project quality plan should have guidelines and instructions on how to manage the defects. Training Requirements Every project team requires some kind of training before the project commences. For this, a skill gap analysis is done to identify the training requirements at the project initiation phase. The project quality plan should indicate these training requirements and necessary steps to get the staff trained.

Automation Testing Automation Testing means using an automation tool to execute your test case suite. The automation software can also enter test data into the System under Test, compare expected and actual results and generate detailed test reports. It demands considerable investments of money and resources. Successive development cycles will require execution of same test suite repeatedly. Using a test automation tool, it's possible to record this test suite and re-play it as required. Once the test suite is automated, no human intervention is required. This improved ROI of Test Automation. The goal of Automation is to reduce the number of test cases to be run manually and not to eliminate Manual testing altogether. Automated software testing is important due to the following reasons:      

Manual Testing of all workflows, all fields, all negative scenarios is time and money consuming It is difficult to test for multilingual sites manually Automation does not require Human intervention. You can run automated test unattended (overnight) Automation increases the speed of test execution Automation helps increase Test Coverage Manual Testing can become boring and hence error-prone.

Automated Testing Process: Following steps are followed in an Automation Process

Test tool selection:Test Tool selection largely depends on the technology the Application under Test is built on. For instance, QTP does not support Informatics. So QTP cannot be used for testing Informatics applications. It's a good idea to conduct a Proof of Concept of Tool on AUT.

Define the scope of Automation:The scope of automation is the area of your Application under Test which will be automated. Following points help determine scope:       

The features that are important for the business Scenarios which have a large amount of data Common functionalities across applications Technical feasibility The extent to which business components are reused The complexity of test cases Ability to use the same test cases for cross-browser testing

Planning, Design, and Development During this phase, you create an Automation strategy & plan, which contains the following details     

Automation tools selected Framework design and its features In-Scope and Out-of-scope items of automation Automation test bed preparation Schedule and Timeline of scripting and execution Deliverables of Automation Testing

Test Execution:Automation Scripts are executed during this phase. The scripts need input test data before there are set to run. Once executed they provide detailed test reports. Execution can be performed using the automation tool directly or through Test management tool which will invoke the automation tool. Example: Quality center is the Test Management tool which in turn it will invoke QTP for execution of automation scripts. Scripts can be executed in a single machine or a group of machines. The execution can be done during the night, to save time.

Maintenance:As new functionalities are added to the System under Test with successive cycles, Automation Scripts need to be added, reviewed and maintained for each release cycle. Maintenance becomes necessary to improve the effectiveness of Automation Scripts.

Project Management Tools When applying any of the techniques to the project you need to accomplish, you also need to use specific tools for successfully implementing the technique. Here’s a list of software tools that are used in project management on different work steps.

Organizing Workflow & Planning It is the core part of starting a project: it defines how the project will be performed, and how the quality of its outcome will be ensured. Large companies often use such comprehensive solution as MS Project. For smaller teams, various alternatives are available. They don’t provide all the rich functionality typical to complex and expensive tools, but they have planning and roadmap features that are sufficient for visualizing future project progress. There are also free project management solutions that suit best for start-ups and small businesses.

Communication Being the key point in many techniques and methodologies, communication within a project team needs to be properly organized. While using email for formal communication and important messages, it’s also essential to have a corporate messenger – Slack and Skype are the most popular ones. And, if your team members use different tools to communicate, eliminate the pain of having multiple messengers by integrating them.

Scheduling When allocating resources and planning for future, it’s crucial to know who on the team is available for specific dates. Use scheduling software for that! Such tools as actiPLANS provide a clear visual chart of absences for upcoming dates, and allow to see all necessary details to team members’ leaves and time off.

Time management Knowing where your team’s time goes not just helps managing current project risks. It also provides valuable information for future planning and estimating. Time management tools, such as actiTIME, help managers understand both individual time expenses and team’s results for any period. Informative reports with time & cost summaries and notes to the logged time provide insights into how time is used and what can be optimized.

Finance & Accounting For any project manager and business owner, understanding financial outcome of the projects is crucial for analysis and future planning. Most used accounting tools – QuickBooks, Zoho, and Fresh books – help collect this information. For smaller project teams, other accounting solutions can be reasonable. They require less investment, but also provide insights on project profitability, teams’ performance, and estimation accuracy.