Release
This document was uploaded by user and they confirmed that they have the permission to share it. If you are author or own the copyright of this book, please report to us by using this DMCA report form. Report DMCA
Overview
Download & View Release as PDF for free.
More details
- Words: 3,674
- Pages: 7
Release Notes: What Do Test Engineers Need to Know? Author’s Introduction Recently, I was asked to investigate and create a Release Notes Standards document to improve the content, format and consistency of my company’s Release Notes documentation. After searching the Web and various Software Engineering books, I discovered that there is little information on this topic. As a result, I had to rely on information obtained during interviews I conducted within my company. Our experienced Test Engineers and Software Engineers were asked what they considered ‘good’ and ‘bad’ practices with regard to Release Notes. The data was compiled and the Release Notes Standards document that resulted became the basis for this article.
Overview My personal experiences indicate that clear and concise communication between Test Engineers and Software Engineers facilitate a successful handoff of a software release. In most cases, the Release Notes become the most important document during the transition of the software from the development phase to the test phase. Missing or incomplete Release Notes often result in wasted time, missed schedules, and inadequate testing. Release Notes provide the mechanism for Software Engineers to convey all of the
December 2002
by Marc René
information that is necessary for acceptance and evaluation of the software release by the Test Engineers. In addition, Release Notes provide a historical record of the release, tracing release content through the evolution of the software. By establishing an agreed upon scope, content, and format for Release Notes, as well as identifying the resources required to create them, GTECH has greatly improved the transition from development to test.
Release Day Today is release day. Cheryl and Craig arrive at work at 7 AM to find a note on the door of the Test Lab. It reads, “Built last night around 4:00 p.m. Get the new stuff on the M: drive. Defects are marked as ‘fixed’ in the defect tracking system. Nancy.” Not knowing how long the install will take or how many defects have been corrected within this release, the Test Engineers proceed, hoping to begin testing before lunch. Cheryl begins installing the ‘new’ version of the software and immediately runs into trouble. She ends the installation and now must wait for Nancy to arrive to resolve the issue. Craig begins printing out the defects
http://www.testinginstitute.com
marked as ‘fixed’ in the defect tracking system and tries to prioritize the retest effort. Craig finds thirty defects that are marked as ‘fixed’. However, he realizes that some of these defects were marked as ‘fixed’ after 4:00 p.m. yesterday. Nancy’s note had indicated the build occurred at 4:00 p.m. As a result, Craig too, must wait for Nancy to arrive to determine which ‘fixed’ defects are included within this release. It is now 9:00 a.m. Nancy arrives at work to find Craig and Cheryl waiting anxiously for her. After a few minutes, Nancy is able to resolve Cheryl’s issue and provide Craig with the correct list of ‘fixed’defects that are included within this release. At 9:15 a.m., Cheryl and Craig again begin their tasks. Cheryl follows Nancy’s verbal directions for the installation. Hoping everything installed successfully, Cheryl informs Craig that they can finally begin testing. It is 10:00 a.m. and approximately three hours (six person hours) of test time is wasted because the Test Engineers were not given all of the information required to commence the testing activities. Meanwhile, Craig’s prioritization efforts are fruitless. Many of the ‘fixed’ descriptions indicate only “problem resolved”.
Journal of Software Testing Professionals
1
The descriptions do not provide details on any possible test considerations. Without meaningful guidance from the Software Engineers, Craig prioritizes the ‘fixed’ defects based solely on their severity level. High severity defects will be verified first. After retesting several ‘fixed’ defects, Cheryl uncovers a serious problem. It takes her over an hour to isolate the steps to reproduce the problem. Cheryl is recording the defect in the defect tracking system when Nancy enters the lab to see how the testing is progressing. When Nancy sees the problem that Cheryl is logging, she indicates that she “knew about the problem last week” and that she “fixed it this morning”. Frustrated, Cheryl and Craig decide to go to lunch; it has been a very tough morning.
What do Test Engineers Need to Know? Cheryl and Craig’s problems are not unique. Many Test Engineers struggle with releases because they do not receive adequate Release Notes. In some cases, Test Engineers do not receive any release guidance at all! Often this occurs because Software Engineers do not appreciate the importance of Release Notes, or do not know what should be included in them. Release Notes can expedite the testing associated with a software release. Release Notes provide installation instructions, including a method to determine if the installation worked and any back-out steps if problems are encountered. They also provide information to assist Test Engineers in the prioritization of test/retest efforts that include details on any reviews performed on fixes and any special test considerations. Release Notes also provide information about known issues so that Test Engineers do not waste valuable testing time identifying and reproducing them. If Cheryl and Craig had received this type of guidance, their release day would have been much more productive and efficient. December 2002
What are Release Notes? Release Notes are a release description document prepared to identify the content and changes made for each release of software. This information aids the Test Engineer in determining which areas and/or items that must be verified or retested. Release Notes allow Software Engineers to convey the exact code that has been changed, the reason that the code has been changed, and any special testing conditions that the Test Group should exercise. In addition, information relating to code reviews and design reviews for the released software is included within the Release Notes. Review information will assist Test Engineers with the prioritization of retest efforts. Fixes or new functionality that have undergone design or code review may be considered lower risk and prioritized for test after more risky functions or fixes. At GTECH, we require Release Notes for all software releases. We do not allow exceptions. Developers supply Release Notes to the Test Engineers prior to the software being released for testing. The Test Engineers reject any software release if Release Notes are missing or incomplete. The suggested Release Note content includes the information specified on the next page. The sequence listed is for the sake of presentation in this article, and the actual order in the Release Notes might be different. For example, the ordering may be different if utilizing defect-tracking tools to generate portions of the Release Notes. Defect-tracking tools often allow for the generation of lists of defects addressed and any special test considerations specified when the problem was corrected. In this case, the sections containing defects addressed and test considerations would be combined.
http://www.testinginstitute.com
Why Create a Release Note Standards Document? In order to gain consistency in the format and content of Release Notes, a Release Note Standards document is written for use throughout an organization. Release Note Standards provide general guidelines for the creation of Release Notes. Adhering to the guidelines set forth in the standard’s document will enable the following goals to be attained: • Uniformity in the presentation and the content of Release Notes across projects • Facilitation of the hand-off of software to the Test Group • Increase visibility into release content A Release Note Standards document has benefits for Software Engineers and Project Managers as well as Test Engineers. Guidance on Release Note presentation and content frees Software Engineers from the mundane and repetitive tasks of determining what information is relevant to the Test Engineers and how to present that information. Once Release Note standards are set, Software Engineers complete the appropriate sections providing the information indicated avoiding the need to ‘re-invent the wheel’ for each new release. This frees Software Engineers to do what they are best at, create software. When Test Engineers receive complete Release Notes, they are less likely to interrupt Software Engineers because they already have the release information required. Consistent Release Notes also allow historical release information to be stored and if necessary retrieved later. Project Managers also benefit from consistent and complete Release Notes as the content assists with the determination of schedule accuracy. Since Release Note content includes functionality added to a release, the Project Manager can compare the schedule and the listed functionality to ensure the project is on schedule.
Journal of Software Testing Professionals
7
Suggested Release Note Content
8
1. Release Identifier
Specific Version Number. In addition, a clear description of the type of software that is being released must be provided (i.e., the specific functional area or component names). If the released software is part of a larger release, then information regarding the Overall Release Name and the Overall Release Number is also included.
2. Release History
Document the revision number, the date of the release, and a brief explanation for all previous software releases. For convenience, this section is listed in reverse chronological order (newest first.)
3. Release References
Include all references to earlier revisions of the software or other products. Software Requirements Specification(s) including version number, and Design Documentation including version number.
4. Release Overview
Includes a narrative description of the functionality contained in the software release. If applicable, include comparisons to older releases of the software. If the release occurs during an incremental build/delivery cycle, the Release Notes refer to the specific build/delivery release by name and list specific items scheduled in the current release.
4.1 Functionality Added
Describe all ‘new’ functionality that has been added to the software in the release. For initial releases, it is not necessary for all functionality to be listed. However, any planned functionality not being released can be listed, making future reference to this functionality easier.
4.2 Functionality Removed
Document any and all functionality that was removed within the software release along with rationale for the removal.
4.3 Known Problems Not Documented by Defects
Documented by Defects include any known problem(s) the software might have that have not been previously documented in the defect tracking system.
5. Release Operational Considerations
Describe the changes relating to intra-system interfaces or system operation that may be dependent upon another product’s release or any non-software solutions. These dependencies are documented to identify any change that the Test Group should make to the Test Environment that are not included as part of the standard release (i.e., Operating System patch or hardware upgrade).
6. Defects and/or Enhancements Addressed
Include a list of all defects and/or enhancements that have been addressed and released for testing. Some defect-tracking systems provide the ability to generate this list.
7. File Descriptions
Provide a comprehensive list of the specific files or, at a minimum, the directories that are being released. These lists assist the Test Engineers with determining the impact of the changes and additions and assist with assuring correct installation.
7.1 Files Modified in Release
Document the names and purpose of source, executable, data, and command files (as applicable) that have been modified within the software release. This subsection also includes an overview of code changes that were made as well as a list of the files that were modified.
7.2 New Files in Release
Document the names and purpose of source, executable, data, and command files (as applicable) that are ‘new’ to the software release.
7.3 Files Removed in Release
Document the names and purpose of source, executable, data, and/or command files (as applicable) that are ‘removed’ since the previous release.
Journal of Software Testing Professionals
http://www.testinginstitute.com
December 2002
Suggested Release Note Content (continued) 8. Tools and Utilities
Document the names and the impact of all ‘new’ tools and/or utilities contained within the release. If changes were made to existing tools and/or utilities, they must also be clearly described. If changes within the release impact testing tools, these changes are listed in detail. This section also indicates if any special tools or utilities are required to utilize this release or to test it.
9. Code/Design Reviews Performed
Document the results of all code or design reviews associated with the software release. The results are detailed in this section of the Release Notes documentation or a ‘path’ is provided where this information can be found. If the inspection data is stored in a tool, specify the location in the tool where the inspection data can be found.
10. Unit/Integration Tests Performed
Provide detailed documentation on how the code was tested by the Software Engineers. The results of Unit/Integration Testing are documented in detail in this section of the Release Notes or the Software Engineer provides a ‘path’ where this information can be found. Additionally, special test harnesses used to perform Unit Testing must be referenced and described along with their results.
11. Test Considerations
Include any special test or setup considerations the Test Engineer should be aware of when testing the release. The Software Engineer must list any special procedures for testing each correction and document any unique set-up and/or tools required. The defect tracking system may provide the ability to generate this information for each defect in the release.
12. Installation Details
Provide specific details on the set-up and verification procedures for the installation of the software release.
12.1 Location
Provide a detailed description of the location (i.e., file path name) of the software release.
12.2 Procedure
Describe the systematic procedure for installing the released software. This subsection also includes any special release installation instructions if the release instructions differ significantly from previous releases.
12.3 Schedule and Duration
Schedule and Duration Include the software release schedule details (date, time and location) as well as an estimate of the expected duration of the installation for the release.
12.4 Post Installation Checklist
Provide the information necessary so that Test Engineers can be assured that the release has been installed successfully and that testing may begin.
13. Contingencies
Include a detailed recovery or ‘back-out’ process. This section provides instructions for Disaster Control and the method to be used to return to an earlier release version if necessary. If a release cannot be returned to a previous version because of a file format change or other major redesign, it must be specified in this section of the Release Notes.
14. Sign Off
Contains authorization by the Software Engineer, Lead Test Engineer, Project Manager and any other interested parties. All parties must agree on and accept the contents of the Release Notes. Sign Off does not necessarily imply an actual signature, but stresses the importance of agreement by that all parties that the Release Notes are complete and correct.
December 2002
http://www.testinginstitute.com
Journal of Software Testing Professionals
9
What is a Release Note Standards Document? A Release Note Standards document describes the basic conventions and guidelines to be used for the generation of Release Notes. These conventions ensure uniform content of Release Notes that result in improved maintainability and testability of software releases. The document is written for the technical staff that is responsible for software development. GTECH’s® Release Note Standards document contains information on content, responsibilities and potential areas for tailoring the standards. In addition, an example template is provided to illustrate a potential format for presentation (refer to the example provided). The content defined in the standard is as illustrated in the above table. This set was chosen because of the complexity of GTECH’s® systems and our need to manage geographically separated development and test teams. Smaller teams or less complex systems or products may not need the complete set above. As defined in the GTECH® standard, the responsibility for creating Release Notes falls on the Software Engineers. Project Managers are responsible for ensuring that Software Engineers follow the standards. The level and extent of tailoring is specified in the standard. In general, it is expected that Release Notes will contain all sections defined in the Release Notes Standards document. If a particular section is not applicable for a release, it is suggested that the section remain and its content be listed as ‘Not Applicable’. The GTECH® standard expects and encourages projects to expand upon the standard as needed for their specific environment. However, the project must not violate any convention within the Release Note Standards document.
10
Summary Release Notes serve an important purpose facilitating the successful handoff of a software release. Release Notes provide the mechanism for the Software Engineers to convey to the Test Engineers all of the information that is necessary to accept and evaluate the software release. Project Teams that fail to understand the importance of complete, clear and concise Release Notes often waste time, miss schedules and perform inadequate testing. Creating a Release Note Standards document is a mechanism to establish the content of and responsibilities for the creation of Release Notes. These standards assure that Test Engineers have the information required to test a release. They assist the Software Engineer by providing a checklist of information to include in a release. They allow the Project Manager greater visibility into release content and schedule. It should be noted that although this article stresses the importance of Release Notes as a communication vehicle for software releases between the Software Engineers and the Test Group, many of the same concepts can be applied for Release Notes given to customers as well.
Release Day – Revisited Today is release day. Cheryl and Craig arrive at work at 7 AM to find a complete set of Release Notes tacked to the Test Lab door. Cheryl begins installing the ‘new’ version of the software following the detailed installation procedures. Once installed, Cheryl follows the procedure to verify that the software has been installed correctly. The software installed without incident. Craig starts prioritizing the retest effort based upon the fix descriptions, test considerations, design review and code review information. Craig finds that all of the HIGH severity defects have been
Journal of Software Testing Professionals
through design and code reviews, so these can be prioritized lower on the list. It is 7:30 a.m and Cheryl and Craig are ready to start testing the new release. After retesting several ‘fixed’ defects, Cheryl uncovers a serious problem. She checks the Release Notes and learns that the problem, although serious, is known by the development team. She moves on to the next defect and does not spend any time trying to reproduce the problem. By the time Cheryl and Craig decide to go to lunch, they have verified all the highrisk fixes and only found one bad fix. It has been a very productive morning.
Bibliography Guide to the Software Engineering Body of Knowledge IEEE – Trial Version 1.00 – May 2001 Software Release Methodology Michael E. Bays, Prentice Hall PTR, 1999. This book illustrates best practices for every stage of a successful product release. It includes carefully designed, practical solutions that enhance quality, reduce costs, and increase time to market.
About the Author Marc René is a Software Quality Assurance Engineer for GTECH Corporation. He is responsible for the definition, collection and evaluation of metrics to analyze product and process quality. Marc works with the SEPG and is responsible for the generation of tools as well as creating many corporate processes and procedures. His experience with Software Engineering, SQA and Testing spans 12+ years. Marc has spoken at STAREAST 2000 and other SQA conferences. Marc received a Bachelors Degree in Computer Science and Psychology from Rhode Island College and is a member of the RIC Computer Science Program’s Industry Advisory Board.
http://www.testinginstitute.com
December 2002
Project Name:
Release Name:
Date of Release:
Release Version Number:
Overall Release Name:
Overall Release Version Number:
Contents of Release
Description:
Release History (repeat as often as required) Revision Number :
Date of the Software’s Release:
Release Descript ion:
Release References Reference(s) to earlier versions of the software or other products:
Software Requi rements Specification(s) with Version Number(s):
Design Documentation with Version Number(s):
Release Overview Functionality contained in the Release:
Functionality Added :
Functionality Removed:
Known problems not documented by defects:
Detailed Release Description
Defect and/or Enhancements Addressed (repeat as often as required) Number:
Short Description:
Fix Description: Test Considerations
Product Name:
Release Operational Considerations
Code/Design Reviews “Code Review Description and Location?
December 2002
http://www.testinginstitute.com
Journal of Software Testing Professionals
11
File Descriptions Files Modified (repeat as often as required) Name of file: Overview changes:
of
Code ‘New’ Files in Release (repeat as often as required)
Name and purpose of file: ‘Removed’ Files in Release (repeat as often as required) Name and Purpose of file:
Tool & Utilities Name and Impact of ‘new’ Tools and/or Utilities
Name and Impact of ‘existing’ Tools and/or Utilities changed:
Special Tools and/or Utilities required for Release:
Overview My personal experiences indicate that clear and concise communication between Test Engineers and Software Engineers facilitate a successful handoff of a software release. In most cases, the Release Notes become the most important document during the transition of the software from the development phase to the test phase. Missing or incomplete Release Notes often result in wasted time, missed schedules, and inadequate testing. Release Notes provide the mechanism for Software Engineers to convey all of the
December 2002
by Marc René
information that is necessary for acceptance and evaluation of the software release by the Test Engineers. In addition, Release Notes provide a historical record of the release, tracing release content through the evolution of the software. By establishing an agreed upon scope, content, and format for Release Notes, as well as identifying the resources required to create them, GTECH has greatly improved the transition from development to test.
Release Day Today is release day. Cheryl and Craig arrive at work at 7 AM to find a note on the door of the Test Lab. It reads, “Built last night around 4:00 p.m. Get the new stuff on the M: drive. Defects are marked as ‘fixed’ in the defect tracking system. Nancy.” Not knowing how long the install will take or how many defects have been corrected within this release, the Test Engineers proceed, hoping to begin testing before lunch. Cheryl begins installing the ‘new’ version of the software and immediately runs into trouble. She ends the installation and now must wait for Nancy to arrive to resolve the issue. Craig begins printing out the defects
http://www.testinginstitute.com
marked as ‘fixed’ in the defect tracking system and tries to prioritize the retest effort. Craig finds thirty defects that are marked as ‘fixed’. However, he realizes that some of these defects were marked as ‘fixed’ after 4:00 p.m. yesterday. Nancy’s note had indicated the build occurred at 4:00 p.m. As a result, Craig too, must wait for Nancy to arrive to determine which ‘fixed’ defects are included within this release. It is now 9:00 a.m. Nancy arrives at work to find Craig and Cheryl waiting anxiously for her. After a few minutes, Nancy is able to resolve Cheryl’s issue and provide Craig with the correct list of ‘fixed’defects that are included within this release. At 9:15 a.m., Cheryl and Craig again begin their tasks. Cheryl follows Nancy’s verbal directions for the installation. Hoping everything installed successfully, Cheryl informs Craig that they can finally begin testing. It is 10:00 a.m. and approximately three hours (six person hours) of test time is wasted because the Test Engineers were not given all of the information required to commence the testing activities. Meanwhile, Craig’s prioritization efforts are fruitless. Many of the ‘fixed’ descriptions indicate only “problem resolved”.
Journal of Software Testing Professionals
1
The descriptions do not provide details on any possible test considerations. Without meaningful guidance from the Software Engineers, Craig prioritizes the ‘fixed’ defects based solely on their severity level. High severity defects will be verified first. After retesting several ‘fixed’ defects, Cheryl uncovers a serious problem. It takes her over an hour to isolate the steps to reproduce the problem. Cheryl is recording the defect in the defect tracking system when Nancy enters the lab to see how the testing is progressing. When Nancy sees the problem that Cheryl is logging, she indicates that she “knew about the problem last week” and that she “fixed it this morning”. Frustrated, Cheryl and Craig decide to go to lunch; it has been a very tough morning.
What do Test Engineers Need to Know? Cheryl and Craig’s problems are not unique. Many Test Engineers struggle with releases because they do not receive adequate Release Notes. In some cases, Test Engineers do not receive any release guidance at all! Often this occurs because Software Engineers do not appreciate the importance of Release Notes, or do not know what should be included in them. Release Notes can expedite the testing associated with a software release. Release Notes provide installation instructions, including a method to determine if the installation worked and any back-out steps if problems are encountered. They also provide information to assist Test Engineers in the prioritization of test/retest efforts that include details on any reviews performed on fixes and any special test considerations. Release Notes also provide information about known issues so that Test Engineers do not waste valuable testing time identifying and reproducing them. If Cheryl and Craig had received this type of guidance, their release day would have been much more productive and efficient. December 2002
What are Release Notes? Release Notes are a release description document prepared to identify the content and changes made for each release of software. This information aids the Test Engineer in determining which areas and/or items that must be verified or retested. Release Notes allow Software Engineers to convey the exact code that has been changed, the reason that the code has been changed, and any special testing conditions that the Test Group should exercise. In addition, information relating to code reviews and design reviews for the released software is included within the Release Notes. Review information will assist Test Engineers with the prioritization of retest efforts. Fixes or new functionality that have undergone design or code review may be considered lower risk and prioritized for test after more risky functions or fixes. At GTECH, we require Release Notes for all software releases. We do not allow exceptions. Developers supply Release Notes to the Test Engineers prior to the software being released for testing. The Test Engineers reject any software release if Release Notes are missing or incomplete. The suggested Release Note content includes the information specified on the next page. The sequence listed is for the sake of presentation in this article, and the actual order in the Release Notes might be different. For example, the ordering may be different if utilizing defect-tracking tools to generate portions of the Release Notes. Defect-tracking tools often allow for the generation of lists of defects addressed and any special test considerations specified when the problem was corrected. In this case, the sections containing defects addressed and test considerations would be combined.
http://www.testinginstitute.com
Why Create a Release Note Standards Document? In order to gain consistency in the format and content of Release Notes, a Release Note Standards document is written for use throughout an organization. Release Note Standards provide general guidelines for the creation of Release Notes. Adhering to the guidelines set forth in the standard’s document will enable the following goals to be attained: • Uniformity in the presentation and the content of Release Notes across projects • Facilitation of the hand-off of software to the Test Group • Increase visibility into release content A Release Note Standards document has benefits for Software Engineers and Project Managers as well as Test Engineers. Guidance on Release Note presentation and content frees Software Engineers from the mundane and repetitive tasks of determining what information is relevant to the Test Engineers and how to present that information. Once Release Note standards are set, Software Engineers complete the appropriate sections providing the information indicated avoiding the need to ‘re-invent the wheel’ for each new release. This frees Software Engineers to do what they are best at, create software. When Test Engineers receive complete Release Notes, they are less likely to interrupt Software Engineers because they already have the release information required. Consistent Release Notes also allow historical release information to be stored and if necessary retrieved later. Project Managers also benefit from consistent and complete Release Notes as the content assists with the determination of schedule accuracy. Since Release Note content includes functionality added to a release, the Project Manager can compare the schedule and the listed functionality to ensure the project is on schedule.
Journal of Software Testing Professionals
7
Suggested Release Note Content
8
1. Release Identifier
Specific Version Number. In addition, a clear description of the type of software that is being released must be provided (i.e., the specific functional area or component names). If the released software is part of a larger release, then information regarding the Overall Release Name and the Overall Release Number is also included.
2. Release History
Document the revision number, the date of the release, and a brief explanation for all previous software releases. For convenience, this section is listed in reverse chronological order (newest first.)
3. Release References
Include all references to earlier revisions of the software or other products. Software Requirements Specification(s) including version number, and Design Documentation including version number.
4. Release Overview
Includes a narrative description of the functionality contained in the software release. If applicable, include comparisons to older releases of the software. If the release occurs during an incremental build/delivery cycle, the Release Notes refer to the specific build/delivery release by name and list specific items scheduled in the current release.
4.1 Functionality Added
Describe all ‘new’ functionality that has been added to the software in the release. For initial releases, it is not necessary for all functionality to be listed. However, any planned functionality not being released can be listed, making future reference to this functionality easier.
4.2 Functionality Removed
Document any and all functionality that was removed within the software release along with rationale for the removal.
4.3 Known Problems Not Documented by Defects
Documented by Defects include any known problem(s) the software might have that have not been previously documented in the defect tracking system.
5. Release Operational Considerations
Describe the changes relating to intra-system interfaces or system operation that may be dependent upon another product’s release or any non-software solutions. These dependencies are documented to identify any change that the Test Group should make to the Test Environment that are not included as part of the standard release (i.e., Operating System patch or hardware upgrade).
6. Defects and/or Enhancements Addressed
Include a list of all defects and/or enhancements that have been addressed and released for testing. Some defect-tracking systems provide the ability to generate this list.
7. File Descriptions
Provide a comprehensive list of the specific files or, at a minimum, the directories that are being released. These lists assist the Test Engineers with determining the impact of the changes and additions and assist with assuring correct installation.
7.1 Files Modified in Release
Document the names and purpose of source, executable, data, and command files (as applicable) that have been modified within the software release. This subsection also includes an overview of code changes that were made as well as a list of the files that were modified.
7.2 New Files in Release
Document the names and purpose of source, executable, data, and command files (as applicable) that are ‘new’ to the software release.
7.3 Files Removed in Release
Document the names and purpose of source, executable, data, and/or command files (as applicable) that are ‘removed’ since the previous release.
Journal of Software Testing Professionals
http://www.testinginstitute.com
December 2002
Suggested Release Note Content (continued) 8. Tools and Utilities
Document the names and the impact of all ‘new’ tools and/or utilities contained within the release. If changes were made to existing tools and/or utilities, they must also be clearly described. If changes within the release impact testing tools, these changes are listed in detail. This section also indicates if any special tools or utilities are required to utilize this release or to test it.
9. Code/Design Reviews Performed
Document the results of all code or design reviews associated with the software release. The results are detailed in this section of the Release Notes documentation or a ‘path’ is provided where this information can be found. If the inspection data is stored in a tool, specify the location in the tool where the inspection data can be found.
10. Unit/Integration Tests Performed
Provide detailed documentation on how the code was tested by the Software Engineers. The results of Unit/Integration Testing are documented in detail in this section of the Release Notes or the Software Engineer provides a ‘path’ where this information can be found. Additionally, special test harnesses used to perform Unit Testing must be referenced and described along with their results.
11. Test Considerations
Include any special test or setup considerations the Test Engineer should be aware of when testing the release. The Software Engineer must list any special procedures for testing each correction and document any unique set-up and/or tools required. The defect tracking system may provide the ability to generate this information for each defect in the release.
12. Installation Details
Provide specific details on the set-up and verification procedures for the installation of the software release.
12.1 Location
Provide a detailed description of the location (i.e., file path name) of the software release.
12.2 Procedure
Describe the systematic procedure for installing the released software. This subsection also includes any special release installation instructions if the release instructions differ significantly from previous releases.
12.3 Schedule and Duration
Schedule and Duration Include the software release schedule details (date, time and location) as well as an estimate of the expected duration of the installation for the release.
12.4 Post Installation Checklist
Provide the information necessary so that Test Engineers can be assured that the release has been installed successfully and that testing may begin.
13. Contingencies
Include a detailed recovery or ‘back-out’ process. This section provides instructions for Disaster Control and the method to be used to return to an earlier release version if necessary. If a release cannot be returned to a previous version because of a file format change or other major redesign, it must be specified in this section of the Release Notes.
14. Sign Off
Contains authorization by the Software Engineer, Lead Test Engineer, Project Manager and any other interested parties. All parties must agree on and accept the contents of the Release Notes. Sign Off does not necessarily imply an actual signature, but stresses the importance of agreement by that all parties that the Release Notes are complete and correct.
December 2002
http://www.testinginstitute.com
Journal of Software Testing Professionals
9
What is a Release Note Standards Document? A Release Note Standards document describes the basic conventions and guidelines to be used for the generation of Release Notes. These conventions ensure uniform content of Release Notes that result in improved maintainability and testability of software releases. The document is written for the technical staff that is responsible for software development. GTECH’s® Release Note Standards document contains information on content, responsibilities and potential areas for tailoring the standards. In addition, an example template is provided to illustrate a potential format for presentation (refer to the example provided). The content defined in the standard is as illustrated in the above table. This set was chosen because of the complexity of GTECH’s® systems and our need to manage geographically separated development and test teams. Smaller teams or less complex systems or products may not need the complete set above. As defined in the GTECH® standard, the responsibility for creating Release Notes falls on the Software Engineers. Project Managers are responsible for ensuring that Software Engineers follow the standards. The level and extent of tailoring is specified in the standard. In general, it is expected that Release Notes will contain all sections defined in the Release Notes Standards document. If a particular section is not applicable for a release, it is suggested that the section remain and its content be listed as ‘Not Applicable’. The GTECH® standard expects and encourages projects to expand upon the standard as needed for their specific environment. However, the project must not violate any convention within the Release Note Standards document.
10
Summary Release Notes serve an important purpose facilitating the successful handoff of a software release. Release Notes provide the mechanism for the Software Engineers to convey to the Test Engineers all of the information that is necessary to accept and evaluate the software release. Project Teams that fail to understand the importance of complete, clear and concise Release Notes often waste time, miss schedules and perform inadequate testing. Creating a Release Note Standards document is a mechanism to establish the content of and responsibilities for the creation of Release Notes. These standards assure that Test Engineers have the information required to test a release. They assist the Software Engineer by providing a checklist of information to include in a release. They allow the Project Manager greater visibility into release content and schedule. It should be noted that although this article stresses the importance of Release Notes as a communication vehicle for software releases between the Software Engineers and the Test Group, many of the same concepts can be applied for Release Notes given to customers as well.
Release Day – Revisited Today is release day. Cheryl and Craig arrive at work at 7 AM to find a complete set of Release Notes tacked to the Test Lab door. Cheryl begins installing the ‘new’ version of the software following the detailed installation procedures. Once installed, Cheryl follows the procedure to verify that the software has been installed correctly. The software installed without incident. Craig starts prioritizing the retest effort based upon the fix descriptions, test considerations, design review and code review information. Craig finds that all of the HIGH severity defects have been
Journal of Software Testing Professionals
through design and code reviews, so these can be prioritized lower on the list. It is 7:30 a.m and Cheryl and Craig are ready to start testing the new release. After retesting several ‘fixed’ defects, Cheryl uncovers a serious problem. She checks the Release Notes and learns that the problem, although serious, is known by the development team. She moves on to the next defect and does not spend any time trying to reproduce the problem. By the time Cheryl and Craig decide to go to lunch, they have verified all the highrisk fixes and only found one bad fix. It has been a very productive morning.
Bibliography Guide to the Software Engineering Body of Knowledge IEEE – Trial Version 1.00 – May 2001 Software Release Methodology Michael E. Bays, Prentice Hall PTR, 1999. This book illustrates best practices for every stage of a successful product release. It includes carefully designed, practical solutions that enhance quality, reduce costs, and increase time to market.
About the Author Marc René is a Software Quality Assurance Engineer for GTECH Corporation. He is responsible for the definition, collection and evaluation of metrics to analyze product and process quality. Marc works with the SEPG and is responsible for the generation of tools as well as creating many corporate processes and procedures. His experience with Software Engineering, SQA and Testing spans 12+ years. Marc has spoken at STAREAST 2000 and other SQA conferences. Marc received a Bachelors Degree in Computer Science and Psychology from Rhode Island College and is a member of the RIC Computer Science Program’s Industry Advisory Board.
http://www.testinginstitute.com
December 2002
Project Name:
Release Name:
Date of Release:
Release Version Number:
Overall Release Name:
Overall Release Version Number:
Contents of Release
Description:
Release History (repeat as often as required) Revision Number :
Date of the Software’s Release:
Release Descript ion:
Release References Reference(s) to earlier versions of the software or other products:
Software Requi rements Specification(s) with Version Number(s):
Design Documentation with Version Number(s):
Release Overview Functionality contained in the Release:
Functionality Added :
Functionality Removed:
Known problems not documented by defects:
Detailed Release Description
Defect and/or Enhancements Addressed (repeat as often as required) Number:
Short Description:
Fix Description: Test Considerations
Product Name:
Release Operational Considerations
Code/Design Reviews “Code Review Description and Location?
December 2002
http://www.testinginstitute.com
Journal of Software Testing Professionals
11
File Descriptions Files Modified (repeat as often as required) Name of file: Overview changes:
of
Code ‘New’ Files in Release (repeat as often as required)
Name and purpose of file: ‘Removed’ Files in Release (repeat as often as required) Name and Purpose of file:
Tool & Utilities Name and Impact of ‘new’ Tools and/or Utilities
Name and Impact of ‘existing’ Tools and/or Utilities changed:
Special Tools and/or Utilities required for Release:
Unit/Integration Testing Description and Location of Unit/Integration Test and Special Test Harnesses:
Description of Unit/Integration Test Results:
to
Installation Details Detailed Description of location of the Software Release:
Step-by-step installation procedure:
Release Schedule Details & Estimated Duration:
Post Installation Checklist:
Contingency Plans – recovery process:
Sign Off
12
Software Engineer:
Integration Engineer (if applicable):
Test Lead:
Technology Lead:
Journal of Software Testing Professionals
http://www.testinginstitute.com
December 2002
