Simon Morley June09 Challenge

Test Republic Testing Challenge:: June-July 2009 Name: Simon Morley Email: xxxxxxxxxxxxxxxxxxxxx Contact no: xxxxxxxxxxxxxxxxxxxxx Test Republic name: Simon Morley

Bug reports (for the above test challenge) are contained on the following pages. Notes are included in the page following the problem reports. The last pages contain the "raw" notes made when executing the test. Time spent on the initial bug investigation: 20 minutes + 15 follow-up / reproduction later Time spent on bug report writing: 3 hours (majority of time spent due to not using a pre-formatted report template and review of the report – worthwhile as demonstrated in one of the final notes.)

Issue Report ID: Report Heading: details

SM00001 Dialogue box for location gives no context for the entry

Configuration: Client-side:

Server-side:

HP Compaq nx7000 Windows XP Home Ed version 5.1.2600 SP3 Web browser: Google Chrome 2.0.172.33 Web site: http://maps.google.com Version: Not available

Problem location:

Web site: http://maps.google.com

Report author: Report observation date: Report submitted date:

Simon Morley 30 June 2009 17 July 2009

How/When found: Reproducible:

Review of first page of web site. Yes

Severity:

Small (non critical)

Customer impact: Incorrect results returned. (Potential to enter data in the customer context – not necessarily the context expected of the web site.) Description: On the start page for the site (http://maps.google.com/) there is no indication of the format required for the entry of the required location. Temporary solution: None Suggested solution: Introduce a roll-over to give the possibilities for data entry. Include a note to state that the default location/map on the first/start page is the default country context, i.e. if another country than the default is intended for search then this should be entered. Indicate the formats that are recognised: country-specific postal codes, GPS coords, etc. Expected design & verification cost:  Minimal-Medium depending on the implementation.  Minimal: Introduction of a roll-over with basic information for data entry.  Medium: Tweaks to the page to include additional help symbols/buttons. Expected benefit to customer:  Enhanced comfort/security about using the site - especially first-time users  More return users instead of searching for alternative map sites.  More recommendations for the site. Expected benefit to vendor:  Reduced number of "how-to" questions to vendor  Enhanced "reputation" for accessibility and usability to a wider audience - wider age and demographic range of users (e.g. allowing for web newbies / beginners)

Notes on report ID: SM00001-2 1. The first thing I noticed on the page after entering the address was that the map appeared over the USA. I wanted to try an address in the UK and had one place to enter this in “Set default location”. BUT, what format should I use? I know that UK post codes and US zip codes are different, but would the site know this? 2. I guessed that it might – but hadn’t tested this yet. I went back one step – suppose that I was completely new to using map websites and had never been to the USA I would just enter the postal code as I was used to using it.

3. So, my first set of tests was about entering postal codes (valid and potentially invalid) in “raw” format – without any country identifier… 4. I tried a valid UK postal code then followed it with an invalid UK postal code (maybe valid somewhere else in the world) 5. I did the same with some Swedish postal codes (without any country context) – bingo! I got an unexpected result. 6. This then led me to think, “Why postal codes?” The page is not giving me any context information about how the data should be entered and is making assumptions about what I want to know – which I managed to get the wrong result by!

Issue Report ID: Report Heading:

SM00002 Country context should only reflect the default location

Configuration: Client-side:

Server-side:

HP Compaq nx7000 Windows XP Home Ed version 5.1.2600 SP3 Web browsers: Google Chrome 2.0.172.33 Firefox 3.0.11 IE8 version 8.0.6001.18702 Web site: http://maps.google.com

Problem location:

Web site: http://maps.google.com

Report author: Report observation date: Report submitted date:

Simon Morley 30 June 2009 17 July 2009

How/When found: Reproducible:

Testing of the location data & Review of results. Yes

Severity:

Medium (non critical, liable to provide erroneous results)

Customer impact: Incorrect results returned. (Potential to enter data in the customer context – not necessarily the context expected.) Description: It was noticed that postcodes from two different countries could be entered successfully without indicating the country. However, when another valid post code from one of the countries was entered a non-expected result was returned (result for a third country.) Reproduction steps (on any of the browsers mentioned above): Pre-requisite: Clear the browser cache Step 1: Enter URL http://maps.google.com Result: Map displayed over USA Step 2: Enter a UK postal code (in “Set default location”). Enter LS9 Result: Map displays a location over UK. Note: The default location is now set to the UK. Step 3: Enter a valid Swedish postal code (in “Change default location”). Enter 11121 Result: Map displays a location over Sweden Note: The default location has changed from UK to Sweden. Step 4: Enter a different valid Swedish postal code. Enter 17831 Result: Map displays a location over USA Note: The default location was Sweden and the search for “17831” did not happen in Sweden – this is a separate fault. Default location is now over the US. Step 5: Qualify the Swedish postal code: Enter 17831, Sweden Result: Map displays a location over Sweden

Temporary solution: Indicate a note/warning that the required country should be specified. Suggested solution: Do not allow ambiguous entries. Where a default location is set (country) then if a country context is not entered it will be assumed the location data entered refers to the default country. If the location data is not found for the default country then an error “not found” response is returned. Include an information response to state that the location was searched for in the default country () and that if this was not the intended country for search then the country should also be entered. E.g. assuming the default country was set to the USA then the following search/results might be expected. Search location = LS9 Result = “Location not found in USA” Search location = 178 31 Result = Pennsylvania Search location = 178 31, Sweden Result = Ekerö, Stockholm Expected design & verification cost:  Medium. Retest of valid and non-valid postcode-country pairings. Expected benefit to customer:  Clearer and more accessible usage of the site - the customer learns to know exactly what is expected and how to enter data. This is done in a very low-tech and low-cost way to the customer so that it is not a deterrent to usage.  It is much better that the customer learns to enter the location data with location context (if different from the default) than to receive a mis-leading result (from the customer's perspective). The customer may not realise that the result is not related to the location context (country) that they were thinking about.  Reduced customer confusion and frustration. Expected benefit to vendor:  Enhanced customer experience. Happier customers that return (don't search for alternatives) and recommend the site to their friends.

Issue Report ID: Report Heading:

SM00003 No version / revision handling on the website

Configuration: Client-side:

Server-side:

HP Compaq nx7000 Windows XP Home Ed version 5.1.2600 SP3 Web browser: Google Chrome 2.0.172.33 Web site: http://maps.google.com Version: Not available

Problem location:

Web site: http://maps.google.com

Report author: Report observation date: Report submitted date:

Simon Morley 17 July 2009 17 July 2009

How/When found: Reproducible:

Review of results from previous testing. Yes

Severity:

Small (non critical)

Customer impact:

Difficult to provide accurate feedback to vendor.

Description: There is no apparent versioning information on the web site - at least visible from the start page. This means it is not possible to give accurate feedback from the customer on issues observed. It is also not possible for the customer to track issues and fixes in replies from the vendor (e.g. issue SM0003 fixed in version x.y.z) Temporary solution: Giving precise information on detection date. However, this may be cumbersome or not always practical for the customer. Suggested solution: Introduce a footer with the version information of the website. Expected design & verification cost: Minimal Expected benefit to customer:  Additional tracking for issues reported and fixes implemented.  Allows the customer to have a separate tracking notation from the vendor and still have a common reference point. Two customers may report similar problems – identified as one root cause and one fix supplied by the vendor – then the customers can track “their” problems without having to interpret a generic vendor issue. Expected benefit to vendor:  Allows for clearer tracking of reported problems- less ambiguity from the customer (reported in vA.B.C rather than “I saw this the other day…”)  Enhances the customer “service” and openness from the vendor to the customer – ultimately, enhances the perception of accessibility.

Notes on Report ID SM0003 1. This report came from a review of the basic data for the report formats. 2. I could give version information for the client-side but not the server-side 3. Google encourages feedback - but if I try something out 2 weeks ago and report it now how does google know which version of their website was used? Suppose I'm not sure about exactly when I saw the issue, "I saw this the other day..."

General notes I first looked at the site over 2 weeks ago. I made some initial notes and stored them in the "cloud" using google docs. I then looked at them later, from a different PC, made some editorial changes and re-stored. Then later at the time of writing the report I used another PC (borrowed) to write this document and stored in google docs. I didn't use a set template for the problem/issue reports - I took some headings from Cem Kaner's bug advocacy paper and some from my own previous experience. I have not tried to make the reports "best-in-class", but merely as examples of presenting information in a clear and understandable way. I found a more “serious” fault by reviewing my problem description – this is mentioned under report SM00002 (Description, Step4, Note), but I did not have the time/space (10 pages) to fit in an extra bug report. But this was very interesting to – an example of the power of post-analysis and reviews – whilst writing a description of one problem I noticed another that I hadn’t spotted originally!

Below are the "raw" notes that I made whilst looking at the site. Readers can see the exact steps I went through (the above reports are a consolidation of those steps and groupings of problems) and how I looked at the problems. The notes above give a summary of the steps but it's sometimes useful to look at the "raw data" - not always as readable but good for reference.

Raw Notes Notation/clarification (added 17 July 2009): Sx: Means test step - it's not really a test case - the test cases could be created from certain test steps, but for the purpose of this exercise I'm just interested to differentiate between a test step, an observed result and my comment or additional information for the observation. R: Observed result. C: Comment or deduction on what was observed. Px: Pre-requisite, pre-condition or pre-amble to step Sx. The below steps and observations were made 30 June 2009. Steps S3-S6 were tried on browsers Chrome, Firefox & IE8 with the same results. Chrome - logged out of my google account S1: Opened a tab with (http://maps.google.com/) R: Map opened over USA. Scale set to 500mi/500km S2: Clicked "Set default location" link R: Dialogue box "Set default location" C: No information on how the location should be entered: Country, City, GPS coords, Ordnance Survey map coords, post/zip code or any other variant. C: Better dialogue is needed, with a pop-up/roll-over for how a location should be entered.... S3: Invalid GB postcode entered (ls9 9qw) R: Map is blank. C: No information that the postcode could not be located. C: The scale changes from the opening page (200ft/100m in this case.) If the map is blank then the scale should be blank or removed - otherwise this is misleading. It could be interpreted that something is displayed and that the scale/zoom needs to be altered to see the display C: Link has changed from "Set default location" to "Change default location". This is misleading implying that my invalid entry is now a "default".

S3.1: Invalid GB postcode entered (ls9 9qw). Then press "Satellite" display button. R: Map displays the message "We are sorry, but we don't have imagery at this zoom level for this region. Try zooming out for a broader look." R: Scale is blank. C: Mis-leading information. Implies that there is something to be seen at a different zoom level. C: If there is nothing to be displayed then the response message should be different.

S4: Re-enter address http://maps.google.com R: Satellite image appears over UK (postcode LS9 9) - however, the default location appears as "ls9 9qw" (invalid).

P5: Settings are Swedish. S5: Enter a valid Swedish postcode as the location (111 21 - Stockholm Central Train Station) R: Map is displayed of the areas.

S6: Enter another valid Swedish postcode (178 31 - Ekerö - one of Stockholm's islands) R: Map displays a location in Pennsylvania, USA C: No obvious connection to the site's settings and the search algorithm used S6.1: Enter the country also "178 31, sweden" R: Correct map displayed. C: Inconsistent use of the web browser's settings - picking the Swedish location for the postcode "111 21" and the US location for "178 31" until the country was entered...