This post continues the discussion of Compatibility testing from our test team. --Steven

In the previous blog post "Application Compatibility Testing for Windows 7" we talked about the importance of Application Compatibility and work we are doing to engineer this in Windows 7. In this post we will examine the challenge that emerges as we consider the world wide audience that Windows serves.

This blog post will cover the following areas:

• Overall International App Compatibility Strategy
• Approach to International App Compatibility
• Application Acquisition
• Testing Applications
• Measuring our Success
• What it means to “Rescue An Application”

For Windows 7 we have made significant investment in application compatibility, ensuring applications that worked on Vista, continue to work on Windows 7 and we’ve also rescued some applications that were broken in Vista to work on Windows 7 (more on that later). As we’ve talked about, there are some applications that are OS version specific by design (utilities, firewalls, security, etc.) and those are not included in this discussion.

Approach

One of the biggest challenges in International Application Compatibility is what applications we test, the scale of testing, and what it means for us to say that an application “works”. For Windows 7 we are testing over 1200 applications across 25 specific markets. We have improved our coverage over Vista by adding over 300 more international applications.

We look at applications in 3 buckets.

1. Global ISV (GISV) Applications – Localized software sold by major ISVs in several international markets as well as the United States.
2. Microsoft (MS) Localized Applications – Microsoft software that has been localized for use in other markets other than the United States.
3. 3rd Party Local Applications – Software where the user interface language is not-English and the application is sold in non-English speaking markets (for example, IchiTaro – Japanese Word Processor, 8 – Russian ERP system,)

Categories 1 & 2 are pretty straightforward. There are a known set of key applications and scenarios used around the world and we must ensure these applications function in Windows 7. Category #3 is where there is some complexity.

The applications list we build for 3rd Party Local Applications is built using a number of methods. First, we build on the list of applications we have used in previous versions Windows (XP/Vista, etc). If it worked on Vista, it must work on Windows 7.

Next we work with our teams in markets around the world to rank top applications in particular markets. It is amazing to see the diversity in application use around the world. The application testing list is based on a combination of market data where it is available, individual knowledge of markets, culture, revenue, usage and even sometimes just “word on the street”. The cultural knowledge in these markets is probably most critical to our success. For example, casual gaming in Korea is hugely popular and we need to ensure our Windows 7 testing accounts for this.

Our goal in selecting applications is to test as many applications as we can that will expose the most issues across different scenarios and markets.

These scenarios include:

• Productivity
• Memories (photo editing and sharing apps, etc.)
• Graphics
• Productivity
• Music
• Fundamentals (security, data backup, etc.)
• TV/Movies

Application Acquisition

Once we build the list of applications we need to test the next process is acquiring them. We acquire applications in a variety of ways but many times we have to buy an application from a retail store just as any end user would. Other methods we use to acquire applications include downloading full featured trial versions, purchasing software, and working with ISVs to acquire their applications to ensure compatibility.

Testing Applications

Testing applications means more than just installing them and making sure they launch. Every application gets a unique test plan written for it to cover as much functionality as we can. We write test cases to cover primary and secondary application functions – for our word processing example this would include opening a file, typing a letter, adjusting formatting, save, and print, emailing a copy to someone, etc. These applications go through 6 or more test passes during the product cycle.

Now, we can’t test every piece of every application and we do run into some interesting challenges when we focus on a worldwide audience. Many applications depend on location specific information (meaning if you aren’t testing the application in that location – you aren’t likely to have the information needed). Examples include Brazilian citizen’s CPF ID, or Brazilian personal number of identification which would be required to test something like tax preparation software. We run into similar problems with SMS applications requiring active local mobile phone accounts.

What it means to Rescue an Application

Along with the core tenet of ensuring that any application that worked on Windows Vista also work on Windows 7 we have a stretch goal to “raise the bar” and make applications work on Windows 7 that never worked on Windows Vista. For Windows 7, we have some good news early in the development cycle. So far we have made over 30 applications that were “broken” on Vista work on Windows 7. This means that Windows 7 will have higher application compatibility than Windows Vista. We are continuing to push this number up. Below is a table of the # of applications by language that we have made to work on Windows 7 but didn’t’ work on Vista.

Language Number of
Apps Fixed Example Applications Arabic 1 Khalifa Cartoon Characters Creator Chinese (Simplified) 1 Arcsoft WebCam Companion Chinese (Traditional) 3 Asure Purchase/Sale/Stock Master 2008

Cyberlink DVD Suite v6

Asure Accounting Master 2008

Czech 1 J.K.R. BYZNYS Danish 1 Bogskabet 3.2 German 2 QuickTime 7.1.6

Haufe Personal Office Professional - Haufe Formular-Manager

Hebrew 3 Compedia Timmy in English World

Compedia Moomins: The Search for the Ruby

Compedia The Puzzling Time Quest

Hungarian 1 Infocentrum Road Register Italian 5 Finson Costo del Lavoro Italian v2

Finson Falco 6

Finson Progetto Condominio

Finson Contintasca 7

Finson ContinBanca

Japanese 5 PostPet v3

Kenchako Adventure 9.0

WZ Editor 5.0

QuickTime 7.1.6

Overland LOKI: with Japanese Manual

Norwegian 1 Visma Avendo Fakturering Polish 2 WF-Fakturka dla Windows

Nahlik eTeacher 5

Portugese 1 Mr. Escola Win Port Spanish 3 Mexico Federal Taxes Simplified SAT: Individual Taxes

Monografias Spanglish

IKEA Home Kitchen Planner

Turkish 1 MYTR Filter 2.6 Along with ensuring these applications work on Windows 7 we have taken an extra step for our existing Vista customers. Of the applications outlined in the above table, 27 of the fixes we made have been back ported to Windows Vista for possible inclusion in future updates. We really wanted to raise the bar for application compatibility and go beyond just looking at Vista as the baseline.

Takeaway

There is a lot of information here and hopefully gives you some insight into what it means for us to make the application experience (application compatibility) on Windows 7 as high as possible for users around the world. We started out with a goal of making sure if an application worked on Windows Vista it should work on Windows 7. We have taken that further by bringing applications that never worked on Vista to work on Windows 7 and even future updates to Vista.



More...

Posted By: dmex
09 Mar 2009