Defect Tracking Tools

After a long time got a good chance of writing on my interesting experience.

Can you imagine a Testers job without a Bug Tracking System?

I can just say it would be this way!!

Earlier i use to work for a project where everything is systematic and we have a well maintained Defect tracking tool called DSolver (The customized version of the Bugzilla). Now i am into a project where we don't have any bug tracking tool. As am the only QA, and after getting used to the well maintained tool it has been really a tough time for me to report and then track the bugs.

Finally i started installing Bugzilla the famous open source defect tracking tool.

Seriously started following step by step procedure mentioned here 

Bugzilla installation needs the following to be installed:

• Apache
• MySQL
• Pearl

Good to see almost all the open source tools use Apache as the webserver and My SQL as the Database. So thought of installing all one after another. But luckily came to know from my collegues about another open source tool  XAMPP.

XAMPP's name is an acronym for:

• X (meaning cross-platform)
• Apache HTTP Server
• MySQL
• PHP
• Perl

kool luckily with the installation of XAMPP two steps (installation of Apache and MySQL) reduced to one. :)

Even after lot of struggle could not make Bugzilla up.In parallel we have started searching for another open source tool and as this is a short term project we are looking for easy web based tool.

Then came across another tool FlySpray

Now this again need 

• Apache
• MySQL
• PHP

As i already installed XAMPP i was able to reuse the same Apache and MYSQL and with minimal setup i was able to get a bug tracking tool in the team.

Now All izz well..

So for a short term projects we can use the open source tool called FlySpray and for the larger projects i would say try  Bugzilla.

Software Testing Principles (GLENFORD J.MYERS)

1. A necessary part of a testcase is a definition of the expected output or result.
2. A programmer should avoid attempting to test his or her own program.
3. A programming organization should not test its own program.
4. Thoroughly inspect the results of each test.
5. Test cases must be written for input conditions that are invalid and unexpected, as well as for those that are valid and expected.
6. Examining a program to see if it does not do what it is supposed to do is only half the battle; the other half is seeing whether the program does what it is not supposed to do.
7. Avoid throwaway testcases unless the program is truly throwaway program.
8. Do not plan a testing effort under the tacit assumption that no errors will be found.
9. The probability of the existence of more errors in a section of a program is proportional to the number of errors already found in that section.
10. Testing is extremely creative and intellectually challenging task.
    

FURPS+ for Software Testing

The following is the acroynm for the FURPS model and what it covers:

Functionality- Feature set, Capabilities, Generality, Security

Usability- Human factors, Aesthetics, Consistency, Documentation

Reliability- Frequency/severity of failure, Recoverability, Predictability, Accuracy, Mean time to failure

Performance- Speed, Efficiency, Resource consumption, Throughput, Response time

Supportability-Testability, Extensibility, Adaptability, Maintainability, Compatibility, Configurability, Serviceability, Installability, Localizability, Portability

But now a days in software industry FURPS+ Model is widely used:

The '+' includes:

Design requirement: Eg: If you specify that DB, Web server etc. are required its a design constraint

Implementation requirement: Eg: This includes reuired coding standards, Implementation languages and resource limits

Interface requirement: Specifies the external item with which the system must interact

Physical requirement: What hardware must the system be deployable on?

Multibyte Testing

MultiByte Testing!!!!!! hmm What it is? Do we need to do this for all kind of products? Why should we do this?

Hey We all say proudly that we work for a MNC. Haa MNC, a Multi National Company. If any of our friends or relatives ask us we say, we have our company in Netherlands, China, Germany, UK, US etc..(the list goes on).

But did we ever think on how your product works on a Chinese environment or a German Environment ?

Yup that's what we do as part of Multibyte testing. We test our software compatibility with respect to other languages.

You might get a doubt on what i mean by Chinese environment and German Environment etc..

let me explain with an exmaple.

Microsofts Windows OS is very popularly known OS and is used almost world wide.

As English is the common language used world wide we generally see the machines in English.

But one of my chinese customer want my software to be seen in Chinese charecter support as shown in the image. For a typical business software, it may even extend to even the Chinese DataBase server also, where as chinese charecters might be considered as some special charecters in normal DB server (English).

This kind of testing is done in this typical kind of environments where your business software is installed in that respective environment, it might be either chinese or german or french etc..

and the functionality of the software remains unchanged.

Interesting Video on Exploratory Testing

Friends,

I found an interesting video on Exploratory Testing in you tube and thought of sharing with you. :)

Go through it.. damn gud

Documentation Testing

From the childhood we are used to learn things from books. Hope all of you agree with me. :)

The enthusiasm in leaning new things will grow in us as we growup.

In this software era, how will we be learning about the softwares which hits our eye attention?

Documentation (ofcourse Soft copy of the BOOK again) helps us in doing this.

The Basic three questions which come into our mind when we think of Documentation testing:

What is Documentation?

Why should it be tested?

How should it be tested?

The general user's perspective:

Say One of my hobby is photo editing.

Hey!!! There is a new software, photo editer 1.0 released into the market with all good features like adding Beautiful frames to our image,making the image more attractive with different special effects etc.. and now i can create an album of my own and suprise everyone with my new album..

Now there comes many questions to the users mind.

How should i install this software? "Installation guide" helps me in doing this

oh great installation is successful :) but now...

How should i use it?? "Product Documentation/Help"

wow here this product has a very big documentation.. i think every feature is explained clearly..

and they had a search feature also..

let me check on "how to add frame to my photo".

hey its very clear and easy to use.. now let me try the remaining features..search -> Special effects to the photo.

kool my album is ready with all what i have expected.. All Happies!!!!! :D

Here ends the story of User.

While using the Software, the user does not think on who had developed the software and how did he develop it.. he just thinks of "HOW CAN I USE THIS?" to put it in other words "HOW IS THIS SOFTWARE USEFUL TO ME?"

This is the general tendency of the End User.

It might be a simple software like picasa, vlc media player or complex softwares like Java, any Business softwares..the user will just check how userfriendly and useful it is for him or his Business?

Finally the only thing that matters for us is "Customer/EndUser Satisfaction"

Regarding testing of Documentation:

The test engineers should keep in mind the following things:

User Perspective:

• Whether the user who does not have any idea on the product but wants to try it will be able to use the software easily with the documentation help provided?
• Are there any broken links in the documentation? etc..

Product Perspective:

• Did we document all the features that our product has? so that end user can make use of it use it fully.

Documentation should be tested thouroughly, as this also plays a primary role in selling the software/Gaining the user attention.

Migration

Migration Testing:

Migration testing also includes up gradation testing.

After reading the first statement most of us get a doubt on what i mean by Migration / Up gradation?  

Let me explain my view with an example.

Say i have developed a software and named it as SW1

and now after i release it into the market, good that i got few customers for it. Happy Happy!!! :)

Now as my customers started using it.. they even started reporting the issues.. As there it is a fact that there is no bug free Software in this world.

and after giving some fixes to the customer, to make him happy i got another idea. let me now Compile all fixes into a pack and now give it as a fix pack release. This fix pack may again have only the fixes or may include simple feature which helps customer to enhance his business.

and finally now the fix pack say "FP1" is released.

As business keeps on growing up day by day there will be many more new business requirements.

Now gathering all the new business requirements i planned to release another version of my SW1. And now here is SW2 ready for customers.

again Fix packs on SW2...... so on.. the story continues...

Now coming to the testing side of it

There are two aspects:

• Up gradation testing: SW1 -> FP1 ->FP2..
• Migration Testing: (SW1+FP) -> SW2

and coming to migration testing:

There are different categories:

• Runtime content testing
• Design time content testing
• Packaged content testing (This is nothing but the backward compatibility check)

1. Runtime content testing:  Now say you have your working application which has some design time and runtime content. Now immediately after migration you execute your application and check whether it is fine or not. This is Runtime testing.
2. Design time content testing: In the design time testing you check whether your content is migrated properly say if you have some HTML pages or XForms and some business process etc.. whether they are migrated properly or not.

Design time content migration check is equally important with the runtime check. The reason is, business keeps enhancing day by day with lot many new customers, and there comes a day where you need to enhance your business process to handle many more. Its not only in the case of business process, but may happen in any content used. Now to edit it you need to have the design time content migrated properly.

Backward compatibility Testing

Backward Compatibility:

What does we mean by it??

Backward compatibility testing is the kind of testing which ensures software pakages(ISVP's) built on your previous versions work on the latest versions.

Eg: you have version 1 of your software and you have developed some applications on top of it for your customer, which generally is delivered as an Idependent Service Vendor Package . 

Now you have Version 2 of your software.

And if you just want to check if the ISV given to your customer works fine on the newer version. This check/verification is called Backward compatibility check.

This also ensures that the previous build functionality is working fine on the latest release. 

Different Types of Testing for a product

In my view/experience..

Below are the different testing types done during the product validation phase.

Categories may again vary from product to product.

1. Installation Testing
2. Business case execution
3. Backward Compatibility
4. Migration
5. Performance
6. Security
7. Multi byte
8. HTTPS
9. SSO
10. High Availability
11. Documentation
12. Usability testing
13. Stability and Robustness Testing

1. Installation Testing : 

• For every product we have an installation guide. In the installation guide it specifies what kind of environment it supports.
• Now check for the supported environment and make the combinations of it and then decide the best combination which covers many aspects of it.
• Customer way of using it, may also be one of the major influencing factor while choosing the best combination for testing.

For  Example: 

Software might be as simple as Picasa or as complex as some Typical business application Software.

Picasa: Please by going  through the link(http://picasa.google.com/support/bin/answer.py?hl=en&answer=106563) we understand that Picasa can be installed on two OS environments. (Windows and Mac)

In This case the software is only dependent on OS.

But in case of typical Business application software's there might be dependency on the DB Server, Web Server, Driver, Browser etc.. along with OS.

In this case again the Combination list goes on increasing. 

Operating Systems: Windows(2003 Server, XP, Vista, Server 2008.. and all the latest releases), Linux (RHEL, Oracle Enterprise Linux)

Database Server: MS SQL Server (2000, 2005), Oracle(9i, 10g, 11i... latest releases), My SQL (5.0)

Drivers: OLEDB, JDBC

Browsers: IE6, IE7, IE8, Google Chrome, Mozilla FireFox,Apple Safari, SR Ware Iron etc.

While installation it is not that important that the browser should be part of your combination but while testing the software which is browser based, it is very important to test it in all the browsers if the software claims that it is Multi Browser compatible. As the implementation vary from mind to mind.

Sample Combination table:

Sl. No.	Operating System	Database	Connectivity	Web Server
1.	RHEL 4	Oracle 10g	JDBC	Apache 2.2.9
2.	RHEL 4	MySQL 5.0.67	JDBC	Apache 2.0.59
3.	MS Windows Vista SP 1 Business Edition	SQL Server 2005 SP2	SQLNCLI	IIS 7.0
4.	MS Windows XP SP 2	SQL Server 2005 SP2	SQLNCLI	IIS 5.1
5.	MS Windows XP SP 2	SQL Server 2000 SP4	JDBC	Apache 2.2.4
6.	MS Windows XP SP 2	SQL Server 2000 SP4	OLEDB	IIS 5.1
7.	MS Windows 2003 Server SP2	Oracle 10g	OLEDB	IIS 6.0
8.	MS Windows 2003 Server SP2	SQL Server 2005 SP2	SQLNCLI	IIS 6.0

Apart from this the product may also support multiple installations like:

• Primary Distributed
• Master Contributor

These will explained in detailed in the coming posts.

2. Business Case Execution:

 In my view, there are two cases which we can think of:

• Based on the Business requirement(BR) that is gathered from the field, the product is developed, so during the validation of the product it is equally important to validate against the business requirement.
• Apart from just validating the business requirement, developing the scenario making use of multiple Business Requirement's will be an added value to the product as this is the way customer uses it. In general we don't have customers for each Business requirement, but we have them for the product.