Network Dictionary – Bias Testing

This is a generally derogatory term used to describe “independent test results” from well known analyst firms where the tests are not of their own design.

Such tests have usually been devised by the vendor to show their products in the best possible context after extensive internal testing. Each test is carefully selected and proven to be in favour or your product, and then the test plan is sent to an external firm to run those tests.

Of course, the external and independent analyst firm will run these tests and confirm exactly what the vendor intended ie. our wotsit is more gooder than your thingummer. The analyst firm then dutifully reports their “independent testing” and collects a fat fee for their “work”. The vendor gets to say that independent test confirm that our wotsit is gooderer than anyone else’s.

Somewhere in this process, analyst firms don’t seem to regard this as a conflict of interest. A very, very thin line is not crossed because the testing is independent even though the “testing plans” are not.

It can be very difficult for a non-informed consumer to tell what happening. The more cynical industry watchers can pick it from a mile away due to the nature of the tests performed.

As a general rule, independent testing is rarely without bias and should be regarded cynically at all times until proven otherwise. Truly independent testing costs a lot of money to perform and there are not enough clients to pay for it.