Are Translation Tests a Good Idea?

It is common for buyers of translation services to resort to free or paid tests as part of the procurement process to check if the agency is worth its salt. After all, if you wanted to hire a mechanic for your garage, you would first ask them to fix one car and see how they do. Or if you wanted to buy a thousand chairs for your hotel, you’d start by asking the supplier to bring a sample to choose from.

Does it make sense to approach buying translation this way? Here’s our take.

Whenever we have a chance to win a new client and reach the point where they ask for a test translation, we get both excited and apprehensive. Especially, if the translation test has a time limit. Excited, because it means that we have passed the initial evaluation and have been shortlisted. Apprehensive, because we know that we may fail the test.

We worry we might fail the test for many reasons that have little to do with whether or not Andovar would be a good fit for that client.

• We may fail, because the client insisted on very quick turnaround time to “check if we work well under pressure”. That’s all good, but does it really reflect what the cooperation would be like long-term? It doesn’t, because long-term, we would be able to carefully select the best project managers and linguists for the job. In a timed translation test situation, we often have to use the first translator available, even if they are not the most suitable.
• We may fail, because the client doesn’t like the style of our translation. They prefer the test from another vendor, because of choice of words or “feel” of the translation. It’s true that style is a very important aspect of quality translation, but the problem is that the client rarely explains beforehand what style they like. This leads to a situation where one provider will be lucky to match their preference and another won’t.
• We may fail, because the text provided was sent without context or reference materials. This is especially deadly in the case of software strings, which are often short and cryptic. During the course of a normal relationship with a client, we develop translation memories, termbases, style guides, screenshots and other reference materials that help linguists understand what they are translating, but usually none of this is provided with the test.
• We may fail, because instructions from the client were incomplete in other ways. Maybe there are HTML tags or proper names in the source and we don’t know whether they should be amended in any way. If we notice this, our project manager will of course reach out to the client for clarifications, but in the meantime, the clock keeps ticking.

So judge for yourself whether a test translation is a fair way to compare vendors.

Our opinion is that it is not. Such tests don’t follow the same process as a standard project would. They often don’t include reference materials, such as style guides or glossaries, that are essential in getting the translation quality right. When the tests are timed, translation vendors are forced to work with translators who happen to be available to take it immediately, not the most suitable ones.

Finally, I’ll let you in on a little secret.

All agencies want to do well on test translations and they do everything they can to get an advantage. This means they may send the same text to a few different linguists, select the best translation, have it edited, re-edited and double-checked before delivering. This gives them the best shot at winning the client, but is definitely not the process used in a real-world situation. This is the final reason why a test translation is a bad indicator or who’s right and who isn’t for you.

What is a better way to evaluate different translation vendors? That will be explained in an upcoming article!