This is the third post in the series on how to choose a localization vendor. I started with pointing out the shortcomings of translation tests, then suggested some alternatives, and today I would like to give some pointers on how a translation test can be improved, so that it serves its purpose better.
Translation tests can be helpful in evaluating translation providers, but only if they’re done well. Otherwise, too much is left to chance and you will learn little about the provider and how they would approach a long-term cooperation with you. In a way, this list is a reverse of the one from the first post in this series, since that one listed the problems with tests.
- Choose the test sample carefully. Most vendors will be happy to translate one A4 page worth of text - that’s about 300-350 words - free of charge. If they see a good opportunity, they may do more than that. Choose text that is both challenging and typical of your content. Challenging, because you want to give the vendors a chance to shine. Typical, because you want someone who can handle what you will actually need translated later. An option worth considering is using text that has already been translated previously, as this will make the job of evaluation much easier. Just make sure that the translation is not available somewhere online or the LSPs will find it! At least the smart ones will.
- Provide reference. If you have existing termbases, translation memories or style guides, include them with the sample text. Most professional translation work in the world is done with these assets available and any LSP worth its salt will recommend using them to you, so why wouldn’t you want to see how they manage them in the test? If you don’t have any of these, at least provide context of the sample, target audience, screenshots, URLs of websites with more information and anything else that will make it easier for linguists to understand what they’re looking at and what to do with it.
- Timing. While insisting on unreasonable turnaround time is pointless, I do recommend setting clear deadlines. You should let the LSPs know in advance what day and what time they will receive the test bundle and by when they should return it. Without this constraint, you may end up comparing good translation from someone who did it quickly and excellent polished translation from someone who took forever. Which one is better? Apples to oranges.
- Be transparent. Let all shortlisted vendors know what criteria matter to you in the procurement process. How much will you value the test, as compared to other factors, like industry experience or client references? Will you give extra points to those who deliver quicker or those who provide three translations instead of one? Additionally, if one of the LSPs asked an important question, share your answer with others involved as well. The race is only fair if the rules are clear to everyone.
- Get serious about review. If you expect the vendors to do their best on the tests, you should also do your best on evaluation. Make sure you have native-speaking reviewers and you brief them on how they should evaluate the tests. It is the nature of translation that everyone will have comments and corrections, but you need to be clear what constitutes an error in translation and what’s merely a stylistic preference. If it’s the latter, was it included in a style sheet beforehand? And if it wasn’t, is it fair to reward or penalize for it?
- Shortlist. Before you invite LSPs to a test, make sure you’ve only shortlisted the ones that meet your other criteria, as explained in the last post. It may make little difference if you’re sending the same test to three or ten providers, but evaluation of the results does take time if you want to do it well.
- The value of free. Consider paying a nominal fee to each provider taking part in the test. It’s true that you don’t have to, and you can argue that it is up to the vendors’ cost of opportunity. However, if there is even a small payment involved, all parties will take things much more seriously. Your side will be more careful in preparing and evaluating the test, so your money doesn’t go to waste, and the LSPs will be able to commit more resources and provide better outcome.
This was the last in this three-part series on selecting a localization vendor. I hope it was useful and will help everyone make better choices!
