If a speech engine can not recognize X finity or C omcast and those are important terms for your use case, the other accuracy is irrelevant.

This is important to measure in addition to the WER and Word Accuracy as the most important words to get transcribed correct are the terms you need to spot or search for. These are standard measures used in information retrieval science. Recall is % of the words you are looking for that were found (so 80% means we found 8 out of 10 were found and 2 were missed). Precision is the % of the hits we found actually were valid hits (so 90% means 9 out of results in the list were true and one was a false positive). Basically you will use this method to compare a machine transcription from each speech engine to a perfect human transcription of that file.įor keyword spotting accuracy, which is important to measure since that is what many people use transcription for, you should be using precision and recall. The algorithm used is called the Levenshtein distance, it is calculated by aligning the reference with hypothesis and counting the words that are Insertions, Deletions and Substitutions (and correct). ‘WER’ ( Word Error Rate) and ‘Word Accuracy’ are the best measurements to take when comparing two accuracies, these are typically values in % and are derived by comparing a reference transcript with the ASR transcript (or hypothesis) for the audio. Every use case is different, but keep in mind the features that will matter most to you when starting this process. Generally, phonetic-based engines tend to only be used in very niche use cases, while text/large-vocabulary based engines tend to be more advanced.

To learn more about the difference of these two methods, check out our previous blog Phonetic vs. Phonetic based speech engines are built with a smaller grammar set and use phonemes as the basis for recognition and search, while fixed vocabulary engines are built with a larger, fixed, predefined vocabulary. The first thing to determine when comparing the accuracy of speech engines is the type you are assessing. There are two main methods to building speech recognition software phonetic-based and text-based / fixed vocabulary speech engines.

Speech technology is a game of ‘what is most likely to have been said here’ and the winner is the speech engine that can predict the results most accurately.

Compare speech to text software