Plum IVR Support

Hello

I noticed what looked like a gross mismatch when using an XMl grammar.

sensitivity at .65
confindence at .40

Please see the examples below. I am less concerned with mismatches - they happen. I am quite concerned with the confidence however. I would expect a 60% or lower match - not in the nineties.

Please advise !

Thank you


user says "bart oakland"

and gets

try 1 - mismatch, reprompt
try 2 - mismatch, reprompt
try 3 - 91.9% confidence match to "bart concord"

the sound file is here
http://grovestreet.net/pv-bad-matches/bart-oakland.wav

another such mismatch is

user says "bart powell"

and got this matched immediately to "bart colma" at 97.6% confidence

the sound file is here
http://grovestreet.net/pv-bad-matches/bart-powell.wav

Hi,

We don't generally recommend setting a confidence level below 0.5, as our platform is finely tuned to minimize the number of false positives above 0.5. You might want to try raising your confidence level, as using a high confidence level is useful for when you are expecting a precise grammar match. You could also try raising your sensitivity level.

Hope this helps!

Regards,
Plum Support

Sorry that suggestion is not helpful.

The confidence level only determines at what %confidence PV considers an utterance to have a match or not.

My point is that the attached files show that PV determined a 90+% confidence. Even if I raise the confidence as you suggest, the problem remains the same: PV is matching the wrong thing at a very high confidence.

With these kind of mismatches I can't take my application to production. Could an engineer please take a look and advise on what can be done. I am a paying customer and need your help

s

Hi,

About your speech recognition concerns, there are going to be concerns since your grammar contains 3 similar sounding matches (and they all contain "bart"). Also, you have to consider that as humans have trouble recognizing speech (bart colma (has a long "O" sound), bart oakland (also has a long "O" sound)), an IVR speech recognition engine will also have issues.

If you want to ensure that the speech recognition is correct, you could add a confirmation prompt in your application such as "You said _______________, is that correct? Press 1 for yes or 2 for no."

If you need additional help, please feel free to post your code here and we could make some suggestions to assist you.

Regards,
Plum Support

Hi

I am aware that recognition can't be perfect.

But i would expect that IVR can recognize a material difference between

"Colma"

"Oakland"

the consonants are very different. If we go only by vowels, well. IVR wouldn't function.

Again, I would expect there to be a confidence in the 60s, based on your points, but getting, in the same call, first 2 no-matches (meaning confidence was below 40 in my case), and then a 97% match to "bart colma" when uttering "bart oakland" seems very suspect.

On the IVR engine side there have got to be methods of tuning recognition. At what point is such tuning available to customers (I realize we have a low volume account)?

Hi,

Just to clarify, when a grammar is matched, this is based on the performance of the speech recognition engine (Nuance), not the IVR. So, there is no "tuning" behavior available for the speech recognition (the only attributes you can adjust are "confidencelevel" and "sensitivity").

Also, when using speech recognition, you have to be careful about the grammar you implement. For example, speech recognition works pretty well for a grammar containing the 50 states of America (since none of them sound similar). However, since your grammar contains "Bart" for each of the utterances that the user can say, it could lead to false positives for the speech recognition engine (as you have seen with your confidence and match levels). Just as a suggestion, you can try removing the "Bart"s from your grammar and test it for just "Colma" and "Oakland".

Finally, as you've already mentioned, speech recognition is not perfect and should be taken into account for your application. You can set up a confirmation prompt within your application to state the following, "You said _________. Is this correct? Press 1 for yes or 2 for no." This will ensure that you get the correct input from the user.

Regards,
Plum Support