Spanish voice recognition problem

[shanthint@paydq.com]

Hello,
We have purchased a spanish voice 'Paulina' recently.
She is working fine. the problem we have is our engine doesn't understand voice input in spanish.
I have grammer for both English and spanish voice input.
English is working fine but not spanish

here I provide the code snippet of my grammer/

<grammar type="application/srgs+xml" root="ROOT" mode="voice">
<rule id="ROOT">
<one-of>
<item>yes</item>
<item>YES</item>
<item>si</item>
<item>ci</item>
<item>see</item>
<item>sea</item>
<item>yeah</item>
<item>sure</item>
</one-of>
</rule>
</grammar>

ANother thing is date in spanish. for example 11/12/2008 it says 11 12 2008 instead of Nov 12 the 2008.

Please let me know how can I make it work.

Thank you.

[support]

Hi,

The reason why the date is not working is because you have to use this format for the date for Spanish:

day/month/year (DD/MM/YYYY)

Here is an IVR code example that should help you get your IVR grammar working. Note how the date is set up in the <filled>
block.

spanishcode.php

Code: Select all

<?php
header("Content-type: text/xml");
echo("<?xml version=\"1.0\"?>\n");
?>
<vxml version="2.0">
<form>
<field name="answer">
<grammar type="application/srgs+xml" root="ROOT" mode="voice">
<rule id="ROOT">
<one-of>
<item>sí<tag>answer='sí'</tag></item>
<item>si<tag>answer='sí'</tag></item>
<item>ci<tag>answer='sí'</tag></item>
<item>see<tag>answer='sí'</tag></item>
</one-of>
</rule>
</grammar>
<prompt>
<speak xml:lang="es-MX"><voice name="Paulina" gender="female">
¿Hacen usted tienen gusto de los huevos?
</voice></speak>
</prompt>

<filled>
<prompt>
<speak xml:lang="es-MX"><voice name="Paulina" gender="female">
 Usted dijo <value expr="answer"/>. La fecha está hoy <say-as type="date">31/10/2008</say-as>.
</voice></speak>
</prompt>
</filled>
</field>
</form>
</vxml>

Hope this helps.

Regards,
Plum Support

[shanthint@paydq.com]

Hello,
Thank you for the help. I fixed the date issue.
And the Grammer you gave worked with little changes.
But Still our engine has problem with recognizing the voice input.
If I talk loud, them it takes the input, otherwise it catches noinput event.
Is there any way I could fix this?

Thank you,

[support]

Hi,

About your noinput event, can you tell us which type of phone line you are using that could be generating this?

Also, the language that you wanted recognized by your IVR grammar should also be specified by using the "xml:lang" attribute.

Here's what the updated IVR code should look like:

Code: Select all

<?php
header("Content-type: text/xml");
echo("<?xml version=\"1.0\"?>\n");
?>
<vxml version="2.0">
<form>
<field name="answer">
<grammar type="application/srgs+xml" root="ROOT" mode="voice" xml:lang="es-MX">
<rule id="ROOT">
<one-of>
<item>sí<tag>answer='sí'</tag></item>
<item>si<tag>answer='sí'</tag></item>
<item>ci<tag>answer='sí'</tag></item>
</one-of>
</rule>
</grammar>
<prompt>
<speak xml:lang="es-MX"><voice name="Paulina" gender="female">
¿Hacen usted tienen gusto de los huevos?
</voice></speak>
</prompt>

<filled>
<prompt>
<speak xml:lang="es-MX"><voice name="Paulina" gender="female">
 Usted dijo <value expr="answer"/>. La fecha está hoy <say-as type="date">31/10/2008</say-as>.
</voice></speak>
</prompt>
</filled>
</field>
</form>
</vxml> 

Regards,
Plum Support

[shanthint@paydq.com]

hello,

We are using Single Line (digital) type of phone line.
Even after added the language attibute, still it is not recognizing the voice input clearly atleast not at the first time.


Thank you.

[support]

Hi,

Could you PM (private message) us the phone number that you are dialing into to test this IVR application? This would help us better determine what's going on with this IVR issue.

Regards,
Plum Support

[shanthint@paydq.com]

Hello,
I sent PM to you with our Customer ID - Spanish Voice recognition Issue.

Thanks for the help.

[shanthint@paydq.com]

Hello,
This issue seems to be long time issue. Right?
Anyway, I tried lot of ways to make this work.
I wasn't successful at all.Here I am going to give you what are the things I tried. Please instruct me any other ways to make this good.

1. If we raise the sensitivity to 0.6 or 0.7, both English and Spanish versions are taking anything voice input even if it is not specified in the grammar.
For example, it is taking 'no' as 'si' (In our application, we don't want to process the request if the input is 'NO') and also for Spanish it throws noinput or nomatch event or after two trials, it gets the input.

2. If we don't raise the sensitivity, it works fine with English voice recognition but Spanish has really hard time to get through.

3. I tried the example code you provided with little changes Instead of this, <item>sí<tag>answer='sí'</tag></item>
<item>si<tag>answer='sí'</tag></item>
<item>ci<tag>answer='sí'</tag></item>
<item>see<tag>answer='sí'</tag></item>
I used the below code because when I validate the above, parser gives me error "Message: Expected end of tag 'tag'"
<item>sí<tag>answer='sí'</tag></item>
<item>si<tag>answer='si'</tag></item>
<item>ci<tag>answer='ci'</tag></item>
<item>see<tag>answer='see'</tag></item>

It doesn't work at all. It is not recognizing the input not even once.

4. I tried the grammer with and without xml:lang="es-MX". It didn't make any improvement.

Now I have a question about the voice recognition engine. Do we have to have separate voice recognition engine for spanish?
Because if I use the lang attirbute, it is really taking very long time to recognize the voice input only if it is in English like 'si' ,'c' 'see'.
if I try 'sí', it doesn't take the input at all.

Please advise me any other solutions. It is really urgent.

Thank you.

[shanthint@paydq.com]

One more Thing:

if I use xml:lang="es-MX" , it takes long time to recognize the input. atlease 10 seconds. for any knid of input 'no' or 'si'.

Thank you.

[support]

Hi,

1) You could try adding more words into your IVR grammar and adding an <if> clause within your <block> block to get better accuracy. Also, you could try setting a higher confidence level of 0.75.

2) You should make sure to keep your English grammar and Spanish grammar separate from each other to ensure that the IVR grammar engine understands which grammar it is dealing with. This could be why it is easier for it to understand the English voice input and not the Spanish voice input.

3) For the example IVR code we provided for you, the IVR code was validated. It seems a bit strange that you received a "Message: Expected end of tag 'tag'" message.

4) By specifying the xml:lang within the grammar, it helps the grammar engine better understand the language it is dealing with (since English is the default).

We recommend that you test the following IVR code first and see if you are still experiencing the same issues:

englishspanish.vxml

Code: Select all

<?xml version="1.0"?>
<vxml version="2.0">

<property name="incompletetimeout" value="2s"/>
<property name="recordcall" value="true"/>
<property name="sensitivity" value="0.6"/>
<property name="interdigittimeout" value="300ms"/>
<property name="timeout" value="5s"/>
<property name="confidencelevel" value="0.75"/>
<!--property name="inputmodes" value="dtmf voice"/-->

<form>
<field name="introduction">
<grammar type="application/srgs+xml" root="ROOT" mode="dtmf">
<rule id="ROOT">
<one-of>
<item>1</item>
<item>2</item>
</one-of>
</rule>
</grammar>
<prompt>
Hello. Welcome! For English, press one.
</prompt>
<prompt>
<speak xml:lang="es-MX"><voice name="Paulina" gender="female">
Para el español, prensa dos.
</voice></speak>
</prompt>
<filled>
<if cond="introduction==1">
  <goto next="#english"/>
  <else/>
    <goto next="#spanish"/>
</if>
</filled>
</field>
</form>

<form id="english">
<field name="answer">
<grammar type="application/srgs+xml" root="ROOT1" mode="voice" xml:lang="en-US">
<rule id="ROOT1">
<one-of>
<item>yes<tag>answer='yes'</tag></item>
<item>yeah<tag>answer='yes'</tag></item>
<item>okay<tag>answer='yes'</tag></item>
<item>no<tag>answer='no'</tag></item>
<item>nope<tag>answer='no'</tag></item>
<item>nuh uh<tag>answer='no'</tag></item>
</one-of>
</rule>
</grammar>

<prompt>
  Would you like this to be processed?
</prompt>

<filled>
<if cond="answer=='yes'">
  <prompt>
    You said <value expr="answer"/>. This will be processed.
  </prompt>
  <else/>
    <prompt>
      You said <value expr="answer"/>. This will not be processed.
    </prompt>
</if>
</filled>
</field>
</form>

<form id="spanish">
<field name="answer">
<grammar type="application/srgs+xml" root="ROOT2" mode="voice" xml:lang="es-MX">
<rule id="ROOT2">
<one-of>
<item>sí<tag>answer='sí'</tag></item>
<item>si<tag>answer='sí'</tag></item>
<item>ci<tag>answer='sí'</tag></item>
<item>no<tag>answer='no'</tag></item>
<item>non<tag>answer='no'</tag></item>
</one-of>
</rule>
</grammar>

<prompt>
  <speak xml:lang="es-MX"><voice name="Paulina" gender="female">
    ¿Usted tienen gusto de esto para ser procesados?
  </voice></speak>
</prompt>

<filled>
<if cond="answer=='sí'">
  <prompt>
    <speak xml:lang="es-MX"><voice name="Paulina" gender="female">
      Usted dijo <value expr="answer"/>. Esto será procesada.
    </voice></speak>
  </prompt>
  <else/>
    <prompt>
      <speak xml:lang="es-MX"><voice name="Paulina" gender="female">
        Usted dijo <value expr="answer"/>. Esto no será procesada.
      </voice></speak>
    </prompt>
</if>
</filled>
</field>
</form>
</vxml>

If you are still experiencing these problems, please provide us with the vxml script that you used to test this and any changes that you might have made to it.

Hope this helps.

Regards,
Plum Support

[shanthint@paydq.com]

Hello,

The example you gave is not working either. If use 'sí', the engine throws 'noinput' or nomatch error. So I am using 'si' instead of sí.


Now I totally separate the English and spanish input as two different files. Even though, I still have the same problem. When I tried all these differernt ways, it seems our plum server got confused and started to disconnect the call when it comes to the point where it gets the voice input. So we restarted the server. Now our production applications works fine in English.

But I still have the voice recognition problem.
Another issue is, if statement.
The below code works weird.
<if cond="confirm$.inputmode=='voice'">
<if cond="confirm=='si'"> -->
<prompt>
<audio expr="'recordings/'+language+'/global/messages/msg_process_wait.wav'" >
</audio>
</prompt>
<goto next="#achpayment"/>
<else/>
<goto next="RemoveAccountInfoAction.do?dest=menu"/>
</if>
<else/>
<goto next="RemoveAccountInfoAction.do?dest=menu"/>
</if>
even when it gets the input 'si', the if statement is going to the else part.
Here I am using if inside another if.
That's because, we are getting 'yes' as voice input to process the request.
Press 2 dtmf input to cancel the request.

If you guys want to test in our plum server, please let me know.

Thank you,

[shanthint@paydq.com]

Hi,
I forgot to include the vxml script. If you want the entire vxml, I will PM to you.
Please let me know.

Thank you.

[support]

Hi,

Please test the following IVR code to see if Spanish ASR is properly working for your onsite server. It simply tests an IVR boolean built-in grammar in a <field>.

spanishasrtest.vxml

Code: Select all

<?php
<?xml version='1.0'?>
<vxml version="2.0" xml:lang="es-MX">

<property name="incompletetimeout" value="2s"/>
<property name="recordcall" value="true"/>
<property name="sensitivity" value="0.5"/>
<property name="interdigittimeout" value="300ms"/>
<property name="timeout" value="5s"/>
<property name="confidencelevel" value="0.75"/>
<!--property name="inputmodes" value="dtmf voice"/-->

<form id="spanish">
<field name="answer" type="boolean">

<prompt>
  <speak xml:lang="es-MX"><voice name="Paulina" gender="female">
    Si or no?
  </voice></speak>
</prompt>

<filled>
  <prompt>
    <speak xml:lang="es-MX"><voice name="Paulina" gender="female">
      Usted dijo <value expr="answer"/>.
    </voice></speak>
  </prompt>
</filled>
</field>
</form>
</vxml>

From this IVR code, if you say "si", you should see, "Usted dijo 1" in your IVR call log. If you say "no", you should see, "Usted dijo 0" in your IVR call log.

Please let us know the results of this test.

Regards,
Plum Support

[shanthint@paydq.com]

Hi,
I got the following result when I tried the code

for No:

first try:
received event: noinput
second try:
Found grammar match
hypothesis #0: false (0.8486)
hypothesis #1: false (0.4070)
hypothesis #2: true (0.2738)
bargein set to true
INPUTMODES set to "DTMF VOICE"
Audio segment added to prompt queue from TTS application/synthesis+ssml for:
---------
<?xml version='1.0'?><speak xml:lang="es-MX"><speak xml:lang="es-MX"><voice gender="female" name="Paulina"><speak>
Usted dijo 0 .
</speak></voice></speak></speak>


For 'Si':
I included the entire Log here

Call Start Event: ANII DNIS 8217 VURL /usr/local/plumvp/config/director.vxml
DocumentParser::FetchDocument()
DocumentParser::FetchDocument(/usr/local/plumvp/config/director.vxml)
Attempting to fetch file:///usr/local/plumvp/config/director.vxml
Click here to view saved VoiceXML script
VXI::var_element(name="ani" expr = "session.telephone.ani")
VXI::var_element(name="dnis" expr = "session.telephone.dnis")
VXI::var_element(name="id" expr = "session.id")
VXI::var_element(name="callee_type" expr = "session.telephone.callee_type")
file:///usr/local/plumvp/config/1225396537_2060460469.vxml
DocumentParser::FetchDocument(file:///usr/local/plumvp/config/1225396537_2060460469.vxml)
Attempting to fetch file:///usr/local/plumvp/config/1225396537_2060460469.vxml
Click here to view saved VoiceXML script
Loading Builtin grammar: builtin:grammar/boolean
Loading Builtin grammar: builtin:dtmf/boolean
VXI::queue_prompts()
bargein set to true
INPUTMODES set to "DTMF VOICE"
Audio segment added to prompt queue from TTS application/synthesis+ssml for:
---------
<?xml version='1.0'?><speak xml:lang="es-MX"><speak xml:lang="es-MX"><voice gender="female" name="Paulina"><speak>
Si or no?
</speak></voice></speak></speak>
---------
VXI::field_element - activating grammars for form = 'spanish' formitem = 'answer'
VXI::do_recognition()
PromptManager::Play()
Newly queued prompts are now being played


Wed 10 Dec 2008 02:44:12 PM EST:

received event: noinput
VXI::queue_prompts()
bargein set to true
INPUTMODES set to "DTMF VOICE"
Audio segment added to prompt queue from TTS application/synthesis+ssml for:
---------
<?xml version='1.0'?><speak xml:lang="es-MX"><speak xml:lang="es-MX"><voice gender="female" name="Paulina"><speak>
Si or no?
</speak></voice></speak></speak>
---------
VXI::field_element - activating grammars for form = 'spanish' formitem = 'answer'
VXI::do_recognition()
PromptManager::Play()
Newly queued prompts are now being played


Wed 10 Dec 2008 02:44:18 PM EST:

received event: nomatch
bargein set to true
INPUTMODES set to "DTMF VOICE"
Audio segment added to prompt queue from TTS text/plain;charset=wchar_t for:
---------
Sorry, I didn't understand you.
---------
VXI::queue_prompts()
bargein set to true
INPUTMODES set to "DTMF VOICE"
Audio segment added to prompt queue from TTS application/synthesis+ssml for:
---------
<?xml version='1.0'?><speak xml:lang="es-MX"><speak xml:lang="es-MX"><voice gender="female" name="Paulina"><speak>
Si or no?
</speak></voice></speak></speak>
---------
VXI::field_element - activating grammars for form = 'spanish' formitem = 'answer'
VXI::do_recognition()
PromptManager::Play()
Newly queued prompts are now being played


Wed 10 Dec 2008 02:44:26 PM EST:

received event: nomatch
bargein set to true
INPUTMODES set to "DTMF VOICE"
Audio segment added to prompt queue from TTS text/plain;charset=wchar_t for:
---------
I still don't understand.
---------
VXI::queue_prompts()
bargein set to true
INPUTMODES set to "DTMF VOICE"
Audio segment added to prompt queue from TTS application/synthesis+ssml for:
---------
<?xml version='1.0'?><speak xml:lang="es-MX"><speak xml:lang="es-MX"><voice gender="female" name="Paulina"><speak>
Si or no?
</speak></voice></speak></speak>
---------
VXI::field_element - activating grammars for form = 'spanish' formitem = 'answer'
VXI::do_recognition()
PromptManager::Play()
Newly queued prompts are now being played


Wed 10 Dec 2008 02:44:34 PM EST:

received event: noinput
bargein set to true
INPUTMODES set to "DTMF VOICE"
Audio segment added to prompt queue from TTS text/plain;charset=wchar_t for:
---------
Sorry, I didn't hear you.
---------
VXI::queue_prompts()
bargein set to true
INPUTMODES set to "DTMF VOICE"
Audio segment added to prompt queue from TTS application/synthesis+ssml for:
---------
<?xml version='1.0'?><speak xml:lang="es-MX"><speak xml:lang="es-MX"><voice gender="female" name="Paulina"><speak>
Si or no?
</speak></voice></speak></speak>
---------
VXI::field_element - activating grammars for form = 'spanish' formitem = 'answer'
VXI::do_recognition()
PromptManager::Play()
Newly queued prompts are now being played


Wed 10 Dec 2008 02:44:42 PM EST:

received event: nomatch
bargein set to true
INPUTMODES set to "DTMF VOICE"
Audio segment added to prompt queue from TTS text/plain;charset=wchar_t for:
---------
Please say it one last time.
---------
VXI::queue_prompts()
bargein set to true
INPUTMODES set to "DTMF VOICE"
Audio segment added to prompt queue from TTS application/synthesis+ssml for:
---------
<?xml version='1.0'?><speak xml:lang="es-MX"><speak xml:lang="es-MX"><voice gender="female" name="Paulina"><speak>
Si or no?
</speak></voice></speak></speak>
---------
VXI::field_element - activating grammars for form = 'spanish' formitem = 'answer'
VXI::do_recognition()
PromptManager::Play()
Newly queued prompts are now being played


Wed 10 Dec 2008 02:44:49 PM EST:

received event: nomatch
bargein set to true
INPUTMODES set to "DTMF VOICE"
Audio segment added to prompt queue from TTS text/plain;charset=wchar_t for:
---------
I'm sorry. I just can't understand your response.
---------
VXI::exit_element()
Newly queued prompts are now being played


Wed 10 Dec 2008 02:44:53 PM EST:

Call End Event
Ending session
Ending Session On Channel 10

Please let me know whether our engine works fine or not for Spanish.

Thank you for you help.

[support]

Try raising the sensitivity value to 0.6 or 0.7 in your IVR code and please try testing again for "si".

Regards,
Plum Support