Unexpected result returned from grammar

[dwollberg]

I am testing Spanish speech recognition, and have run into an issue. This is the code I am using:

Code: Select all

<?xml version="1.0" encoding="utf-8"?>
<vxml version="2.1" xml:lang="es">
	<form>
		<field name="keyword">
			<grammar type="application/srgs+xml" root="KEYWORDEN" mode="voice" xml:lang="en">
				<rule id="KEYWORDEN" scope="public">
					<tag>lang='en';</tag>
					<one-of>
						<item>baseball<tag>id='1';</tag></item>
						<item>burrito<tag>id='4';</tag></item>
						<item>llama<tag>id='5';</tag></item>
					</one-of>
				</rule>
			</grammar>
			<grammar type="application/srgs+xml" root="KEYWORDES" mode="voice" xml:lang="es">
				<rule id="KEYWORDES" scope="public">
					<tag>lang='es';</tag>
					<one-of>
						<item>manzana<tag>id='manzana';</tag></item>
						<item>burrito<tag>id='burrito';</tag></item>
						<item>llama<tag>id='llama';</tag></item>
						<item>lámpara<tag>id='lámpara';</tag></item>
					</one-of>
				</rule>
			</grammar>
			<prompt>
				diga la palabra clave.
			</prompt>
			<nomatch>
				<log>nomatch</log>
				<prompt>No entiendo.</prompt>
				<clear/>
			</nomatch>
			<filled>
				<log>match</log>
				<log>lang: <value expr="keyword.lang"/></log>
				<foreach item="result" array="application.lastresult$">
					<log>confidence: <value expr="result.confidence"/></log>
					<log>utterance: <value expr="result.utterance"/></log>
					<log>interpretation: <value expr="result.interpretation"/></log>
				</foreach>
				<prompt>tu dijiste <value expr="keyword.id"/></prompt>
				<clear/>
			</filled>
		</field>
	</form>
</vxml>

When I say "lámpara", it matches me, but it doesn't return all of the tags properly. I get the "id=lámpara" part, but not the "lang=es" part. When I say any of the other options, I get both. This happens because of the accent mark "á" - any time I try to return something in the tag with an accent mark, I get the same result. Here are my results from the logs:

LOCAL: confidence: 0.79
LOCAL: utterance: lámpar
LOCAL: interpretation: {id:lámpara lang:es
LOCAL: confidence: 0.03
LOCAL: utterance: llama
LOCAL: interpretation: {id:5 lang:en}

Notice that there is a missing "}" on the "lámpara" interpretation line. The results of this are that lang:es is ignored.

Is there some way I should be escaping this text, or encoding it differently to get it to return the expected results?

[support]

Hi,

Thank you for reaching out.
When we run your code, we see this returned in the log:

Code: Select all

LOCAL: confidence: 0.783
LOCAL: utterance: llama
LOCAL: interpretation: {id:llama lang:es}
LOCAL: confidence: 0.165
LOCAL: utterance: lámpara
LOCAL: interpretation: {id:lámpara lang:es}

We have not been able to recreate the missing "}" on the "lámpara" interpretation line.
To help us further our investigation can you provide exact steps to recreate and which TTS Engine you are using?

Regards,
Plum Support

[dwollberg]

We are using the AT&T Natural Voices 1.4, but I have also tried Realspeak 4.0, I believe. We plan on using Nuance Vocalizer when that is released.

I have this as an XML file in Plum File Repository for my account under /testgrammars/SpanishEnglishTest.xml. I have it saved as UTF-8, as that sounded like a solution for other's issues here in the forum.

My steps to recreate were simply to call in and say "lámpara". Not only did it produce the results I mentioned in the OP, but I also got a:

Code: Select all

LOCAL: lang: 

without an "es" for the line:

Code: Select all

<log>lang: <value expr="keyword.lang"/></log>

[support]

Hi,

We were able to reproduce this issue with the ASR Engine set to Nuance Recognizer 9.
We suggest changing your application configuration so the ASR Engine is Nuance OSR 3.0.
Please test and let us know if this resolves the issue.

Regards,
Plum Support

[dwollberg]

Thanks, switching to OSR fixed that issue. However, I cannot get OSR to properly recognize my Spanish utterances. "Lámpara" returns "llama" every time. "Manzana" returns "llama" as well.

Do you have any more information on the differences between these engines, with regards to accuracy and multilingual support?

[support]

Hi,

Can you provide samples of utterance not being properly recognized?
We have been unable to recreate this behavior.

There shouldn't be any major differences between the ASR Engines with regards to accuracy and multilingual support.
We have also noticed that our documentation on these engines is lacking.
We will be sure to work on this so you can be properly informed about the software you are using.

Regards,
Plum Support

[dwollberg]

Sent a PM with instructions on testing.

I recently noticed that I can actually have two grammars of different languages active at the same time, and this worked with Recognizer. Is this a supported scenario?

[dwollberg]

I see that your test was successful. We did more testing from different phones, with different people, and different pronunciations. We have received mixed results, even when speaking very similarly between calls on the same device. From our desk phones, it seems to work pretty consistently. From our cell phones, results were mixed, but I did get it to work a couple times.

[support]

Hi,

To answer your first question, yes, you can have multiple grammars of different languages active at the same time.

In regards to the testing, we have conducted multiple tests and experienced similar results. The outcome is dependent on background noise, accent of the speaker, and the phone being used. Unfortunately this is a limitation of the ASR Engines, as it relies heavily on the user input and environment.

Please reach back out to us if you have any further questions.

Regards,
Plum Support