`

5 steps to customize an ASR with VDK’s Grammar Editor Plugin

Favicon Vivoka Author

Written by Nathan Janezcko

Nathan Janezcko is Technical Project Manager at Vivoka. He handles product's development as well as customer projects with the R&D and technical team.

11 May 2021

 

You wish to fully customize your own ASR model? You want it to be able to understand words that even yourself have difficulties ton get? Stay with us, today we are going to learn how to create an automatic speech recognition with the Grammar Editor plugin which is available in the Voice Development Kit.

Quick reminder, the Voice Development Kit (VDK) is a toolkit specialized in offline speech technologies. It allows companies to easily and quickly integrate automatic speech recognition, language processing and voice synthesis into their applications and devices.

 

1) Start by creating your new project with VDK

 

Choosing the type of project you want to build

 

Project Wizard Voice Development Kit

 

To begin this tutorial, we will first start with the creation of the project. With the help of the wizard, you need to create a custom application that will allow you to use the Grammar Editor.

 

Setting the voice project information and languages

 

 

For the creation of the project, you will be asked for 3 pieces of information. You will have to set up the project with a name, a directory and the languages you need.

More than 50 languages are available with the Voice Development Kit. You can even create multilingual voice assistants.

 

Define the technologies to be developed and integrated (ASR in order to customize speech recognition grammar)

 

 

In this new step, you have to choose between 4 available technologies: Wake up Word (WUW), Automatic Speech Recognition (ASR), Natural Language Understand (NLU) or Text to Speech (TTS).

For this case, we will need to use ASR to work on grammar.

 

2) Discovering Speech Recognition Grammar Edition Plugin

 

Speech Recognition Grammar Editor VDK

 

We will now proceed to the grammar edition this step will take place in 5 steps.

  1. Click on the file to open the Grammar Editor
  2. Select the language of your grammar (mostly if you work on a multilingual grammar)
  3. Write your customized grammar here
  4. Save and Compile to use your grammar
  5. Click on “Test” and experiment your grammar with your microphone with pre-recorded audio files

 

Some element useful to write a grammar:

Every lines should end with a semicolon ;

<x> -> Rule name

[x] -> Optional value

| -> “or”

!function(X, Y)

Useful functions:

  • !tag(X, Y); tags (X) words (Y) to make the interpretation of result way easier
  • !repeat(X, Y, Z); Repeat the words (X) at least Y times up to Z times
  • !pronounce “X” PRONAS “Y”; Replace the pronunciation of a word (X) to the pronunciation of another word (Y). You can put | PRONAS “Z” to add various pronunciations.

 

3) Speech Recognition Grammar Edition : How-to

 

<main>: I want a pizza;

Now your assistant will be able to detect an order of a pizza.

Let’s make this assistant smarter.

 


 

<main>: I want a [pizza] !tag(PIZZA_TYPE, <pizza>);

<pizza>: margherita | proscuiutto e funghi | capricciosa | vegetariana | calzone;

And now my assistant allows me to select a type of pizza from a predefined list.

 


 

To improve the detection of the pizza name when pronouncing the american way or the italian way we added the !pronounce function

!pronounce “capricciosa” PRONAS “caprikiosa” | PRONAS “caprichioza”;

This allow us to setup more expected pronunciation to a word

 


 

We improve even more our grammar to allow different verbs of action and the possibility to order a number of pizza

<main>: <verb> (a | <number>) [pizza] <pizza>;

<verb>: I (want | would like);

<number>: !tag(NUMBER, 1 | 2 | 3);

<pizza>: [pizza] !tag(PIZZA_TYPE, margherita | proscuiutto e funghi | capricciosa | vegetariana | calzone);

We also moved the !tag function to keep the <main> rule lisibility

 


 

Let’s implement the repeat function so our assistant can take the order for a whole group of person at once

<main>: <verb> !repeat((a | <number>) <pizza> [and [<verb>]], 1, *);

The !repeat function allows the repetition of a segment of our sentence.

As you can see, we added [and [<verb>]] at the end of the bracket. This segment is totally optional but allows more interaction with the assistant.

Examples of valid sentence:

  • I want a margherita
  • I would like a pizza capricciosa and 2 vegetariana
  • I want a margherita, 2 capricciosa and I would like a calzone

 


 

With all that food, some drinks would be appreciate

<main>: <verb> !repeat((a | <number>) (<pizza> | <drinks>) [and [<verb>]], 1, *);

<drinks>: [!tag(DRINK_FORMAT, glass | bottle) of] !tag(DRINK_TYPE, water | coca | wine | beer);

 


 

And finally, let’s add another action to our assistant, to make it more complete. I want to be able to request the remaining time for my order to be ready.

I create a new rule <time_left> and move the content of <main> to a new rule <order>

 

The finished and customized speech recognition grammar

 

#BNF+EM V2.1;

!grammar ASRENG-US;

!start <main>;

!pronounce “capricciosa” PRONAS “caprikiosa” | PRONAS “caprichioza”;

<main>: <order> | <time_left>;

<time_left>: How (much | many) time (left | remaining) [for (our | my) order];

<order>: <verb> !repeat((a | <number>) (<pizza> | <drinks>) [(and | with) [<verb>]], 1, *);

<verb>: I (want | would like);

<number>: !tag(NUMBER, 1 | 2 | 3);

<pizza>: [pizza] !tag(PIZZA_TYPE, margherita | proscuiutto e funghi | capricciosa | vegetariana | calzone);

<drinks>: [!tag(DRINK_FORMAT, glass | bottle) of] !tag(DRINK_TYPE, water | coca | wine | beer);

 

Now let’s make an order:

“I want a crapricciosa and a bottle of water.”

Since we put tag, this result is really easy to interpret.

This was a short example showing you one of the ways to think a grammar. We recommend iterative work, as it usually allows to cover a maximum of use cases very easily.

 

Thank you for reading this article. We hope it will help you find cool ideas for your projects and have fun building grammar!