Introduction
Imagine that Power Virtual Agents is like a chatbot superhero. It is a Microsoft platform that allows you to create chatbots quickly and easily, without needing to be a programming expert. It's like having your own personalized virtual assistant.
On the other hand, the recognition of conversational language is like this superhero's superpower. It is a technology that allows chatbots to understand and interact in human language. So, basically, it gives your chatbot the ability to understand and respond to what users are saying or writing.
Think of the integration of conversational language recognition into Power Virtual Agents as the special training our chatbot superhero needs. It's like learning a new superpower.
By integrating conversational language recognition, your chatbot can not only understand and respond to basic queries, but it can also understand the context, emotions, and intentions behind users' words. It's as if your chatbot can read users' minds.
This integration makes interactions with your chatbot more natural and fluid, improving the user experience. Additionally, it allows your chatbot to handle more complex queries and provide more precise answers.
Continuing with the superhero analogy, we could say that Azure Cognitive Services is like the superhero academy where our chatbot acquires its superpowers. Azure Cognitive Services is a collection of artificial intelligence services that allow developers to incorporate intelligent algorithms into their applications to see, hear, speak, understand, and even interpret user needs.
In the case of our chatbot, one of the most impressive superpowers you can acquire in this academy is the recognition of conversational language. But that's not all, you can also learn to be multilingual. Imagine that our chatbot is like a superhero with the incredible ability to speak and understand multiple languages. This means you can interact with users from all over the world, regardless of what language they speak.
So, by integrating conversational language recognition and multilingual recognition into Power Virtual Agents via Azure Cognitive Services, you're equipping your chatbot with some seriously impressive superpowers. Not only will you be able to understand and respond to user queries, but you will also be able to do so in multiple languages.
It's like having a team of international superheroes ready to help your users at any time!
Creating an initial bot
To begin, we are going to start from the base that we have a fully functional Bot that has already been created and that follows the following usage diagram:
As you can see, the objective of the bot is that we can tell it what we want to do and go through 2 levels depending on our choice. Note that all these choices will be given by the user's phrases, in no case by selection since otherwise the operation is reduced to something very simple.
So at the end of each node we expect the Bot to propose a way to act; In our case it simply shows us a phrase to confirm that we have reached the end of the correct node.
To do this we will have a theme for each node and subnode with the following structure:
As you can see, we have had to add trigger phrases so that the Bot recognizes that topic. This step is the main one affected by our new functionality. Until now we introduced phrases and Power Virtual Agents tried to identify the topics based on it but without the possibility of training or refining it.
And each main node an entity with which to identify the subnodes:
Depending on the user's response we would redirect the bot to the specific final topic:
Where each final node would have the following structure:
As you see, the final result is a phrase identifying each node so that we know that it has passed through the correct place
Analysis of Limitations and Potential Solutions in Bot Implementation
Multilingual Limitation and Synonym Management:
Challenge: The Bot is currently restricted to a single language, making it difficult to understand multiple languages and synonyms.
Proposed Solution:
Implementation of Multilingual Models: Explore advanced NLP solutions that allow the implementation of multilingual models, enabling the Bot to understand various languages.
Synonym Management: Use sophisticated NLP techniques, such as the creation of multilingual synonym databases or embeddings, to address synonyms in different languages.
Perfection of the Predictive Model:
Challenge: The current predictive model is not efficiently trainable or perfectable, which affects the accuracy of the Bot.
Proposed Solution:
Continuous Training: Implement a continuous training system for the Bot's predictive model, allowing progressive improvements based on data and user patterns.
User Feedback: Establish a mechanism for users to provide feedback on inaccurate responses, leveraging this information to refine and perfect the model.
Problems with Upper and Lower Case:
Challenge: Entities are case-sensitive, which can lead to incorrect results.
Proposed Solution:
Text Normalization: Normalize all user input to lower case before processing, ensuring consistency in the treatment of input, regardless of the user's use of upper or lower case.
And this is where this new preview functionality comes in!!
Creation of your own natural language model
Prerequisites:
Azure Subscription
Create a Language Studio resource in the same region where our Power Virtual Agents bot resides: https:// aka.ms/languageStudio
Once we have the Language Studio resource created we can proceed to create a conversational language recognition project:
Project Name:
Specifies a meaningful name for the project. Be sure to copy this value, as it will be important for future reference.
Main Language of Phrases (Utterances):
Select the main language of the model. If you want to create a multilingual bot, check the corresponding option to ignore the primary language.
Enable Multiple Languages in the Project:
Check this option to allow the model to recognize and process multiple languages in user sentences.
With these steps we are ready to start working on our new model for the bot.
Action planning
To begin we have to be clear about 2 premises:
To ensure that PVA works correctly, we must create intents for the Power Virtual Agent system themes and for any custom themes
Create any custom entity that we want to use in our Power Virtual Agents.
But choosing the methodology is not easy. So let's tell a little theory about how to structure themes and entities. To do this, I have asked GPT to explain it to us with an example:
Imagine you are building a chatbot that understands and answers users' questions. To do this, you need to define two things: “intentions”, which are the actions or queries that users may want to do, and “entities”, which are the details or information that the robot needs to perform those actions.
For example, if you are creating a chatbot for an online store, you could have an intent called “Buy” and an entity called “Product.” When a user says “I want to buy a book,” the robot recognizes “Buy” as the intention and “book” as the entity.
There are different ways to define intentions and entities. One way is to think of actions as intentions and information as entities. Another way is to think of information as intentions and actions as entities. The important thing is to be consistent and not mix different approaches.
With this, therefore, and for our example we will have to create the following themes:
The topics that are not underlined correspond to generic system topics that we will also have to generate.
As far as entities are concerned, we have the following:
Let's get to work then!!
If we enter the Language Studio project that we just created we will see that in the Schema definition section we can create Themes and Entities. This is when we have to start creating the same thing we had in Power Virtual Agents until it looks something like this:
It is important to keep the same name of topics and entities since later you will see that for it to work we have to reference it by name.
Once the themes and entities have been created, we can begin to feed our model based on phrases and languages.
We go to the Data labeling section
We began to introduce phrases for each Theme, in the main language of the project, the more the better!!!
And this is where we can give the system a twist and take advantage of Azure OpenAI or any AI tool to help us in this task. As you can see, there is a button to upload a file of "phrases" and another button for Azure OpenAI to suggest phrases for a particular topic. If we press it we will see the following:
With this tool we can save a lot of time when entering phrases into different languages since AI will make our work easier. If you still cannot have this resource, you can use ChatGPT as follows:
1. We PROMPT the example of the JSON file that the system expects:
[
{
"language": "en-us",
"intent": "Quote",
"text": "I left my heart in Alexandria Egypt
},
{
"language": "en-us",
"intent": "BookFlight",
"text": "Book me a flight to Alexandria Egypt
}
]
2. ChatGPT will recognize that it is a phrase file for a Topic
3. We ask you to generate examples for multiple languages for the Topic X we want.
4. We save the output in a JSON file and import it into Language Studio
A result would be the following:
[
{
"language": "es-es",
"intent": "AX2012",
"text": "Necesito hacer un desarrollo para AX2012."
},
{
"language": "es-es",
"intent": "AX2012",
"text": "Estoy trabajando en un proyecto de desarrollo para AX2012."
},
{
"language": "es-es",
"intent": "AX2012",
"text": "¿Puedes ayudarme con un desarrollo para AX2012?"
},
{
"language": "es-es",
"intent": "AX2012",
"text": "Estoy programando una aplicación para AX2012."
},
{
"language": "en-us",
"intent": "AX2012",
"text": "I need to do a development for AX2012."
},
{
"language": "en-us",
"intent": "AX2012",
"text": "I'm working on a development project for AX2012."
},
{
"language": "en-us",
"intent": "AX2012",
"text": "Can you assist me with a development for AX2012?"
},
{
"language": "en-us",
"intent": "AX2012",
"text": "I'm coding an application for AX2012."
},
{
"language": "fr-fr",
"intent": "AX2012",
"text": "Je dois faire un développement pour AX2012."
},
{
"language": "fr-fr",
"intent": "AX2012",
"text": "Je travaille sur un projet de développement pour AX2012."
},
{
"language": "fr-fr",
"intent": "AX2012",
"text": "Pouvez-vous m'aider avec un développement pour AX2012 ?"
},
{
"language": "fr-fr",
"intent": "AX2012",
"text": "Je code une application pour AX2012."
},
{
"language": "de-de",
"intent": "AX2012",
"text": "Ich muss eine Entwicklung für AX2012 machen."
},
{
"language": "de-de",
"intent": "AX2012",
"text": "Ich arbeite an einem Entwicklungsprojekt für AX2012."
},
{
"language": "de-de",
"intent": "AX2012",
"text": "Können Sie mir bei einer Entwicklung für AX2012 helfen?"
}
]To complete this step you need many sentences, the more the better. This way we will ensure that our Bot clearly recognizes what we want to do.
And the question...in all languages?
The answer is NO. One of the advantages of Language Studio is that it is able to recognize the language and translate it to find matches. We will only have to add phrases in other languages if we see that in those languages the successes are very low and we need to refine them.
Now we have acted on the Themes. Let's start with the entities.
We create the entities with the same name that we have in Power Virtual Agents:
And now we have several ways to configure these entities:
Learning
Precompiled entity
Regular expression
List
The last 3 already exist in Power Virtual Agents, so we are going to investigate the one that does not exist there and, for me, the most interesting.
We are going to configure the DEVELOP ENTITY entity that we just created following the learning method:
We open the Data labeling section where we have all our phrases.
We will see that on the right there is a section with the entities that we have created, and next to them the symbol of a marker.
We select the marker and start playing!!!
In each of the sentences in which we see our entity identified, we underline the part of the sentence.
And we are doing the same with all the phrases and all the entities. I repeat the same as before, the more the better.
This same action can be done in the JSON created previously and then import it identifying the entities.
You can consult the best practices when modeling entities at the following link:
Creating a consistent model is not an easy task since it requires a lot of training and having a robust and well-thought-out scheme before starting with it.
Model training
Once we have the project completed we have to train it so that the Machine Learning tool creates the model and we can start using it. To do this, we go to the Training Jobs section and execute the training according to the parameters we want:
Once trained we can go to the Model performance section where we can see the model analysis, prediction percentages and other very useful information.
For those more knowledgeable on this topic, you can also consult the confusion matrix used and the results:
And again we ask GPT to explain to us what the confusion matrix is:
The confusion matrix is a tool used in the field of artificial intelligence to evaluate the performance of a classification model, that is, a system that has been trained to distinguish between different categories or classes.
Imagine that you have a system that has been trained to distinguish between cats and dogs. When you show it an image, the system will make a prediction: is it a cat or a dog?
The confusion matrix is a table that helps us understand how well our system is doing. It is made up of four parts:
True positives (TP): The system predicted “cat” and the image was really of a cat.
False positives (FP): The system predicted “cat”, but the image was actually of a dog.
True negatives (TR): The system predicted “dog” and the image was actually of a dog.
False Negatives (FN): The system predicted “dog”, but the image was actually of a cat.
These four numbers give us a complete idea of the performance of the system. For example, if false positives are very high, it means that the system is incorrectly classifying many images of dogs as cats.
And once the results are good enough we can implement the model so that it is available for use:
We go to the Deploying a model section
We add a name to the implementation (we copy this name for later)
We accept and wait for the implementation to finish
With this completed we can return to Power Virtual Agents!!!
Mapping CLU Intents and Entities
In order to assign the CLU model to Power Virtual Agents we will follow these steps:
We go to the language tab of our Bot
We create a connection with Azure Cognitive Services and select it
We save
At that moment it will tell us that we can migrate the Bot to CLU, we save a snapshot of the Bot for what may happen and we mark the check
It will ask us for the name of the project and the name of the implementation. It is at this moment where you have to introduce the 2 components copied previously and which must match those configured in Language Studio
Once this is done we already have our Bot pointing to our CLU model. We only have to map each topic and each entity with its corresponding one in CLU.
To do this we will go to each topic, click on the trigger and enter the same name that the topic has in CLU:
If what we have is a very complex Bot, there is the mass import tool in the language tab:
We export the project from Language Studio
We import into the tool
And multilanguage?
You may have wondered how the multilingual issue solves all this. Yes, we have configured topics and entities, but the Bot still understands in one language and responds in that language.
This is where there is an additional topic that the system has created when we moved to CLU. The topic is called "Analyze text.
Everything that the user is saying goes through this topic, sends it to Language Studio and returns the conclusion: topic and recognized entities.
If we enter we will see that it makes a call to Language Studio:
In which one of the parameters is the language. Therefore we will modify that theme to introduce one more box:
This box will call a Power Automate that will return the language of the user's message.
On the way back and for each message that the Bot says to the user we will do the opposite step, translate into the user's language.
Power Virtual Agents with Machine Learning
We already have our Bot completed so let's see it work.
As you will see in the following video, although our Bot is created in English, we are writing to it in Spanish and it correctly recognizes our prompt.
Conclusions
Conversational Language Recognition (CLU) is an Azure AI service that allows you to create custom natural language understanding models for conversational applications2.
Power Virtual Agents is a tool that allows you to design and publish your own chatbot without needing to know how to program.
You can integrate your CLU model with your Power Virtual Agents bot to take advantage of the intents and entities that you have defined in your CLU project.
To integrate your CLU model with your bot, you must take into account the following aspects:
Choose a coherent scheme to define intentions and entities, either modeling actions as intentions and information as entities, or vice versa.
Prepare your Azure environment and your language resource with the CLU feature enabled.
Train and deploy your CLU project in the same region as your Power Virtual Agents resource.
Assign CLU intents and entities to your bot's topics and questions.
Update bot trigger phrases to link each topic to a corresponding CLU intent3.
Manually manage the relationship between the CLU model and Power Virtual Agents4.
The integration of CLU with Power Virtual Agents offers you several advantages, such as:
Improve your bot's accuracy and coverage by using a custom natural language recognition model.
Easily extend existing functionality to new targets or actions by creating new entities or intents.
Combine multiple CLU child applications or other Azure AI services into one orchestrated application.