Use Copilot Studio Topics to Guide Users to Your Agents

…continued from last article Build the 007 Agent and Wire It Up With the Knowledge Base

After the agent is created, we can add and modify topics init to incorporate our custom logics to make the agent more intelligent and user-friendly.

You can think of "topics" in Copilot Studio as functions in traditional programming, each can do something autonomously and can have input and output parameters. For more information on the basics, please refer to this article.

For our 007 agent, we need to identify the best matching agent using information from user’s input and the agents list in SharePoint site and provide users with details about these agents. If users want to try out the agents, we should give them instructions on how to proceed.

Something like:

1. End user comes to 007 agent to ask for "Which agent should I use to engage with IT support?"
2. 007 replies: "The best agent for your case is: IT Helpdesk, empowers employees to resolve issues and effortlessly create/view support tickets. Would you like to try it out?"
3. If the user response is "Yes", the 007 will provide them with the direct link to the IT Helpdesk agent.

For all the requirements above, we need to create a custom topic to:

• Get end user's prompt
• Call Generative AI (Copilot in this case) to query our agents list in SharePoint.
• Get the matching agent's information and send response back to end users.

Sounds simple enough, and it is almost the case with a few tricks I will go through before I give you the full YAML code. 😉

Trick #1 – Returning structured data

Within a topic, you can send prompts to gen-AI and receive responses by using the “Generative answer” node.

An agent has many attributes, like name, description, URL, etc. When returning the information from called the Generative Answer node, you want to get back a structured data that can be converted to a Copilot Studio record or table to be used in future information processing.

Luckily this can be done, and I have published an article about it at this link.

Following that logic, the prompt we use in 007 agent to query the Gen-AI is:

"Response with information about the agent that is best for '"& System.Activity.Text &"'. Response with the agent's Name, Description, Category, Type, Trigger and URL values as a JSON formatted string."

Where System.Activity.Text is a system variable in Copilot Studio to represent the initial prompt that the end user sent to our agent.

Trick #2 – Dealing with gen-AI’s inaccuracy and hallucination

Keep in mind that generative AI can be inaccurate or hallucinate: it may not return the data exactly as they are in the knowledge sources and sometime may even make things up completely.

A few things can be done to help ease this situation.

• Content moderation level
  Set it to High to tell Gen-AI that you favor relevancy over number of answers.

• Use PowerFx expressions to filter out unwanted characters in responses
  In our case, the data in “Type” field often returned as “Autonomous IT” other than “Autonomous” as it should be. I need to use Find("Autonomous", Trim(Topic.AgentInfo.Category)) expression to pre-process it before sending it to a condition node otherwise the information may get misinterpreted.

Below is the full YAML code for the topic:

1kind: AdaptiveDialog
2beginDialog:
3  kind: OnRecognizedIntent
4  id: main
5  intent:
6    triggerQueries:
7      - agent, job, task, search, find, data, process
8
9  actions:
10    - kind: SetVariable
11      id: setVariable_7Psnuh
12      variable: Topic.aiPrompt
13      value: ="Response with information about the agent that is best for '"& System.Activity.Text &"'. Response with the agent's Name, Description, Category, Type, Trigger and URL values as a JSON formatted string."
14
15    - kind: SearchAndSummarizeContent
16      id: search-content
17      latencyMessageSettings:
18        allowLatencyMessage: false
19
20      autoSend: false
21      variable: Topic.Answer
22      userInput: =Topic.aiPrompt
23      applyModelKnowledgeSetting: false
24      sharePointSearchDataSource: {}
25      fileSearchDataSource:
26        searchFilesMode:
27          kind: DoNotSearchFiles
28
29      knowledgeSources:
30        kind: SearchSpecificKnowledgeSources
31        knowledgeSources:
32          - cr756_copilotAgentForAgents.topic.AgentListdocx_7QlPosRZZ7d5xECzgr7Qp
33
34      responseCaptureType: TextOnly
35
36    - kind: ConditionGroup
37      id: has-answer-conditions
38      conditions:
39        - id: has-answer
40          condition: =!IsBlank(Topic.Answer)
41          actions:
42            - kind: SetVariable
43              id: setVariable_7Dtj2Q
44              variable: Topic.JSONString
45              value: =Mid(Topic.Answer, Find("{", Topic.Answer), Find("}", Topic.Answer, Find("{", Topic.Answer)) - Find("{", Topic.Answer) + 1)
46
47            - kind: ParseValue
48              id: xSRHQ8
49              variable: Topic.AgentInfo
50              valueType:
51                kind: Record
52                properties:
53                  Category: String
54                  Description: String
55                  Name: String
56                  Trigger: String
57                  Type: String
58                  URL: String
59
60              value: =Topic.JSONString
61              errorPolicy: ThrowError
62
63            - kind: ConditionGroup
64              id: conditionGroup_A8TZdH
65              conditions:
66                - id: conditionItem_jTcg2j
67                  condition: =!IsBlank(Topic.AgentInfo)
68                  actions:
69                    - kind: SetVariable
70                      id: setVariable_XtvVpt
71                      variable: Topic.isAuto
72                      value: =Find("Autonomous", Trim(Topic.AgentInfo.Category))
73
74                    - kind: ConditionGroup
75                      id: conditionGroup_hf6hj4
76                      conditions:
77                        - id: conditionItem_XKQzBG
78                          condition: =!IsBlank(Topic.isAuto)
79                          actions:
80                            - kind: SendActivity
81                              id: sendActivity_DJsI4q
82                              activity: "{Topic.AgentInfo.Name} can help. It is an autonomous agent and this is how the agent can be triggered: {Topic.AgentInfo.Trigger}"
83
84                      elseActions:
85                        - kind: Question
86                          id: 9xz7wI
87                          interruptionPolicy:
88                            allowInterruption: true
89
90                          variable: init:Topic.TryAgent
91                          prompt: "The best agent for you is: {Topic.AgentInfo.Name}, {Topic.AgentInfo.Description}. Would you like to try it out?"
92                          entity:
93                            kind: EmbeddedEntity
94                            definition:
95                              kind: ClosedListEntity
96                              items:
97                                - id: Yes
98                                  displayName: Yes
99
100                                - id: No
101                                  displayName: No
102
103                        - kind: ConditionGroup
104                          id: U9vsF1
105                          conditions:
106                            - id: yksUs9
107                              condition: =Topic.TryAgent = 'cr756_copilotAgentForAgents.topic.AgentFinder.main.9xz7wI'.Yes
108                              actions:
109                                - kind: SendActivity
110                                  id: 4CbXLj
111                                  activity: "Please follow this link to try it out: <a href=\"{Topic.AgentInfo.URL}\">{Topic.AgentInfo.Name}</a>"
112
113                            - id: lm1XT1
114                              condition: =Topic.TryAgent = 'cr756_copilotAgentForAgents.topic.AgentFinder.main.9xz7wI'.No
115
116              elseActions:
117                - kind: SendActivity
118                  id: sendActivity_zfD4XJ
119                  activity: No valid agent found!
120
121      elseActions:
122        - kind: GotoAction
123          id: fVfLOK
124          actionId: sendActivity_zfD4XJ
125
126    - kind: CancelAllDialogs
127      id: 8fUleJ
128
129inputType: {}
130outputType: {}

To create a new topic:

• In Copilot Studio open the agent we just created.
• Navigate to the "Topic" section and then click on the "Add a topic" button and select the "From blank" option.

• Once in the new topic's canvas click on the".../more" button and Open code editor.

• Paste the YAML code above to the code view and save it. Name the new topic "Agent Finder".
• You may see a few error messages and that's expected. Click on the "Save" button to confirm it.

Here is how to fix those errors:

• Missing data sources for Create generative answers node: click on "Edit" under "Data Sources" and in the popped-up dialog select the knowledge source.

• Invalid conditions
  Just type "Yes" to the left condition and "No" to the right one will fix the errors.

Save the fixed topic and you can now test your topic with some prompts below:

I need to verify my documents against branding standards.

Which agent should I use to engage with IT support?

Where to find store procedures and policies?

If you use the information and format in the first section of this series to create your knowledge source you should see some results like below. Otherwise, you may see a "No valid agent found" message.

In the next section, we will publish the “007 Agent” to let end users to use it in Teams and Microsoft 365 Copilot Chat apps.

…to be continued.

Happy piloting Copilot!