Building AI-powered Microsoft Copilot with SignalR and other open-source tools

Establish low-latency communication channel using SignalR: SignalR is used to create a low-latency communication channel between the client and the server. This allows for real-time interactions and quick responses from the AI models.

Describe and render UI using Adaptive Cards + Markdown: The UI is described and rendered using Adaptive Cards and Markdown. This allows for a flexible and customizable UI that can handle different types of inputs and outputs.

Application flow: The application flow starts with the user providing input, which is then sent to the server. The server processes the input using AI models and generates a response. The response is sent back to the client, which then displays it to the user.

Deep dive: how we use SignalR? We use SignalR to establish a connection between the client and the server. This connection allows for real-time communication and quick response times. We also use SignalR to send events with responses back to the client, allowing for interactive and dynamic user experiences.

Speech input -> speech output: Users can provide input using their voice, and the server converts it to text. The text is then processed by the AI models, and the generated response is converted to speech and read out to the user.

Users can also use images for input: In addition to textual and voice input, users can also provide input in the form of images. These images are processed by the AI models and generate a response that is then sent back to the client.

One connection per user message: Instead of using a single connection for all messages, we use one connection per user message. This simplifies the client-side code and allows for easier debugging and testing.

Overwriting messages: In some cases, the entire response is sent in each update message. This allows for easier processing and handling of the response on the client side.