What I Learned About Building a Custom Copilot with Sam Nasr

February 12, 2025

What I Learned About Building a Custom Copilot with Sam Nasr

Earlier this week, our Azure User Group had the pleasure of hosting a talk by Sam Nasr, a Microsoft MVP and experienced software engineer. Sam walked us through the process of creating our own “Copilot” using Azure AI Foundry (previously known as Azure AI Studio). If you’ve been curious about extending generative AI to fit your organization’s specific needs, here’s a recap from his talk.

1. What Is Copilot?

Microsoft’s Copilot is essentially a generative AI service that can handle conversational prompts and provide intelligent, context-aware responses. You may have heard of GitHub Copilot (which suggests code in Visual Studio) or Bing Chat (an AI-enhanced search). Now, that same technology is available to integrate into your own applications.

2. Why Build a Custom Copilot?

A custom Copilot allows you to:

Tailor the AI to your own data and brand voice.
Improve user experience by providing focused answers, rather than general web searches.
Control the tone, style, and behavior of the AI responses.
Restrict or allow certain content based on business rules, ensuring compliance and brand safety.

3. Azure AI Foundry: The Hub of AI Models

Sam introduced us to Azure AI Foundry, a web-based environment that integrates seamlessly with Azure services. It includes:

Model Catalog: Thousands of models from various providers (OpenAI, Meta, Microsoft, etc.).
Playgrounds: Sandboxes to test Chat, Speech, and Image features.
Deployment Tools: Options to publish as a web app or call from an API in your own application.

Hubs and Projects

Hub: A container for multiple AI projects. It also centralizes security, resource usage, and data connections.
Project: Each project inherits settings from the hub, including data connections and AI resources.

4. Setting Up Your Copilot

Prerequisites

Azure Subscription: You may need to request access for Azure OpenAI.
Hub & Project: Created in Azure AI Foundry and linked to a Resource Group.
Data Indexing: If you want the AI to answer from your own docs, you’ll need an Azure Cognitive Search index.

Steps Sam Demonstrated

Create a Hub and Project in Azure AI Foundry.
Select a Model (e.g., GPT-3.5 Turbo) for your AI.
Customize Model Instructions (the AI’s “personality” and style).
Add Safety System Messages to filter out harmful or off-topic content.
Import Custom Data and connect it to your AI using Azure Cognitive Search.
Test in the Playground by asking questions and seeing references to your indexed data.
Deploy as a web app or retrieve code samples (C#, Python, etc.) for integration.

5. Key Takeaways

User-Friendly Tuning: You can literally type “You are a friendly AI that…” and the AI changes its style. No advanced code needed.
Resource Costs: Every AI query consumes “tokens,” so keep track of usage to stay within budget.
Flexible Deployment: Switch among GPT models, or add features like speech recognition/response.
Security & Permissions: Set up role-based access in Azure to protect data and control usage.
Scalability: Start small (even on the free tier), then add features (speech, advanced models, more data) as needed.

6. Next Steps and Resources

If you’re ready to explore building your own custom Copilot, definitely give Azure AI Foundry a try. It’s a powerful way to bring conversational AI into your projects with minimal friction.

Huge thanks to Sam Nasr for sharing his expertise! If you have any follow-up questions or would like to see more sessions on AI and Azure, feel free to reach out.

Thanks for reading! If you found this helpful, share it with your fellow Azure enthusiasts or join us at our next user group meeting.