Unleashing Innovation: AI Agent Development with Azure AI Foundry
Creating AI agents using Azure AI Foundry is a game-changer for businesses and developers looking to harness the power of artificial intelligence. These AI agents can automate complex tasks, provide insightful data analysis, and enhance customer interactions, leading to increased efficiency and productivity. By leveraging Azure AI Foundry, organizations can build, deploy, and manage AI solutions with ease, ensuring they stay competitive in an ever-evolving technological landscape. The importance of creating AI agents lies in their ability to transform operations, drive innovation, and deliver personalized experiences, making them an invaluable asset in today's digital age. Let's take a look at how to create an agent on Azure AI Foundry. We'll explore some of the features and experiment with its capabilities in the playground. I recommend by creating a new resource group with a new Azure OpenAI resource. Once the Azure OpenAI Resource is created, follow these steps to get started with Azure AI Foundry Agents. Implementation Overview Open Azure AI Foundry and click on the Azure AI Foundry link at the top right to get to the home page where you'll see all your projects. Click on + Create project then click on Create new hub Give it a name then click Next and Create New resources will be created with your new project. Once inside your new project you should see the Agents preview option on the left menu Select your Azure OpenAI Service resource and click Let's go We can now get started with implementation. A model needs to be deployed. However, it's important to consider which models can be used and their regions for creating these agents. Below is a quick summary of what's currently available. Supported models in Azure AI Agent Service - Azure AI services | Microsoft Learn Other models supported include Meta-Llama-405B-Instruct, Mistral-large-2407, Cohere-command-r-plus, and Cohere-command-r. I've deployed gpt-4 as Global Standard and can now create a new agent. Click on +New agent. A new Agent will be created and details such as the agent instructions, model deployment, Knowledge and Action configurations, and model settings are shown. Incorporating knowledge into AI agents is to enhance their ability to provide accurate, relevant, and context-specific responses. This makes them more effective in automating tasks, answering complex queries, and supporting decision-making processes. Actions enable AI agents to perform specific tasks and interact with various service and data sources. Here we can leverage these abilities by adding a Custom Function, OpenAPI 3.0 specified tool, or an Azure function to help run tasks. The Code Interpreter feature within Actions empowers the agent to read and analyze datasets, generate code, and create visualizations such as graphs and charts. In the next section we'll go deeper with code interpreters' abilities. Code Interpreter For this next step I'll leverage weatherHistory.csv file from Weather Dataset for code interpreter to perform on. Next Actions click on + Add then click on Code interpreter and add the csv file. Update the Instructions to "You are a Weather Data Expert Agent, designed to provide accurate, up-to-date, and detailed weather information." Lets explore what Code interpreter can do. Click on Try in playground on the top right. I'll start by asking "can you tell me which month had the most rain?", code interpreter already knows that I'm asking a question in reference to the data file I just gave it and will breakdown the question into multiple steps to provide the best possible answer. We can see that based on the dataset, August 2010 has the most where 768 instances of rainfall were recorded. We'll take it a step further and create a graph using a different question. Let's ask the agent "ok, can you create a bar chart that shows the amount of rain fall from each year using the provided dataset?" in which the agent will respond with the following: This is just a quick demonstration of how powerful code interpreter can be. Code interpreter allows for efficient data interpretation and presentation as shown above, making it easier to derive insights and make informed decisions. We'll create and add a Bing Grounding Resource which will allow an agent to include real-time public web data into their responses. Bing Grounding Resource A Bing Grounding Resource is a powerful tool that enables AI agents to access and incorporate real-time data from the web into their responses and also ensures that the information provided by the agents is accurate, current, and relevant. An agent will be able to perform Bing searches when needed, fetching up-to-date information and enhancing the overall reliability and transparency of its responses. By leveraging Bing Grounding, AI agents can deliver more precise and contextually appropriate answers, significantly improving user satisfaction and trust. To add a Bing Ground Resource to the agent: Create the Resource: Navigate to the Azure AI Foundry portal and create a new Bing Grounding resource. Add Knowledge: Go to your agent in Azure AI Foundry, click on + Add next to Knowledge on the right side, select Grounding with Big Search, + Create connection. Add connection with API key. The Bing Grounding resource is now added to your agent. In the playground I'll add first ask "Is it raining over downtown New York today?". I will get a live response that also includes the links to the sources where the information was retrieved from. The agent responds as shown below: Next i'll ask the agent "How's should I prepare for the weather in New York this week? Any clothing recommendations?" in which the agent responds: The agent is able to breakdown the question using gpt-4 in detail by leveraging the source information from Bing and providing appropriate information to the user. Other the capabilities of custom functions, OpenAPI 3.0 specified tools, and Azure Functions significantly enhance the versatility and power of Azure AI agents. Custom functions allow agents to perform specialized tasks tailored to specific business needs, while OpenAPI 3.0 specified tools enable seamless integration with a wide range of external services and APIs. Azure Functions further extend the agent's capabilities by allowing it to execute serverless code, automating complex workflows and processes. Together, these features empower developers to build highly functional and adaptable AI agents that can efficiently handle diverse tasks, drive innovation, and deliver exceptional value to users. Conclusion Developing an AI Agent on Azure AI Foundry is a swift and efficient process, thanks to its robust features and comprehensive tools. The platform's Bing Grounding Resource ensures that your AI models are well-informed and contextually accurate, leveraging vast amounts real-time of data to enhance performance. Additionally, the Code Interpreter simplifies the integration and execution of solving complex tasks involving data analysis. By utilizing these powerful resources, you can accelerate the development of intelligent agents that are not only capable of understanding and responding to user inputs but also continuously improving through iterative learning. Azure AI Foundry provides a solid foundation for creating innovative AI solutions that can drive significant value across various applications. Additional Resources: Quickstart - Create a new Azure AI Agent Service project - Azure AI services | Microsoft Learn How to use Grounding with Bing Search in Azure AI Agent Service - Azure OpenAI | Microsoft Learn
Prompt Engineering for OpenAI’s O1 and O3-mini Reasoning Models
Important Attempting to extract the model's internal reasoning is prohibited, as it violates the acceptable use guidelines. This section explores how O1 and O3-mini differ from GPT-4o in input handling, reasoning capabilities, and response behavior, and outlines prompt engineering best practices to maximize their performance. Finally, we apply these best practices to a legal case analysis scenario. Differences Between O1/O3-mini and GPT-4o Input Structure and Context Handling Built-in Reasoning vs. Prompted Reasoning: O1-series models have built-in chain-of-thought reasoning, meaning they internally reason through steps without needing explicit coaxing from the prompt. In contrast, GPT-4o often benefits from external instructions like “Let’s think step by step” to solve complex problems, since it doesn’t automatically engage in multi-step reasoning to the same extent. With O1/O3, you can present the problem directly; the model will analyze it deeply on its own. Need for External Information: GPT-4o has a broad knowledge base and access to tools (e.g. browsing, plugins, vision) in certain deployments, which helps it handle a wide range of topics. By comparison, the O1 models have a narrower knowledge base outside their training focus. For example, O1-preview excelled at reasoning tasks but couldn’t answer questions about itself due to limited knowledge context. This means when using O1/O3-mini, important background information or context should be included in the prompt if the task is outside common knowledge – do not assume the model knows niche facts. GPT-4o might already know a legal precedent or obscure detail, whereas O1 might require you to provide that text or data. Context Length: The reasoning models come with very large context windows. O1 supports up to 128k tokens of input, and O3-mini accepts up to 200k tokens (with up to 100k tokens output), exceeding GPT-4o’s context length. This allows you to feed extensive case files or datasets directly into O1/O3. For prompt engineering, structure large inputs clearly (use sections, bullet points, or headings) so the model can navigate the information. Both GPT-4o and O1 can handle long prompts, but O1/O3’s higher capacity means you can include more detailed context in one go, which is useful in complex analyses. Reasoning Capabilities and Logical Deduction Depth of Reasoning: O1 and O3-mini are optimized for methodical, multi-step reasoning. They literally “think longer” before answering, which yields more accurate solutions on complex tasks. For instance, O1-preview solved 83% of problems on a challenging math exam (AIME), compared to GPT-4o’s 13% – a testament to its superior logical deduction in specialized domains. These models internally perform chain-of-thought and even self-check their work. GPT-4o is also strong but tends to produce answers more directly; without explicit prompting, it might not analyze as exhaustively, leading to errors in very complex cases that O1 could catch. Handling of Complex vs. Simple Tasks: Because O1-series models default to heavy reasoning, they truly shine on complex problems that have many reasoning steps (e.g. multi-faceted analyses, long proofs). In fact, on tasks requiring five or more reasoning steps, a reasoning model like O1-mini or O3 outperforms GPT-4 by a significant margin (16%+ higher accuracy). However, this also means that for very simple queries, O1 may “overthink.” Research found that on straightforward tasks (fewer than 3 reasoning steps), O1’s extra analytical process can become a disadvantage – it underperformed GPT-4 in a significant portion of such cases due to excessive reasoning. GPT-4o might answer a simple question more directly and swiftly, whereas O1 might generate unnecessary analysis. The key difference is O1 is calibrated for complexity, so it may be less efficient for trivial Q&A. Logical Deduction Style: When it comes to puzzles, deductive reasoning, or step-by-step problems, GPT-4o usually requires prompt engineering to go stepwise (otherwise it might jump to an answer). O1/O3 handle logical deduction differently: they simulate an internal dialogue or scratchpad. For the user, this means O1’s final answers tend to be well-justified and less prone to logical gaps. It will have effectively done a “chain-of-thought” internally to double-check consistency. From a prompt perspective, you generally don’t need to tell O1 to explain or check its logic – it does so automatically before presenting the answer. With GPT-4o, you might include instructions like “first list the assumptions, then conclude” to ensure rigorous logic; with O1, such instructions are often redundant or even counterproductive. Response Characteristics and Output Optimization Detail and Verbosity: Because of their intensive reasoning, O1 and O3-mini often produce detailed, structured answers for complex queries. For example, O1 might break down a math solution into multiple steps or provide a rationale for each part of a strategy plan. GPT-4o, on the other hand, may give a more concise answer by default or a high-level summary, unless prompted to elaborate. In terms of prompt engineering, this means O1’s responses might be longer or more technical. You have more control over this verbosity through instructions. If you want O1 to be concise, you must explicitly tell it (just as you would GPT-4) – otherwise, it might err on the side of thoroughness. Conversely, if you want a step-by-step explanation in the output, GPT-4o might need to be told to include one, whereas O1 will happily provide one if asked (and has likely done the reasoning internally regardless). Accuracy and Self-Checking: The reasoning models exhibit a form of self-fact-checking. OpenAI notes that O1 is better at catching its mistakes during the response generation, leading to improved factual accuracy in complex responses. GPT-4o is generally accurate, but it can occasionally be confidently wrong or hallucinate facts if not guided. O1’s architecture reduces this risk by verifying details as it “thinks.” In practice, users have observed that O1 produces fewer incorrect or nonsensical answers on tricky problems, whereas GPT-4o might require prompt techniques (like asking it to critique or verify its answer) to reach the same level of confidence. This means you can often trust O1/O3 to get complex questions right with a straightforward prompt, whereas with GPT-4 you might add instructions like “check your answer for consistency with the facts above.” Still, neither model is infallible, so critical factual outputs should always be reviewed. Speed and Cost: A notable difference is that O1 models are slower and more expensive in exchange for their deeper reasoning. O1 Pro even includes a progress bar for long queries. GPT-4o tends to respond faster for typical queries. O3-mini was introduced to offer a faster, cost-efficient reasoning model – it’s much cheaper per token than O1 or GPT-4o and has lower latency. However, O3-mini is a smaller model, so while it’s strong in STEM reasoning, it might not match full O1 or GPT-4 in general knowledge or extremely complex reasoning. When prompt engineering for optimal response performance, you need to balance depth vs. speed: O1 might take longer to answer thoroughly. If latency is a concern and the task isn’t maximal complexity, O3-mini (or even GPT-4o) could be a better choice. OpenAI’s guidance is that GPT-4o “is still the best option for most prompts,” using O1 primarily for truly hard problems in domains like strategy, math, and coding. In short, use the right tool for the job – and if you use O1, anticipate longer responses and plan for its slower output (possibly by informing the user or adjusting system timeouts). Prompt Engineering Techniques to Maximize Performance Leveraging O1 and O3-mini effectively requires a slightly different prompting approach than GPT-4o. Below are key prompt engineering techniques and best practices to get the best results from these reasoning models: Keep Prompts Clear and Minimal Be concise and direct with your ask. Because O1 and O3 perform intensive internal reasoning, they respond best to focused questions or instructions without extraneous text. OpenAI and recent research suggest avoiding overly complex or leading prompts for these models. In practice, this means you should state the problem or task plainly and provide only necessary details. There is no need to add “fluff” or multiple rephrasing of the query. For example, instead of writing: “In this challenging puzzle, I’d like you to carefully reason through each step to reach the correct solution. Let’s break it down step by step...”, simply ask: “Solve the following puzzle [include puzzle details]. Explain your reasoning.” The model will naturally do the step-by-step thinking internally and give an explanation. Excess instructions can actually overcomplicate things – one study found that adding too much prompt context or too many examples worsened O1’s performance, essentially overwhelming its reasoning process. Tip: For complex tasks, start with a zero-shot prompt (just the task description) and only add more instruction if you find the output isn’t meeting your needs. Often, minimal prompts yield the best results with these reasoning models. Avoid Unnecessary Few-Shot Examples Traditional prompt engineering for GPT-3/4 often uses few-shot examples or demonstrations to guide the model. With O1/O3, however, less is more. The O1 series was explicitly trained to not require example-laden prompts. In fact, using multiple examples can hurt performance. Research on O1-preview and O1-mini showed that few-shot prompting consistently degraded their performance – even carefully chosen examples made them do worse than a simple prompt in many cases. The internal reasoning seems to get distracted or constrained by the examples. OpenAI’s own guidance aligns with this: they recommend limiting additional context or examples for reasoning models to avoid confusing their internal logic. Best practice: use zero-shot or at most one example if absolutely needed. If you include an example, make it highly relevant and simple. For instance, in a legal analysis prompt, you generally would not prepend a full example case analysis; instead, just ask directly about the new case. The only time you might use a demonstration is if the task format is very specific and the model isn’t following instructions – then show one brief example of the desired format. Otherwise, trust the model to figure it out from a direct query. Leverage System/Developer Instructions for Role and Format Setting a clear instructional context can help steer the model’s responses. With the API (or within a conversation’s system message), define the model’s role or style succinctly. For example, a system message might say: “You are an expert scientific researcher who explains solutions step-by-step”. O1 and O3-mini respond well to such role instructions and will incorporate them in their reasoning. However, remember that they already excel at understanding complex tasks, so your instructions should focus on what kind of output you want, not how to think. Good uses of system/developer instructions include: Defining the task scope or persona: e.g. “Act as a legal analyst” or “Solve the problem as a math teacher explaining to a student.” This can influence tone and the level of detail. Specifying the output format: If you need the answer in a structured form (bullet points, a table, JSON, etc.), explicitly say so. O1 and especially O3-mini support structured output modes and will adhere to format requests. For instance: “Provide your findings as a list of key bullet points.” Given their logical nature, they tend to follow format instructions accurately, which helps maintain consistency in responses Setting boundaries: If you want to control verbosity or focus, you can include something like “Provide a brief conclusion after the detailed analysis” or “Only use the information given without outside assumptions.” The reasoning models will respect these boundaries, and it can prevent them from going on tangents or hallucinating facts. This is important since O1 might otherwise produce a very exhaustive analysis – which is often great, but not if you explicitly need just a summary. Ensure any guidance around tone, role, format is included each time. Control Verbosity and Depth Through Instructions While O1 and O3-mini will naturally engage in deep reasoning, you have control over how much of that reasoning is reflected in the output. If you want a detailed explanation, prompt for it (e.g. “Show your step-by-step reasoning in the answer”). They won’t need the nudge to do the reasoning, but they do need to be told if you want to see it. Conversely, if you find the model’s answers too verbose or technical for your purposes, instruct it to be more concise or to focus only on certain aspects. For example: “In 2-3 paragraphs, summarize the analysis with only the most critical points.” The models are generally obedient to such instructions about length or focus. Keep in mind that O1’s default behavior is to be thorough – it’s optimized for correctness over brevity – so it may err on the side of giving more details. A direct request for brevity will override this tendency in most cases. For O3-mini, OpenAI provides an additional tool to manage depth: the “reasoning effort” parameter (low, medium, high). This setting lets the model know how hard to “think.” In prompt terms, if using the API or a system that exposes this feature, you can dial it up for very complex tasks (ensuring maximum reasoning, at the cost of longer answers and latency) or dial it down for simpler tasks (faster, more streamlined answers). This is essentially another way to control verbosity and thoroughness. If you don’t have direct access to that parameter, you can mimic a low effort mode by explicitly saying “Give a quick answer without deep analysis” for cases where speed matters more than perfect accuracy. Conversely, to mimic high effort, you might say “Take all necessary steps to arrive at a correct answer, even if the explanation is long.” These cues align with how the model’s internal setting would operate. Ensure Accuracy in Complex Tasks To get the most accurate responses on difficult problems, take advantage of the reasoning model’s strengths in your prompt. Since O1 can self-check and even catch contradictions, you can ask it to utilize that: e.g. “Analyze all the facts and double-check your conclusion for consistency.” Often it will do so unprompted, but reinforcing that instruction can signal the model to be extra careful. Interestingly, because O1 already self-fact-checks, you rarely need to prompt it with something like “verify each step” (that’s more helpful for GPT-4o). Instead, focus on providing complete and unambiguous information. If the question or task has potential ambiguities, clarify them in the prompt or instruct the model to list any assumptions. This prevents the model from guessing wrongly. Handling sources and data: If your task involves analyzing given data (like summarizing a document or computing an answer from provided numbers), make sure that data is clearly presented. O1/O3 will diligently use it. You can even break data into bullet points or a table for clarity. If the model must not hallucinate (say, in a legal context it shouldn’t make up laws), explicitly state “base your answer only on the information provided and common knowledge; do not fabricate any details.” The reasoning models are generally good at sticking to known facts, and such an instruction further reduces the chance of hallucinationIterate and verify: If the task is critical (for example, complex legal reasoning or a high-stakes engineering calculation), a prompt engineering technique is to ensemble the model’s responses. This isn’t a single prompt, but a strategy: you could run the query multiple times (or ask the model to consider alternative solutions) and then compare answers. O1’s stochastic nature means it might explore different reasoning paths each time. By comparing outputs or asking the model to “reflect if there are alternative interpretations” in a follow-up prompt, you can increase confidence in the result. While GPT-4o also benefits from this approach, it’s especially useful for O1 when absolute accuracy is paramount – essentially leveraging the model’s own depth by cross-verifying. Finally, remember that model selection is part of prompt engineering: If a question doesn’t actually require O1-level reasoning, using GPT-4o might be more efficient and just as accurate. OpenAI recommends saving O1 for the hard cases and using GPT-4o for the rest. So a meta-tip: assess task complexity first. If it’s simple, either prompt O1 very straightforwardly to avoid overthinking, or switch to GPT-4o. If it’s complex, lean into O1’s abilities with the techniques above. How O1/O3 Handle Logical Deduction vs. GPT-4o The way these reasoning models approach logical problems differs fundamentally from GPT-4o, and your prompt strategy should adapt accordingly: Handling Ambiguities: In logical deduction tasks, if there’s missing info or ambiguity, GPT-4o might make an assumption on the fly. O1 is more likely to flag the ambiguity or consider multiple possibilities because of its reflective approach. To leverage this, your prompt to O1 can directly ask: “If there are any uncertainties, state your assumptions before solving.” GPT-4 might need that nudge more. O1 might do it naturally or at least is less prone to assuming facts not given. So in comparing the two, O1’s deduction is cautious and thorough, whereas GPT-4o’s is swift and broad. Tailor your prompt accordingly – with GPT-4o, guide it to be careful; with O1, you mainly need to supply the information and let it do its thing. Step-by-Step Outputs: Sometimes you actually want the logical steps in the output (for teaching or transparency). With GPT-4o, you must explicitly request this (“please show your work”). O1 might include a structured rationale by default if the question is complex enough, but often it will present a well-reasoned answer without explicitly enumerating every step unless asked. If you want O1 to output the chain of logic, simply instruct it to — it will have no trouble doing so. In fact, O1-mini was noted to be capable of providing stepwise breakdowns (e.g., in coding problems) when prompted. Meanwhile, if you don’t want a long logical exposition from O1 (maybe you just want the final answer), you should say “Give the final answer directly” to skip the verbose explanation. Logical Rigor vs. Creativity: One more difference: GPT-4 (and 4o) has a streak of creativity and generative strength. Sometimes in logic problems, this can lead it to “imagine” scenarios or analogies, which isn’t always desired. O1 is more rigor-focused and will stick to logical analysis. If your prompt involves a scenario requiring both deduction and a bit of creativity (say, solving a mystery by piecing clues and adding a narrative), GPT-4 might handle the narrative better, while O1 will strictly focus on deduction. In prompt engineering, you might combine their strengths: use O1 to get the logical solution, then use GPT-4 to polish the presentation. If sticking to O1/O3 only, be aware that you might need to explicitly ask it for creative flourishes or more imaginative responses – they will prioritize logic and correctness by design. Key adjustment: In summary, to leverage O1/O3’s logical strengths, give them the toughest reasoning tasks as a single well-defined prompt. Let them internally grind through the logic (they’re built for it) without micromanaging their thought process. For GPT-4o, continue using classic prompt engineering (decompose the problem, ask for step-by-step reasoning, etc.) to coax out the same level of deduction. And always match the prompt style to the model – what confuses GPT-4o might be just right for O1, and vice versa, due to their different reasoning approaches. Crafting Effective Prompts: Best Practices Summary To consolidate the above into actionable guidelines, here’s a checklist of best practices when prompting O1 or O3-mini: Use Clear, Specific Instructions: Clearly state what you want the model to do or answer. Avoid irrelevant details. For complex questions, a straightforward ask often suffices (no need for elaborate role-play or multi-question prompts). Provide Necessary Context, Omit the Rest: Include any domain information the model will need (facts of a case, data for a math problem, etc.), since the model might not have up-to-date or niche knowledge. But don’t overload the prompt with unrelated text or too many examples – extra fluff can dilute the model’s focus Minimal or No Few-Shot Examples: By default, start with zero-shot prompts. If the model misinterprets the task or format, you can add one simple example as guidance, but never add long chains of examples for O1/O3. They don’t need it, and it can even degrade performance. Set the Role or Tone if Needed: Use a system message or a brief prefix to put the model in the right mindset (e.g. “You are a senior law clerk analyzing a case.”). This helps especially with tone (formal vs. casual) and ensures domain-appropriate language. Specify Output Format: If you expect the answer in a particular structure (list, outline, JSON, etc.), tell the model explicitly. The reasoning models will follow format instructions reliably. For instance: “Give your answer as an ordered list of steps.” Control Length and Detail via Instructions: If you want a brief answer, say so (“answer in one paragraph” or “just give a yes/no with one sentence explanation”). If you want an in-depth analysis, encourage it (“provide a detailed explanation”). Don’t assume the model knows your desired level of detail by default – instruct it. Leverage O3-mini’s Reasoning Effort Setting: When using O3-mini via API, choose the appropriate reasoning effort (low/medium/high) for the task. High gives more thorough answers (good for complex legal reasoning or tough math), low gives faster, shorter answers (good for quick checks or simpler queries). This is a unique way to tune the prompt behavior for O3-mini. Avoid Redundant “Think Step-by-Step” Prompts: Do not add phrases like “let’s think this through” or chain-of-thought directives for O1/O3; the model already does this internally. Save those tokens and only use such prompts on GPT-4o, where they have impact. Test and Iterate: Because these models can be sensitive to phrasing, if you don’t get a good answer, try rephrasing the question or tightening the instructions. You might find that a slight change (e.g. asking a direct question vs. an open-ended prompt) yields a significantly better response. Fortunately, O1/O3’s need for iteration is less than older models (they usually get complex tasks right in one go), but prompt tweaking can still help optimize clarity or format. Validate Important Outputs: For critical use-cases, don’t rely on a single prompt-answer cycle. Use follow-up prompts to ask the model to verify or justify its answer (“Are you confident in that conclusion? Explain why.”), or run the prompt again to see if you get consistent results. Consistency and well-justified answers indicate the model’s reasoning is solid. By following these techniques, you can harness O1 and O3-mini’s full capabilities and get highly optimized responses that play to their strengths. Applying Best Practices to a Legal Case Analysis Finally, let’s consider how these prompt engineering guidelines translate to a legal case analysis scenario (as mentioned earlier). Legal analysis is a perfect example of a complex reasoning task where O1 can be very effective, provided we craft the prompt well: Structure the Input: Start by clearly outlining the key facts of the case and the legal questions to be answered. For example, list the background facts as bullet points or a brief paragraph, then explicitly ask the legal question: “Given the above facts, determine whether Party A is liable for breach of contract under U.S. law.” Structuring the prompt this way makes it easier for the model to parse the scenario. It also ensures no crucial detail is buried or overlooked. Provide Relevant Context or Law: If specific statutes, case precedents, or definitions are relevant, include them (or summaries of them) in the prompt. O1 doesn’t have browsing and might not recall a niche law from memory, so if your analysis hinges on, say, the text of a particular law, give it to the model. For instance: “According to [Statute X excerpt], [provide text]… Apply this statute to the case.” This way, the model has the necessary tools to reason accurately. Set the Role in the System Message: A system instruction like “You are a legal analyst who explains the application of law to facts in a clear, step-by-step manner.” will cue the model to produce a formal, reasoned analysis. While O1 will already attempt careful reasoning, this instruction aligns its tone and structure with what we expect in legal discourse (e.g. citing facts, applying law, drawing conclusions). No Need for Multiple Examples: Don’t supply a full example case analysis as a prompt (which you might consider doing with GPT-4o). O1 doesn’t need an example to follow – it can perform the analysis from scratch.. You might, however, briefly mention the desired format: “Provide your answer in an IRAC format (Issue, Rule, Analysis, Conclusion).” This format instruction gives a template without having to show a lengthy sample, and O1 will organize the output accordingly. Control Verbosity as Needed: If you want a thorough analysis of the case, let O1 output its comprehensive reasoning. The result may be several paragraphs covering each issue in depth. If you find the output too verbose or if you specifically need a succinct brief (for example, a quick advisory opinion), instruct the model: “Keep the analysis to a few key paragraphs focusing on the core issue.” This ensures you get just the main points. On the other hand, if the initial answer seems too brief or superficial, you can prompt again: “Explain in more detail, especially how you applied the law to the facts.” O1 will gladly elaborate because it has already done the heavy reasoning internally. Accuracy and Logical Consistency: Legal analysis demands accuracy in applying rules to facts. With O1, you can trust it to logically work through the problem, but it’s wise to double-check any legal citations or specific claims it makes (since its training data might not have every detail). You can even add a prompt at the end like, “Double-check that all facts have been addressed and that the conclusion follows the law.” Given O1’s self-checking tendency, it may itself point out if something doesn’t add up or if additional assumptions were needed. This is a useful safety net in a domain where subtle distinctions matter. Use Follow-Up Queries: In a legal scenario, it’s common to have follow-up questions. For instance, if O1 gives an analysis, you might ask, “What if the contract had a different clause about termination? How would that change the analysis?” O1 can handle these iterative questions well, carrying over its reasoning. Just remember that, if the project you ar working on, the interface doesn’t have long-term memory beyond the current conversation context (and no browsing), each follow-up should either rely on the context provided or include any new information needed. Keep the conversation focused on the case facts at hand to prevent confusion. By applying these best practices, your prompts will guide O1 or O3-mini to deliver high-quality legal analysis. In summary, clearly present the case, specify the task, and let the reasoning model do the heavy lifting. The result should be a well-reasoned, step-by-step legal discussion that leverages O1’s logical prowess, all optimized through effective prompt construction. Using OpenAI’s reasoning models in this way allows you to tap into their strength in complex problem-solving while maintaining control over the style and clarity of the output. As OpenAI’s own documentation notes, the O1 series excels at deep reasoning tasks in domains like research and strategy– legal analysis similarly benefits from this capability. By understanding the differences from GPT-4o and adjusting your prompt approach accordingly, you can maximize the performance of O1 and O3-mini and obtain accurate, well-structured answers even for the most challenging reasoning tasks.Learn about Azure AI during the Global AI Bootcamp 2025
The Global AI Bootcamp starting next week, and it’s more exciting than ever! With 135 bootcamps in 44 countries, this is your chance to be part of a global movement in AI innovation. 🤖🌍 From Germany to India, Nigeria to Canada, and beyond, join us for hands-on workshops, expert talks, and networking opportunities that will boost your AI skills and career. Whether you’re a seasoned pro or just starting out, there’s something for everyone! 🚀 Why Attend? 🛠️ Hands-on Workshops: Build and deploy AI models. 🎤 Expert Talks: Learn the latest trends from industry leaders. 🤝 Network: Connect with peers, mentors, and potential collaborators. 📈 Career Growth: Discover new career paths in AI. Don't miss this incredible opportunity to learn, connect, and grow! Check out the event in your city or join virtually. Let's shape the future of AI together! 🌟 👉 Explore All Bootcamps
Built-in Enterprise Readiness with Azure AI Agent Service
Ensure enterprise-grade security and compliance with Private Network Isolation (BYO VNet) in Azure AI Agent Service. This feature allows AI agents to operate within a private, isolated network, giving organizations full control over data and networking configurations. Learn how Private Network Isolation enhances security, scalability, and compliance for mission-critical AI workloads.Announcing Provisioned Deployment for Azure OpenAI Service Fine-tuning
You've fine-tuned your models to make your agents behave and speak how you'd like. You've scaled up your RAG application to meet customer demand. You've now got a good problem: users love the service but want it snappier and more responsive. Azure OpenAI Service now offers provisioned deployments for fine-tuned models, giving your applications predictable performance with predictable costs! 💡 What is Provisioned Throughput? If you're unfamiliar with Provisioned Throughput, it allows Azure OpenAI Service customers to purchase capacity in terms of performance needs instead of per-token. With fine-tuned deployments, it replaces both the hosting fee and the token-based billing of Standard and Global Standard (now in Public Preview) with a throughput-based capacity unit called provisioned through units (PTU). Every PTU corresponds to a commitment of both latency and throughput in Tokens per Minute (TPM). This differs from Standard and Global Standard which only provide availability guarantees and best-effort performance. With fine-tuned deployments, it replaces both the hosting fee and the token-based billing of Standard and Global Standard (now in Public Preview) with a throughput-based capacity unit called a PTU. 🤔 Is this the same PTU I'm already using? You might already be using Provisioned Throughput Units with base models and with fine-tuned models they work the same way. In fact, they're completely interchangeable! Already have quota in North Central US for 800 PTU and an annual Azure reservation rate? PTUs are interchangeable and model independent meaning you can get started with using them for fine-tuning immediately without any additional steps. Just select Provisioned Managed (Public Preview) from the model deployment dialog and set your PTU allotment. 📋 What's available in Public Preview? We're offering provisioned deployment in two regions for both gpt-4o (2024-08-06) and gpt-4o-mini (2024-07-18) to support Azure OpenAI Service customers: North Central US Switzerland West If your workload requires regions other than the above, please make sure to submit a request so we can consider it for General Availability. 🙏 🚀 How do I get started? If you don't already have PTU quota from base models, the easiest way to get started and shifting your fine-tuned deployments to provisioned is: Understand your workload needs. Is it spiky but with a baseline demand? Review some of our previous materials on right-sizing PTUs (or have CoPilot summarize it for you 😆). Estimate the PTUs you need for your workload by using the calculator. Increase your regional PTU quota, if required. Deploy your fine-tuned models to secure your Provisioned Throughput capacity. Make sure to purchase an Azure Reservation to cover your PTU usage to save big. Have a spiky workload? Combine PTU and Standard/Global Standard and configure your architecture for spillover. Have feedback as you continue on your PTU journey with Azure OpenAI Service? Let us know how we can make it better!Capacity's AI Answer Engine® leveraged Phi to deliver better results for their customers, faster
Capacity an all-in-one Support Automation Platform, provides organizations with the ultimate Answer Engine®. They needed a way to help unify diverse datasets across tens of millions of search results and billions of interactions and make information more easily accessible and understandable for their customers. By leveraging Phi—Microsoft’s family of powerful small language models offering groundbreaking performance at low cost and low latency—Capacity provides the enterprise with an effective AI knowledge management solution that democratizes knowledge on large teams securely and in a way that maximizes value to the customer. With Phi, Capacity’s Answer Engine® improved results quality and scale, so customers save both time and money by more quickly finding the rich information they invested in to do their best work. What was the challenge? Enterprise employees struggle to find the data they need searching through isolated, untagged content, leading to frustration and wasted time. To address this, Capacity’s Answer Engine® retrieves information across diverse enterprise systems, repositories and sources, instantly delivering the exact answers needed to inform work and make faster decisions. At the same time, AI can only go so far to unify and enrich this data. Capacity addressed the challenge by leveraging Phi using Azure Serverless API to experiment on the effectiveness of Language Model-based tagging infrastructure. They applied prompt engineering, adherence workflows, and at-scale testing to better prepare Answers for search and create a more universal Answer Engine®. Why did Capacity choose Phi? Capacity chose Phi-3.5-mini for its speed, cost-effectiveness, and deployment flexibility. With Azure Models as a Service (MaaS), Capacity was able to use the Phi family models without having to provision GPUs or manage back-end operations, saving their team time, effort, and cost. They used prompt engineering and metadata tagging to optimize search results, ultimately improving development speed and query processing efficiency. Additionally, the favorable MIT Open Source licensing of the Phi family provided a strong long-term strategy for their private cloud deployment, vectorization, and query routing activities. "From our initial experiments, what truly impressed us about the Phi was its remarkable accuracy and the ease of deployment, even before customization. Since then, we've been able to enhance both accuracy and reliability, all while maintaining the cost-effectiveness and scalability we valued from the start." ~ Steve Frederickson, Head of Product, Answer Engine How did they solve for it? To achieve their goal, Capacity implemented Phi-3-mini and Phi-3.5-mini Model-as-a-Service, using both 4k and 128K variants with some prompt engineering. This allowed them to accelerate development on their AI-powered Answer Engine and help their enterprise customers deliver the right information to their end users quickly and accurately. When presenting an Answer to their customer’s end user, Capacity wanted their AI Answer engine to instantly present the full Answer along with all the content metadata around it, so the end user could feel confident in their search results. To accomplish this, Capacity engineers split the tasks for Phi into preprocessing and real-time flows. In preprocessing, they generated metadata such as title summaries for answers, keyword tags for search, and other information to the index. This pre-work was done offline and ahead of time. Depending on the tagging task required for each Answer, they calculated the needed token size then rerouted the query to the appropriate Phi model. At query time, Phi models pre-process the query to retrieve the most relevant content. The split tasks for Phi enabled repeatable performance, keeping the responsive query times users expect while enhancing results with new functionality and increased retrieval relevance. At the same time, the cost-efficiency of Phi was able to produce the same or better qualitative results for preprocessing with a 4.2x cost savings as compared to the competing workflow. The considerable cost savings on the preprocessing allows Capacity to scale to ever-growing datasets. While the increased retrieval relevance fosters sustained growth and enhances user satisfaction. After integrating Phi, Capacity observed significant improvements in both performance and customer satisfaction. The AI-powered solutions provided faster and more accurate information retrieval, which reduced time users spent searching for information. Additionally, the seamless integration of datasets with the Phi-3.5-mini model as a service significantly empowered Capacity to address a wide range of use cases with enhanced efficacy, ultimately elevating the user experience. Steve Frederickson, Capacity's Head of Product, Answer Engine, noted, “Integrating our datasets with the Phi-3.5-mini model was effortless. We have found new opportunity in its speed, and the enriched customer experience of GenAI enables us to resolve customer issues more effectively, delivering a superior user experience." Capacity also shared some valuable tips for other organizations looking to implement similar AI solutions. They recommended designing the system to optimize for query performance and retrieval accuracy, including adding metadata and keyword tags to optimize search efficiency. They also emphasize the importance of choosing the right AI model based on the capability and scalability, to balance speed and cost-effectiveness. The next step Implementing Phi has revolutionized Capacity’s approach to knowledge management, providing their enterprise customers with efficient and accurate information retrieval solutions. Their success highlights the potential of the Phi model family to transform enterprise operations and improve user experiences. Looking ahead, Capacity plans to explore additional state-of-the-art models such as Phi-4-multimodal and Phi-4-mini for more complex reasoning tasks like multilingual support and image understanding scenarios. They also aim to fine-tune their solutions to enhance their knowledge graph and improve interoperability among different institutional knowledge bases. By continuously innovating and leveraging advanced AI technology, the Capacity Answer Engine® is well-positioned to remain at the forefront of knowledge management solutions, helping organizations do their best work with the complexities of information retrieval and discovery. Learn more about the Phi family of models here: About Phi Learn about the latest updates Download the modelsRAG Time: Your Guide to Mastering Retrieval-Augmented Generation!
RAG Time is a brand-new AI learning series designed to help developers unlock the full potential of Retrieval-Augmented Generation (RAG). If you’ve been looking for a way to build smarter, more efficient AI systems—join us in RAG Time, every Wednesday 9AM PT from March 5 through April 2 on Microsoft Developer YouTube. What's in RAG Time? RAG Time is a five-part learning journey, with new videos and blog posts releasing every week in March. The series features: 🔥 Expert-led discussions breaking down RAG fundamentals and best practices 🎤 Exclusive leadership interviews with AI leaders ⚡ Hands-on demos & real-world case studies showing RAG in action 🎨 Creative doodle summaries making complex concepts easier to grasp and remember 🛠 Samples & resources in the RAG Time repository so you can start building today What You’ll Learn The series is structured into five learning journeys, each tackling a crucial aspect of RAG-powered AI: 📌 March 5th, 9AM PT - Journey 1: RAG and Knowledge Retrieval Fundamentals – Start with the basics! Learn how RAG, search indexing, and vector search work together to create smarter AI retrieval systems. 📌 March 12th, 9AM PT - Journey 2: Build the Ultimate Retrieval System for RAG – Go beyond the fundamentals with hybrid search, semantic ranking, and relevance tuning to refine how AI retrieves the most relevant information. 📌 March 19th, 9AM PT - Journey 3: Optimize Your Vector Index for Scale – Learn how to scale vector search efficiently, optimize storage, and implement advanced techniques like quantization and Matryoshka learning for large-scale AI applications. 📌 March 26th, 9AM PT - Journey 4: RAG for All Your Data: Multimodal and Beyond – Move beyond text-based retrieval! Discover how to integrate images, audio, and structured data into your RAG workflows and leverage multimodal pipelines for next-level AI capabilities. 📌 April 2nd, 9AM PT - Journey 5: Hero Use Cases for RAG – Explore real-world implementations, industry-leading examples, and best practices, while diving into Responsible AI considerations to ensure ethical and impactful solutions. Why You Should Watch If you're a developer, data scientist, or AI enthusiast, this series is built for you! Whether you’re just getting started or looking to master enterprise-grade retrieval systems, RAG Time delivers practical knowledge, hands-on resources, and expert insights to help you stay ahead. Journey starts here 🚀 Start your journey from the RAG Time repo: https://aka.ms/rag-time. You'll find all the information about the video series, samples, documentation and doodles in the repo! Share your experience and feedback on GitHub discussions.Introducing Azure AI Agent Service
Introducing Azure AI Agent Service at Microsoft Ignite 2024 Discover how Azure AI Agent Service is revolutionizing the development and deployment of AI agents. This service empowers developers to build, deploy, and scale high-quality AI agents tailored to business needs within hours. With features like rapid development, extensive data connections, flexible model selection, and enterprise-grade security, Azure AI Agent Service sets a new standard in AI automationCode First Distillation with Stored Completions in Azure OpenAI Service
We are thrilled to announce the Public Preview release of the Stored Completions API and SDK in Azure OpenAI Service! Following our recent announcement on the enhanced Azure OpenAI Service distillation and Fine-Tuning Capabilities, we are excited to introduce a set of new API capabilities and an SDK experience that will empower our customers to interact with Stored Completions through code. What were we supporting before? Model distillation empowers developers to use the outputs of large, complex models to fine-tune smaller, more efficient ones. This technique allows the smaller models to perform just as well on specific tasks, all while significantly cutting down on both cost and latency. Azure OpenAI Service distillation involves three main components: Stored Completions: Capture and store input-output pairs from models like GPT-4o to generate datasets for evaluation and fine-tuning. The feature offers an interface for reviewing, filtering, and exporting data based on predefined criteria. Evaluation: Create and run custom evaluations to measure model performance using data from Stored Completions or existing datasets. The Evaluation experience provides a UI-based approach to score data. Fine-tuning: Integrate Stored Completions and Evaluation with Azure OpenAI fine-tuning. Use datasets from Stored Completions in fine-tuning jobs and run evaluations on fine-tuned models. Together, these steps create a comprehensive distillation process: collecting live traffic, filtering and subsetting it in the Stored Completions UI, exporting it to the Evaluation UI for quality scoring, and fine-tuning from the collected data. What is new? In addition to our UI offering for Stored Completions, we are now providing a code-first approach to an end-to-end distillation flow in Azure OpenAI Service. We are introducing Stored Completions API that help our customers to: Retrieve stored completions by ID: GET /chat/completions/:id Get stored completions messages: GET /chat/completions/:id/messages Get a full list of stored completions: GET /chat/completions Update metadata of stored completions: POST /chat/completions/:id Delete stored completions by ID: DELETE /chat/completions/:id These new capabilities will enable developers to seamlessly integrate stored completions into their workflows, providing greater flexibility and control over their data and models in the Distillation Process. Maximizing Efficiency and Control with Stored Completions API and SDK The value of using the API and SDK for retrieving, updating, or deleting stored completions lies in the enhanced flexibility, efficiency, and control they offer developers. The Stored Completions UI provides an interface for reviewing, filtering, and exporting data based on predefined criteria. The new APIs not only enable the retrieval of stored completions but also support update and delete functionalities. Retrieving Stored Completions: The API and SDK allow developers to programmatically access stored completions, enabling seamless integration into their workflows. This is particularly useful for automated dataset generation, ensuring that the data is always up-to-date and relevant for training machine learning models. Updating Stored Completions: By using the API and SDK to update metadata or other attributes of stored completions, developers can maintain accurate and organized datasets. This helps in refining the data for better evaluation and fine-tuning of models. Deleting Stored Completions: The API and SDK provide the ability to delete stored completions that are no longer needed, ensuring that the dataset remains clean and manageable. This is crucial for maintaining data quality and relevance. Overall, the API and SDK enhance the efficiency of managing stored completions, making it easier for developers to generate, update, and maintain high-quality datasets for their machine learning projects. Getting Started with the Stored Completions API and SDK Here is an example of an API call to get list of stored completions: Example Request and Response: To get started with the SDK, you can follow these steps: Install the SDK, Initialize the client instance to interact with the APIs and start using the APIs to interact with stored completions. And here is an example of how to use the SDK: The introduction of the Stored Completions API and SDK in Azure OpenAI Service marks a significant milestone in our journey to enhance the capabilities of our platform. These new functionalities provide developers with greater flexibility, efficiency, and control over their data and models, enabling them to create high-quality datasets seamlessly. We are excited to see how our customers will leverage these new features to drive innovation and achieve their goals. Stay tuned for more updates and happy coding! Want to learn more? How to use Azure OpenAI Service Stored Completions and Distillation
Using Semantic Kernel to control a BBC Microbit
Background The BBC Micro:bit has proved very popular in UK schools as a cheap and simple device that can be used to demonstrate basic coding, either using a building block approach or by using Python code in an editor. There are browser-based tools to allow editing of the code in either mode. The BBC Micro:bit is then connected via USB and the browser is then able to upload the code to the BBC Micro:bit. Thus making it a very interactive experience. What if the power of Azure OpenAI prompts could be combined with the simplicity of the BBC Micro:bit to allow prompts to directly program the BBC Micro:bit? This is what this blog explains. Semantic Kernel This blog assumes the reader has a basic understanding of Azure OpenAI and that there is an API that allows you to send requests and get responses from a model provisioned in Azure OpenAI. If not please look here A little background on Semantic Kernel may be useful. This is an open-source framework for the building of AI agents. Semantic Kernel makes use of plugins as a mechanism to perform specific tasks. A plugin is a specific piece of code that is tagged with metadata in a way such that Semantic Kernel will know whether to use the plugin and if so, how to call the plugin. For those conversant with OpenAI function calling, Semantic Kernel automates the process of registering a function with Azure OpenAI and calling that function when the response from Azure OpenAI believes that the user's query can be best answered by calling a function. This mechanism is extremely powerful as it opens up huge possibilities for an Azure OpenAI model to integrate with any number of things. In this example, the BBC Microbit will be a plugin to Semantic Kernel. The Demo Semantic Kernel app In order to keep things really simple, this demonstration is a console application that runs a Semantic Kernel planner with a plugin for the BBC Micro:bit. The code for this is really simple. In the first step below, create the Semantic Kernel with your model, endpoint and key. You will need an instance of Azure OpenAI with a model like gpt-4o provisioned. #pragma warning disable SKEXP0050 #pragma warning disable SKEXP0060 using System.ComponentModel; using Microsoft.SemanticKernel; using Microsoft.SemanticKernel.ChatCompletion; using Microsoft.SemanticKernel.Connectors.OpenAI; using Microsoft.Extensions.Logging; using Microsoft.Extensions.DependencyInjection; using System; // Create the kernel var builder = Kernel.CreateBuilder(); builder.AddAzureOpenAIChatCompletion( "YOUR_MODEL", "YOUR_ENDPOINT", "YOUR_KEY"); Next register the plugin builder.Plugins.AddFromType<LightsPlugin>(); builder.Plugins.AddFromType<MicrobitPlugin>(); in the sample there are 2 plugins, one that is for the BBC Micro:bit and another one that emulates the grid of lights on the BBC Micro:bit, so you can get Semantic Kernel to control this if you don't have a BBC Micro:bit connected. We will, however, concentrate on the BBC Micro:bit in this blog. Next we enable automatic function calling and a chat history object that we will initialise with some instructions to explain to the model what it can do: Kernel kernel = builder.Build(); // Retrieve the chat completion service from the kernel IChatCompletionService chatCompletionService = kernel.GetRequiredService<IChatCompletionService>(); // 2. Enable automatic function calling OpenAIPromptExecutionSettings openAIPromptExecutionSettings = new() { ToolCallBehavior = ToolCallBehavior.AutoInvokeKernelFunctions }; // Create the chat history ChatHistory history = new ChatHistory(""" You have a matrix of 5 rows and columns each of which can have a brightness of 0 to 9. Rows make up the horizontal axis. The top row has coordinates 0,1 to 4, 0. Columns represent the vertical axis. The left most column is 0,0 to 0, 4 and the right column is 4,0 to 4,4. """); This is all the initialisation needed and so the final step is to run the prompt loop where you can prompt the application and it will respond. string? userInput; do { // Collect user input Console.Write("User > "); userInput = Console.ReadLine(); if (userInput is not null) { // Add user input history.AddUserMessage(userInput); // Get the response from the AI with automatic function calling var result = await chatCompletionService.GetChatMessageContentAsync( history, executionSettings: openAIPromptExecutionSettings, kernel: kernel); // Print the results Console.WriteLine("Assistant > " + result); // Add the message from the agent to the chat history history.AddMessage(result.Role, result.Content ?? string.Empty); } } while (userInput is not null); As can be seen above, this is just a loop that requests input, passes to the chatCompletionService, gets the response, displays it and then it adds this to the history. In this manner the agent is conversational and so will take into account previous prompts when answering the current one. In this case, the conversation history is not not persisted. BBC Microbit plugin The BBC Micro:bit has a USB port and when plugged into a PC, presents itself as a serial port device to your PC. The port can be found out from Windows Device Manager, but it is often COM3. The plugin uses a feature of the BBC Micro:bit where commands can interactively be sent. This is referred to as Read-Evaluate-Print-Loop REPL The plugin opens a connection to a COM port on initialisation, so this should error if the BBC Micro:bit is not plugged-in or presents itself on a different COM port. The code that may need amending is in the MicrobitPlugin.cs SerialPort serialPort; public MicrobitPlugin() { serialPort = new SerialPort("COM3", 115200); serialPort.Open(); //send crt-c to stop any running program serialPort.Write(new byte[] { 0x03 }, 0, 1); } The above opens the serial port for later use so that commands may be sent As explained in the Semantic Kernel description, meta data is used against a plugin function in order to indicate to Semantic Kernel and then Azure OpenAI what that function does. The simplest function is to set a specific pixel on the BBC Micro:bit display [KernelFunction("set_microbit_light_brightness")] [Description("sets the brightness of a microbit pixel by its row and column ID.")] public async Task<int> SetLightBrightness( Kernel kernel, int rowid, int columnid, int brightness ) { SendCommand(serialPort, $"display.set_pixel({rowid},{columnid},{brightness})"); return brightness; } This then calls the serial port to send the command to the BBC Microbit using the serialPort object that was previously opened: private void SendCommand(SerialPort serialPort, string command) { //send crt-c to stop any running program serialPort.Write(new byte[] { 0x03 }, 0, 1); // Wait for the micro:bit to stop any running program System.Threading.Thread.Sleep(50); // Adjust the delay as needed // Send the command to the micro:bit serialPort.Write(command); serialPort.WriteLine(" "); // send a carriage return to execute the command serialPort.Write(new byte[] { 0x0d }, 0, 1); // Wait for the micro:bit to finish executing the command System.Threading.Thread.Sleep(50); // Adjust the delay as needed // Read the response from the micro:bit var response = serialPort.ReadExisting(); // Print the response to the console Console.WriteLine(response); } As can be seen above, there is metadata which describes the purpose of the function and the parameters are inspected too. For the light, these are the row and column and brightness. This then sends a specific display.set_pixel command to the BBC Micro:bit. The number of rows and columns is not fixed here, nor the brightness levels. These are explained to the model in the original prompt (chat history) before the main Semantic Kernel planner loop is run. Some sample prompts Once the program is built and run. You can try any number of prompts: set the top left light to brightness 4 set it the last light to a lower brightness turn off all lights draw a circle make the circle bigger Draw the letter "W" Using the application, you can see that even with this basic plugin function, the planner can do more complicated things than expected, it has history so can refer to a previous prompt, but can do much more interesting things like draw shapes or letters that require multiple calls to the plugin function! Extending the plugin The REPL interface to the BBC Micro:bit can send all sorts of other commands besides the display.set_pixel one, so it makes sense to expand the plugin to give it more broad capability: [KernelFunction("set_microbit_command")] [Description("send a command to the microbit using REPL.")] public void SendGenericCommand( Kernel kernel, string command ) { SendCommand(serialPort, command); } The above plugin function allows any arbitrary command to be sent to the BBC Microbit. Allowing a wider set of use cases or more compact commands to be sent to the BBC Microbit. Some more prompts As this extra plugin function need not be doing things just to the lights you can ask it to: make a sound ask the current temperature ask it if one of the buttons is pressed For the lights too, there are extra possibilities: display a word with each letter in sequence display a scrolling word or sentence Summary What can be seen in this demonstration is how simple it is for Semantic Kernel to control an external device using a plugin and a simple planning loop. Many of the models on Azure OpenAI have enough training data to natively understand what REPL commands can be sent to the BBC Micro:bit to accomplish a whole task. The main limiting factor seen is the ability of the REPL interface to accept complex commands from the planner. This is largely to do with Python as a language and how control flow get expressed as indentations in the code (rather than using some form of brackets like C#). Further reading There are some labs that may be used to explore this further. In addition, for those who do not have access to a BBC Micro:bit, there are earlier labs that are essentially the same but with an array of virtual lights which represent the light matrix on the BBC Micro:bit. Have a play 🙂
