Prompt Engineering Simplified: AI Toolkit's Prompt Builder
In the age of generative AI, crafting effective prompts is no longer a nice-to-have, it's a must-have. Understanding how to communicate with these underlying models is the key to unlocking their true potential and getting the results we need. What are Prompts? Every time we want to communicate to the language model, we give set of instructions to these models, we refer to these inputs as Prompts. Prompts play a very crucial role while working with the GenAI models. The quality of a prompt directly impacts the output of GenAI models. Precise and well-crafted prompts are crucial for achieving desired results. What factors crafts an optimal Prompt? Crafting an optimal requires balancing clarity, specificity and context. Besides these, constraints are a critical factor in crafting effective prompts. Specificity Clearly define the expectations. The prompt should leave no room for misinterpretation. Precise language is the key. Avoid vague language. Vague prompts lead to vague or irrelevant responses. e.g., “Tell me about history” ➔ “Explain the economic causes of the French Revolution”. Clarity Use simple, unambiguous language. Avoid jargon unless your audience expects it, recommended to use action verbs like "write," "summarize," "explain," "translate". Context Provide background for e.g., “As a beginner in coding, how do I write a Python loop?”. Give the LLM enough context to understand the situation. Include relevant details, keywords, and background information Conciseness Trim unnecessary words (e.g., “Describe photosynthesis” vs. “Can you tell me about how plants use sunlight?”). Ensure the prompt remains relevant to the desired output Tone & Audience Alignment Match the tone to the goal (formal, casual, instructive). Example: For kids, “Explain how rainbows form in simple terms.” Explicit Instructions Directly state what is needed e.g., “Compare X and Y”, “List pros and cons,” “Write a poem about…”. Guiding Constraints Limit scope to avoid overly broad answers e.g., “Focus on environmental impacts, not economic ones”. Constraints reduce ambiguity, focus responses, and improve relevance. Few example constraints, Format: “Summarize in 3 bullet points.” Length: “Explain in 2 sentences.” Scope: “Focus on environmental impacts, not economic ones.” Style/Tone: “Write a casual email,” or “Use non-technical terms.” Technical limits: “Keep code examples under 50 lines.” Few advanced Considerations for AI/LLM Prompts Examples or Demonstrations Include examples to set expectations for e.g., “Write a limerick like this: There once was a cat from Peru…”. Step-by-Step Guidance Break complex tasks into steps for e.g., “First analyze the Python code, then suggest solutions”. Role Assignment Assign roles to guide the AI for e.g., “Act as a historian explaining World War 2”. Avoid Bias Neutral phrasing ensures fair responses for e.g., “Discuss pros and cons of renewable energy” vs. “Why is solar energy bad?” the former is a well-formed prompt. Prompt engineering is an iterative process. Experiment with different phrasings and structures to see what works best. Analyze the LLM's responses and refine prompts accordingly. Make adjustments to improve the accuracy and relevance of the output. Prompt Builder: From the above section, we know that crafting effective prompts is essential for robust AI engagement. Prompt Builder tool on AI Toolkit helps in this enhancement by streamlining the whole process of crafting prompts. Prompt builder helps the users by helping in the following areas, o Prompt Creation, Modification, and Evaluation: Customize prompts through an accessible and straightforward interface. o AI-Assisted Prompt Generation: Articulate the project concept using everyday language, and the AI-powered feature will produce prompts for your exploration. o Organized Output Capability: Craft the prompts to yield outputs in a consistent, standardized and predictable manner. o Automated Code Generation for Prompt Usage: Following model and prompt experimentation, transition to coding immediately by accessing automatically generated, executable Python code. This tool has three sections on the UI. Prompt configuration Response History Prompt Configuration Section: In the Prompt configuration section, there are 4 major sub sections, Model System Prompt User Prompt Add Prompt Model: The Model section is the first subsection of the Prompt Configuration. Here, we select the model to use. The AI Toolkit offers a wide range of models, including remote models served from GitHub and those from providers such as OpenAI, Google, Anthropic, and Nvidia. For this tutorial we will be using OpenAI GPT-4o mini via GitHub System Prompt: In System prompt section, we provide instructions with relevant context to guide the system response. We can think of a system prompt as the "role" we give an AI before we ask it anything, like telling an actor what character to play. Generate Prompt: Upon choosing cloud-based / GitHub / Remote models, a new tool called as “Generate Prompt” is enabled, this is an AI Powered tool especially useful for crafting AI Powered well defined prompts which can be used in the “System Prompt” Section. Upon clicking on the “Generate Prompt” we can see a small window that pops up and asks for the input prompt. This can generate a prompt template by sharing basic details about the task. In this tutorial, let’s ask the LLM to generate prompt about “Professor in university teaching math”. Once the message is updated click on “Generate” button, and in a few seconds, we will have a well-structured prompt in the “System Prompt” section. The prompt that we generated is as follows Provide a detailed syllabus for a university-level mathematics course, including course objectives, weekly topics, assessment methods, and required materials. The syllabus should cover all essential components such as the course title, description, prerequisites, learning outcomes, weekly schedules, and any relevant policies regarding attendance, grading, and participation. # Steps 1. **Course Title and Description**: Clearly state the title of the course and provide a brief description of what the course will cover. 2. **Prerequisites**: List any required courses or knowledge necessary for students to enroll. 3. **Learning Outcomes**: Define what students are expected to learn by the end of the course. 4. **Weekly Schedule**: Outline topics for each week, along with any associated readings or assignments. 5. **Assessment Methods**: Describe how students will be evaluated (e.g., exams, quizzes, projects). 6. **Required Materials**: Include information on textbooks and other resources needed for the course. 7. **Course Policies**: State attendance, grading, and participation rules. # Output Format The output should be formatted as a structured syllabus, presented in clear sections with headings for each part. The document should be detailed yet concise, ideally around 3-5 pages in length. # Examples **Example 1** **Input:** Create a syllabus for a Calculus I course. **Output:** - **Course Title**: Calculus I - **Description**: An introduction to limits, derivatives, and integrals. - **Prerequisites**: Pre-Calculus or equivalent. - **Learning Outcomes**: Students will be able to calculate limits, differentiate basic functions, and understand the Fundamental Theorem of Calculus. - **Weekly Schedule**: - Week 1: Introduction to Limits - Week 2: Continuity - Week 3: Derivatives - ... - **Assessment Methods**: Midterm exam (30%), Final exam (40%), Weekly quizzes (20%), Participation (10%). - **Required Materials**: "Calculus: Early Transcendentals" by James Stewart. - **Course Policies**: Attendance required, late assignments will incur a penalty. **Example 2** **Input:** Design a syllabus for a Linear Algebra course. **Output:** - **Course Title**: Linear Algebra - **Description**: Study vector spaces, matrices, and linear transformations. - **Prerequisites**: None. - **Learning Outcomes**: Mastery of matrix operations and ability to solve systems of linear equations. - **Weekly Schedule**: - Week 1: Introduction to Vector Spaces - Week 2: Matrix Operations - Week 3: Determinants - ... - **Assessment Methods**: Two midterms (50%), Homework assignments (30%), Attendance (20%). - **Required Materials**: "Linear Algebra Done Right" by Sheldon Axler. - **Course Policies**: Participation in class discussions is mandatory. # Notes Ensure that the syllabus is comprehensive and tailored to the specific course topic. Consider including any unique teaching methods or technologies that will be employed during the course. User Prompt: User prompt is the specific question, instruction, or request that a person provides to the AI to elicit a response. It's the direct input from the user that initiates the AI's processing and generation of text. In AI Toolkit for a few models that support the multimodal feature, we can also upload images in this section. For this tutorial let’s input “Explain to me the Fourier equation in simple terms” Add Prompt: If any additional prompt needs to be added, we can configure more User or assistant prompt. So, in a conversation, we have: User Prompt: What the human says. Assistant Prompt: What the AI says. The major configuration part is now completed through this window, its now time to test the responses based on the LLM’s knowledge, in this case how well does GPT 4o mini behave in the role as university-level mathematics professor. In order to test it, we navigate to the next window, the Response section. Response Section: The Response section is where we finally get to see the responses. This section has the “Run” and “View Code” buttons. We can also choose the type of response we need. It can be a simple text or json schema. Upon choosing Json Schema, user will be prompted to “Prepare Schema”. Users can define their own schema or select from example. There are a few examples for the user to choose from. For this tutorial we will be using the simple text format. As we have our setup ready, we can directly click on the “Run” button, In a few seconds we have our well formatted and accurate answer on the screen, AI Toolkit‘s markdown capability can neatly format all the mathematical signs and equations. We can also add this to the “Assistant Prompt” by using the button provided. It provides better example for the LLM in the code later. The result from the LLM now seems very satisfactory with our well-crafted prompt. We can now proceed with the Code generation feature of the Prompt Builder tool of AI Toolkit. Upon clicking the “View Code” button, user is prompted to choose the SDK of their choice. This SDK lets us communicate with the API from the code. For this tutorial, we will use Azure AI Inference SDK. For more details on this SDK refer here. The code requires azure-ai-inference. Install the library by pip install azure-ai-inference """Run this model in Python > pip install azure-ai-inference """ import os from azure.ai.inference import ChatCompletionsClient from azure.ai.inference.models import AssistantMessage, SystemMessage, UserMessage from azure.ai.inference.models import ImageContentItem, ImageUrl, TextContentItem from azure.core.credentials import AzureKeyCredential # To authenticate with the model you will need to generate a personal access token (PAT) in your GitHub settings. # Create your PAT token by following instructions here: https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens client = ChatCompletionsClient( endpoint = "https://models.inference.ai.azure.com", credential = AzureKeyCredential(os.environ["GITHUB_TOKEN"]), api_version = "2024-08-01-preview", ) response = client.complete( messages = [ SystemMessage(content = "Provide a detailed syllabus for a university-level mathematics course, including course objectives, weekly topics, assessment methods, and required materials.\n \nThe syllabus should cover all essential components such as the course title, description, prerequisites, learning outcomes, weekly schedules, and any relevant policies regarding attendance, grading, and participation.\n \n# Steps\n \n1. **Course Title and Description**: Clearly state the title of the course and provide a brief description of what the course will cover.\n2. **Prerequisites**: List any required courses or knowledge necessary for students to enroll.\n3. **Learning Outcomes**: Define what students are expected to learn by the end of the course.\n4. **Weekly Schedule**: Outline topics for each week, along with any associated readings or assignments.\n5. **Assessment Methods**: Describe how students will be evaluated (e.g., exams, quizzes, projects).\n6. **Required Materials**: Include information on textbooks and other resources needed for the course.\n7. **Course Policies**: State attendance, grading, and participation rules.\n \n# Output Format\n \nThe output should be formatted as a structured syllabus, presented in clear sections with headings for each part. The document should be detailed yet concise, ideally around 3-5 pages in length.\n \n# Examples\n \n**Example 1** \n**Input:** \nCreate a syllabus for a Calculus I course. \n**Output:** \n- **Course Title**: Calculus I \n- **Description**: An introduction to limits, derivatives, and integrals. \n- **Prerequisites**: Pre-Calculus or equivalent. \n- **Learning Outcomes**: Students will be able to calculate limits, differentiate basic functions, and understand the Fundamental Theorem of Calculus. \n- **Weekly Schedule**: \n - Week 1: Introduction to Limits \n - Week 2: Continuity \n - Week 3: Derivatives \n - ... \n- **Assessment Methods**: Midterm exam (30%), Final exam (40%), Weekly quizzes (20%), Participation (10%). \n- **Required Materials**: \"Calculus: Early Transcendentals\" by James Stewart. \n- **Course Policies**: Attendance required, late assignments will incur a penalty.\n \n**Example 2** \n**Input:** \nDesign a syllabus for a Linear Algebra course. \n**Output:** \n- **Course Title**: Linear Algebra \n- **Description**: Study vector spaces, matrices, and linear transformations. \n- **Prerequisites**: None. \n- **Learning Outcomes**: Mastery of matrix operations and ability to solve systems of linear equations. \n- **Weekly Schedule**: \n - Week 1: Introduction to Vector Spaces \n - Week 2: Matrix Operations \n - Week 3: Determinants \n - ... \n- **Assessment Methods**: Two midterms (50%), Homework assignments (30%), Attendance (20%). \n- **Required Materials**: \"Linear Algebra Done Right\" by Sheldon Axler. \n- **Course Policies**: Participation in class discussions is mandatory. \n \n# Notes\n \nEnsure that the syllabus is comprehensive and tailored to the specific course topic. Consider including any unique teaching methods or technologies that will be employed during the course."), UserMessage(content = [ TextContentItem(text = "Explain to me the Fourier equation in simple terms"), ]), ], model = "gpt-4o-mini", response_format = "text", max_tokens = 4096, temperature = 1, top_p = 1, ) print(response.choices[0].message.content) This Python code is ready to be modified and used in any Generative AI application. It can be modified with any Orchestration framework like Semantic Kernel to add more features or even make an agentic application. History Section: We also have the “History” and “New Prompt”. History shows all the previous sessions; we can revisit and resume working or perhaps check the output or regenerate the code. History” and “New Prompt” In essence, the Prompt Builder tool significantly streamlines the process of crafting effective prompts, saving developers valuable time. Beyond prompt creation, it also facilitates output evaluation, model behavior analysis, and generates quality code to accelerate application development. Stay tuned for upcoming blog posts, where we'll delve into even more advanced techniques for building powerful generative AI applications. You can also join our AI Sparks series to learn more about the capabilities of the AI Toolkit for Visual Studio Code.120 Days Study Plan to Become an AI-Focused Full-Stack Software Engineer
Hello there, my name is Oumaima, and I am an MLSA student ambassador from Morocco, studying at the University Of The People. Welcome to the first step in my exciting, unpredictable journey, one I’ve chosen to embark on with you! For the past three years, I’ve watched the AI industry evolve dramatically. Generative AI has shifted from a fascinating experiment to an integral part of our everyday lives, whether at school, work, or even in our personal routines. In fact, my ChatGPT app is now my go-to therapist, lawyer, and all-around advisor! As a software engineering student for over three years, I’ve seen the growth of generative AI up close. But this shift didn’t just inspire me; it made me realize that I don’t want to remain only a consumer of this technology. I want to contribute to it! Seeing AI’s ability to mimic human thought, draw connections from vast amounts of information, and deliver impressive results sparked something in me. It showed me that the best way to break into AI might just be to use AI itself as my guide. That’s when the idea came to ask ChatGPT O1-preview for a personalized study plan, crafted uniquely for me. It takes into account my available time, coding background, learning preferences, mental health, and energy. Here’s how my journey began with a simple prompt: I want to become an AI-focused full-stack software engineer and have 120 days to dedicate to this goal. Please create a detailed 120-day study plan tailored for me, dedicating 3-4 hours daily. The study plan should: - Cover all essential topics including programming foundations, data structures and algorithms (DS&A), mathematics for AI, machine learning fundamentals, deep learning, advanced AI topics, integrating AI into applications, web development basics for AI integration, advanced web development, full-stack project development, scripting, DevOps, and career development. - Include weekly breakdowns and daily tasks. - Provide recommended resources for each topic (e.g., online courses, tutorials, documentation). - Suggest hands-on projects or exercises to apply the concepts learned. - Incorporate tips for success, such as active engagement, seeking feedback, balancing depth and breadth, and maintaining well-being. - Emphasize developing all the skills that will make me an irreplaceable software developer, including scripting and DevOps skills. - Conclude with a summary and final advice. Please ensure the plan is structured, comprehensive, and practical for someone balancing work and study. Then it generated the following plan, that I tried to follow by using Microsoft Learn learning paths that offer in depth trainings on each topic I got: Days 1–25: Programming Foundations & Data Structures and Algorithms (DS&A) Microsoft Learn path suggestion: Python for beginners Days 26–50: Mathematics for AI & Machine Learning Fundamentals Microsoft Learn path suggestion: Introduction to machine learning Days 51–80: Deep Learning & Advanced AI Topics Microsoft Learn path suggestion: Train and evaluate deep learning models Days 81–100: Integrating AI into Applications Microsoft Learn path suggestion: Microsoft Azure AI Fundamentals: Generative AI Days 101–115: Advanced Web Development & Full-Stack Project Development Microsoft Learn path suggestion: Build an AI web app by using Python and Flask Days 116–120: Portfolio Projects and Industry Trends. Not going to lie, the roadmap turned out to be even more exciting than I’d expected! When I asked for it, I specified that it should guide me through developing problem-solving skills directly tied to full-stack development. I wanted a path that not only sharpens my abilities but also allows me to build interesting, hands-on applications where I can see the results of what I’m learning. And now, my friends, the journey has officially begun! I’ll be following the roadmap closely, documenting my weekly progress to learn AI, noting the challenges, and celebrating the accomplishments. The goal is to see if artificial intelligence can really help create a customized study plan that aligns with my personal goals, circumstances, and unique learning rhythm. So, stay tuned — this is only the beginning! See you in my first step with DSA!Prepare and get ready for AI-900 Certification
The Azure AI Fundamentals Training program is designed to provide a foundational understanding of artificial intelligence (AI) concepts and how AI services can be utilized on Azure. This training is ideal for individuals new to AI, aiming to build a solid understanding of AI concepts, practical applications, and the Responsible AI considerations involved. Throughout the program, participants will explore various AI services available on Azure. By the end of the program, attendees will be equipped with the knowledge to implement and manage AI solutions using Azure's tools and services The training program will run from 7th January, and we will have live sessions on YouTube and Discord from 8 - 9pm GMT+3 Earn a Certification Voucher Upon successful completion of the program, Kenyan participants will receive a certification voucher. By earning this certification, you'll be able to showcase your knowledge and skills to potential employers and colleagues, giving you a competitive edge in the job market. Key Program Takeaways Gain a foundational understanding of AI concepts and their applications. Explore Azure's AI services and tools. Learn how to implement practical AI solutions using Azure. Understand the responsible AI considerations and best practices for developing and deploying AI projects. Learner Checklist As a learner/participant, this is how you can participate in the program: Catch up and rewatch our previous AI sessions on Microsoft Reactor. Link: https://aka.ms/aifundamentalstraining-reactors Sign up and participate in the Microsoft Learn Challenge: https://aka.ms/aifundamentalstraining-csc - closing soon! Certification vouchers are up for grabs once you complete the challenge for all Kenyan participants. Once you sign up or complete the challenge, fill the form https://aka.ms/aifundamentalstraining-voucher to be eligible for a certification voucher. Join the Discord Community to interact with other learners: https://aka.ms/aifundamentalstraining-discord Sign up to the AINSI Skills Navigator to customize your learning journey at https://aka.ms/aifundamentalstraining-navigator Continue learning and exploring: https://aka.ms/aifundamentalstraining-collection Online Sessions Calendar The training program will run from 7th January, and we will have live sessions on YouTube and Discord from 8 - 9pm GMT+3 Week Topic Live Sessions Link Description 7 Jan Introduction to Artificial Intelligence and Azure AI Services YouTube Embark on a journey to explore the fundamentals of artificial intelligence (AI) with Azure. 9 Jan Microsoft Azure AI Fundamentals: Computer Vision YouTube Dive into the world of computer vision with Azure and discover how to harness the power of AI to analyze and interpret visual data. 14 Jan Microsoft Azure AI Fundamentals: Natural Language Processing YouTube Dive into the fascinating world of natural language processing (NLP) with Azure and learn how to build intelligent applications that can understand and interpret human language 16 Jan Generative AI pt 1 - Fundamentals of Generative AI YouTube Step into the world of generative AI with Azure and discover how to create new content such as text, images, music, and code using advanced AI models. 23 Jan Responsible generative AI YouTube Explore the principles and practices of responsible AI with Azure. 28 Jan Document Intelligence and Knowledge Mining YouTube Discover the power of Azure AI Search and learn how to build intelligent search solutions that can transform your data into actionable insights 29 Jan Generative AI pt 2 - Introduction to Azure AI Foundry YouTube Step into the world of generative AI with Azure and discover how to create new content such as text, images, music, and code using advanced AI models in Azure AI Foundry. 30 Jan Certification Readiness Session Discord Prepare to ace your Microsoft certification exams with this comprehensive walkthrough session. What are you waiting for? Rewatch and engage with live sessions https://aka.ms/aifundamentalstraining-reactors Join in and learn together with us! Remember, certification vouchers are up for grabs! Preparing for the AI-900 Certification can be a rewarding experience that opens up new opportunities in the field of AI and ML. By following the tips and utilizing the resources provided, you'll be well on your way to achieving your certification. Stay motivated, keep learning, and good luck on your journey to becoming AI-900 certified! All learning resources can be found at: MS Learn Collection: https://aka.ms/aifundamentalstraining-collection Enjoyed the session? Send us your feedback, the good, the bad and the ugly at: https://aka.ms/aifundamentalstraining-feedbackThe Launch of "AI Agents for Beginners": Your Gateway to Building Intelligent Systems
🌱 Getting Started Each lesson covers fundamental aspects of building AI Agents. Whether you're a novice or have some experience, you'll find valuable insights and practical knowledge. We also support multiple languages, so you can learn in your preferred language. To see the available languages, click here. If this is your first time working with Generative AI models, we highly recommend our "Generative AI For Beginners" course, which includes 21 lessons on building with GenAI. Remember to star (🌟) this repository and fork it to run the code! 📋 What You Need The course includes code examples that you can find in the code_samples folder. Feel free to fork this repository to create your own copy. The exercises utilize Azure AI Foundry and GitHub Model Catalogs for interacting with Language Models: Github Models - Free / Limited Azure AI Foundry - Azure Account Required We also leverage the following AI Agent frameworks and services from Microsoft: Azure AI Agent Service Semantic Kernel AutoGen For more information on running the code for this course, visit the Course Setup. 🙏 Want to Help? We welcome contributions from the community! If you have suggestions or spot any errors, please raise an issue or create a pull request. If you encounter any difficulties or have questions about building AI Agents, join our Azure AI Community on Discord. 📂 Each Lesson Includes A written lesson located in the README (Videos Coming March 2025) Python code samples supporting Azure AI Foundry and Github Models (Free) Links to extra resources to continue your learning 🗃️ Lessons Overview Intro to AI Agents and Use Cases Exploring Agentic Frameworks Understanding Agentic Design Patterns Tool Use Design Pattern Agentic RAG Building Trustworthy AI Agents Planning Design Pattern Multi-Agent Design Pattern Metacognition Design Pattern AI Agents in Production 🌐 Multi-Language Support We offer translations in several languages and will updating these on a regular basis. 🚀 Go Fork or Clone this repo and get started on your AI Agents journey 🤖 at https://aka.ms/ai-agents-beginnersStep-by-Step Tutorial: Building an AI Agent Using Azure AI Foundry
This blog post provides a comprehensive tutorial on building an AI agent using Azure AI Agent service and the Azure AI Foundry portal. AI agents represent a powerful new paradigm in application development, offering a more intuitive and dynamic way to interact with software. They can understand natural language, reason about user requests, and take actions to fulfill those requests. This tutorial will guide you through the process of creating and deploying an intelligent agent on Azure. We'll cover setting up an Azure AI Foundry hub, crafting effective instructions to define the agent's behavior, including recognizing user intent, processing requests, and generating helpful responses. We'll also discuss testing the agent's conversational abilities and provide additional resources for expanding your knowledge of AI agents and the Azure AI ecosystem. This hands-on guide is perfect for anyone looking to explore the practical application of Azure's conversational AI capabilities and build intelligent virtual assistants. Join us as we dive into the exciting world of AI agents.Building AI Agent Applications Series - Using AutoGen to build your AI Agents
In the previous content, we learned about AI Agent. If you didn't read it, please read my previous content - Understanding AI Agents. We have many different frameworks to implement AI Agents. AutoGen from Microsoft is a relatively mature AI Agents framework. Now AutoGen is mainly based on two programming languages .NET and Python. The more mature version is the Python version. The content in this article is mainly based on the Python version https://microsoft.github.io/autogen. If you want to learn the .NET version, you can visit here https://microsoft.github.io/autogen-for-netUnleashing the Power of AI Agents: Transforming Business Operations
Let "Get Started with AI Agents," in this short blog I want explore the evolution, capabilities, and applications of AI agents, highlighting their potential to enhance productivity and efficiency. We take a peak into the challenges of developing AI agents and introduce powerful tools like Azure AI Foundry and Azure AI Agent Service that empower developers to build, deploy, and scale AI agents securely and efficiently. In today's rapidly evolving technological landscape, the integration of AI agents into business processes is becoming increasingly essential. Lets delve into the transformative potential of AI agents and how they can revolutionize various aspects of our operations. We begin by exploring the evolution of LLM-based solutions, tracing the journey from no agents to sophisticated multi-agent systems. This progression highlights the growing complexity and capabilities of AI agents, which are now poised to handle wide-scope, complex use cases requiring diverse skills. Lets now look at agentic AI capabilities. AI agents can significantly enhance employee productivity and process efficiency, making our operations faster and more effective. Lets examine the key applications of AI agents across industries, such as travel booking and expense management, employee onboarding, personalized customer support, and data analytics and reporting. However, developing AI agents is not without its challenges. Some of the primary considerations, including tool integration, interoperability, scalability, real-time processing, maintenance, flexibility, error handling, and security. These challenges underscore the need for robust platforms that enable rapid development and secure deployment of AI agents. To this end, we introduce Azure AI Foundry and Azure AI Agent Service. These tools empower developers to build, deploy, and scale AI agents securely and efficiently. Azure AI Foundry offers a comprehensive suite of tools, including model catalogs, content safety features, and machine learning capabilities. The Azure AI Agent Service, currently in public preview, provides flexible model selection, extensive data connections, enterprise-grade security, and rapid development and automation capabilities. When building multi agent or agentic based systems there is a huge importance of multi-agent orchestration. Tools like AutoGen and Semantic Kernel facilitate the orchestration of multi-agent systems, enabling seamless integration and collaboration between different AI agents. In conclusion, the transformative potential of AI agents in driving productivity, efficiency, and innovation. By leveraging the capabilities of Azure AI Foundry and Azure AI Agent Service, we can overcome the challenges of AI agent development and unlock new opportunities for growth and success. Resources Azure AI Discord - https://aka.ms/AzureAI/Discord Global AI community - https://globalai.community Generative AI for beginners – https://aka.ms/genai-beginners AI Agents for beginners - https://aka.ms/ai-agents-beginners Attend one of the Global AI Bootcamp near you - https://globalai.community/bootcamp/ Build AI Tour open content - https://aka.ms/aitour/repos Build your first Agent with Azure AI Agent Service - Slide deck and code - https://github.com/microsoft/aitour-build-your-first-agent-with-azure-ai-agent-serviceData-driven Analytics for Responsible Business Solutions, a Power BI introduction course:
Want to gain inside on how students at Radboud University are introduced, in a praticle manner, to Power BI? Check out our learning process and final project. For a summary of our final solution watch our Video Blog and stick around till the end for some "wise words"!Using Advanced Reasoning Model on EdgeAI Part 1 - Quantization, Conversion, Performance
DeepSeek-R1 is very popular, and it can achieve the same capabilities as OpenAI o1 in advanced reasoning. Microsoft has also added DeepSeek-R1 models to Azure AI Foundry and GitHub Models. We can compare DeepSeek-R1 ith other available models through GitHub Models Playground Note This series revolves around deployment of SLMs to Edge Devices 'Edge AI' we will focus on the deployment advanced reasoning models, with different application scenarios. You can learn more in the following session AI Tour BRK453. In this experiement we want to deploy advanced reasoning models to the edge, so that they can run on edge devices with limited computing power and offline environments. At this time, the recommendation is to use the traditional ONNX model . We can use Microsoft Olive to convert the DeepSeek-R1 Distrill model. Getting started with Microsoft Olive is very straightforward. Install the Microsoft Olive library through the command line and Python 3.10+ (recommended) pip install olive-ai The DeepSeek-R1 Distrill model series has different parameters such as 1.5B, 7B, 8B, 14B, 32B, 70B, etc. This article is mainly based on the 1.5B, 7B, and 14B models (so a Small Language Model). CPU Inference Let's discuss 1.5B and 7B, which are models with lower parameter. We can directly use the CPU as computing for inference to test the effect (hardware environment Azure DevBox, AMD EPYC 7763 64-Core + 64GB Memory + 2T SSD) Quantization conversion olive auto-opt --model_name_or_path <Your DeepSeek-R1-Distill-Qwen-1.5B/7B local location> --output_path <Your Convert ONNX INT4 Model local location> --device cpu --provider CPUExecutionProvider --precision int4 --use_model_builder --log_level 1 You can download it directly from my Hugging face Repo (Note: This model is for testing and has not been fully tested by AI Content Safety or provided as an Offical Model) DeepSeek-R1-Distill-Qwen-1.5B-ONNX-INT4-CPU DeepSeek-R1-Distill-Qwen-7B-ONNX-INT4-CPU Running with ONNX Runtime GenAI Install ONNX Runtime GenAI and ONNX Runtime CPU support libraries pip install onnxruntime-genai pip install onnxruntime Sample Code https://github.com/kinfey/EdgeAIForAdvancedReasoning/blob/main/notebook/demo-1.5b.ipynb https://github.com/kinfey/EdgeAIForAdvancedReasoning/blob/main/notebook/demo-7b.ipynb Performance comparison 1.5B vs 7B We compare two different inference scenarios explain 1+1=2 1.5B quantized ONNX model memory occupied, time consumption and number of tokens generated: 7B quantized ONNX model memory occupied, time consumption and number of tokens generated 2. Find all pairwise different isomorphism groups with order 147 and no elements with order 49 1.5B quantized ONNX model memory occupied, time consumption and number of tokens generated: 7B quantized ONNX model memory occupied, time consumption and number of tokens generated Results of the numbers Through the test, we can see that the 1.5B model of DeepSeek is more suitable for use on CPU inference and can be deployed on traditional PCs or IoT devices. As for 7B, although it has better inference, it is not very effective on CPU operation. GPU Inference It is ideal if we have a GPU on the edge device. We can quantize and convert it to an ONNX model for CPU inference through Microsoft Olive. Of course, it can also be converted to a model for GPU inference. Here I take the 14B DeepSeek-R1-Distill-Qwen-14B as an example and make an inference comparison with Microsoft's Phi-4-14B Quantization conversion olive auto-opt --model_name_or_path <Your Phi-4-14B or DeepSeek-R1-Distill-Qwen-14B local path > --output_path <Your converted Phi-4-14B or DeepSeek-R1-Distill-Qwen-14B local path > --device gpu --provider CUDAExecutionProvider --precision int4 --use_model_builder --log_level 1 You can download it directly from my Hugging face Repo (Note: This model is for testing and has not been fully tested by AI Content Safety and not an Official Model) DeepSeek-R1-Distill-Qwen-14B-ONNX-INT4-GPU Phi-4-14B-ONNX-INT4-GPU Running with ONNX Runtime GenAI CUDA Install ONNX Runtime GenAI and ONNX Runtime GPU support libraries pip install onnxruntime-genai-cuda pip install onnxruntime-gpu Compare the results in the GPU environment with Gradio It is recommended to use a GPU with more than 8G memory To increase the comparison of the results, we compare it with Phi-4-14B-ONNX-INT4-GPU and DeepSeek-R1-Distill-Qwen-14B-ONNX-INT4-GPU to see the different results. We also show we use OpenAI o1-mini (it is recommended to use o1-mini through GitHub Models), Sample Code https://github.com/kinfey/EdgeAIForAdvancedReasoning/blob/main/notebook/Performance_AdvancedReasoning_ONNX_CPU.ipynb You can test any prompt on Gradio to compare the results of Phi-4-14B-ONNX-INT4-GPU, DeepSeek-R1-Distill-Qwen-14B-ONNX-INT4-GPU and OpenAI o1 mini. DeepSeek-R1 reduces the cost of inference models and produces more instructive results on professional problems, but Phi-4-14B also has advantages in reasoning and uses lower computing power to complete inference. As for OpenAI o1 mini, it is more comprehensive and can touch all problems. If you want to deploy to Edge Device, Phi-4-14B and quantized DeepSeek-R1 are good choices for you. This blog is just a simple test and the first in this series. Please share your feedback and continue the discussion in the Microsoft AI Discord Channel. Feel free to me a message or comment. We look forward to sharing more around the opportunity of EdgeAI and more content in this series. Resource DeepSeek-R1 in GitHub Models https://github.com/marketplace/models/azureml-deepseek/DeepSeek-R1 DeepSeek-R1 in Azure AI Foundry https://ai.azure.com/explore/models/DeepSeek-R1/version/1/registry/azureml-deepseek Phi-4-14B in Hugging face https://huggingface.co/microsoft/phi-4 Learn about Microsoft Olive https://github.com/microsoft/olive Learn about ONNX Runtime GenAI https://github.com/microsoft/onnxruntime-genai Microsoft AI Discord Channel BRK453 Exploring cutting-edge models: LLMs, SLMs, local development and more https://aka.ms/aitour/brk453
