GPT-4 Turbo with Vision on Azure OpenAI Service
GPT-4 Turbo with Vision on Azure OpenAI service is coming soon to public preview. It can analyze images and provide textual responses to questions about them. It incorporates both natural language processing and visual understanding. This integration allows Azure users to benefit from Azure's reliable cloud infrastructure and OpenAI's advanced AI research.
Azure AI Speech launches Personal Voice in preview
Today at Ignite 2023 conference, Microsoft is taking customization one step further with its new 'Personal Voice' feature. This innovation is specifically designed to enable customers to build apps that allow their users to easily create their own AI voice, resulting in a fully personalized voice experience.
GPT-4 Turbo with Vision is now available on Azure OpenAI Service!
We are excited to announce that GPT-4 Turbo with Vision is now available for public preview on Azure OpenAI Service! This advanced multimodal AI model retains all the powerful capabilities of GPT-4 Turbo while introducing the ability to process and analyze image inputs. This provides the opportunity to utilize GPT-4 for a wider range of tasks, including accessibility improvements, visual data interpretation and analysis, and visual question answering (VQA). All existing Azure OpenAI Service customers now have access to this service. GPT-4 Turbo with Vision can be accessed in the following Azure regions: Australia East, Sweden Central, Switzerland North, and West US. GPT-4 Turbo with Vision + Azure AI Service Additionally, we are releasing curated Azure AI Service enhancements for GPT-4 Turbo with Vision, which introduces an array of advanced functionalities, including: Optical Character Recognition (OCR): Extracts text from images, integrating it with the user's prompt and image to enrich the context. Object grounding: Enhances text responses from GPT-4 Turbo with Vision by identifying and outlining key objects within images. Video prompts: Allows GPT-4 Turbo with Vision to answer questions using the most relevant frames from a video based on the user's prompt. Azure OpenAI Service on your data with images: By combining GPT-4 Turbo with Vision, Azure AI Search, and Azure AI Vision, images can now be added with text data, utilizing vector search to develop a solution that connects with user’s data, enabling an improved chat experience. Example of GPT-4 Turbo with Vision + Azure AI Service (Object grounding) Guide to Deploying GPT-4 Turbo with Vision To deploy GPT-4 Turbo with Vision from the Studio UI, select "GPT-4" and then choose the "vision-preview" version from the dropdown menu. This preview version has a separate quota from the existing GPT-4 versions, which allows you to experiment without affecting your current deployments. Pricing Model Input Output GPT-4 Turbo with Vision 1 $0.01 per 1000 tokens $0.03 per 1000 tokens + Enhanced add-on features for OCR $1.50 per 1000 transactions + Enhanced add-on features for Object Grounding $1.50 per 1000 transactions + Enhanced add-on feature for “Add your Image” Image Embedding $0.10 per 1000 transactions + Enhanced add-on feature for Video prompts integrating Video Retrieval $0.05 per minute for indexing $0.25 per 1000 transactions 2 1 GPT-4 Turbo with Vision pricing explained in detail here. 2 Additional input and output tokens for video prompts: Processing videos will involve the use of extra tokens to identify key frames for analysis. The number of these additional tokens will be roughly equivalent to the sum of the tokens in the text input plus 700 tokens. Tips for Tailoring System Prompts for Enhanced Accuracy and Efficiency Guidelines for Crafting Effective System Prompts with GPT-4 Turbo with Vision To unlock the full potential of GPT-4 Turbo with Vision, it's essential to skillfully tailor system prompt to your specific needs. Here are some guidelines to enhance the accuracy and efficiency of your prompts: Contextual Specificity: For instance, if you're working on image descriptions for a product catalog, ensure your prompt reflects this. A prompt like “Describe images for an outdoor hiking product catalog, focusing on enthusiasm and professionalism” guides the model to generate responses that are both accurate and contextually rich. This level of specificity aids in focusing on relevant aspects and avoiding extraneous details. Task-Oriented Prompts: If your project involves analyzing videos for auto insurance claims, your prompt should be precisely tailored to this task. For example, “Analyze this car damage video for an auto insurance report, focusing on identifying and detailing damage.” This prompt steers the model to concentrate on elements crucial for insurance assessments, thereby improving accuracy and relevancy. Handling Refusals: When the model indicates an inability to perform a task, refining the prompt can be an effective solution. More specific prompts can guide the model towards a clearer understanding and better execution of the task. Prompt Examples for Various Use Cases: Use Case Example System Prompt Image Description "As an AI assistant, provide a clear, detailed sentence describing the content depicted in this image." Image Tagging "Identify and list prevalent tags associated with the content of this image." Defect Detection "Act as a professional defect detector. Compare this test image with a reference image and state 'No defect detected' or 'Defect detected', providing detailed reasoning." Car Insurance Damage Report Writing "Function as a car insurance and accident expert. Extract detailed information about the car's make, model, damage extent, license plate, airbag deployment status, etc., and present the results in JSON format." These guidelines and examples demonstrate how tailored system prompts can significantly enhance the performance of GPT-4 Turbo with Vision, ensuring that the responses are not only accurate but also perfectly suited to the specific context of the task at hand. Preview Note The first version of GPT-4 Turbo with Vision, "gpt-4-vision-preview" is in preview and will be replaced with a stable, production-ready release in the coming weeks. Customer deployments using "gpt-4-vision-preview" will be automatically updated to the GA version of GPT-4 Turbo upon the launch of the stable version. To Get Started, Explore the Following Resources Apply now for access to Azure OpenAI Service Learn more about GPT-4 Turbo with Vision on Azure OpenAI Service AI Studio Quickstart: Get started using GPT-4 Turbo with Vision on your images and videos in Azure AI Studio Azure Open AI Quickstart: Quickstart: Use GPT-4 Turbo with Vision on your images and videos with the Azure Open AI Service Azure Open AI How-To Guide: How to use the GPT-4 Turbo with Vision model on Azure OpenAI Service RAG with GPT-4V Turbo with Vision using your own data: Azure OpenAI on your data with images using GPT-4 Turbo with Vision Use Azure AI Search and GPT-4 Turbo with Vision on your image data (e.g., charts and graphs, like financial reports) using the Retrieval Augmented Generation pattern: GitHub samples repository GPT-4 Turbo with Vision pricing explained in detail: Text and Image tokens Responsible AI: Transparency Note for Azure OpenAI ServiceVideo Retrieval: GPT-4 Turbo with Vision Integrates with Azure to Redefine Video Understanding
Microsoft is thrilled to unveil the Azure AI Vision Video Retrieval preview. This innovative feature revolutionizes video search, enabling the exploration of thousands of hours of video content through advanced multi-modal vector indexing of vision and speech. Further enhancing the Azure OpenAI GPT-4 Turbo with Vision, Video Retrieval seamlessly integrates, providing customers with the capability to craft solutions that can both perceive and interpret video content. This opens novel possibilities and use cases. It simplifies the process for developers, allowing them to effortlessly incorporate video input into their applications, skipping complex video processing and indexing code. This is the power of Azure OpenAI Service and Azure AI Services working together.Join Microsoft experts for our Make Azure AI Real livestream series
Are you a developer who wants to learn how to use Microsoft AI tools and services, such as Azure AI Cognitive Search, Azure OpenAI Service, and more? Do you want to get insights into the latest AI trends and best practices from Microsoft AI experts? If you answered yes to any of these questions, then you should join Microsoft Reactor’s Make Azure AI Real livestream series kicking off today! We’re thrilled to bring you a collection of AI livestreams that are focused on in-depth AI technical content from experts at Microsoft where you will discover how to make AI real for your projects and goals and have the opportunity to ask questions directly to experts. Register now https://aka.ms/MAIR/RSVP
