Your Video Insights, Promptly Extracted: Azure AI Video Indexer's Preview of Prompt-Ready API
Have you ever watched an online course and wished you could ask questions on the entire course, or have a comprehensive summary of a video? This can all now be achieved with Azure AI Video Indexer and an LLM (Large Language Model) – powering each other. LLMs are powerful language models that can capture the essence of text and allow natural language question-answering and much more. In Azure AI Video Indexer, we understand videos – video content is more than just words, and a single shot can contain a wealth of insights that are critical for its understanding. Coupling these two powerful tools can lead to great results in video understanding, and downstream tasks in natural language. Our new API extracts and processes all the multi-modality insights of a video into prompt-ready format, that can be easily used with LLMs.Microsoft Ignite 2023 Announcements for Azure AI Video Indexer
We are excited to share the latest updates that we have added to Azure AI Video Indexer (VI), the video analysis solution that helps you to extract insights from your video and audio files, based on AIs. Here are some of the highlights: Edge version of Azure VI, enabled by Arc: You can use Azure VI on your own edge on premise environment, without sending your video data to the cloud. This will ensure you adhere to local regulation and privacy constrains or optimize your hardware use, while still enjoying the same functionalities that Azure VI offers on the cloud. It the first hybrid Video AI managed solution in Azure. Bring-your-own AI feature: You can use models from Hugging Face, Azure Florence, or any other AI models and connect them to VI insights. This will allow you to customize and enhance your video analysis with your own models, and to benefit from a single joint experience. Enrich your video metadata and video search experience: You can now add custom tags and free text as video metadata for any specific video in your account. This will help you to annotate your videos with any information that is relevant to your business. You can also search for videos based on the custom tags and free text that you have added to them. Features improving the efficiency of customized people model (subjected to limited access features program) All features were delivered as preview features.Build 2022: Azure Video Indexer Enhancements
At MS Build 2022, Azure Video Indexer service is becoming the primary video analytics offering Applied AI services, a GA service for analyzing stored videos (e.g. media libraries), with a single endpoint, more than 35 video and audio AIs - mainly based on Azure Cognitive Services. We have released set of quality, security and supportability-related enhancements for Build 2022.When AI meets fashion: Azure Video Indexer’s new model for detecting Featured Clothing in videos
What happens when a Data Scientist combines complex neural networks with classic ML (machine learning) models? Spoiler – the results are super interesting and fashionable! But let's not jump ahead and talk about clothes in videos.The AI technology that predicts which clothing items appearing in a video you're likely to shop
Azure Video Indexer recently released a new feature named "Featured Clothing". This feature allows publishers and advertisers to leverage deep contextual understanding in-order to place the most relevant ads at the most relevant timing where viewers will be most receptive to engage with the ad. "Featured Clothing" insight uses advanced AI me identifies the key moments and frames that contains the most interesting people wearing the most interesting outfits that can be used for contextual advertisements, this enables to show Ads that contains similar items like the ones that are showing in the video.
Ignite 2022: What's new in Azure Video Indexer?
For Ignite 2022, we expand the product insights and functionality by shipping preview version of featured clothing detection, clapperboard, digital patterns and textless slate detection. In addition, we added more languages support, and release as GA the ability to manage account based on Azure resource management.Build 2023: What’s new in Azure Video Indexer
Microsoft Build is always a great opportunity to share innovations and it’s been an exciting start to 2023 for Azure Video Indexer! We aim to be customer obsessed and have listened to and worked with our customers and partners to deliver impactful improvements. The overarching message from them was to make Video Indexer cheaper, faster, and safer. There is also great interest in integration with Chat GPT/Open AI, smarter insights, and an overall experience that’s easier to use and manage. We were asked to help our customers “Do more with less” and lower the cost of indexing projects to enable the indexing of huge media projects as well as the video content powering fast growing applications and social platforms. We were asked to make it quicker to index with Video Indexer in all scenarios – be it through our amazing webapp and embeddable widget or our API when indexing vast media archives! We were asked to raise the bar regarding our commitment to enable secure indexing with Video Indexer and allow customers to have more control of when to opt out of using sensitive AI’s. We were asked to continue innovating with new AI’s and Open AI integration while making it easier to index and monitor your Video Indexer resources safely and confidently. Finally, we were asked to provide a solution that can index your media wherever it resides, be it in Azure or on the Edge With that in mind, we are happy to share the latest and greatest from Video Indexer: Cheaper Price Reduction We lowered the price of our most popular presets by 40%, making it easier than even to innovate with Video Indexer. By reducing prices and the introduction of a new Advanced SKU, we offer competitive pricing while providing more options for customers to balance costs and features. Additionally, as we continue to improve and add more AI capabilities, customers will be able to take advantage of these cost savings when performing new or re-indexing operations. Faster Ten uploads at a time! You wanted the ability to upload and index through our webapp more than one video at a time and we heard you! We completely revamped the webapp upload experience, which now enables you to upload and index 10 videos at a time! It is also more intuitive and consistent with your overall Azure experience. Observed people indexing duration optimization and quality improvements We have optimized the memory usage of the observed people model, resulting in a 60% reduction in indexing duration when using the advanced video analysis preset. You can now process your video footage more efficiently and get results faster. In addition, Observed people now supports people who are sitting. This improvement makes observed people model more versatile and suitable for a wider range of use cases. We have also improved the model re-identification and grouping algorithms by 50%. The model can now more accurately track and group people across multiple camera views. Secure and Sensitive Support for storage behind firewall It is good practice to lock storage accounts and disable public access to enhance or comply with enterprise security policy. Video Indexer can now access non-public accessible storage accounts using the Azure Trusted Service exception using Managed Identities. Exclude Sensitive AI’s In line with our commitment to Responsible AI, Azure Video Indexer now allows you to exclude specific AI models when indexing media files. The list of sensitive AI models includes face detection, observed people, emotions, and labels identification. Innovative AI, Smarter/Easier to use and manage ChatGPT & VI for content discovery We have released code samples, demo’s, and articles on how to use ChatGPT and OpenAI together with Video Indexer for content discovery and insight extraction. New Speech model experience including Pronunciation training Azure Video Indexer has added a new custom speech model experience that reduces the need for timely and costly transcript edits and fixes. The experience includes ability to use custom pronunciation datasets to improve recognition of mispronounced words, phrases, or names. The custom models can be used to improve the transcription quality of content with industry specific terminology. Topics fully mapped to IPTC standards Video Indexer topics are mapped to the media Ontology of IPTC. This hierarchical set of concepts is the industry standard of media topics which is used for content categorization. We now support all five levels of IPTC ontology for enhanced video recognition, search, and recommendations applications. Additional language support We frequently add support for additional source languages and so far in 2023 have added support for 10+ new languages. Textual logo detection Textual logo detection enables you to customize text logos to be detected within videos. Notification experience The Azure Video Indexer webapp now has a notification panel where you can have visibility to your recent operations and upload status and stay informed of important product updates, such as service impacting events, and more. Resource Health Integration and Diagnose and Solve Azure Video Indexer is now integrated with Azure Resource Health enabling you to see the health and availability of each of your Azure Video Indexer resources. Azure Resource Health also helps with diagnosing and solving problems and you can set alerts to be notified whenever your resources are affected. Index everything everywhere…. Azure Arc enabled extension There are scenarios where content can’t be sent to Azure and customers need a way to unlock insights of this content as well! With this in mind, we are super excited to share that Video Indexer will be extending beyond Azure and opening to preview an Azure Arc enabled extension that can run on-premises, leveraging the Azure Arc platform. Customers will be able to bring the Video Indexer experience they know and love to their media wherever it is, delivering multiple AI’s in an easy to install and use solution. The initial scope of the preview is our Basic Audio preset. If you are interested in learning more and joining the preview, reach out to us at vipreviews@microsoft.com . Want to explore Video Indexer and stay up to date on all releases? Here are some helpful resources: Use Azure Video Indexer website to access product website and get a free trial experience. Visit Azure Video Indexer Developer Portal to learn about our APIs. Search the Azure Video Indexer GitHub repository Review our product documentation. Get to know the recent features using Azure Video Indexer release notes. Use Stack overflow community for technical questions. To report an issue with Azure Video Indexer (paid account customers) Go to Azure portal Help + support. Create a new support request. Your request will be tracked within SLA. Read our recent blogs in Azure Tech Community.Special 4 NAB 2022: Sweet 16 Azure Video Indexer Features Every Media Company Should Leverage
What’s new in Microsoft Media AI? Azure Video Analyzer for Media (AVAM) aka Azure Video Indexer (VI) is an applied-AI cloud solution, hyper-scale, enterprise grade, productive media workflow solution orchestrating intelligent video and audio AI pipelines. At NAB we are announcing 16 new features, all based on our customer feedback. Meet Azure Video Indexer team at NAB show 2022 and see the latest product features in action.
