Step-by-step: Integrate Ollama Web UI to use Azure Open AI API with LiteLLM Proxy
Introductions Ollama WebUI is a streamlined interface for deploying and interacting with open-source large language models (LLMs) like Llama 3 and Mistral, enabling users to manage models, test them via a ChatGPT-like chat environment, and integrate them into applications through Ollama’s local API. While it excels for self-hosted models on platforms like Azure VMs, it does not natively support Azure OpenAI API endpoints—OpenAI’s proprietary models (e.g., GPT-4) remain accessible only through OpenAI’s managed API. However, tools like LiteLLM bridge this gap, allowing developers to combine Ollama-hosted models with OpenAI’s API in hybrid workflows, while maintaining compliance and cost-efficiency. This setup empowers users to leverage both self-managed open-source models and cloud-based AI services. Problem Statement As of February 2025, Ollama WebUI, still do not support Azure Open AI API. The Ollama Web UI only support self-hosted Ollama API and managed OpenAI API service (PaaS). This will be an issue if users want to use Open AI models they already deployed on Azure AI Foundry. Objective To integrate Azure OpenAI API via LiteLLM proxy into with Ollama Web UI. LiteLLM translates Azure AI API requests into OpenAI-style requests on Ollama Web UI allowing users to use OpenAI models deployed on Azure AI Foundry. If you haven’t hosted Ollama WebUI already, follow my other step-by-step guide to host Ollama WebUI on Azure. Proceed to the next step if you have Ollama WebUI deployed already. Step 1: Deploy OpenAI models on Azure Foundry. If you haven’t created an Azure AI Hub already, search for Azure AI Foundry on Azure, and click on the “+ Create” button > Hub. Fill out all the empty fields with the appropriate configuration and click on “Create”. After the Azure AI Hub is successfully deployed, click on the deployed resources and launch the Azure AI Foundry service. To deploy new models on Azure AI Foundry, find the “Models + Endpoints” section on the left hand side and click on “+ Deploy Model” button > “Deploy base model” A popup will appear, and you can choose which models to deploy on Azure AI Foundry. Please note that the o-series models are only available to select customers at the moment. You can request access to the o-series models by completing this request access form, and wait until Microsoft approves the access request. Click on “Confirm” and another popup will emerge. Now name the deployment and click on “Deploy” to deploy the model. Wait a few moments for the model to deploy. Once it successfully deployed, please save the “Target URI” and the API Key. Step 2: Deploy LiteLLM Proxy via Docker Container Before pulling the LiteLLM Image into the host environment, create a file named “litellm_config.yaml” and list down the models you deployed on Azure AI Foundry, along with the API endpoints and keys. Replace "API_Endpoint" and "API_Key" with “Target URI” and “Key” found from Azure AI Foundry respectively. Template for the “litellm_config.yaml” file. model_list: - model_name: [model_name] litellm_params: model: azure/[model_name_on_azure] api_base: "[API_ENDPOINT/Target_URI]" api_key: "[API_Key]" api_version: "[API_Version]" Tips: You can find the API version info at the end of the Target URI of the model's endpoint: Sample Endpoint - https://example.openai.azure.com/openai/deployments/o1-mini/chat/completions?api-version=2024-08-01-preview Run the docker command below to start LiteLLM Proxy with the correct settings: docker run -d \ -v $(pwd)/litellm_config.yaml:/app/config.yaml \ -p 4000:4000 \ --name litellm-proxy-v1 \ --restart always \ ghcr.io/berriai/litellm:main-latest \ --config /app/config.yaml --detailed_debug Make sure to run the docker command inside the directory where you created the “litellm_config.yaml” file just now. The port used to listen for LiteLLM Proxy traffic is port 4000. Now that LiteLLM proxy had been deployed on port 4000, lets change the OpenAI API settings on Ollama WebUI. Navigate to Ollama WebUI’s Admin Panel settings > Settings > Connections > Under the OpenAI API section, write http://127.0.0.1:4000 as the API endpoint and set any key (You must write anything to make it work!). Click on “Save” button to reflect the changes. Refresh the browser and you should be able to see the AI models deployed on the Azure AI Foundry listed in the Ollama WebUI. Now let’s test the chat completion + Web Search capability using the "o1-mini" model on Ollama WebUI. Conclusion Hosting Ollama WebUI on an Azure VM and integrating it with OpenAI’s API via LiteLLM offers a powerful, flexible approach to AI deployment, combining the cost-efficiency of open-source models with the advanced capabilities of managed cloud services. While Ollama itself doesn’t support Azure OpenAI endpoints, the hybrid architecture empowers IT teams to balance data privacy (via self-hosted models on Azure AI Foundry) and cutting-edge performance (using Azure OpenAI API), all within Azure’s scalable ecosystem. This guide covers every step required to deploy your OpenAI models on Azure AI Foundry, set up the required resources, deploy LiteLLM Proxy on your host machine and configure Ollama WebUI to support Azure AI endpoints. You can test and improve your AI model even more with the Ollama WebUI interface with Web Search, Text-to-Image Generation, etc. all in one place.
Azure Virtual Desktop
Hi There Hope you are doing great! I am new to Azure and Virtual Desktop and require some help to set it up correctly. I have created and setup the Virtual Desktop but looking for an option to automatically shut it down and deallocate, there is only 1 machine running and usually 1 user connects to it. if that user is not accessing the machine it needs to be deallocated after 30 or 15 min. Is it something anyone can help? Many Thanks
Best practise for managing deploys to VMs (environments vs agent pools)
Hello, I wanted to ask about best pracitices when it comes to deploying IIS sites and windows services to virtual machines. Let's say we have a setup of 2 web vms that host a website and 1 vm that's running some windows services. I can see two options for deploy setup: 1. create environment (i.e. dev, prod etc.), register single agent per virtual machine (specific for site - so each site can be deployed independently, reducing downtime) 2. create Agent Pool per VM and register multiple agents So for Web I would go with 1. - adding 2 vm's to environments with proper tags, so we can deploy with rolling deployment strategy to reduce downtime. For services I would create an agent pool and register there multiple agents so we can reduce deployment time (considering we have a bunch of services) and deploy them in parallel. Service deploy would run under specific environment but will just use agent pool. I dont see a point of using environments here, as we only have 1 VM and it's harder to manage multiple agents there - seems like it was not created for this purpose. I hope I have explained it clear, what are your thoughts? RafalHow can I deploy software in Azure for my business?
Hello, I would like to deploy 10 software for a part of my company. The 10 software should be accessible from 40 laptops at the same time. I want to do it with a VM, because then I do not always have to install the software everywhere, but only once. Is this a good idea? What performance would I need for e.g. a KUKA robot simulation software. Can I also forward a dongle? What kind of performance do I need for 40 connections to a software?
Influx in Azure Marketplace
Hi, I have subscribed to the Influx DB from Azure Marketplace, as per the documentation of the Influx, when the subscription and account creation is done, Influx cloud will use the AKS and VM's from the Azure. But I didn't see any AKS or VM's created for the Influx. Thanks, Shashi.
VMs Created when Creating New Host Pool Aren't in Pool
I've created a golden image in accordance with Create an Azure Virtual Desktop golden image | Microsoft Docs and sysprepped it IAW Generalize a VM before creating an image - Azure Virtual Machines | Microsoft Docs When I create a pooled Host Pool with the golden image, and create VMs as part of that process, the VMs get created however they aren't in the pool. The image is based on the Windows 10 Enterprise multi-session, version 21H2 + Microsoft 365 Apps - Gen 2. Customizing the image, I remove unnecessary pre-installed apps like Windows Mail and install a few other apps like Adobe Reader. Nothing fancy. If I RDP directly into one of the VMs as admin, it is named properly and joined to the domain as defined when creating the pool. What am I missing?
Windows 10 90-day Evaluation on Public Cloud
Hi Everyone, I'm wondering if I can, without breaking any licensing, download 90-days Windows 10 Evaluation and deploy it as a VM on a Public Cloud Infrastructure (for example Google Cloud or AWS) ? Only as a Proof of Concept, with no production what-so-ever. VM would be running less than 1 day, after that it would be destroyed. The end-goal would be to exercise such operation + create an step-by-step tutorial of how to deploy OS by bringing your own ISO file.Deploying Jupyterhub on Azure for Individual Students or Class Groups
A simple JupyterHub distribution for a small (1-400) number of users on a single server. Based on JupyterOrg http://tljh.jupyter.org/en/latest/ We recommend reading When to use The Littlest JupyterHub to determine if this is the right tool for you.
