Use AI for Free with GitHub Models and TypeScript! 💸💸💸

Artificial Intelligence is becoming more accessible to developers. However, one of the biggest challenges remains the cost of advanced model APIs, such as GPT-4o and many others. Fortunately, GitHub Models is here to change the game! Now, you can experiment with AI for free, without needing a paid API key or downloading large models on your local machine.

In this article, we will explain in detail what GitHub Models is and how to use it for free with TypeScript in a practical project. We chose the Microblog AI Remix project as an example, an open-source microblog with AI-powered features.

We will explore the structure of this project and provide a step-by-step guide on how to integrate GitHub Models, replacing the need for paid LLMs. This includes before and after code comparisons. We will also cover how to configure and run the project locally while discussing the advantages and limitations of GitHub Models for AI project prototyping.

By the end of this article, you will have a solid understanding of GitHub Models and how to use it in your own projects, enabling you to explore AI affordably and freely.

Let's go!

What is GitHub Models?

GitHub Models is a GitHub initiative that provides a collection of ready-to-use AI models integrated into the platform. Think of GitHub Models as an AI Model Marketplace, where developers can discover large language models (LLMs) from different providers, test their capabilities in an interactive playground, and integrate them into applications easily.

There are models from various sources and scales, such as OpenAI GPT-4o, open-source models like Meta Llama 3.1, Microsoft Phi-3, Mistral Large 2, and many others. All of them can be accessed for free for experimentation.

One of GitHub Models' key advantages is allowing free model usage during the prototyping phase. This means you can test and build a proof of concept (POC) without incurring costs, using GitHub's provided infrastructure. There are two main ways to interact with the models:

• Playground (Web Interface): In this playground, you can test models directly from your browser. You can ask questions, get real-time responses, adjust parameters (temperature, max tokens, etc.), and even compare the output of two models side by side.

 

• Via API/SDK: If you need to integrate a model into a project, GitHub Models also provides a REST API and SDKs for various languages, including Python, JavaScript/TypeScript, Java, C#, and REST. Each model has a public inference endpoint, allowing you to make HTTP calls or use SDKs (such as the Azure OpenAI SDK or GitHub Models SDK) in multiple languages.

Authentication is simple—just use a personal GitHub access token (PAT) without requiring separate API keys. Generate a PAT in your GitHub account (without special scopes, using the Beta option) and use it in your requests. In other words, your GitHub token acts as the credential for accessing models within the free usage limits.

Advantages and Limitations

However, as with all free services, there are limitations. Let's break down GitHub Models' constraints.

Currently, there are restrictions on requests per minute and per day, token limits per request, and concurrent request limits. For example, smaller models (low category) allow around 15 requests per minute and 150 per day, while high-category models (like GPT-4o) have slightly stricter limits due to their higher computational demand.

But what if you want to deploy the model in production? In that case, GitHub Models suggests migrating to a paid Azure endpoint—the good news is that you only need to swap the GitHub token for an Azure key, and the rest of the code remains unchanged!

In summary, GitHub Models is an excellent way to find and experiment with state-of-the-art AI models for free. Developers can integrate AI features into TypeScript (or other language) projects using just a GitHub account. Next, we'll explore the Microblog AI Remix example and then demonstrate how to use GitHub Models in the project.

Microblog AI Remix with GitHub Models

Microblog AI Remix is a sample project combining a microblogging web application with AI capabilities. It showcases how to build modern, scalable web applications using Microsoft Azure's stack alongside Server-Side Rendering (SSR) and generative AI techniques. At a high level, Microblog AI allows users to create and view short blog posts (microblogs), with an AI model assisting content generation based on user suggestions.

I highly encourage you to try the project, fork it, and contribute improvements. The project is open source and can be tested in GitHub Codespaces. Leave a star ⭐️ and contribute enhancements!

Originally, the project used Azure OpenAI as its AI provider, but we will replace it with GitHub Models to offer a free and accessible alternative.

Step-by-Step Guide to Setting Up Microblog AI

I recorded a video showing the step-by-step migration to GitHub Models. The video is available on my YouTube channel (in Portuguese) and can be watched here:

 

Before starting, we need to clone the project and set up dependencies. Follow these steps:

1. Clone the official Microblog AI Remix repository:

git clone https://github.com/Azure-Samples/microblog-ai-remix.git
cd microblog-ai-remix

 

2. Install the project dependencies:

npm install
cd server
npm install

 

3. Create a .env file in the project root and add the following environment variables:

GITHUB_MODELS_ENDPOINT=https://models.inference.ai.azure.com
GITHUB_MODELS_TOKEN=YOUR_TOKEN

 

You can generate this token in your GitHub account under Settings > Developer Settings > Personal Access Tokens > Generate new token (beta).

 

4. In the /server directory, create a local.settings.json file with the following content:

{
  "IsEncrypted": false,
  "Values": {
    "AzureWebJobsStorage": "UseDevelopmentStorage=true",
    "FUNCTIONS_WORKER_RUNTIME": "node",
    "GITHUB_MODELS_ENDPOINT": "https://models.inference.ai.azure.com",
    "GITHUB_MODELS_TOKEN": "YOUR_TOKEN"
  },
  "Host": {
    "LocalHttpPort": 7071,
    "CORS": "*",
    "CORSCredential": true
  }
}

 

5. Now, go to the file: app/services/openaiService.ts and make the following changes:

• Importing the OpenAI Client: Replace the import of AzureOpenAI with OpenAI:

import { OpenAI } from "openai";

 

• Renaming the Class and Client: The AzureOpenAIService import has been renamed to GitHubModelsService. The AzureOpenAI client instance has also been renamed to OpenAI. Additionally, a default modelName (gpt-4o) was added for completion requests. However, you can choose any other model name, such as Llama 3.1 or Mistral 7B.

class GitHubModelsService {
  private client: OpenAI;
  private readonly toneGuidelines: ToneGuidelines;
  private readonly modelName: string = "gpt-4o";

(...)

 

• Client Configuration: Azure-specific environment variables (AZURE_OPENAI_API_KEY, AZURE_OPENAI_ENDPOINT, etc.) have been replaced with GitHub variables (GITHUB_TOKEN and GITHUB_MODELS_ENDPOINT).

this.client = new OpenAI({
      baseURL: process.env.GITHUB_MODELS_ENDPOINT || "https://models.inference.ai.azure.com",
      apiKey: process.env.GITHUB_TOKEN,
    });

 

• Exporting the Class Instance: The exported instance of the class has been renamed from azureOpenAIService to GitHubModelsService.

export const azureOpenAIService = new GitHubModelsService();

 

And that's it! Now you can run the project locally and test the AI features with GitHub Models. If you want more details on what was changed, I have made a branch available called feat/github-models-usage with all the modifications. You can compare it with the main branch to see what has been updated.

 

6. Finally, to run the project, execute the following command in the project's root directory:

npm run build:all
npm run dev

 

Now, you can access the application at the URL: http://localhost:5173/ and start creating your microblogs with the help of GitHub Models!

Conclusion

GitHub Models is an excellent alternative for those who want to experiment with AI for free. It allows testing advanced models like GPT-4o without having to pay for APIs or set up complex infrastructure. In the case of Microblog AI Remix, we successfully replaced the paid Azure OpenAI API with GitHub Models with minimal code changes, making the application accessible to any developer.

Of course, if you want to put it into production, GitHub Models suggests migrating to a paid Azure endpoint. However, for prototyping and learning purposes, it is a powerful and free tool.

If you enjoyed this article, don’t forget to try Microblog AI Remix and leave a ⭐ on the repository! We’d love to hear your thoughts on this approach and how you plan to use AI in your projects.

Now it’s your turn: clone the repository, test the changes, and explore GitHub Models for free. Let’s code with AI without spending a dime! 💸