Setting Up NextChat with GitHub's Free LLM API

I've been testing different kinds of web-based interfaces to interact with various AI models, including smaller but powerful models like Mistral AI's Ministral-3B. After experimenting with several options including Open WebUI, LobeChat, and others, I found NextChat (also known as ChatGPT-Next-Web) to be a solid choice for connecting to GitHub's free LLMs.

With LobeChat, the popup at the beginning of every chat is quite annoying. What makes NextChat particularly appealing is its clean interface, straightforward setup, and compatibility with OpenAI-compatible API endpoints, which includes GitHub's Models API. GitHub offers free access to various cutting-edge models from OpenAI, Anthropic, Meta, Mistral, and more, making it an excellent option for developers and AI enthusiasts.

What is GitHub Models API?

GitHub Models API provides free access to state-of-the-art AI models for developers. It's an OpenAI-compatible endpoint, meaning any tool that works with OpenAI's API can be configured to use GitHub's models instead.

Key benefits:

• Free access to premium models

• Multiple providers: OpenAI, Anthropic, Meta, Mistral, Google, and more

• OpenAI-compatible API structure

• Suitable for testing and development

Prerequisites

Before starting, ensure you have:

• Docker installed on your system

• A GitHub account

• A GitHub Personal Access Token (PAT) with appropriate permissions

Step 1: Obtaining your GitHub PAT

To use GitHub Models API, you need to generate a Personal Access Token:

1. Visit GitHub's token settings at: https://github.com/settings/tokens

2. Click "Generate new token"

3. Give it a descriptive name

4. Set the expiration date according to your needs

5. Select the required scopes/permissions

6. Generate the token and copy it immediately (you won't see it again).
   Important: Store this token securely. It will look something like: github_pat_11HHXXSXT1...

Step 2: Deploying NextChat with Docker

NextChat can be deployed using a simple Docker command. Here's the command I used to get it running:

docker run -d -p 3000:3000 
-e OPENAI_API_KEY=your_github_token 
-e BASE_URL=https://models.github.ai/inference 
-e CODE=password_of_your_choice 
yidadaa/chatgpt-next-web

Environment variable breakdown:

• OPENAI_API_KEY - Your GitHub Personal Access Token

• BASE_URL - GitHub's model inference endpoint

• CODE - Access password for your NextChat instance (change this to something secure)

Step 3: Accessing the NextChat UI

Once the container is running, you can access NextChat through your web browser:

• On the same machine: http://localhost:3000

• From another device on your network: http://YOUR_SERVER_IP:3000 (like, http://192.168.0.111:3000)

You'll be prompted to enter the access code you set in the CODE environment variable.

Step 4: Configuring Available Models

NextChat comes with a predefined list of AI providers (OpenAI, Azure, Google, Anthropic, etc.). Since GitHub's API is OpenAI-compatible, we'll use the OpenAI configuration:

1. In the NextChat interface, look for the settings icon (usually a gear icon in the bottom-left corner)

   

2. Select Custom Endpoint and navigate to Model Provider

3. Select "OpenAI" from the provider dropdown

Ensure the API endpoint is set to https://models.github.ai/inference. This should be inherited from your Docker configuration, but may require manual entry if missing.

In my case, I entered the model name manually in the Custom Models input: mistralai/Ministral-3B

Available Models on GitHub

GitHub provides access to a wide range of models from various providers. Here are some of the models you can use:

OpenAI Models:

Anthropic (Claude) Models:

Meta (Llama) Models:

Mistral AI Models:

Note: For a complete and up-to-date list of available models, visit https://github.com/marketplace?type=models or call the GitHub API endpoint directly like:

curl -H "Authorization: Bearer YOUR_GITHUB_TOKEN" \
     -H "X-GitHub-Api-Version: 2022-11-28" \
     https://models.github.ai/catalog/models

Troubleshooting common issues

Authentication Redirect Loop

Symptom: Browser redirects to /auth?redirect=%2F but shows nothing.
Cause: NextChat requires authentication but no access code was set.
Solution: Add the CODE environment variable and refresh browser cache: docker run -d -p 3000:3000 -e CODE=your_password ...

Model Returns 422 Error

Symptom: Error message: Extra inputs are not permitted with details about stream_options or user parameters.
Cause: Some models (especially Mistral variants) don't accept certain optional parameters that NextChat sends.
Solution: Try switching to a different model that has better parameter compatibility:

• openai/gpt-4o or openai/gpt-4o-mini

• anthropic/claude-sonnet-4

• meta-llama/Llama-3.3-70B-Instruct

Container running cut can't access

Symptom: Docker logs show Ready but browser shows connection error.
Diagnostic steps:

• Check container status: docker ps

• View logs: docker logs <container_id>

• Test locally: curl http://localhost:3000

• Clear browser cache and try incognito mode

• Verify firewall settings aren't blocking port 3000

Invalid API Key error

Symptom: API returns authentication error.
Solution:

• Verify your GitHub token is still valid

• Check that the token has the necessary permissions

• Ensure no extra spaces were copied with the token

• Regenerate the token if needed and update the Docker container

Best Practices and Tips

• Choose the right model: Start with smaller models like Ministral-3B for quick responses, and use larger models like GPT-4 or Claude Opus for complex tasks.

• Secure your instance: Always set a strong CODE password and don't expose your instance to the public internet without additional security measures.

• Monitor usage: GitHub's free tier has rate limits. Keep track of your API usage to avoid hitting limits.

• Keep tokens secure: Never commit your GitHub token to version control or share it publicly.

• Regular updates: Periodically pull the latest NextChat image to get bug fixes and new features: docker pull yidadaa/chatgpt-next-web:latest

Advanced Configuration

For more control over your NextChat deployment, consider using docker-compose:

version: '3.8'
services:
  nextchat:
    image: yidadaa/chatgpt-next-web
    ports:
      - "3000:3000"
    environment:
      - OPENAI_API_KEY=your_github_token
      - BASE_URL=https://models.github.ai/inference
      - CODE=your_secure_password
    restart: unless-stopped

Save this as docker-compose.yml and run: docker-compose up -d

Conclusion

NextChat provides a clean, user-friendly interface for interacting with GitHub's free LLM API.

While I explored several alternatives including Open WebUI and LobeChat, NextChat struck the right balance between simplicity and functionality. The setup process is straightforward, and once configured, it provides reliable access to a wide range of cutting-edge AI models.

Whether you're testing different models, building prototypes, or just exploring what various AI systems can do, this setup gives you a powerful, free platform to work with. The key is understanding the authentication flow, choosing compatible models, and knowing how to troubleshoot common issues when they arise.

For the latest information on available models and API changes, always refer to the official GitHub Models documentation and the marketplace.

Now, you know! 🚀