LibreChat Github LLM Endpoint

LibreChat is a powerful, open-source alternative to proprietary AI chat platforms. It provides a unified interface for interacting with multiple LLM providers (including OpenAI, Anthropic, Google, and now GitHub Models) all from a single, self-hosted application.

What makes it compelling is its flexibility… instead of being locked into one vendor's ecosystem, you can switch between models seamlessly, compare responses, and maintain complete control over your data and infrastructure. Whether you're a developer experimenting with different models, a team building internal AI tools, or someone who values privacy and customization, it offers a production-ready solution that runs entirely on your own hardware.

One of the key capabilities I've been searching for is multi-user support, something that NextChat lacks. LibreChat excels here with built-in user authentication, individual conversation histories, and role-based access controls.
This makes it ideal not just for personal use, but for teams and organizations that need to provide AI access to multiple users while maintaining data separation and security. Each user gets their own account, their own conversation threads, and can even have different API key configurations if needed.

The project is particularly valuable for those exploring GitHub Models, which provides free access to cutting-edge LLMs including GPT-4o, Meta's Llama models, and DeepSeek. By configuring LibreChat to work with GitHub's inference endpoint, you get enterprise-grade AI capabilities without the typical cost barriers.

What’s the catch?

Once you've installed LibreChat, you'll find that there's no configuration option to add GitHub LLMs using the UI:

This is because GitHub Models uses a custom OpenAI-compatible API endpoint that isn't natively recognized by LibreChat's built-in providers. While LibreChat has pre-configured support for major providers like OpenAI, Anthropic, and Google, GitHub's inference endpoint requires you to define a custom endpoint with specific authentication and routing parameters.

The following configuration approach keeps your API credentials secure, allows dynamic model updates without rebuilding containers, and gives you full control over which models appear in your interface.

Configuring LibreChat to Use GitHub LLMs

After installing and running LibreChat, you need to make a few configuration changes so it can authenticate against GitHub Models and expose those LLMs inside the UI. This is done by defining a custom endpoint and mounting it into the LibreChat Docker container.

Important: If LibreChat is already running, you must stop the containers before making these changes.
From the LibreChat directory, run:

docker compose down

Stopping the LibreChat containers is important because Docker does not reliably apply new volume mounts to running containers. Bringing the stack down first ensures the new files are mounted correctly and read during startup. If this step is skipped, it may start normally, but GitHub Models will not appear in the interface.

Step 1: Create the LibreChat configuration file

From the LibreChat root directory, create a new configuration file:

nano librechat.yaml

Paste the following content into the file:

version: 1.2.8
endpoints:
  custom:
    - name: "GitHub Models"
      apiKey: "YOUR_GITHUB_TOKEN"
      baseURL: "https://models.github.ai/inference"
      models:
        default: ["gpt-4o-mini","meta-llama-3.1-8b-instruct","gpt-4o","deepseek-r1","gpt-4.1-nano","gpt-4.1-mini","gpt-4.1"]
        fetch: true
      titleConvo: true
      titleModel: "current_model"

What this does:

name This is how the provider will appear in LibreChat's model selector.
apiKey Your GitHub Personal Access Token with access to GitHub Models. Replace YOUR_GITHUB_TOKEN with the real token.
baseURL: Points LibreChat to GitHub's inference endpoint.
models.default Defines the models available by default in the UI.
fetch: true Allows LibreChat to dynamically fetch available models from GitHub.
titleConvo / titleModel Improves conversation titles by reflecting the active model.

Step 2: Mount the configuration into the container

LibreChat runs inside Docker, so it cannot see local files unless they are mounted. That's why we need to override the default Docker Compose configuration.

Edit the override file:

nano docker-compose.override.yml

Add the following content:

services:
  api:
    volumes:
      - ./librechat.yaml:/app/librechat.yaml

This is required because LibreChat expects its configuration at /app/librechat.yaml inside the container. The volume mount takes the local librechat.yaml and injects it into the running container.

Without this step, LibreChat will start normally but the GitHub Models will never appear, because the config is never loaded.

This approach also lets you update models without rebuilding the image and keep secrets out of the Docker image itself.