Using Copilot's LLM for your agent

About Copilot's Language Learning Model (LLM)

Copilot's Language Learning Model (LLM) is a powerful, large-scale language model that is trained on a diverse range of data sources, including code, documentation, and other text. Copilot's LLM underpins the functionality for GitHub Copilot, and is used to power all of Copilot's features, including code generation, documentation generation, and code completion.

You have the option to use Copilot's LLM to power your agent, which can be useful if you want your agent to be able to generate completions for user messages, but you don't want to manage your own LLM.

Using Copilot's LLM for your agent

You can call Copilot's LLM deployment at https://api.githubcopilot.com/chat/completions with a POST request. The request and responses should be in the same format as the OpenAI API.

To authenticate, use the same X-Github-Token header sent to your agent. For more information, see "Configuring your Copilot agent to communicate with GitHub."

Here is an example of how Copilot's LLM deployment is used by the Blackbeard extension to generate completions for a user message:

  // Use Copilot's LLM to generate a response to the user's
  //  messages, with our extra system messages attached.
  const copilotLLMResponse = await fetch(
    "https://api.githubcopilot.com/chat/completions",
    {
      method: "POST",
      headers: {
        authorization: `Bearer ${tokenForUser}`,
        "content-type": "application/json",
      },
      body: JSON.stringify({
        messages,
        stream: true,
      }),
    }
  );

To see this example in its full context, see the Blackbeard extension.