Open-Source LLM API

Powered by Llama, delivering ChatGPT level performance.

5 times cheaper than GPT-4o - From $0.50 per 1 million tokens.

Our OpenAI-compatible chat API is trained to be a helpful assistant. We are supporting state of the art open-source AI models (Llama 3.1 8B and Llama 3.3 70B). They can be utilized for a wide range of applications and excel in tasks such as commonsense reasoning, world knowledge, reading comprehension, code-related tasks, and more. Test it for yourself in our AI API playground.

API Usage

Our OpenAI-compatible API takes a list of messages as input and provides an AI-generated (assistant) message as output. If you haven't already, you will need to create an API key to authenticate your requests.

// npm install --save openai or yarn add openai
import OpenAI from "openai";

const openai = new OpenAI({
  apiKey: "YOUR_API_KEY",
  baseURL: "https://api.lemonfox.ai/v1",
});

const completion = await openai.chat.completions.create({
  messages: [
    { role: "system", content: "You are a helpful assistant." },
    { role: "user", content: "How many days are in a year?" },
  ],
  model: "llama-8b-chat",
});

console.log(completion.choices[0].message.content);

API Response

{
  "id": "cmpl-8a9ba025b8a744e881636351a26e4642",
  "object": "chat.completion",
  "created": 1697721484,
  "model": "llama-8b-chat",
  "choices": [
      {
          "index": 0,
          "message": {
              "role": "assistant",
              "content": "There are typically 365 days in a year. However, in a leap year, which occurs every four years, there are 366 days. Leap years are used to account for the extra fraction of a day that is not included in a regular year."
          },
          "finish_reason": "stop"
      }
  ],
  "usage": {
      "prompt_tokens": 30,
      "total_tokens": 90,
      "completion_tokens": 60
  }
}

You can obtain only the generated assistant message with the following code:

response['choices'][0]['message']['content']

API Parameters

The API POST https://api.lemonfox.ai/v1/chat/completions takes the following parameters:

• messages
  array, required
  An array of message objects representing the ongoing conversation, each containing a role (system, user, assistant) and content. See above for an example.

• model
  string, required
  Specify the model ID to be used. We support llama-8b-chat and llama-70b-chat.

• max_tokens
  integer, optional, default: infinity
  Set the output's token limit, ensuring cost efficiency.

• stop
  string / array, optional, default: null
  A sequence of tokens that instructs the API to stop the generation when it occurs in the output.

• stream
  boolean, optional, default: false
  When the parameter is activated, incremental message updates are transmitted as server-sent events with data-only messages. Tokens are delivered upon availability. The stream's termination is signaled by data: [DONE].

• frequency_penalty
  number, optional, default: 0
  Ranging from -2.0 to 2.0. Positive values decrease the probability of generating identical text by penalizing tokens based on how frequent they were used in the text before.

• presence_penalty
  number, optional, default: 0
  Ranging from -2.0 to 2.0. Positive values increase the probability of using different words by penalizing tokens that were used in the text before, increasing the probability to mention new topics.

• temperature
  number, optional, default: 1
  Ranging from 0.0 to 2.0. Higher values, like 1.5, introduce randomness to the output, whereas lower values, like 0.3, bring focus and determinism.

• top_p
  number, optional, default: 1
  Top p, also known as nucleus sampling, shapes token selection. Adjust it to focus or diversify results. For instance, a top p of 1 includes all tokens, whereas a lower value like 0.2 prioritizes high-probability tokens for a more focused output.

Tips

• Conversations often begin with a system message, followed by alternating user and assistant messages.

• The system message shapes the assistant's behavior and can be customized, but it's not mandatory.

• User messages make requests or comments, while assistant messages store prior responses or demonstrate desired behavior.

• It can be helpful to include conversation history when user instructions refer to past messages, as models have no memory of previous requests.