# Open-Source LLM API

### Powered by Llama, delivering ChatGPT level performance.

5 times cheaper than GPT-4o - From $0.50 per 1 million tokens.

Our OpenAI-compatible chat API is trained to be a helpful assistant. We are supporting state of the art open-source AI models (Llama 3.1 8B and Llama 3.3 70B). They can be utilized for a wide range of applications and excel in tasks such as commonsense reasoning, world knowledge, reading comprehension, code-related tasks, and more. Test it for yourself in our [AI API playground.](/content/playground/chat/index.html)

### API Usage

Our OpenAI-compatible API takes a list of messages as input and provides an AI-generated (assistant) message as output. If you haven't already, you will need to [create an API key](/content/apis/keys/index.html) to authenticate your requests.

```javascript
// npm install --save openai or yarn add openai
import OpenAI from "openai";

const openai = new OpenAI({
  apiKey: "YOUR_API_KEY",
  baseURL: "https://api.lemonfox.ai/v1",
});

const completion = await openai.chat.completions.create({
  messages: [
    { role: "system", content: "You are a helpful assistant." },
    { role: "user", content: "How many days are in a year?" },
  ],
  model: "llama-8b-chat",
});

console.log(completion.choices[0].message.content);
```

### API Response

```
{
  "id": "cmpl-8a9ba025b8a744e881636351a26e4642",
  "object": "chat.completion",
  "created": 1697721484,
  "model": "llama-8b-chat",
  "choices": [
      {
          "index": 0,
          "message": {
              "role": "assistant",
              "content": "There are typically 365 days in a year. However, in a leap year, which occurs every four years, there are 366 days. Leap years are used to account for the extra fraction of a day that is not included in a regular year."
          },
          "finish_reason": "stop"
      }
  ],
  "usage": {
      "prompt_tokens": 30,
      "total_tokens": 90,
      "completion_tokens": 60
  }
}
```

You can obtain only the generated assistant message with the following code:

```
response['choices'][0]['message']['content']
```

### API Parameters

The API `POST https://api.lemonfox.ai/v1/chat/completions` takes the following parameters:

- **messages**  
  *array, required*  
  An array of message objects representing the ongoing conversation, each containing a role (system, user, assistant) and content. See above for an example.

- **model**  
  *string, required*  
  Specify the model ID to be used. We support `llama-8b-chat` and `llama-70b-chat`.

| Model | Description | Price / 1M tokens |
| --- | --- | --- |
| `llama-8b-chat` | Llama 3.1 8B is a state-of-the-art AI model that can be used for the most complex tasks. The model is suitable for most tasks. | $0.50 |
| `llama-70b-chat` | Llama 3.3 70B is the most powerful AI model available. The model is recommended for tasks that require the most accurate results. | $1.25 |

- **max_tokens**  
  *integer, optional, default: infinity*  
  Set the output's token limit, ensuring cost efficiency.

- **stop**  
  *string / array, optional, default: null*  
  A sequence of tokens that instructs the API to stop the generation when it occurs in the output.

- **stream**  
  *boolean, optional, default: false*  
  When the parameter is activated, incremental message updates are transmitted as [server-sent events](https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events/Using_server-sent_events#event_stream_format) with data-only messages. Tokens are delivered upon availability. The stream's termination is signaled by `data: [DONE]`.

- **frequency_penalty**  
  *number, optional, default: 0*  
  Ranging from -2.0 to 2.0. Positive values decrease the probability of generating identical text by penalizing tokens based on how frequent they were used in the text before.

- **presence_penalty**  
  *number, optional, default: 0*  
  Ranging from -2.0 to 2.0. Positive values increase the probability of using different words by penalizing tokens that were used in the text before, increasing the probability to mention new topics.

- **temperature**  
  *number, optional, default: 1*  
  Ranging from 0.0 to 2.0. Higher values, like 1.5, introduce randomness to the output, whereas lower values, like 0.3, bring focus and determinism.

- **top_p**  
  *number, optional, default: 1*  
  Top p, also known as nucleus sampling, shapes token selection. Adjust it to focus or diversify results. For instance, a top p of 1 includes all tokens, whereas a lower value like 0.2 prioritizes high-probability tokens for a more focused output.

### Tips

- Conversations often begin with a system message, followed by alternating user and assistant messages.

- The system message shapes the assistant's behavior and can be customized, but it's not mandatory.

- User messages make requests or comments, while assistant messages store prior responses or demonstrate desired behavior.

- It can be helpful to include conversation history when user instructions refer to past messages, as models have no memory of previous requests.