Reports

OpenAI does not officially support a model named llama3.1-8b.Therefore, attempting to use llama3.1-8b with OpenAI's API may result in unexpected behavior. Here's a list of OpenAI's supported models as of March 2025: https://platform.openai.com/docs/models

Together AI's API is compatible with OpenAI's libraries, making it easy to try open-source llms.You can get the API key by registering at Together AI website. Here's the code example:

import openai

client = openai.OpenAI(api_key=""TOGETHER_API_KEY"", base_url="https://api.together.xyz/v1")

response = client.chat.completions.create(
    model="meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo",
    messages=[{"role": "user", "content": "Please tell me a joke"}],
    stream=True  
)

for chunk in response:
    print(chunk.choices[0].delta.content, end="", flush=True)

ref: OpenAI library with Together AI documentation

79523629