Top Coding Models
Don't worry about staying up-to-date with the latest coding model to use in your favorite IDE. This mix will use the top coding models ranked in the LMSys Chatbot Arena for the Coding category
API
Example
Models
This mix uses the models below:
chatgpt-4o-latest
Provided by multiple sources
gpt-4o-2024-05-13
Provided by multiple sources
claude-3-5-sonnet-20240620
Provided by multiple sources
gemini-1.5-pro-exp-0827
Provided by multiple sources
meta-llama/Meta-Llama-3.1-405B-Instruct
Provided by multiple sources
gemini-1.5-pro-002
Provided by multiple sources
Readme
LMSys Coding Leaderboard Mix
Don't worry about staying up to date with the latest coding model to use in your favorite IDE. This mix is based on the top coding models ranked in the LMSys Chatbot Arena for the Coding category. The weight is a function of the Elo score adjusted for number of votes and variance. Models with more consistently high votes will be weighed more heavily.
Note: o1-mini and o1-preview are not included in this mix because they do not support system prompts and other parameters.
Categories
- 👩🏽💻 Coding
- 🏆 Leaderboard
Composition
This mix produces responses from the following models:
| Model | Weight % |
|---|---|
| "chatgpt-4o-latest" | 53.48% |
| "gpt-4o-2024-05-13" | 15.76% |
| "claude-3-5-sonnet-20240620" | 14.33% |
| "gemini-1.5-pro-exp-0827" | 8.67% |
| "meta-llama/Meta-Llama-3.1-405B-Instruct" | 5.03% |
| "gemini-1.5-pro-002" | 2.73% |
Update Frequency: Based on LMSys (every few weeks)
Source: https://lmarena.ai/