Spanish Mix (Scale.ai Leaderboard)
A mix that picks the highest-ranked model for Spanish prompts, based on Scale's Multilingual Prompts Dataset aimed at measuring a model's proficiency in engaging with Spanish users from Spain, Mexico and the rest of Latin America, reflecting complexity of global communication.
API
Example
import OpenAI from 'openai'const openai = new OpenAI({baseURL: "https://api.crosshatch.app/v1",apiKey: "your_api_key_here"})const completion = await openai.chat.completions({model: "seal-spanish",messages: [{role: "user",content: "Hello there"}]})
Models
This mix uses the models below:
gpt-4o
Provided by multiple sources
gemini-1.5-pro
Provided by multiple sources
gpt-4-turbo-preview
Provided by multiple sources
Readme
SEAL Spanish Mix
This mix is based on the top models ranked for Spanish by Scale.ai's SEAL leaderboard. The weight is a function of the Elo score.
The evaluation process involves assessing model responses across three main dimensions: honesty (understanding and consistency), accuracy (correctness of claims), and helpfulness (instruction following and writing quality). Models are paired against each other and evaluated on these criteria, with a focus on instruction following abilities.
The top-performing models in the Spanish leaderboard are GPT-4o in first place, followed by Gemini 1.5 Pro (May 2024) in second, and GPT-4 Turbo Preview in third.
Composition
This mix produces responses from the following models:
Model Name | Weight % |
---|---|
gpt-4o | 22.51% |
gemini-1.5-pro | 21.54% |
gpt-4-turbo-preview | 23.79% |
Last updated: est. May 2024