Position: LLM AI Quality Analyst (Spanish)
Type: Short-Term Contract
Compensation: $11 per hour
Location: Remote
Commitment: Part-time (30 40 hrs/week)
Role Responsibilities Evaluate a personalization feature for Gemini Design and execute multi-turn conversational prompts that require the AI to utilize personal information and experiences Assess how effectively the model uses past conversations and activity to generate relevant and helpful responses Evaluate model responses based on intent and appropriate personalization Analyze responses for grounding issues, including flawed inferences or hallucinations Assess integration quality to ensure personal data is incorporated naturally into responses Perform side-by-side evaluations and stack-rank model responses based on helpfulness and naturalness Write clear rationales referencing specific conversation turns Extract and verify debug information to confirm correct use of summaries and data sources Maintain data hygiene by deleting evaluation conversations after completion Requirements Experience in data annotation, AI quality evaluation, content moderation, or related roles Strong Spanish proficiency (reading and writing) Willingness to use a primary personal Google account and enable personal data sources for a
