Music Content Reviewer (AI Evaluation)

Gramian Consultancy is a boutique consultancy specializing in IT professional services and engineering talent solutions. With a strong background in software engineering and leadership, we help companies build high-performing teams by matching them with professionals who truly fit their needs.

Role Overview

We are looking for an experienced Music Domain Expert to contribute to the evaluation and improvement of Large Language Models. The role focuses on creating challenging music-related prompts, reviewing AI-generated responses, identifying factual and reasoning errors, and developing high-quality benchmark data.

The work may cover music theory, genres, composers, artists, instruments, production techniques, performance, and music history. This position is suited to professionals with advanced academic knowledge and practical, research, or teaching experience in music.

CONTRACT: Contractor assignment, 8 weeks

COMMITMENT: Full-time, 40 hours per week with at least 4 hours of PST overlap

LOCATIONS: Remote - Bangladesh, Brazil ,Colombia, Egypt, Ghana, India, Indonesia, Pakistan, Turkey, Vietnam

PROCESS: Initial screening followed by domain manager review

Key Responsibilities

• Create advanced music-related prompts across a range of disciplines and difficulty levels.
• Evaluate AI-generated responses for factual accuracy, reasoning quality, completeness, and nuance.
• Identify hallucinations, outdated information, logical inconsistencies, and difficult edge cases.
• Verify claims related to music theory, artists, composers, genres, instruments, and production.
• Develop benchmark datasets and adversarial test cases for model evaluation.
• Provide clear, objective, and evidence-based feedback supported by reliable references.
• Document findings and maintain high annotation and evaluation standards.
• Collaborate with AI researchers and project teams to improve model performance.
• Participate in calibration and quality-review activities.

Requirements

• Master’s degree or higher in Music, Music Theory, Musicology, Composition, Music Production, Performance, or a closely related discipline.
• At least 3 years of professional, research, or teaching experience in a music-related field.
• Strong knowledge of music theory and at least one additional area such as music history, performance, composition, genres, instruments, or production.
• Ability to evaluate complex music-related content for factual, logical, and contextual accuracy.
• Experience conducting research and supporting conclusions with reliable references.
• Excellent written English communication skills.