Senior Research Engineer, LLM Evaluation and Behavioral Analysis

As a Senior Research Engineer, you will be responsible for evaluating and improving the performance of Large Language Models (LLMs) at Together AI. This role involves developing evaluation frameworks, designing high-quality test suites, and collaborating with cross-functional teams to ensure model reliability and quality.

Requirements

• Strong engineering skills with Python, evaluation tooling, and distributed workflows.
• Experience working with LLMs or transformer-based models, particularly in model evaluation, testing, or red-teaming.
• Ability to reason clearly about qualitative behavior, edge cases, and model failure patterns.
• Experience designing experiments, building datasets, and interpreting noisy behavioral signals.
• Understanding of function calling and structured output formats.
• Familiarity with GPU or distributed compute environments.
• Hands-on experience evaluating function-calling models, agentic systems, or tool-augmented LLM pipelines.
• Experience with multi-turn or multi-step reasoning tasks.
• Familiarity with inference systems, distributed infrastructure, or post-training workflows.
• Passion for discovering subtle behaviors, surprising model gaps, or edge-case failures.

Benefits

• Competitive compensation
• Startup equity
• Health insurance
• Other benefits