Mercor is recruiting full-stack generalist engineers to work with Cognition on building a data collection flywheel that will improve the coding capabilities of their AI SWE agent, Devin.

This is not a traditional software engineering or research role. You’ll be building the systems to test how an AI agent writes code, not directly developing the agent or agent’s outputs yourself. This can look like investigating customer usage data, writing robust evaluations to test the AI system, and extensively testing agent behavior.

You will be working directly with the Cognition research team on the following:

• Creating and supervising the creation of evaluations for a wide variety of coding tasks (e.g., migrating repositories from JavaScript to TypeScript).
• These evaluations will be used to measure Devin’s performance on an ongoing basis. You can think of these evaluations as test cases for the AI system.
• Analyzing how Devin performs on real users’ use cases.
• Communicating evaluation results to the research team.

You are a good fit if you:

• Are hard-working and detail-oriented.
• Have several years of professional full-stack engineering experience.
• Are actively looking for a long-term opportunity.
• Are based in the United States or Canada.

Details about the role:

• You will work directly with Cognition’s research team (though this is not a research role).
• The role is fully remote and flexible on specific hours.
• You can start part-time (minimum 15 hours per week).
• The project's duration is long-term (12 months+), with an opportunity to join Cognition as a full-time Applied Engineer upon strong performance.
• You’ll work as both an individual contributor and team lead.

About Mercor:

• Our team is based in San Francisco, CA.
• We specialize in recruiting experts for top AI labs.
• Our investors include Benchmark, General Catalyst, Peter Thiel, Adam D’Angelo, Larry Summers, and Jack Dorsey.

‍