Data Annotation Specialist, Software Engineering

Role details

Who are we? Cohere is the leading security-first enterprise AI company. We build cutting-edge foundation AI models and end-to-end products that are designed to solve real-world business problems. We’re training and deploying frontier models for enterprises who are building AI systems. We believe that our work is instrumental to the widespread adoption of AI and we are looking for folks that want to be part of that. We obsess over what we build. Each one of us is responsible for contributing to increasing the capabilities of our models and the value they drive for our customers. Cohere is a team of researchers, engineers, designers, and more, who are all passionate about their craft. We are a global technology company headquartered in Toronto with key offices in London, New York City, San Francisco, Montreal, Paris, Berlin and Seoul. Join us! Why this role? This role will focus on evaluating coding tasks, requiring you to review and debug code, navigate repository architecture, and analyze model trajectories. Your work will contribute to our model development efforts and the logic our models apply when completing task requests. Please note: This is a part-time independent contractor position available within Canada. We seek candidates who are able to commit to 16 hours per week minimum at a 40 CAD/hour contract rate. This role is BYOD 💻 - Bring Your Own Device (laptop). Remote work within Canada. 12 month contract. Performance incentives included! As a Data Annotation Specialist, you will: - Evaluate the model's ability to respond to coding requests, workflows, and code base-related questions using available tools. - Assess agent trajectories and model capabilities for code generation and debugging requests. - Prompt models to complete complex coding tasks and review the accuracy of generated responses. - Label, proofread, and improve machine-written and human-written software engineering-related outputs. - Report quality and performance trends related to model/agent behaviour and project assignments. You may be a good fit if you have: - Subject matter expertise in software engineering - you have both studied in this field and have 3-5 years of relevant industry experience. - Proficient knowledge and understanding of Python and the following programming languages: Java, JavaScript, Go, and SQL (any dialect). - Prior experience designing, building, and deploying APIs - Prior experience working with code agents (OpenCode, Claude Code, Codex, Cursor) or evaluating agent trajectories is a big plus. The Candidate Journey: Initial Screening - Once you have submitted your application, our Talent Team will review your resume and writing samples. Virtual Annotation Test - This assignment will test your written and technical skills through various language-based tasks, such as a coding take-home assessment, writing sample, and more. Video Screen - If selected to move forward, you will have a short video call with a member of our Operations Team! Offer - Independent Contractor Agreement As an independent contractor, you maintain control over how you complete your work and may work with multiple clients simultaneously. We request that you declare any external work relationships with Cohere’s direct competitors and always maintain the IP confidentiality of the Cohere project. Independent contractors are not eligible for health benefits or other benefits provided to employees. Compensation for services is provided to contractors by self-invoicing for services provided pursuant to the terms of our agreement with the contractor. It is important to understand that, as an independent contractor, continuous work is not guaranteed. The client-contractor relationship is fundamentally project-based, meaning engagements may be temporary, periodic, or intermittent based on our organizational needs and project availability. As an independent contractor, you should anticipate fluctuations in workflow and, therefore, compensation for services when Cohere does not require as many hours of services in a week. Prospective candidates, please be advised: this role involves working with human-generated and model-generated tasks that may involve exposure to not safe for work (NSFW) text content as part of data annotation tasks, including explicit, offensive, or other inappropriate material. If the above qualifications do not perfectly align with your experience, we still encourage you to apply! We value and celebrate diversity and strive to create an inclusive work environment for all. We welcome applicants from all backgrounds and are committed to providing equal opportunities. Should you require any accommodations during the recruitment process, please submit an Accommodations Request Form, and we will work together to meet your needs.