AI-Assisted Developer Workflows (Trace) AI Task Auditor - Freelance AI Trainer Project

Accountabilities

• Evaluate developer workflow tasks for technical accuracy, realism, solvability, reproducibility, and alignment with reliable testing and evaluation criteria.


• Audit AI-assisted development scenarios to identify technical inconsistencies, logic errors, workflow inefficiencies, or issues affecting task quality.


• Assess IDE integrations, developer tooling, telemetry and trace analysis, and AI-assisted coding workflows within assigned areas of specialization.


• Test and troubleshoot complex technical scenarios to determine whether tasks function as intended and can be consistently reproduced.


• Provide clear, detailed, and actionable feedback that enables improvements to AI training and evaluation tasks.


• Apply professional technical judgment to ensure audited workflows are practical and representative of real-world developer environments.

• Demonstrable professional experience and strong technical expertise in modern developer tooling and AI-assisted development workflows.


• Familiarity with AI coding assistants, telemetry or trace analysis, IDE integrations, and broader developer workflow ecosystems.


• Strong analytical and problem-solving abilities, with the capacity to rigorously test, troubleshoot, and assess complex technical scenarios.


• Ability to identify subtle technical issues and communicate findings clearly through concise, actionable feedback.


• Candidates will be assessed for a specific area of specialization based on their professional experience; expertise across every listed domain is not required.


• Strong attention to technical accuracy, reproducibility, and practical applicability.


• Ability to work independently in a remote freelance environment and manage assigned technical evaluation tasks effectively.


• Access to a secure computer and reliable, high-speed internet connection is required.

• Competitive freelance compensation: $70–$100 USD per hour, with the final rate determined by experience, expertise, and geographic location.


• Fully remote: Work remotely from India and contribute to a globally distributed AI training project.


• Flexible contract format: Freelance engagement designed to provide flexibility in how and where you work.


• Technical impact: Apply your professional expertise to help improve the rigor, reliability, and practical quality of AI systems.


• Specialized work: Focus on the developer workflow specialty most closely aligned with your background.


• Contractor status: Health insurance, paid time off, and other company-sponsored employee benefits are not included.


• Equipment: Contractors are responsible for providing a secure computer and high-speed internet connection.

This listing is from Lever. View original listing ↗