About this role
Design coding tasks that challenge frontier AI coding agents in self-contained Docker environments. Build broken software scenarios, write pytest verification, and calibrate difficulty so state-of-the-art agents solve tasks 20-60% of the time. Project-based role, 8-20 hours/week, up to $35/hour.
Must have
3+ years production software development in one backend stack
Python and pytest fluency required regardless of primary stack
Docker authoring with reproducible Dockerfiles and pinned dependencies
Linux and Bash comfort debugging inside containers
AI coding agent experience with Claude Code, Cursor, or similar
English B2+ written proficiency
Never written pytest from scratch is a disqualifier
Technologies
Responsibilities
Invent realistic developer scenarios with real bugs and broken ETL
Build reproducible Docker environments with pinned dependencies
Write pytest tests that verify outcomes deterministically without leaking fixes
Write instruction.md documents that read like Jira tickets
Write reference solve.sh proving tasks are solvable
Calibrate difficulty so agents solve tasks 20-60% of the time
Iterate based on feedback from expert QA reviewers
Review other authors tasks as a QA reviewer later
Benefits
Recruitment process
Project-based participation not permanent employment
90-minute sample-task screen plus short behavioral interview
Onboarding takes approximately 10 hours per first task
Steady state is approximately 5 hours per task with 2-4 parallel tasks
Realistic weekly load is 8-20 hours with higher volume for top performers
Task-based compensation equivalent to hourly rate depending on performance
Some projects include incentive payments
Rates vary based on expertise skills assessment location and project needs
Similar offers
Freelance Data Scraping Engineer (Python)
76,960 – 76,960 USD
/ year
Senior Software Engineer
98,000 – 147,000 CAD
/ year
Senior Application Support Engineer (Remote Ontario)
85,000 – 110,000 CAD
/ year
Senior Engineer, eCommerce
161,640 – 169,995 USD
/ year
Azure DevOps & Infrastructure Lead
114,400 – 166,400 CAD
/ year