extractgigs.com
← Back

Senior Software Engineer – Python (LLM Evaluation & Repository Validation)

via Turing

RemoteContractSoftware Engineering

Key skills

  • Python
  • LLMs
  • Docker

About the job

This is hands-on Python engineering work that helps measure how large language models handle real-world coding problems. It suits developers at tech-lead level who know their way around well-maintained open-source GitHub projects. The contract is with a fast-growing AI company that builds datasets to teach models practical software tasks.

What you'll do

  • Review and prioritize issues from popular open-source libraries
  • Prepare repositories for testing, including Docker and environment configuration
  • Judge how thorough and reliable existing unit tests are
  • Change and run code locally to see how well models fix bugs
  • Work with researchers to choose repositories and issues that push model limits, with a chance to guide junior engineers

What you'll need

  • At least 3 years of software engineering experience
  • Strong Python skills
  • Solid command of Git, Docker and basic pipeline setup
  • Comfort reading and finding your way through large, complex codebases
  • Open-source contribution or evaluation experience helps, and prior LLM evaluation or developer-tool testing is a bonus

Good to know

  • Remote contractor assignment for 3 months, with no medical or paid leave
  • Choose 20, 30 or 40 hours a week, working a minimum of 4 hours daily with 4 hours overlapping Pacific time
  • Open to candidates in India, Pakistan, Nigeria, Kenya, Egypt, Ghana, Bangladesh, Turkey and Mexico
  • Screening is two technical interview rounds

Similar jobs

Keep exploring