Polaris: the evals team that grades Gemini.

01The two roles

Same team (Polaris, DeepMind, Mountain View), same core work: design tasks that test Gemini's ability to complete long-horizon software engineering. A task = a prompt that instructs the model + a grader/rubric that scores the attempt. One well-designed task takes about a week, built by hand — no synthetic pipelines.

02Why you fit

The posting asks for someone who uses coding agents and can build the tasks that expose what models can't do yet. That is your daily work, not a line on a resume:

• You operate coding agents at production scale, daily. You direct a multi-agent engineering team through structured task specs — specs in, production systems out, human approval gates at every step.
• You already design prompt-plus-grader gates. Every agentic build passes acceptance criteria and scoring rubrics you design: a prompt that instructs, a grader that scores, iterated until output meets the bar. That is the Polaris task shape exactly.
• You know where agents fail, from the inside. Ambiguous specs, untestable requirements, graders that reward the wrong behavior — you've spent years building the machinery that catches exactly those failures before they ship.
• You ship the data side too. Structured brief formats, validators, fixtures — most recently protean-sym2p, a dense agent protocol (99-byte wire format vs 481-byte v1 JSON, 4.9× denser) with translator and validation suite.
• No degree required for the Evals role — stated explicitly in the posting. Your McMaster attendance (one year each of BCom and Engineering) is honest and sufficient; the Research Engineer posting lists experience minimums, no degree.

03Your ammo

Proof points to drop in the application and interviews. All live, all checkable:

• protean-sym2p — dense wire protocol for agent-to-agent briefs, translator + validator. GitHub · site
• Open-source builds with docs sites: hydro-os, protean-edu, protean-subagent-viewer, protean-model-sync.
• Production platforms: sqip.askaconsult.com, shed.askaconsult.com, stoner.askaconsult.com (EN/ES), ibf.askaconsult.com.
• Practice: digital.askaconsult.com — the ASKA Digital site.
• Your story: ahraz.askaconsult.com/journey — the full arc, hardware to AI.

The one-liner

"I don't just use coding agents — I run a practice where every agentic build passes rubrics I design. Prompt plus grader, iterated until it meets the bar. That is eval-task design on production work, daily."

04Interview prep

Polaris interviews will probe one thing above all: can you design a good eval task? Expect to walk through a task end-to-end on a whiteboard. The axes they'll push on:

• Realism: does the task look like actual software engineering, or a toy? (Long-horizon, messy context, real constraints.)
• The grader: how do you score a solution numerically without rewarding the wrong behavior? What does a partial-credit rubric look like? Where could a model game it?
• Difficulty calibration: how do you know the task is hard for the right reasons — and how do you verify a human expert can actually solve it?
• Failure modes: where do agents typically break on this task — ambiguous spec, untestable requirements, tool-use errors, long-context loss?
• Iteration: how do you refine the task after watching model attempts? What do you change — the prompt, the grader, or the task itself?

Questions to ask them

• What does the task-approval process look like — who reviews, what gets rejected?
• How do you currently catch grader-gaming, and where is it still a problem?
• What fraction of tasks target training data vs. pure evaluation?
• For the FTC role: what does success look like at month 12?

05Eval-task walkthrough (rehearse this)

A worked example in the Polaris shape. Pick one from your own work and rehearse it out loud — this is the closest thing to a take-home for this team:

The task

Prompt: "Port this 400-line Python CLI tool to TypeScript. It must keep the exact CLI surface (flags, exit codes, stdout format), pass the existing 23-test fixture suite, and handle the three documented edge cases in EDGE_CASES.md. You may restructure internals freely."

The grader

• 40 pts — all 23 fixture tests pass unmodified.
• 25 pts — CLI surface identical: flags, exit codes, stdout byte-format (diffed against Python output on 50 generated inputs).
• 20 pts — the three edge cases from EDGE_CASES.md handled (one hidden test each).
• 15 pts — no network, no new dependencies, builds with tsc --strict clean.

Why it's a good task

• Realistic: ports are real engineering work — messy, judgment-heavy, long-horizon.
• Grader is hard to game: byte-format diffing punishes "close enough" rewrites; hidden edge-case tests punish spec-skimming.
• Calibrated: a strong engineer does it in a day; the fixture suite proves solvability.
• Diagnostic: failures separate cleanly — spec comprehension (edge cases), tool use (tsc strict), vs. sloppy output formatting.

06Comp & logistics

• Location: both roles list Mountain View, CA. You're in Toronto and willing to relocate anywhere — say so early; Google sponsors (TN/H-1B typical for Canadians).
• FTC vs full-time: the 12-month contract pays $350K–525K base ($550K–1.1M total). Full-time is $147K–210K + 15% + equity. The FTC is the money; the FTE is the seat. No rule against pursuing both.
• Referral: a warm intro inside Google/DeepMind beats a cold application. Worth one ask before submitting.
• Materials ready: resume PDF (2 pages, dated, gate-clean), tailored cover letters for both reqs, this briefing.

07Checklist

• Google Careers profile complete (work history matches resume; no degree claimed — Evals posting requires none).
• Apply to Req 114077679900074694 (Research Engineer) with resume + cover letter.
• Apply to Req 95145323399127750 (Evals Engineer, FTC) with resume + cover letter.
• Ask around for a Google/DeepMind referral before or right after submitting.
• Rehearse the eval-task walkthrough out loud (Section 05) — whiteboard-ready.
• Prep two stories: one agent failure you diagnosed, one grader you designed that caught gaming.