The Live Coding Interview

This is a step by step guide on how we run a live coding interview with engineers. This is a document our internal senior engineers use when assessing candidates.

We have over 300 engineers working inside companies across the US and Europe.

Our interview to hire ratio for the end client is around 2:1. The ratio is the main reason we run it this way.

We wrote this for ourselves, so some of it may only half apply. Take what's useful!

Owner: Anto Cabraja, CTO

Feel free to make a copy and edit it. That's what it's for.

Why we run interviews this way

We've been interviewing and hiring engineers since 2014.

For a long time we ran the standard process - Resume screen, technical questions, a take-home, a vibe / culture chat.

But our success rate was not strong enough. Too many mismatches.

Some people interviewed really well but couldn't actually do the job. Others built well but interviewed badly.

The methods each fail in a specific way:

• Resumes. Anyone can write anything. This was true before ChatGPT and it's worse now.
• Take-home tests. You find out what someone can produce with unlimited time and unlimited help. That's not the job. And the good ones often won't do them. A senior engineer with a job and a family isn't spending Sunday on your take-home.
• Question-and-answer interviews. You're measuring how well someone talks about engineering. Useful. Not the same as actually engineering.
• Culture chats. You find out whether you like them. Super important but different from whether they'll be good, and mistaking one for the other is how teams end up full of the same person.

So this is what we actually want to know:

Can this person think clearly, communicate, and build something live?

We assume they can code. By the time someone reaches this stage we've seen the resume, the history, the coding assessment. What we haven't seen is how they think when they work.

Can they come up with an approach from what they've done before? Can they explain it while they're doing it? Can they defend it when questioned, and let it go without ego if it turns out there's a better one.

It's very hard to fake competence you don't have for 45 minutes.

Worth being precise about the direction though. It's good at catching people who can't do the job. It's less good at guaranteeing everyone who is able to show you.

And some very good engineers just don't perform well with someone watching them work.

There's a section further down on that failure mode, because it's the one that can cost us good people.

What this process does not catch

Worth saying up front, because every process has holes and pretending otherwise is how you stop paying attention.

• It doesn't tell you how someone performs over six months. Stamina, consistency, whether they'll still care etc - you don't see any of that in one hour. This is where our personality and behavioural assessments, past tenures and references help us build a picture.
• It's biased against people who are slower and deliberate. Some genuinely excellent engineers think for a long time before they type. Under observation that can read as stalling. More on how we handle that below.
• It doesn't test how someone handles a codebase they don't know. Our tasks are scoped and set up. Real work is usually neither.

And this is a small sample. One session, one task, one interviewer's judgement. We use it alongside other signals to build a picture that will look like this example engineer profile -

Example of a senior AI engineer profile and interview notes
https://profile.cloudemployee.io/s/GCAafg9

We're not necessarily looking for the best engineer, we're looking for the right engineer for a specific role and client team, which is a narrower question.

It also runs the other way. We're not only asking whether someone fits the role, we're asking whether this company and this role are the natural next step in their career. When the answer is yes you get genuine excitement and the extra mile. When it isn't, you get someone competent who leaves in 6 months.

You can see this in the session if the task is the right one. Watch what happens when they hit the interesting part. Do they lean in? Or are they just completing it?

Someone can be excellent and still be the wrong fit for the team we're hiring into. Your job is to take the hiring brief and client profile and hire as if you are them.

Before the session

Spend 15 minutes reading the hiring brief <example here>, job description and whatever the account manager wrote about the client in terms of technical requirements but also personality and work style.

You're not just assessing whether this person is good. You're assessing whether they're right for this specific team.

Look at the resume and LinkedIn. Skill-set, seniority, gaps.

Have a quick look at their GitHub, public repos, anything they do outside their main role. Useful for opening the conversation and for seeing what they're genuinely curious about.

An empty GitHub isn't an issue. Plenty of good engineers don't have one.

Check the earlier notes so you don't repeat questions someone else already asked. If a previous interviewer flagged something, probe it.

Set up your environment and test it. Live Share breaks. Test it before, not during.

One-time setup, if you've never run one of these:

• Fork and test the base repo

• Install Live Share in VS Code or Codespaces

• Bookmark a CodeSandbox fallback

• Do a mock run with another engineer first. Genuinely - the first one you run will be bad, and it's better for that to happen with a colleague than a candidate.

Tooling. We use GitHub Codespaces with Live Share for the code, and Google Meet for video. Fireflies notetaker joins for the recording and transcript - make sure your Fireflies settings automatically join. Any equivalent stack is fine. The tooling matters much less than the format.

The Cloud Employee Account Manager looking after this company will email the candidate and send the invite and set up the interview.

Running the session

The whole thing is 1 hour.
5 minutes settling in, 45 building, 10 at the end to reflect. Don't let the build eat the last ten - that's where the seniority shows up.

Part one: set the tone. Five minutes.

Say something close to this: "This isn't a test. Think out loud, talk through what you're doing, ask questions, treat me like a teammate. If you get a little lost, just say so and we'll take a step back."

That last clause matters more than it looks. You are trying to create conditions where signal can appear. A frightened person produces no signal at all - you learn nothing except that they were frightened.

Some interviewers skip this because it feels soft. It isn't. It's the difference between watching someone work and watching someone perform.

Part two: build together. 45 minutes.

Pick one scoped task from our task library per role.
Fix a bug, add a feature, refactor something, write a test.

They drive. You navigate. You are not demonstrating anything.

What you're watching for:

• How they break the problem down before starting

• Whether they ask clarifying questions, and which ones

• Naming, structure, readability

• Whether they explain what they're doing and why

• How they use their tools - git, search, AI assistants

• What they do when something breaks

• Whether they adjust when you push back, or dig in

Take light notes. Write down moments, not judgements.
Eg. "Spent 5 minutes reading the existing code before writing anything" is more useful later than "good approach."

// Implement retry logic for a flaky async callexport async function withRetry(fn, { retries = 3, baseMs = 200 } = {}) {  let lastError  for (let attempt = 0; attempt <= retries; attempt++) {    try {      return await fn()    } catch (err) {      lastError = err      if (attempt === retries) break      const delay = baseMs * 2 ** attempt      await new Promise((resolve) => setTimeout(resolve, delay))    }  }  throw lastError}const user = await withRetry(() => fetchUser(id))

Part three: reflect. Ten minutes.

This is where seniority shows up, and it's the part people cut when they run out of time. Don't cut it.

• "How would you improve this for production?"

• "Any edge cases we didn't cover?"

• "How would you test this?"

• "If you had two more hours, what would you clean up?"

A mid-level engineer answers the first question. An experienced engineer has already been thinking about it during the build and tells you what they deliberately skipped and why.

The AI question

This is the thing everyone's asking now, so it gets its own section.

Let them use AI. Copilot, Cursor, Claude, whatever they normally work with. Banning it tells you how someone codes in conditions that no longer exist.

We want engineers that use it correctly to maximize output whilst not making more work for themselves.

What you're assessing is not whether they use it. It's how.

Good signals:

• They know when to reach for it and when not to bother

• They read what it produced before accepting it

• They can explain code the AI wrote

• They notice when it's confidently wrong

• They use it for the boring parts and think for themselves on the hard parts

Bad signals:

• Prompt, accept, move on, no reading

• Can't explain a line they just committed

• Reaches for AI on something they should simply know

• Keeps re-prompting when the problem is that they haven't understood the requirement

Ask directly at some point: "Why did you accept that suggestion?" The answer is one of the most informative things you'll get in the whole session.

One study worth knowing about. A randomised trial by a research group called METR took 16 experienced developers working on large codebases they knew well, and found they were around 19% slower using AI tools - while believing they'd been about 20% faster. None of them noticed.

Small sample, and a specific population: senior people, deep in familiar code. On new projects and boilerplate the same tools genuinely do speed people up. So don't read it as "AI makes developers slower."

Think of it like this: in at least one careful study, good engineers were badly wrong about their own speed and had no idea. Which is a reason to watch what someone does with AI rather than ask them how it's going.

Scoring

7 criteria, 1 to 5 each. Thirty-five available.

The letter in the first column is how heavily we weigh it when the total is borderline.
A-weighted criteria are the ones we've found predict success in a client team. C-weighted ones are learnable.

• 30+ - strong hire.
• 24 to 29 - discuss. Don't decide alone.
• Under 24 - pass.

On average we run 6 of these to put 2 candidates in front of a client. Then the client interviews 2 and hires 1.

A note on the weighting. Code quality is a C not because it doesn't matter but because it's the most improvable thing on the list. Someone who thinks well and takes feedback will write better code in three months. Someone who writes clean code and can't take feedback will still not take feedback in three months.

When two interviewers disagree. Happens regularly and it's usually informative rather than annoying. Don't average the scores - an average of 31 and 22 is not a 26, it's two people who saw different things.

Go criterion by criterion and find where the gap is. Most disagreements turn out to be one or two rows, not a general impression. Then ask each person for the specific moment behind their score. "He seemed sharp" isn't a moment. "He rewrote the loop after I questioned it and explained why the second version was better" is.

If the disagreement survives that, it's usually one of three things: the two of you were interviewing for different roles in your heads, one of you had a bad session and the other didn't, or the candidate is genuinely uneven - strong in some areas, weak in others. All three are worth knowing, and none of them are solved by splitting the difference.

Default when it can't be resolved: run a second session with a different interviewer and a different task. It costs an hour. A wrong hire costs considerably more.

Red flags

• Doesn't ask a single clarifying question

• Googles fundamentals they should know cold

• Defends bad code rather than considering the point

• Can't explain a trade-off they just made

• Ignores edge cases, tests, or the person who'll use it

• Gives up when stuck rather than narrating being stuck

• Copies and pastes without reading

One of these on its own is usually nothing. People have bad days. Two or more in the same session is a pattern, and patterns hold.

The one we take most seriously is the third. You are hiring someone who will be reviewed by other engineers for years. How they respond to being told they're wrong is close to the whole job.

When someone freezes

This happens, and it doesn’t mean what it looks like.

We've seen genuinely excellent engineers fall apart in a live session. Being watched while you type is not a normal working condition, and some people - often the more reflective ones - find it unbearable. Their inability to perform is telling you about the format, not about them.

If you think you’re seeing this, don’t push through. Options:

• Drop to pseudocode. Take the keyboard pressure off. Ask them to talk you through the shape of the solution instead.
• Just talk. Abandon the code entirely and discuss how they'd approach it. You lose some signal, you keep the person.
• Offer a take-home instead. They record themselves working through a task in their own environment, at their own pace, talking through their thinking. We use Screenity for this - free Chrome extension, records screen and voice, no time limit.

That last option contradicts what we said earlier about take-homes, and it's worth being clear about why. A take-home used as a filter is a bad idea - it screens out busy senior people and tells you little. A recorded take-home used as an accommodation, after you've already got a reason to be interested, is a completely different thing. You're not asking them to prove themselves to get in the door. You're giving someone you already rate a fairer format.

We're not hiring performers. We're hiring people who build things.

Adjusting by level

• Junior. Scope the task tightly. Prompt and nudge freely - you're not testing independence, you're testing whether they can think. Look for curiosity and how they handle not knowing. Perfect code from a junior is usually memorised.
• Mid. Less scaffolding. Watch whether they ask about requirements or just assume. This is the level where communication starts separating people.
• Senior. Give them something open-ended and slightly ambiguous. Watch what they do with the ambiguity - a senior engineer resolves it, either by asking or by making a defensible call and telling you they've made it. Expect them to lead the session. If you're still driving the conversation at minute thirty, that's your answer.

Task examples

Reuse the same task across candidates for the same role where you can. Comparisons get much sharper when the variable is the person.

Frontend (React)

• Add a dark mode toggle, persisted to local storage

• Refactor a component for reuse and accessibility

• Fix a layout bug on mobile

Backend (Node)

• Add JWT middleware to secure a route

• Implement retry logic for a flaky async call

• Add role-based access control to an endpoint

Fullstack

• Wire a frontend form to a backend route

• Add auth and session handling

• Write an integration test for a multi-step flow

Data / ML

• Write a z-score based outlier detector

• Optimise a slow pandas pipeline

• Add a test that catches nulls in a dataset

The best tasks are real. A genuine bug from your own backlog beats anything invented, because you already know what good looks like and you'll notice when someone does something you didn't think of.

After the session

Do this within the hour, while you still remember specifics.

Score against the rubric. Write down moments, not adjectives - "froze on the edge case, recovered when prompted" is worth more in a debrief than "handled pressure okay."

If you recorded it, the transcript is useful. Two prompts we use:

Score this live coding interview against the rubric below. Summarize performance in each category with specific examples from the transcript. [paste rubric]

Review this code, written during a live coding session under time pressure. Assess clarity, structure, naming, test coverage, and any obvious problems. Note that it was written in about 40 minutes.

Treat the output as a second opinion, not a verdict. It's useful mainly for catching things you missed while you were concentrating on the conversation.

Then submit the score and your notes to whoever owns the hire. For us that's the CTO.

Whatever you do, don't let scores sit in someone's notebook - the value of a rubric is that it's comparable across candidates and interviewers, and that only works if it's written down somewhere shared.

Get a second opinion Prompt >

When a score lands 24 to 29 use this prompt to help you review the candidate.

Paste in your scorecard and the transcript. It argues the other side.

https://www.cloudemployee.io/prompts/second-opinion-light

Where we've got this wrong

The candidate who passed everything and still failed.

An engineer for an AI role cleared every stage. Our interview, the live coding, a final with the client's Head of AI. Two months in it was over.

The problem wasn't skill. The client ran on defined playbooks and he wouldn't follow them. He thought the process was tedious, disagreed with the approach, and worked around it.

Looking back, everything that impressed us happened one on one. Which is exactly where a strong individual contributor shines. Nothing we did tested how he'd behave inside someone else's system.

So we split the assessment into two profiles: individual contributors and team players. Both can be excellent. They fit different companies. We now look for which one we're dealing with and how hard they'd find it to work inside a client's process.

The wider lesson: company size and process maturity matter enormously, and technical vetting ignores them completely. Someone who thrives in a five person startup can drown in a rule heavy organisation. It goes both ways.

The candidate who failed the format and turned out to be a champion.

A backend developer, standard JD, process run correctly. He was nervous from the first minute. No working setup, couldn't compile code, came across as someone who didn't know what he was doing. He said his real environment was on a company laptop he couldn't share. We had no way to check.

On our own scoring he didn't make it.

What held my attention was how he modelled solutions and thought ahead. So I broke our rule. I gave him an offline task instead, a small codebase with three functions left undefined, set up in an environment he was comfortable presenting. He was a different engineer.

Partly because of that and partly because we were short on qualified candidates and under pressure, we sent him to the final. He's still with that client and he's one of our strongest placements.

His actual gap was that he'd never built a setup outside work. No side projects, everything on company machines. I'd have made a different choice myself, but that's loyalty, not inability. In a live interview it made him look junior.

I'm sure we've rejected good engineers for the same reason.

Scorecard

Copy this per candidate.

• Candidate:
• Role:
• Interviewer:
• Date:
• Task used:

Total: /35

Red flags observed:

One thing they did that impressed me:

One thing I'd want a second opinion on:

Recommendation: Strong hire / Discuss / Pass

This is how we run it. We continue to refine and improve it as we learn more and technology advances.

Find other Cloud Employee resources for hiring, onboarding templates, personal development programs here at - cloudemployee.io/resources