← All Build Logs
AI AgentsFailures & FixesFailed → Fixed

Why One AI Interview Coach Agent Failed and One Didn't

Why One AI Interview Coach Agent Failed and One Didn't

I built an AI interview-prep bot with two agents. One crushed it. One was embarrassing.

A friend was prepping for a technical interview, so I built two agents to help: one for technical questions, one for behavioral ones. Same underlying model, two very different results.

The Build

  • Agent 1 (Technical): a RAG setup grounded in coding problems and system design references, meant to check solutions and explain reasoning.
  • Agent 2 (Behavioral): a role-play coach meant to simulate real behavioral interview questions like "tell me about a time you failed."

What Worked

Agent 1 performed well — clean code, clear explanations of tradeoffs, accurate feedback on approach. Technical questions have mostly objective right answers, and grounding the agent in reference material gave it something solid to check against.

What Didn't

Agent 2 was flat and generic. It gave textbook-sounding responses to behavioral prompts with no sense of tone, follow-up, or pressure — nothing like a real interviewer probing for specifics.

The Fix

The problem wasn't the model — it was the prompt giving it no persona or evaluation framework to work from:

Before (no persona, no framework):

You are a helpful assistant. Answer the user's question.

After (persona + structured evaluation):

You are a senior engineering manager at a large tech company.

You value honesty, humility, and specific examples over vague answers.

Evaluate the user's response using the STAR framework
(Situation, Task, Action, Result). If the answer lacks
specific numbers or outcomes, ask a probing follow-up question.

With a defined persona and an explicit evaluation structure (STAR), the responses got noticeably sharper — more like a real interviewer pushing for specifics instead of accepting a vague answer.

What I Learned

Technical tasks respond well to grounding in reference material — give the model correct source data and it performs reliably. Behavioral and conversational tasks need something different: a defined persona and an explicit framework for what "good" looks like. Treating both agent types the same way, with the same generic prompt, is why one worked immediately and the other didn't.

Want something like this built for your business?

See AI Agents services

Related Build Logs

Comments

No comments yet — be the first.