Adding AI skills to a job description has become easy. Assessing them is much harder. Across industries, organizations are updating role requirements to include AI literacy, AI-assisted productivity, and experience with AI tools. Yet interview practices have barely changed.
AI Has Changed What Competence Looks Like
The Real Signal Is Not AI Usage. It Is Judgment
AI Competence Is Less About Prompting and More About Problem Framing
Practical Lessons from the Interview Room
Nicolas Rubliauskas Umaras, Principal Software Developer
Hukivskyi, Bohdan, Principal Software Developer
Most job interviews are still designed to evaluate knowledge and experience in a world where work happened without an AI copilot sitting next to every employee. Here is our recent experience and practical approach to assessing AI skills.
AI Has Changed What Competence Looks Like
Today, hiring managers are no longer evaluating only what a person knows. They are evaluating how effectively that person can work alongside AI. And those are not necessarily the same thing.
Engineering leaders we spoke with described candidates as operating in two modes: independently and AI-assisted. A senior engineer may demonstrate strong judgment, deep technical understanding, and the ability to challenge assumptions without any external help. A junior specialist, meanwhile, may solve problems far above their experience level thanks to effective collaboration with AI.
This creates a new assessment challenge. We still need to understand a candidate’s fundamental expertise, but ignoring AI-assisted performance means ignoring how work is increasingly done in practice. Some teams have already started evaluating candidates in both scenarios: first without AI, then with it. The goal is not to prove that someone can work forever without AI, but to understand where their expertise ends and where AI starts amplifying it.
The Real Signal Is Not AI Usage. It Is Judgment
Many companies still begin with questions such as “Do you use AI?” or “Which AI tools do you know?” The problem is that these questions reveal almost nothing.
Most candidates can list ChatGPT, Claude, Copilot, or another popular assistant. What interviewers care about is something else entirely: judgment.
One engineering lead shared a simple but revealing question: “Tell me about a situation where AI was wrong.” Candidates with real experience rarely struggle to answer. They can describe hallucinations, misleading recommendations, or implementation mistakes and explain how they identified and corrected them. Candidates who present AI as consistently accurate often reveal a lack of practical exposure.
AI Competence Is Less About Prompting and More About Problem Framing
Another misconception is that AI expertise is primarily about writing better prompts.
In practice, experienced professionals increasingly behave more like architects than operators. Instead of starting with instructions, they start with context.
One interviewer used a simple example: an inexperienced user may ask AI to “build a social network.” A more mature practitioner will first define business objectives, clarify requirements, identify constraints, discuss architecture options, and only then use AI to accelerate execution.
Practical Lessons from the Interview Room
While much of the discussion around AI hiring remains theoretical, interviewers are already adapting their techniques.
Rather than asking candidates about AI usage, some teams simply let them use it during a practical exercise. An unfamiliar codebase, a new feature request, or a partially documented project becomes the assessment. The interviewer then observes what happens next. Does the candidate immediately ask AI to explain everything? Or do they first create a plan, narrow the scope, identify key components, and formulate targeted questions?
Interestingly, many interviewers say the strongest signal is no longer knowledge itself but learning velocity. Real projects rarely involve problems people already understand perfectly. More often, they involve unfamiliar technologies, changing requirements, and incomplete information. Strong candidates use AI to explore these unknowns, but they remain actively engaged in the reasoning process. Weak candidates tend to outsource the reasoning itself.
Some engineering teams are already discussing the next step: standardized AI-enabled assessments. Instead of allowing every candidate to bring their own tools, companies could provide access to the same sandbox project, the same AI environment, and the same business challenge. That would make it easier to evaluate not what candidates know, but how they investigate problems, validate information, and make decisions under uncertainty.
Recruiters Should Collect Context, Not Score AI Expertise
Another practical observation is that recruiters should not be expected to score AI competence directly. Evaluating AI workflows, prompting quality, or technical limitations often requires domain expertise.
The Future Interview Will Measure How People Learn
Perhaps the biggest shift is that interviews may increasingly stop measuring what candidates know and start measuring how quickly they learn.
When information is instantly accessible and AI can help explore unfamiliar domains, the differentiator becomes the ability to ask the right questions, evaluate the answers, build understanding, and make sound decisions. In other words, success depends less on recalling information and more on directing the process.
The question organizations ask today is:
“Does this candidate use AI?”
The more useful question is:
“Can this candidate think effectively while using AI?”
As AI becomes universally accessible, access to technology stops being the differentiator. Critical thinking, judgment, learning speed, and the ability to work productively alongside intelligent systems become the qualities that matter most. And those are exactly the capabilities future interview frameworks will need to assess.
