Several years ago, there was a candidate who interviewed for a position on my team.

I and another colleague were the “discuss system design stuff for an hour” team. It became clearer as we spoke that the candidate knew barely a fraction of what they claimed, and pointedly flailed with absolute nonsense answers even when my colleague or I tried to throw them a line to get back on track - like, it was clear they had no idea what the concepts being discussed were. Moreover, after some pointed questioning, the candidate admitted that the systems she said she had built at a previous job were, in fact, largely set up by someone else, and that she had just made some changes after everything was initially set up. A few more pointed questions uncovered a handful of additional blatant misrepresentations, at which point we called that portion of the interview done.

As a result of these discoveries, I submitted written feedback for the candidate. We use a 1-5 rubric - “definitely not” through “holy fuck throw money at this person”, more or less. I gave the candidate a 0, and specified that we had caught them lying about significant aspects of their experience and knowledge, that the candidate essentially admitted as much when we presented them with the obvious incongruencies, and that that level of disingenuous behavior is at odds with not only our technical standards, but ethical ones as well.

Later, we all met to compare notes. Another dev pair evaluated the “take-home project” (which I took when I applied - it’s not that hard) and called it “perfect”. The thing is, we prescribe that you should be able to complete the exercise in a couple hours, and absolutely shouldn’t spend more than like 4-5 on it… but we don’t really track the time taken to do the thing. I am 100% confident that the candidate worked for DAYS on that fucker. Anyways: the chat with the manager, and the “culture fit” meeting apparently went well for them, and because the takehome exercise result was “excellent”, those engineers thought that the candidate would probably be great at cranking out tickets. I objected strongly and consistently in the clearest terms possible that we’d be setting ourselves up for failure by bringing this person in, and diminishing the overall effectiveness of the team.

My concerns were overruled. The candidate has been on my team since then, to my intense frustration. They have an apparent inability to understand normal hit workflows, and insist on using a GUI that has created issues multiple times for the rest of the team. They are the specific reason why we tightened down all of our internal git project permissions from “you’re all devs, you know what you’re doing” to a minimal set of perms tightly scoped to our GitHub workflows. I have wasted weeks of development time un-fucking shit that this dev has broken. They are, by an order of magnitude, the most pain-in-the-ass member of the team, to the extent that even our manager has effectively siloed them off from the rest of us by mostly pointing this dev at peripheral busy tasks so that they don’t touch code that actually matters.

Before you ask: the person in question is a member of four protected categories, so they’re effectively un-fireable. I also found out about a year after the hire that they only even got the position they did because they needed visa sponsorship, and that title was the lowest level that corporate would do that for.

TL;DR: tests/project evals are absolute useful tools that can and often should contribute to a technical hiring decision. But they should NEVER be used as a basis for overriding conclusions reached from actual discussions, unless the technical challenge was executed in a fully-controlled environment (which I have done, and it was actually a much more pleasant process than I was worried it would be).