me and a friend were arguing about this and i think the part i left out before is AI
like even if you accept that leetcode was never supposed to perfectly simulate the job and is just meant to test reasoning/pattern recognition/grind or whatever, does it still work as a hiring signal today?
you now have AI that can look at a self-contained coding problem, reason through it, generate multiple approaches, optimize the brute force solution, explain the complexity, all in minutes. then you have tools like Cluely and similar stuff specifically built to sit alongside interviews
you can’t really just pretend those don’t exist
meanwhile the actual job has also changed. engineers are already using AI to write code, review code, debug things, generate tests, understand unfamiliar systems, etc.
so wouldn’t it make more sense to assess someone inside an actual codebase? give them a scoped repo, a task, maybe even an embedded AI assistant and see what they actually do with it
can they find the relevant code? understand the system? decide what needs changing? tell when the AI is wrong? make the change without breaking something else?
obviously you can still cheat and obviously i’m not saying “here’s our 10 million line production repo, you have 40 minutes”
but surely a whole codebase with context is harder to just outsource to some interview copilot than “here’s one neatly packaged algorithm problem”
so i’m genuinely curious why we’re still optimizing hiring around leetcode instead of adapting the assessment to how software engineering actually works now
I’ve been against live coding in an interview for years. But now with AI it’s something I’ve brought back into interviews, and it is to be done without AI. I refuse to hire someone who can’t write a function by hand, let alone can’t think through a problem on their own. I don’t really care if they get the right answer or even if the code works. I just want to see that they are capable of thinking through something and writing something reasonable.
I’m not going to force anyone on my team to not use the tools day to day, but you’d better understand what’s going on and you’d damn well be able to do your job if there’s an outage. I’m also not going to have my team beholden to the whims of some LLM provider if they choose to change pricing or limitations.
Leetcode is still good for weeding out people who can’t code at all. I would never give someone a “hard” question from leetcode.com because those are more or less “have you practiced this?” but definitely easy and most medium questions are ok. You’d be surprised how many candidates can’t do an easy leetcode.com question.
It is perhaps slightly less relevant with AI being good, but really if you’re happy hiring someone who can’t write a for loop but can talk to AI, what added value are they even bringing?
If the goal is to test AI-era engineering, I’d make the exercise a small unfamiliar repo with one seeded bug and one misleading-but-plausible AI suggestion. Let candidates use any tools, then score the trace: what they inspected, what hypothesis they formed, what tests they added, and why they rejected or accepted the generated change. The final diff matters less than whether they can build evidence. That is harder to fake than a polished take-home and closer to real maintenance work.
Been hiring for many years. I’ve never made a decision based upon of even cared a tiny bit about the results of a programming assignment. I prefer to simply talk through how they think about problems. I’d rather hire a person that knows nothing of algorithms or time complexity that can reason and learn than somebody that can write a pangram verification or heap sort in minutes but generally sucks at communication or reasoning.
Same here. And when I was looking for jobs, I never had to do a programming excercise. Maybe that’s a regional thing? I’m located in Germany.
The most technical thing I had to do during an interview was to give feedback on some code - it took about five minutes and they were happy enough as soon as I criticized the use of field injections. It was a good format though, because it got us talking about technical stuff.
That is not the experience I had with jobs in Germany (remote from abroad). They were extremely rigid and by the book. It made me stop applying for jobs I’m Germany.
Were you doing on location interviews?
On location for my first job, about ten years ago, remote since then.
Then why give the assignment in the first place? I’ve actually started declining to interview if a programming assignment is required. I’m a senior embedded systems engineer; if I can’t give you the confidence that I know what I’m doing in a standard interview (whiteboard problem solving is fine) then it’s unlikely I’ll be able to get the job anyway.
We don’t, but even if we did it would be for the other interviewers I suppose.
Hiring now?
Unfortunately no. Did a few months back but not at the moment. Times are tough out there. It was a very good candidate pool which is candidly not a good sign.
Leetcode was never a good signal for if someone could be a productive developer; even hiring managers from Google have been open about that kind of interview being more about the ego of the interviewer than the quality of the candidate.
It has more to do with rote memorization than problem solving.
My impression is a lot of the people in the hiring pipeline are bad at this job. Maybe they’re good at other parts of their job, but not hiring. Especially not the recruiters who aren’t themselves technical
The problem is that the non-technical are the ones who care the most about hiring, and the technical people mostly hate it. If someone screens down a stack of 500 resumes to 100, and you need to hire 20 of those people in the span of six weeks, that’s a fucking chore. I absolutely despise timed coding interviews but sometimes there’s no way around it. I simply don’t have a spare 100 hours to dedicate to doing in person oral exams.
Bro culture continues
I think they are terrible, but also I suck at them. I think I’ve had to do only one thing remotely like such problems once in my 30 year career, and I figured it out over a couple of days like all the hard problems I encounter.
When I make hiring decisions, I want to see a clear understanding of defensive programming, troubleshooting techniques, curiosity, ability to question assumptions, recognition of local standards, and ideally a proven track record of delivery. Dumb questions are better than no questions. And for Christ’s sake, when I ask you how you figured out a hard problem, don’t fucking tell me you asked AI. If that’s a resource you know how to use well, that’s great, but I want to know you know how to solve hard problems if anthropoic goes out of business.
I had one guy say he asked AI and tried a bunch of things and none of them worked, so they paid more money and asked a smarter AI and that found the problem. Mate, you didn’t solve shit. That’s like saying, “every time I am stumped, I ask John and he fixes it.” Well, ask John if he’s looking for a job because I’m interviewing the wrong person.
I’ve just decided to switch careers for a while. Software is dead for the next 10 years. Find something else to do
What are you doing now?
Leetcode seems pretty useless right now. I think a take home, the results, and a conversation will lead to a really solid hiring signal though.
You just described the correct way to use leetcode type questions, unless you intend “take home” to be something longer. Anything take-home that takes longer than an hour I think is ridiculous to expect people to make the time for.
Because it’s an easy filter to apply. It’s a pretty bad filter, but it cuts down the pile of candidates, making the next step easier. Everyone knows it doesn’t do a good job of only letting the best candidates through, but that’s not the point.
“We want the best”
Cuts out possibly the best candidates, bad candidates, and everything in between. Amazing technique. Might as well just cut half randomly and save everybody some time.
Some of the best interviews I’ve both done and given were “take home” assignments of exactly that. You’d have a scaffold in a relevant web framework and then have a few days to add a feature or debug something and then submit a link to a GitHub repo with your project.
I actually think leetcode type questions are pretty useful, when done as a kind of pair exercise with an interviewer or with an interviewer discussing it afterword. It is useful to filter out the massive amount of charlatans in software and give ample opportunity for questions that let you place someone’s real skill level at coding. This kind of process won’t be obsoleted by AI, and in fact will still work to weed out vibe coders because they won’t be able to talk intelligently about the code they present.
All that said leetcode questions are not a complete solution and the other parts of the interview are equally important.
imo solving leetcode doesn’t helped that much even before AI, it was basically like remembering a lot of algo which you will never use in your entire life.
IMO, Leetcode and other programming exercises are a simulacrum for assessing aptitude. The exact exercise is not the point – though many managers might think otherwise – but rather is the process: can a candidate methodically approach a challenge with sufficient rigor as becoming of an engineer?
I’ve written before about what I look for when conducting interviews, in the context of embedded software engineers. In that realm, I’m usually probing for prior experience with computer architecture in general, not necessarily in the particular framework that the job will deal in. I want to see transferrable skills, because it’s kinda rare to actually find perfect candidates that already meet our final requirements. So instead, I expect candidates to be quick studies, the sort of people that can draw analogies and get an approximate answer. Everything else can reasonably be looked up in man pages and web searches.
But this might just be specific to embedded, where we do actually care about the nitty gritty compiler and assembly details, when there’s only 512 KB of RAM. And to be clear, a candidate that knows bit tricks will likely do well, but if that’s the only tool in their toolbox and they don’t or can’t understand why writing maintainable, mostly-portable code is important in a medium sized organization, then that could be a problem. In this realm, programming exercises are still a view into a mindset. Other CS fields may vary.
