A new Coddy Developer Survey found that four in five developers, 80%, say their use of AI has felt more like a dependence than an advantage.
25year swe here. If AI disappeared tomorrow I’d “Oh no… anyways” in 2 seconds. AI’s a nice tool, it provides slight value. I’d miss it like I miss auto formatting or integrated terminals in IDEs, not really a big deal.
I don’t really derive a sense of accomplishment from it, so I don’t get the dopamine hit. It’s more write a simple bash script to do X because I hate working in that syntax, or this library has terrible documentation, tell me how I’m supposed to use it.
You still need to read and understand everything it writes, but it’s easier to validate something is correct sometimes rather than figure out the correct way to do something.
Also the freaking essays it writes as comments burying the useful information in the equivalent of a recipe blog post when you really just need assumptions about input, and side effects.
I definitely don’t get the high of coding anymore, but I do get a high of solving complex issues in architectural designs. I realized the high I chased when coding previously have shifted from coding design, to systems design.
I never got a high from coding, now compiling and seeing your tests have a green [PASS] next to them is a different story.
I find it anti addictive. Work is pretty aggressive (but now not too aggressive thanks to cost, but still a bit aggressive). So everyone is expected to show some utilization or be shunned. For the first time, I find myself thinking it might be nice to get out of this entire industry, if only I could afford to
Anyway, I managed to consume what was my monthly quota in a week and was quite happy that I could credibly ignore AI for the next three weeks.
Then disappointed as an administrator quintupled my quota the next day.
Though nothing is nearly as annoying as everyone else’s use is AI. I hate using it, and it’s just even worse dealing with the consequences of other people using it.
it might be nice to get out of this entire industry, if only I could afford to
I got pushed out of the industry right before COVID hit, so I haven’t ever dealt with AI. I’m a school bus driver now, which is infinitely more satisfying than programming since I get to drive a big Tonka truck around and gaslight children all day. The only downside is making 1/6 of what I used to make so this is not really a viable alternative for most people. At least I get subsidized health insurance.
Yeah, the other people factor is a pretty good point. Some people in the organization seem to think that they have no liability for their output anymore, so long as AI produced it for them. They will pass on bloated shit, unreviewed shit, laundry lists of glittering generalities, elaborate plans based on fundamental misunderstandings about a tool, … half of the problem never gets noticed because people see a wall of fancy text that they don’t understand, so they think it’s good. Stakeholders aren’t reading that shit, certainly not comprehending it. I know as much, because it’s a steaming pile of horseshit. Nobody is over here actually grading the horseshit. Nobody is having hard discussions about what’s acceptable and what isn’t.
I put a considerable amount of effort into making an LLM do exactly what I want… I use it daily, have for years, and I’m still pretty terrible at it. Just imagine what some people are producing, who actually trust these models… not to mention when they use the fucking model to step into my lane. I will call your shit out in a group conversation if you do that. Don’t fucking do that. If it’s my lane, don’t pollute it with that AI bullshit. Now I have to explain to everyone why that shit doesn’t work. Do a better job.
they use the fucking model to step into my lane
An extension of the “they think it can do everyone’s job (except their own)”. Even weirder, they tend to think that the people enjoy their obnoxious attempts to diminish their jobs. Then you just have one more thing to try to explain to management why the code generation from a sales guy doesn’t actually do what the sales guy thinks it does.
I will say at least around me, it has gotten less bad on this front as of late. Either a lot of these folks got bitten and learned the codegen doesn’t magically make them capable, or they just got bored of trying. Either way, it’s nice to have them quiet down a bit.
It helps if they actually have to use the crap code they vibe out. They can no longer yell at the nearest dev that their impossible requirement isn’t working. So there’s less pushback against bringing back their convenient scapegoats.
I can kind of relate. There are a lot of parts of coding I find a little boring.
- Once the interesting problem is solved and all you need to do is follow through
- Large scale refactorings in ways too complicated for an IDE.
- Fixing annoying dependency / migration issues
When you use an expensive AI agent, like Claude, it can nowadays handle these tasks competently, to the point where I only need to correct small things here and there.
When you give these tasks to an AI agent, it feels a bit like delegating to a junior, but without the guilt of giving someone a menial or boring task. That allows me to work on stuff I find more interesting. To me, THAT’S why it’s adictive.
However, this isn’t without cost. First, there’s the societal costs: Environmental, centralization of power, contribution to hardware shortages and a bubble
Second, there are more personal costs. You’ll come to rely more and more on these tools, and your skills will rust. You may end up avoiding learning things about a codebase because you delegate it away.
It’s a tempting tool. One which in my experience can genuinely help, but is easy to misuse.
Wow this is all so true and well put.
Some colleagues are showing off 20k+ line change commits with no review. Whilst I’m at most doing 100 lines, usually 10 or less. I am reading each change. Every plan is read thoroughly with multiple iterations. If I don’t understand a part, I ask the AI to explain. It is surprisingly good at explaining the work. However I do find mistakes. I have it commit after every change after the tests work and I’d say a third of the commits are prefixed as "fix: ". I’m still on the fence about the whole thing but it feels good and weird and phoney at the same time.
it feels good and weird and phoney at the same time.
I feel like: we’ve been developing these “best practices” of documented traceable requirements and design specs, repository storage of the whole change history, trace matrices showing test coverage / validation of all requirements and specs, code reviews, etc. etc. etc. and… for the most part… if you’ve got exclusively good responsible programmers on your team, most of that is a waste of time. But, when you have personnel turnover, people with … marginal skills, etc. those practices become much more important, even if they more than qunintuple the time required to do a thing, they enable projects to grow and be maintainable at much larger scale than if you don’t do them.
And along comes LLM agents, who strongly resemble those fresh hire colleagues of marginal skills, and they don’t complain about these “best practices” wasting time, and they’re so wicked fast that they can cut through the process that used to take 500% as long in 20% of the time instead… No, they’re not the greatest at getting things right on the first try, but they have been getting good at catching and correcting their own mistakes. And I can type messages like this one while they work on things that don’t need my attention…
Some colleagues are showing off 20k+ line change commits with no review.
Our primary use of the Cursor LLM agent is: code review. Refinement of the review to clear out the misconceptions - improvement of the pull request documentation to make intent more clear for everyone - not just the LLM agents.
We also have come (lately) to rely on it for writing unit tests. A year ago the LLM written tests tended to be ineffective, just “whitewash” coverage that didn’t really check the important aspects of the requirements. Today, they’re probably better than our Sr Sw Eng written unit tests, and you can crank the coverage arbitrarily high with very little effort.
Whilst I’m at most doing 100 lines, usually 10 or less.
One of my big criticisms of the .NET toolchain is that even simple changes can touch 47+ files, thousands of lines of code, and take hours just to have eyes on everything that changed, whereas a similar change in my Qt/C++ might be one line, or up to a dozen here a dozen there in maybe 6 files, but never the mess I’ve been seeing come out of .NET/WPF and friends for the past 20 years.
Second, there are more personal costs. You’ll come to rely more and more on these tools, and your skills will rust.
I mean, if you’re curious and you’re diligent, you can learn from what the AI chugs out. Skills only stagnate when you trust the AI blindly.
I’d say the bigger problem is that when you’re outsourcing junior coder work (even the tedious stuff) to a machine, you’re not investing in new junior coders.
I don’t trust the AI, I just can’t be bothered to argue. At the end of the day it’s no longer my code, so I feel no guilt about pushing in slop.
Trust But Verify is a good policy regardless of where your code comes from.
“AI” code is still human code in some capacity. Human code is what it was trained on. What would you do with a module you grabbed off GitHub or a script you found on Discord? Hopefully the same thing.
Yeah I relate strongly with this. Once you get used to having more time and energy for the tough problems, the interesting stuff, it’s hard to go back to the mechanical and menial.
But I’ve found it’s a lot like riding a bike. You don’t really rust, because the stuff the agents can handle today are the kind of shit that you’ve done so much and from the start, that it has become menial and boring. Which means it’s pretty ingrained into the backbone, at least that’s what I’ve found.
But damn if I don’t feel bad about it, recognizing and being conscious about the societal and environmental cost the convenience has.
Don’t get too bent about the environmental costs… the (US) AI data centers pollute about the same as 11 million office commuters. So, the great corporate leaders who are trying to mandate 30 million work-from-home workers back to the office are advocating for 3X as much pollution increase as the current crop of AI data centers emit.
I keep having the issue where I ask AI to do one small but tedious task, e.g. a refactor, and it sloppifies the rest of the code while it’s at it
Addictive? What the fuck are they coding?
Every time I try to use AI for coding I just get endlessly frustrated and angry.
The code LLMs spit is IME super buggy and gives as much work to correct it as it would have cost writing it from scratch… but it generally does the job. The addictive part is where you click a button and avoid thinking how to code.
This is my experience, but im an amateur and actually enjoy programming things… so, i piss on LLMs taking that from me.
EDIT: i know two persons that actively use LLMs, one does it to avoid having to ibteract with someone that actually knows how to code, the other learned to code to make money, so he really does not care that much.
Don’t forget that when you use LLMs you are helping these types of people:
https://www.wsj.com/tech/ai/claude-dario-amodei-wife-anthropic-e1eeda7d
Cami Clark—who started what she called a ‘revolutionary porn company’ where she sought investment from Jeffrey Epstein—keeps a low profile but is a key adviser to the AI chief. Its coming IPO could top trillions.
I had this experience once. We have a ChatGPT license where I work, and I asked it to configure a switch that I wasn’t familiar with. I simply described in words what network architecture I wanted and it did it! It even made some nice-looking documentation.
But, then I tried the new configs, and they didn’t work. It turns out there were some key syntax things it got wrong. And the documentation was wrong on top of that, with incorrect diagrams, and when I asked it to fix it it made different errors I the diagrams in different places. On balance, I still saved some time over reading all the manuals and figuring out the syntaxes myself, but only because I made my own documentation with the results that worked. If I had trusted the AI I would be sunk.
I’ve concluded that AI gives the illusion of competence, like a overly confident new manager. This can be very attractive to a less experienced person. But it’s really guessing, just like we all are. It can just guess after actually “reading” all the manuals. I haven’t used AI to write anything more than simple configurations and helper scripts. If I did want to use AI for more it would be in more of a pair-programming context. I might have a window open where I describe some things and ask for analysis, but I wouldn’t just run anything it does blindly.
There is also the ‘jagged frontier’ that can throw you off. The AI will solve hard problems for you, and you think, “Wow, it is more useful than I thought,” and then it gets absolutely stumped on a simple problem: completely stuck and unable to continue. If you are not a developer yourself, but just a vibe coder, you will be stuck there with your AI, with no solution.
That’s because the answers to hard problems show up often in the training data, whereas simple problems might not ever ever make it to any documentation since the solution just takes reasoning from what’s in the manual.
But it’s really guessing, just like we all are.
Beyond the moralistic objections I have to LLMs, I have serious issues with being fed confidently incorrect answers. I’ve had my share of configuration hell and I’m not above throwing code at the wall to see what sticks, but at least I have the good sense to drop a comment or mention in my commit that “hey, there’s a chance this isn’t right and could cause problems”.
I have serious issues with being fed confidently incorrect answers.
Except I have plenty of experience dealing with this from humans. (Mainly from the aforementioned new managers, because being wrong with confidence seems to be a key trait to get promoted.) It’s been my experience that when you tell an AI “I just tried that and it didn’t work”, it will accept that more readily than a human would.
Every bit of AI-generated code that I use, even in the smallest and most meaningless context, has to pass my own review first. I have to understand every line, and if I don’t I will ask the bot to explain what it did. By the time I am done with it, I can stand behind it just as if I wrote it all myself. I might note that I got AI help, but if my name is on the commit I will not pass the buck on any errors.
I see a lot of “human type mistakes” in my LLM agent sessions. If you manage these human like mistakes with the same procedures that keep them under control in real humans, the LLMs become much more useful.
This is how you use AI correctly
Then why not just write the fucking code yourself at that point? This is like putting training wheels on a tricycle.
I’ve written the fucking code myself for 30+ years. With AI, doing it right, fully understanding what it’s doing, it’s still 5x faster than doing it myself.
The same reason pair programming exists. You get a better outcome when you have someone (or these days, something) to validate and implement with.
Did you forget pair programming and pair planning exist?
I don’t trust A.I. code at all. If I ever do use it, I use it as a research tool like “please google for me how to do this one obscure thing because IDK what search query to use”; then I type out it’s output manually. Usually as I do so, I come across some subtle error that would cause horrible problems, and fix it as I go.
I tried to use it for a mathematical algorithm once. I might as well have just written
return Math.random();This is a good thing to do if your goal is to gain a deep understanding of something.
If it’s just to get it done, I just enforce TDD on my agent and review it’s output. I don’t need to be an expert in everything (and I am very much a generalist). But if you focus on a very specific thing and only that thing, then yeah what you are doing is a great way to truly understand it. It’s slow, but it’s totally valid.
Can you really say your code is done if you can’t understand it?
It’s still faster than what I did before A.I.; I would spend ages Googling for something obscure and scrutinising one vague StackOverflow post over and over for insights. Also cursing iOS Safari.
I like using it to setup github stuff and save time, like I needed to use rembg, I know you can setup terminal scripts as apps so if you open an app it runs the script. Had it set up a basic app to open videos with, create a folder using ffmpeg and turn it into an image sequence then run through that folder using removebg and/or depth anything (have added options for vectorizing, splats, etc.), afterwards sticying the image sequence back together to the original format, bringing back the audio. I was already doing this with comfyui before I realized they could be installed seprately be run through terminal commands, so I tried to get ai to set this up.
Took about 5 minutes and a penny or 2 using deepseekv4flash with hermes. At it’s core, it’s hella simple, it’s just running existing programs rather than coming up with how to do all the tasks itself. I technically didn’t need it and could manually type these terminal commands myself or figure out how to automate it, but ai setting it up means it actually got done and saved me hours of time.
After noticing most converters are frontends for ffmpeg and most downloaders yt-dlp, I realized you can easily make a gui for anything using the terminal with ai.
one of the programs that we use everyday at work recently added an AI coding tool. I was going to announce it to the team when I noticed with the usual disclaimers about ensuring you know what the macros are doing, but then just deleted my message.
we’re not a team of programmers, and there’s only one or two people on my team that I would trust to write code that could potentially cause us days of rework and tons of thousands of dollars lost to the company.
those other people don’t need an AI coding tool, because they can’t code in the first place, and those aren’t the people that I want modifying thousands of files at once when I know that they barely review the work they’re doing manually already and I have tools in place to semi-automate that review for them.
My coworkers use Claude like an actual brain subscription, and I have started to write off everything they say as if it came straight from the AI. They have gained so much unearned confidence about shit they have no idea about, and have even argued with the development team about it.
I got into an argument about how in band and out of band DTMF work with one of them for a solid half hour before they finally admitted they didn’t actually know but were going off what claude said.
I wanted to punch them for wasting everyone’s time. If you don’t understand stop answering definitively like you’re the expert.
Some customers have switched to using AI emails too. Customers that used to ask extremely low level questions will now submit a 2 page long email with action items and explinations about why our product does X, Y, Z, and I have to read it twice to figure out their problem isn’t even in the action items because the AI hyper focused on the wrong thing.
Like ok thanks you dumped the entire app log into Claude and asked it “why no work” and Claude read an error message that’s benign and now the customer is demanding fixes for something that is not and never has been a problem and won’t actually solve the root issue.
Claude read an error message that’s benign and now the customer is demanding fixes for something that is not and never has been a problem and won’t actually solve the root issue.
When we started cooperating with a team in India, we had to clean up our logs to reclassify those “errors that are not a problem” because we end up endlessly re-explaining to every new engineer they hire how “ERROR doesn’t indicate a problem in this situation.” No, writing docs explaining that and assigning said docs as required training ALSO did not help.
But just think of those billable hours
And that’s why AI tools should only be given to seniors who know where the bullshit is hidden.
I have had a fair amount of success in fixing these kinds of configurations with LLMs when the LLM also has access to independent verification (logs, system info utility output, etc.) that its changes / coding has worked as intended.
When you just give it a job and no way to check its own work, it’s a lot like people: it rarely gets it right the first try.
Considering the vast amounts of knowledge it has at its disposal, I can only conclude that it’s not very smart at applying it. A person with a fraction of that knowledge will produce better results.
So it has more access to information, but the results are poor compared to a person.
The important thing to remember is that it actually has zero access to information, because that’s not how LLMs work.
At their core, they’re vector databases, and they’re trying to probabilistically come up with the next most likely token in a stream of tokens found in the DB. You can manipulate the stream by injecting text such as the content of existing files (which becomes more tokens) into the stream, but it never actually understands any of it.
That’s why hallucinations are inherently unavoidable. It’s really all just hallucinations. It’s just that you can sometimes get useful text from their hallucinations if they happen to comport with reality.
Well, vector fields are information. But they have no understanding. The number 1 might be followed by 2 in 99.999% of cases, but it has no function to explain why, or to contextualize a scenario where that might be wrong.
or to contextualize a scenario where that might be wrong.
Actually, the many dimensions of the vector field are exactly where and how they do this.
Google Gemini prompt: “List common situatuions where 2 would not be expected to folow 1”
Here are common situations where the number 2 would not be expected to follow the number 1:
🔢 Alternative Number Systems
- Binary code: Counts 0, 1, 10, 11 (2 does not exist).
- Odd numbers: Sequential listing skips even numbers (1, 3, 5, 7).
- Prime numbers: Starting a list of odd primes skips 2 (3, 5, 7).
- Fibonacci sequence: The sequence begins 0, 1, 1, 2 (1 follows 1).
🏷️ Identifiers and Classifications
- Software versioning: A patch update moves from version 1.1 to 1.1.1 or 1.2.
- Sports jersey numbers: Roster listings sort by position or last name, not sequence.
- Product models: iPhone models skipped from 8 to X (10), and later from 11 to 12.
- Street addresses: Odd and even numbers sit on opposite sides of the road.
🕒 Time and Measurements
- Military time: The hour 01:00 moves to 02:00, but minutes go from 01 to 02 up to 59.
- Calendar dates: January 1st is followed by January 2nd, but February 1st follows January 31st.
🃏 Games and Sports
- Playing cards: An Ace (1) can be followed by a King in a high-low wrap sequence.
- Leaderboards: Tie scores result in two players holding 1st place, skipping 2nd place entirely.
- Dice rolls: Independent probability means a roll of 1 has no bearing on the next number.
🗣️ Linguistics and Formats
- Alphabetical order: When sorting numbers as text, 1 is followed by 10, 100, and 11.
- Roman numerals: The value I is followed by II, but formatting rules change at IV (4).
To help me narrow down what you are looking for, could you tell me if you need this for a mathematical logic puzzle, a programming algorithm, or a creative writing project?
Those context questions it asked at the end are a prompt to you to feed it dimensional guidance into its vector fields for more specifically applicable responses to your vague and open ended example.
They’re information, but not the same information that was used to create them.
Ypu have no idea what youre talking about. They absolutely have access to “information”
No, they really don’t. That’s not how they work. At least, not if the “information” you’re talking about is real semantic content that real minds can process.
Every piece of information you think an LLM has access to is actually just converted into a stream of additional tokens that are fed into the model to (hopefully usefully) modify the next tokens it predicts. That’s not the same thing as having actual access to information. Tokens are just numbers with statistically more (or less) likely relationships to each other.
I’m not trying to downplay LLMs. They’re architecturally interesting and have genuine uses. I’m just trying to head off a bit of technical inaccuracy.
Absolutely correct and well said. Until you give it a tool to call a websearch (in my case SearxNG), I occasionally break it out (local 27B model) when a search is pulling lots of AI slop, Spy vs Spy style. I make it give me references and it usually indicates a bad search (XY problem)
LLMs don’t. There are tools that can fetch new information and then gets fed into the model as more tokens, but that’s just a special case of what kescusay is saying about injecting text.
This is crazy lol they obviously have access to information.
As I’ve mentioned elsewhere, not if by “information” you mean semantic content that a mind can process. What they have are vector fields (essentially just numbers) with statistically more or less likely relationships.
If I say, “take me out to the ballgame” to an LLM, the tokens representing the words in the next verse of the song are statistically “close” in the vector database, so it’s likely to generate them. But that doesn’t mean it actually knows the lyrics… or even has those lyrics recorded in a regular database anywhere.
That’s why they hallucinate. The model determines that the next token is something nonsensical, but it has no way of understanding that it has made a mistake. In a sense, it actually hasn’t made a mistake. It’s done exactly what it’s designed to do. It’s just that in the case of hallucinations, its output isn’t useful.
even has those lyrics recorded in a regular database anywhere.
Is that required?
Do you have those lyrics recorded in a regular database in your head?
Of course, LLMs are more “human” if they occasionally mis-remember the lyrics…
You are both right. An LLM inherently has access to stuff the same way a brain in a jar has access to stuff. It’s information comes from fine-tuning the models to return syntax that agent code can interpret as a request to invoke a tool. That tool returns information to the context of the conversation. It doesn’t learn and it can’t truly remember things. Every time you start a session it is brand new. It sees your codebase for the first time every time.
The information access they have is whatever the agent allows it to access via tool exposure. Be it built in tools, or MCP servers
It doesn’t learn and it can’t truly remember things. Every time you start a session it is brand new.
Eternal sunshine of the spotless mind… it has its advantages.
I have mine develop and maintain a set of documentation to introduce fresh agents to the project efficiently and correctly.
deleted by creator
Pair programming/writing/creating is exactly how AI is meant to be used. It is an augmentor not a replacement for human competence. The person using it still has to do the thinking, qc, and directing, not take the first output as final.
I’ve concluded that AI gives the illusion of competence, like a overly confident new manager.
I like to call this “confidently incorrect”, and ChatGPT is probably the worst culprit.
I’m Buzz Lightyear, I’m always SURE!
I saw precisely this, that the AI are pretty terrible at generating switch configuration, which surprised me as I thought that would have gone pretty well.
And sure, people might guess and some people guess with similar confidence, but I can’t stand those folks already (extends to AI).
But the plausible looking config with supreme confidence already convinced the non technical management that we should rely heavily upon it.
I’ve concluded that AI gives the illusion of competence
Basically the same conclusion I came to. Which is terrifying when you think of all the devs who depend and all the money thats riding on it.
there were some key syntax things it got wrong
And syntax is the kind of thing llm’s should be awesome at. If they can’t even get that right we’re all cooked.
In fairness, this was all on a switch, where the commands are very tightly tied to the vendor and their underlying in-house shell. So commands may vary by release greatly. I was already explicitly telling the bot what software version and licenses I had, but ultimately I had to resort to the CLI’s help function at times and tell the bot “The command you gave me didn’t work. Here’s where it broke, and here’s the commands it will accept”. Given that information, it could (generally) figure it all out.
I dunno if you had this experience but the one I used made up commands if it didn’t know them.
It basically hallucinated them.
I’ve had them do this for various things, nowhere near the 20% of the time they’ve called “the hallucination rate” but definitely once in a while if it doesn’t know it will guess.
Which, to be fair, is what I’ve been doing for 40+ years in computer programming. When you have the ability to check if you’ve guessed right or wrong in a couple of seconds or even minutes, but researching it to “be sure” (and sometimes still get it wrong) before trying might take hours it’s only natural / efficient to guess a few times before giving up and RTFM.
This to a fucking T. It will be cock sure of accurate reliable results and run you in circles sometimes for hours and even repeat the same things when it doesn’t know. Ask me how I know. 4 hours alone yesterday fixing a production screen problem.
It’s almost like all Big Tech sells is addiction.
A fun game to play is watching any tech talk even remotely about UX and replacing the words engagement or retention with addiction. The best category for this would be the 2010’s smart phone app design talks.
Well said.
Ultimately it’s a capitalism problem, it makes the value proposition drift towards addiction in every single case.
So my preferred agent atm is GitHub Copilot as a jetbrains plugin. The other day when GitHub had a 6 hour outage and I couldn’t log in, my first instinct was “guess I can’t work today”. I realized almost immediately how fucking dumb that initial thought was. Like I had forgotten that I’ve been doing this for over 20 years, and around 17 of those with no AI whatsoever.
Was I slower that day? Yes absolutely. Was I incapable? Not at all.
I wouldn’t want to do without it, because on the positive side it has helped me discover libraries and tools I had no idea existed, and it’s made me a better archtitect. But, maybe we should still actually touch the code a little.
I wonder how long that will still be the case? The more of the mental load you pawn off on the chatbot, the less your brain will prioritize retaining it.
It’s like when you spend years only speaking a second language you’re fluent in, it can be hard to switch back to your native tongue when you need to.
I’m curious what tasks are you using GitHub Copilot for? I left the industry before AI got huge so I was never pressured to use the tools.
All agents are pretty much the same. Think of it like pair programming with someone, except that someone doesn’t have feelings and you can micromanage them.
In general, I give it instructions for what I want to accomplish. It has a “plan mode” that basically instructs the LLM to give me an execution plan to approve before actually doing it. We iterate on the implementation plan together and then when I satisfied I let it generate code.
It generated diffs essentially that I can approve or deny directly, in aggregate or by individual chunk.
I can (and do) provide custom instructions that it loads whenever I start a session. Instructions are basically md files, but it can be any text.
It’s a very different way of writing code, but if you ever pair programmed with a knowledgeable junior then that’s kind of what it feels like.
It can write code decently well and fast, and is amazing at finding stuff in a large project.
But you still 100% need to verify what it does and truly understand it.
This isn’t what I was asking for, I’d like to see something like, “When tasked with X AI did Y” with breakdowns between simple bug fixes to standing up a monolith in a legacy environment.
I mean that you would have to test yourself. To me it seems like AI funnily enough has the same issues as humans, and that’s that it works worse in legacy codebases full of tech debt, and in large monoliths.
But it can still be an invaluable tool.
I got a job once at a video processing software company, camera streams etc. The tasks there involved a lot of reverse engineering of under or undocumented things, very slow - I felt like I was getting 2-3 hours of “accomplishment” accomplished on a typical week there, very demoralizing, but they swore that’s how it always is there and I’m actually unusually quick at getting things done… IDK, I got a better offer and was out of there to a more productive job fairly quickly.
Coding with AI is really leaves you in a similar mental state as if you’ve just scrolled tiktok reels during that time.
Not if you don’t scroll tiktok reels while your agent is coding for you.
The biggest chilling effect I noticed, even as a developer who refuses to use AI, is that I feel like I can no longer ask my co-workers for advice. When I do, I get the the AI equivalent of “let me Google that for you, was that so hard?” So even without using it, I feel cut off from the normal places I would use to learn and improve my coding skills.
The only utility I’ve had with it is as a slightly more fancy document search. “Give me the syntax for [one line of code I want but can’t remember]” or “Give me the syntax to do a c# style .select() call in [language I am less familiar with]”
And then I look at the docs and actually put the effort into understanding how the fuck things work.
Using it for anything more than that is a mistake. You still need to understand the context of what it is you’re doing, you still need to think.
You’ve never tried just giving it the task you did to see how it did?
I hate doing a lot of code review all at once, and AI vomits a lot of code. Hopefully the mistake is obviously big and up front, but all too often the mistake dwells in the details that I would likely be too tired to notice after a lot of plausible code.
With human code to review at least the volume is generally workable and when it’s wrong, it tends to be more obviously wrong. A human that takes care of very detailed facets with care inspires trust in their thoroughness, but codegen looks that way without the thorough consideration.
Yes, and that’s the easiest way to get garbage.
Asking for a single line of syntax and using my brain will always give me better results than checking out and gambling that the the output is based in reality.
But coding is fun and there’s lots of free resources to help. What could be fun about begging an AI to write poorly made code?
Addictions aren’t necessarily fun
Okay fair but wouldn’t it be more aggravating than just making the code that does what you want it to do? If nothing else, they can copy from StackOverflow like everyone else does
This is a small, fairly biased survey with unpublished proper metrics. Whether you agree or disagree with the result, this is bad science.
Ah, but if you agree with the results you probably care much less about bad science than making AI users look bad.
Indeed
Ai coding is like the loom for society but you have to know how to use it.
People claiming its garbage slop are 2 years out of date on current models.
To all those people I suggest you familiarize yourself with it as whats coming is going to ruin the world but not because its slop because its going to be capable.
AI can handle boilerplate and any coding project that has been done ad nauseam. AI can’t handle legacy system quirks, creative problem solving, or massive projects with changing requirements. I don’t think the AI bubble will be around long enough for them to solve these shortcomings or the issues of expanding scope and scale leading to hallucinations.
To me AI is just impressive enough to convince managers and inexperienced developers it is the future. I’ll believe that when a Fortune 500 company rebuilds their whole entire tech stack via AI, and I think AI could solve climate change before it could do that haha.
AI can’t handle legacy system quirks, creative problem solving, or massive projects with changing requirements.
I think that disliking ai - for whatever good reason - makes people shun the reality, cherry-pick articles/studies etc, but we have to be honest: We all do that - even when we try to balance it, so that is normal behaviour within a spectrum. However, current anti-ai leaning do end up blinded to how fast things are moving with this technology because it is moving so fast. There are massive burnouts just from those following AI development every day, and they constantly have to expand/narrow their scope. If anyone takes a few days of in their field of interest, it feels like they have to sprint to catch up. It is accelerating and is no longer confined to model inventions from big corps with mega models - developers are building ‘cognitive architectures’ around the current llm’ limitations.
So to me, these ‘AI can’t …’ type of sentences are always changing as they become yesterdays ‘AI cant …’ sentences within days or weeks. It is a great ‘moving goalpost’ tho, as there are a lot of human level evolutionary abilities lacking from current models. But every second, one+ of millions of people are working towards adding that next ‘AI cant …’ sentence to yesterdays claims, and things are moving fast.
I think it is vital for everyone - in all corners of human ‘belief-space’ - to keep in active touch with this technology, as much as they can muster, agree they are angry but try to shake some hard assumptions/‘loosen up’, see where it is headed and adjust their expectations/plans for the future accordingly. Anger is an easy but dangerous thing as it can make us refuse to acknowledge something that we need to prepare for. We’ve all been there - many many times, and we all know how we think and act in anger. Now, is a bad time for that. Please familiarise your self with ai tech.
Not starting an argument/debate (i’m burned out atmo), just a long-winded observation really…
Haha I feel the long winded observation, I get it. I’m open to being wrong about AI, but you missed the main point of my argument
I don’t think the AI bubble will be around long enough for them to solve these shortcomings or the issues of expanding scope and scale leading to hallucinations.
Right now AI simply can’t (or at least last time I checked because you’re correct the technology moves very fast) do the things I mentioned.
These things, in my mind, are the gatekeepers for profitability if AI can figure these and any other issues and turn a profit before the bubble bursts, then yes we’d have an AI future. If we really get down to brass tacks (and now that I’m really thinking about it) that is the objective issue with AI, it isn’t profitable yet, it doesn’t matter if it can code a simulation of every life form on this planet if it isn’t making money. If it can solve this problem however, then yes I’d agree, everyone saying “AI can’t…” with eat their words seasoned with a little salt.
To address your accusation of cherry picking articles and advice for keeping in touch, I was kind of forced to cherry pick while attempting to keep in touch? When I was researching modern developments in AI all I was fed was an excessive amount of marketing material (this is a red flag for obvious reasons), I had to intentionally break good faith and search for things like “The issues with claude code” to get something written by a person who did not have a conflict of interest when it came to reviewing AI
I’d love to see examples of AI proving the naysayers wrong, but I was either seeing nothing but hype or enough evidence to remain skeptical of the technology.
What I’m trying to say is that I’ve tried to follow AI, I’ve tried to be impartial, but combined with what I am seeing, what I experienced as a software developer, and the economic side of AI, I don’t see any sort of AI revolution akin to the printing press or the internet. Just another race to wealth fueled by hype and some very impressive math.
I do think that outside of coding AI has some really great promises, like being able to design an Excel spreadsheet in plain English is amazing for anyone afraid of Excel, so microsoft copilot has my respect within the M365 suite, granted I don’t know what the Excel wizards have to say about the matter, but I think your average 9 to 5 desk worker whose experience with Excel revolved around accidentally opening it once would love copilot.
I think claude fable could organize, plan, execute, basically any coding task and human level complexity…
I’m asking for specifics, benchmarks, analysis, and data. Your responses are not that. I’ll look on my own I guess.
I disagree with much of your ai cant deal with complexity.
I dunno i am just saying be prepared because its going to blind side everyone thinking its a fad
Could you give some examples of AI handling complexity? From what I’ve read from senior developers is that the more scope AI has to chew on the more mistakes it makes. In terms of complexity I’m not talking about algorithms, differential calculus, or mind breaking recursion. I’m talking about the usual hodgepodge of monolithic microservices too many “Were a tech company that does X” companies have built over the past decade that are layered on top of software built before I was born.
Maybe I’m just creating too complex and specific use case, but my time as a software developer had me working through problems like that.
Furniture used to be made by hand. Then factories came along and furniture makers became furniture assemblers. The quality wasn’t the same but you could create it more quickly and for a cheaper price. For many consumers, the trade off between quality and price was fine.
This draws parallels with what’s happening in the software industry - we’re essentially making software factories and programmers are becoming ‘assemblers’.
The thing is, there is still a market for handmade furniture. The market is smaller than it used to be and there are less furniture makers but some people are willing to pay more for quality.
To say AI is going to ruin the world is a bit of a stretch. There will be software developers who are happy working in the factory assembling lesser quality products and there will be software developers who want to create higher quality products that stand out. I can see a market for both of these things. Whether you can achieve the latter with AI agents is the real question. I’m yet to see it.
It could be the loom or it could be laserdisc
it’s like gambling. you hope it churns out good stuff – and some of the time, it does! lucky!
then you try your luck again. and again with small “fixes”. and again.
That’s only if you have no idea what you’re doing with it. Depending on your skill level in development, that is not the case at all.
i don’t think that applies at all.
if you have no idea what you’re doing, you’re surely going to think all the churned out stuff is good.
if you have no idea what you’re doing, you’re surely going to think all the churned out stuff is good.
That depends on how much of an idiot you are. Intelligent people know when and how they are ignorant and they use that to inform their decisions.
You can think whatever you want, go ahead.
Indeed the more skilled you are the more infuriating it becomes when it does the dumbest shit.
