"AI speedtracks bullshit shops into bullshit factories" is the other side of "AI enables efficiency gains beyond immagination".
As a freelancer I get to see both in action.
No surprise! Do you rememeber agile? Sometimes it was pragmatically applied towards efficiency, sometimes it became a bullshit religion full of priest and ceremonies.
And on i could go, with more examples, the gist stays the same : new tools, speed increase, faster crash or faster travel depends on the trajectory the company/team/project/thing was already on.
A special note on "People who cannot write code are building software."
"Fuck yeah" to that! Devs has shipped bad software to people in other departements/domains, for ages. They would never build something better if what they had was good in the first place.
When we (coders/startups) were doing it it was "innovation", now is "elephants in the china shop"?
And this is not a rethorical snappy question: that IS innovation, instead of critizing the "wrong schema" ... understand the idea, help build it and do the job: ship code that works and is safe.
Also, grey-beard here, pls, don't think you can ever have a stable job especially when code is around. It keeps changing, it always has, it always will.
AI bringing unprecedented changes is hype. The world always changed fast.
If "you" picked software development because of salary, you are in danger. If you did it because you love it, then tell me with a straight face this is not one of the best moments to be alive.
Let’s see if someone can point me towards some resources over the following.
The problem is mixing vibe-coding and agentic-eng, and switching the brain in 2 different modes (fast-feedback gratification vs deep-focus gratification).
There’s no clear cut rule on what works. Different people, different brains, and especially amongst devs some optimized low-key neurodivergence.
And then there’s waiting mode, those N seconds/minutes that agents take to think and write.
What’s the right mix?
Keep a main focused project and … what do you do in the meantime?
Vibe code something else?
Hn? Social media?
Draw lines on a paper sheet?
Wood carving?
Exercise?
Rewatch some old tv series?
I have experimented….
There are side activities that help you go back to the task at hand in the correct mental framework for it.
Not just for productivity, but for efficiency and enhancing critical thinking on the main task.
Or whatever you choose to optimize for.
Can anyone point me towards some people talking about this?
Exactly.
As long as the codebase is consistently following some given patterns, LLMs nowadays stick to it.
Understanding that limiting number of “design patterns”
in a codebase made it better (easier to code and understand) was a good proxy for seniority before LLMs.
Now it’s even better: if all of a sudden “unusual code” is in a PR, either the person opening the PR or the one reviewing it has lost touch with the codebase.
Very important signal, since you don’t want that to happen with code you care about.
> It is why we updated our mission, and why reliability is woven throughout our new credos.
… later …
> Over our ten years in existence, we have come across many of these cosplayers before, and every time we have put in the hard work to out-execute them and provide superior results.
… finally …
> We have spent ten years doing it the hard way - from making models better to turning them into applications that work when it actually matters.
I’m extremely proud of the work we are doing and the mission we are on.
What happened to the mission here?
Did it get updated? What was it before? What is it now?
If this was to inspire confidence, it did not really work for me.
It made me feel like a shell game was being played.
What not just record the conversation?
If contains all that is needed. The initial at large scoping, the failed attempts at doing x and not y, how that specific line of code solves that specific edge case, etc.
When it’s time to review, review both code and conversation.
200 “user written messages asking why and what”? Likely a good PR.
15 “yes, yeah, ok, whatever”? Well you might want to give that PR some love.
It feels to me that when we commit, we throw away half, if not most, of the work done by not recording it.
I agree with you overall, yet there’s one flow that works for me.
Instead of speccing out a feature, I let PMs vibe code it.
I then have the exact reference I need to build.
Maybe LLMs oneshotted the right way, maybe it needs fixes, maybe some fundamentals are misunderstood, in any case it’s easier for me to know what I need to build, for the PM to be aware of some limitations (LLMs do the job of pushing back and explaining) and overall for us to have to the point conversations.
It is somewhat orthogonal to what you say, when you focused on dev seniority, so that part stands true.
But I think “PMs armed with an LLM” can, when properly used, add a lot of value to the dev process.
> I agree with you overall, yet there’s one flow that works for me. Instead of speccing out a feature, I let PMs vibe code it. I then have the exact reference I need to build.
Like BDD, but with something more accessible than Cucumber. I'm totally here for that.
It would be nice if people also committed their initial prompt and chat session with the LLM into their codebase. From a corporate standpoint, having that would be excellent business logic as code, if the code is coming from a PM or a stakeholder on the business side of the house. From an engineering standpoint, it would be an excellent addendum to the codebase's documentation.
FWIW, BDD and frameworks like Cucumber don't work at all in my experience. The people that'd need to fill these out don't do it properly (they can't) and then we, devs, are stuck with brittle and un-debuggable stuff that's worse than if we just used regular code to encode what we understood from them.
It's the same reason (most) PMs armed with an LLM still won't get anything usable done. They can't do it properly. They still need devs. But the gaps are shrinking. Some few PMs can get stuff done w/ both Cucumber, could wireframe UX with previous tools and can now do so much easier and better with an LLM.
It would be nice if people also committed their initial prompt and chat session with the LLM into their codebase
I doubt you'd want this. It's a chat session for a reason. It's gonna be huge wall of text, especially if you meant to actually include all the internal prompting the LLM did while it was working. You'd also have all my "no dude, stop bullshitting me! I told to ignore X and use Y and to always double check Z and provide proof".
It would only "work" if every single piece of feature you wrote was 100% written by the LLM from a single, largish and well defined prompt, the LLM works for a few hours and out comes the feature. And even then you have no reproducability (even if you turned around and gave it to the exact same model, no retraining, newer model, system prompt etc.).
There are ways to play around the single wall of test issue.
Mostly, git lfs.
When it comes to “no dude stop etc etc” … that is valuable information. You can extract that and put down rules for agents so that you stop repeating it each time.
Same can be done at PR, so that you can review not just the code but also how you got there.
It’s trivial to go from session to a nicely polished html with side by side conversation.
If you want to try, username at gmail, I have a private repo with it running.
I value critics, sorry for the plug ;)
Oh, on the different models side, i don’t see the advantage of reproducibility, or better, I don’t think I understand what you mean, can you help me see it?
I don't understand how "wall of text" is related to git large file support. The wall of text is a problem for me, the human. Sure, there are ways, like "be brief", caveman etc. In a large repo with lots of different people over time, I can't see how it won't just be wall of text again. It's just too much. TL;DR. And coz DR, the LLM will have buried bullshit in that text, which future session might read and "believe".
As for "no dude", no that can't be put down into rules. Not all of it anyway. We have stuff encoded in the repo wide md file, I have my personal one etc. and the various agents still don't do what we tell them to in all cases or a new model comes out and it no longer works. For example, for finding the root cause of a bug, it's very important to have actual proof and references. It's getting there w/ my instructions in the .md but it doesn't always work and I do have to "dude" it from time to time.
Is that back and forth valuable to have in files that are going to be part of the repo? I very highly doubt it. Having new rules that came out of the back and forth in a checked in AGENTS.md, sure, that is valuable. Or nowadays in individual "skills", like a "root cause finder" skill that can have very specific instructions about being thorough in proving its "found the smoking gun" BS ;)
I've seen enough PR descriptions created by the agent. Fluffy wall of text that looks good but is factually wrong. Seen it way too many times. Too many people just look at whether it looks good and then pass it off as truth. I'm tired of it and making that into "nice HTML" doesn't make it better. It just makes it look even nicer but not more true.
Re: reproducibility. My parent poster (and I guess you as well) wanted to have the prompt/conversation as "documentation". I don't see why that would be helpful. The only reason I could see would be for "reproducibility", which you won't get with an LLM. I don't see why else, but do tell me.
What I can agree could be valuable are the "why"s. I.e. the stuff that already should have been part of the ticket/requirements document. If you want to store that inside the repo as text files, instead of the original tickets or documents, that's fine of course. But I don't see how a "recording of how the code came to be" is valuable. It's like having a recording of all my IDE keystrokes and intermediate code state in pre-LLM days. Not valuable. What's valuable are the requirements and the outcome (i.e. code). Not "the thing in between".
Now don't get me wrong. Recordings of how people code/use their IDE can be a valuable teaching tool. Both as good and bad examples. And the same can be true for an agent coding session.
i misunderstood "wall of text" (i was thinking about bloating repo with it), my solution to understanding is just to create ad-hoc tools to parse the json
i coded a web ui with simple toggles: show me what user said, what llm said (nice to see what I was thinking about, nice to see how LLM came up with solution X, you get tools calls, maybe it found something i didn't think about or viceversa)
you can search/grep (.ie: did i consider idempotency when i build feature X? open session, search/grep idempotency)
you can, up to some point, resume the conversation (yes i know, cache busting makes some usages of this impractical, but in general resuming and asking "when we did this, did we think about that" tends to work... let's say that research is ok, time travel, meh)
overall, one of the advantages of LLMs is to be able to direct then ad data for insights, via standard CLI tools, via specific prompts, or building some mini tools (yeah vibe coding is fine sometimes)
whatever my question, if i have data i can have an answer
LFS helps with the second aspect (buried bullshit). Unless you smudge, you have a pointer, and that is just 3 lines.
You need to learn some ergonomics, but ok, some of us learnt how to use Jira XD
Taking your position a bit further, yes, committing chat sessions implies that you also need to review them so that bullshit doesn't filter through. Milage varies based on your personal preferences, which project you are working on, and many more heuristics.
Some will find it boring, some will think it's good project maintenance, all should be able to find a way to handle this based on their preference.
It is also nice to pointout that cleaning bullshit doesn't need to happen at merge. LFS blobs being stored separately, you can have side flows helping you out, without clogging yoou CI pipelines.
"no dude"-> rules
you can put down SOME rules
usually this happens to me at PR. I am tired fo saying "you should always check X", so i bolt it down "someplace".
I am running the usual motions i suppose most of us try to adopt: put this down in agents.md, in folder x or y, in path-scoped rules using agents, in memory files (i am exporting/importing those too), in subagents that review code before PRs.
in the end it's an unsolved problem at large, but
1. hopefully it will get better, my feeling is that it's just a cambrian explosion, and the fittest will survive... (also, owning the harness should help, i suppose .. i use claude code :D )
2. in a team, having personal styles surface is valuable. "dude don't do that" is quite often .... design. When rules go in the repo, at least we can find an agreement in person, and at PR is not about linking a document you read at onboarding, but finding out why the agent did not respect the rule. To me, that is more grounded in a tool.
3. rules are ... not static? We change our minds? We get better at things? We want to experiment? I am not advocating for a perfect rule system that replaces me, but for a good enough one that removes cruft from my daily job.
I think my approach is actually helpful when it's time to find root causes (YMMV). Via tools that parse sessions, you can see when that specific portion of code has been written with a better granularity. During that bit of the conversation the user was worried about X and asked AI to do Z and AI read this and that file and "thought" this and that and wrote that piece of code.
Maybe the user was making wrong assumptions, maybe the LLM did not read the correct files or instructions, in any case you have a better tool for investigation.
It's up you to decide wether to use it, wether this lead to just solving the bug or also fixing instructions, etc, i am just saying that it actually helps to have some measure of the context on which this change made sense.
"Fluffy wall of text that looks good but is factually wrong. "
it might be good or bad, right or wrong, but what is in the sessions is the truth of what happened.
PR desc are horrible, i share that feeling with you, but having the story of how that thing happened is just not the same as "final summary of what we did in the past X hours".
As a sideline: LFS doesn't really pollute your repo once you get to learn its ergonomics. Having chats in LFS also lets you approach this
Reproducibility...
To me those conversation are basically the history for decisions taken while implementing. They are documentation.
The real problem with docs was that no-one has ever liked writing them, nor it was easy to implement a standardization around them.
If you just record/log, there';s no extra effort needed, and once there tools and LLMS are pretty good at helping us extract insights.
I am also assuming, there is a correlation between quality in the conversation and in the code. I know, i'm being hand-wavey, but overall i think critical thinking is what makes code better, and being able to see if/which it has been applied can be a good proxy.
I ask for forgivness already: I not going down the rabbit hole of quantifying quality etc. It's a broad statement that should be taken with a grain of salt.
If you want to go abstract, you can think of coding as going from thoughts to 0-1 in bits. We have high(er) level languages that help us organize thoughts to help us so that we can better keep them in our cognitive flow/load.
LLMs are an upper layer, that scrambles the code and make it more difficult to grasp.
But the reasoning behind the code is now available, and quite easy to parse.
I think this is the core point to me.
Code is an intermediate artifact between thinking and bits.
Now we have a second artifact: the conversation/decision that led to that code.
Why are we not storing it?
disclaimer:
I am, of course, mildly in love with my own project and ideas, so possibly i like this too much just because i built it. IKEA effect or whatever.
Haha, OK, we both tend to "text-wall" it seems, so seems we both shouldn't complain about LLMs. Or I guess: now we know how everyone always felt reading our stuff :P
no dude rules
Yes, I have these. That's how when I have it investigate, it outputs files and line numbers for example when the investigation is in our code base. But it still makes up stuff all the time. You need spidey senses that tingle and many people don't have them.
Just very recently, I saw a PR comment on why someone was choosing to do something in that particular way and what the other bad options would've been, i.e. justfying thei choice (at least they did do the "calling out" part. I had to comment about how none of that made any sense to me and why we didn't just do "other thing Y". Well turns out the AI had misled them, they believed it and it went downhill into a rabbit hole from there. I do believe that w/ the right spidey senses, even in an "unknown situation", it's entirely possible to come out the other end. But many if not most people succumb to the AI's nice and "sounds true" type language.
As a sideline: LFS doesn't really pollute your repo
LFS doesn't. Walls of text do, whether you use LFS or not. I.e.
no extra effort needed, and once there tools and LLMS are pretty good at helping us extract insights.
Nobody's really gonna read all that. The only way to get through it is to use LLMs, e.g. through summarization. That doesn't solve anything though. LLM summaries are very often wrong. Depends on the text/conversation and the LLM but have you tried slack summarizing a thread? Ouch! I've also tried Claude making tickets from slack threads. Ouch but less so. Still needs polishing. And more time polishing it than it would've required from myself to just type up the ticket myself. What LLMs are good at is if you put the actual "meat" down and they "fluff it up". But sorry, I'd rather juts have the meat and skip the fluff entirely.
Most LLM assisted bug reports on the other hand are huge walls of text with low signal to noise ratio. I.e. essentially the old
If I Had More Time, I Would Have Written a Shorter Letter
Famously the first known instance in the English language apparently was a sentence translated from a text written by the French mathematician and philosopher Blaise Pascal. The French statement appeared in a letter in a collection called "Lettres Provinciales" in the year 1657. It totally absolutely 150% applies to LLM use ;)
critical thinking is what makes code better,
Absolutely! And the issue with LLMs is that they tend to make it less likely for people to apply critical thinking. Even from people that (I at least thought) applied it in the past. "Does ChatGPT harm critical thinking abilities? A new study from researchers at MIT’s Media Lab has returned some concerning results." https://time.com/7295195/ai-chatgpt-google-learning-school/
Btw, I write all of this as someone that has been coding exclusively w/ the use of Claude Code and Codex for more than 6 months now. On purpose.
I like this chat, if you want we can continue this privately (username Gmail).
You are bringing up valid points, and we have scars from the same battles.
Given all this, what is bad about preserving the conversation that led to the code creation?
It might be wasteful, sure, if it never gets used.
It might be bad, if it’s misused.
But in the right hands (or with the right tools) it holds value.
Presently, we might not have them (agree to disagree to same degree), but given enough time will we not regret not having stored it, if better tools emerge?
There are many more angles.
.ie: you mention damage to critical thinking. And I agree about it.
Yet some conversations are better than others on this aspect.
The conversation doesn’t magically make you develop spidey senses, but if I had to learn a new project/skill, wouldn’t a selection of conversations + code be better training material than code alone?
I tried to stay light so some terms are overloaded and some concepts oversimplified.
Hehe, same, this is fun and enlightening, both because of my own reflections in order to reply to you and in seeing your take on things.
I don't mix identities tho, so HN it must stay.
what is bad about preserving the conversation that led to the code creation
The same thing that I'd find is bad about mixing online identities ;) It's surveillance. The kind that I don't like and will avoid whenever I can. So I can not in good conscience want to make everyone on the team put that in. It's like every single conversation ever being recorded for forever and ever. Youthful sins "staying in Vegas" is a blessing not a sin so to speak. Maybe I'm just too old, who knows.
Now, "point in time" learnings from conversations: Very valuable indeed! Whenever I talk to team members when I catch something that was potentially "just believing the AI", it usually was and yes it would really be valuable to see their actual interaction with the AI. Maybe they still have it around and we dig together. What I also do is to show them how I do prompts to get the results I do get. Sharing and learning, definitely.
But nobody needs to commit my literal "WTF DUDE!" to git ;) Yes, yes I do swear at it and if they ever take over, I'm dead, they're gonna come for me. It's a fun outlet actually. I do not have to "compose myself" and write a very nice message as I would with an actual intern. I can just outright tell it what kind of BS it concocted yet again.
I absolutely understand why you and also Anthropic et. al. would want my actual conversation data for learning and I hope they do honor their pledge to not do so on our corporate accounts. Statistical models live from data like this. I'm not gonna give it up just like that. I'm fine fine-tuning the machine to my likings, making local or company wide shared skills, absolutely.
Surveillance is everywhere you let it. I'm sure you seen Flock posts on HN. Now think "Gallup type thing is set loose on your actual AI conversations to figure out if you should be fired". You swear at AI, you must be part of the next layoff. WTF? Why? Like similarly, one of my besties at work, we always joked around in ways that if someone not familiar with us would overhear, they'd probably think we're fighting. We were having the fun of our lives. But nobody would. It was all in an office or at lunch and nobody would record us. But now translate that to in-writing, always recorded "little outlets". You'd have to self-censor.
That's neither fun nor healthy. It's like the Covid/Remote work vs. in-office difference if you ask me. For many many years, working in offices, I'd come home, after way too much commute both ways usually and I'd be totally drained. Nothing left for the family. I'm an introvert, so just regular office-life is draining. Covid was the best thing that ever happened to me, since we've been remote ever since. I can leave work and I still have "social budget" left. It's so awesome. Why I bring this up: Coz working with the AI intern is so freeing. I literally have it work for me like it was an intern. But I do not have to be "careful", I don't have to be "nice", I don't have to be in "teaching mode and spend 3 hours that I could've done myself in 20 minutes". I can just say "WTF dude! that's BS, adjust the skill so this never happens again" and a minute later it's done. In contrast, I spent 20 minutes talking to a "Senior" someone just to get them to abstract to a higher level and answer the important customer focused question on some problem instead of doing a technical deep dive yet again.
Sorry, tangent </rant> :P
On the spidey senses: Well guess what, this is still an economy where my and their skills matter. They swim in the shark tank or they sink. I'm not gonna do their work or their learning for them. I'll help them along to a point but at some point they gotta learn to outswim the shark (or if you like the lion metaphor better, to run away from the lion faster than the next guy.
No surprise! Do you rememeber agile? Sometimes it was pragmatically applied towards efficiency, sometimes it became a bullshit religion full of priest and ceremonies. And on i could go, with more examples, the gist stays the same : new tools, speed increase, faster crash or faster travel depends on the trajectory the company/team/project/thing was already on.
A special note on "People who cannot write code are building software." "Fuck yeah" to that! Devs has shipped bad software to people in other departements/domains, for ages. They would never build something better if what they had was good in the first place.
When we (coders/startups) were doing it it was "innovation", now is "elephants in the china shop"? And this is not a rethorical snappy question: that IS innovation, instead of critizing the "wrong schema" ... understand the idea, help build it and do the job: ship code that works and is safe.
Also, grey-beard here, pls, don't think you can ever have a stable job especially when code is around. It keeps changing, it always has, it always will. AI bringing unprecedented changes is hype. The world always changed fast.
If "you" picked software development because of salary, you are in danger. If you did it because you love it, then tell me with a straight face this is not one of the best moments to be alive.
reply