Train Your Tenacity

I saw something scary recently: AI can confidently tell me, repeatedly, for days, to give up.

I spent five days reproducing a bug in the Mermaid graphing library, an hour or two here and there on my own time. Unusually, I can't think of a single step Claude took that actually helped reproduce the bug. It kept suggesting the same half dozen css reset tricks.

I have a lot of experience so I ignored Claude and tried simple steps like "does this work in the mermaid live editor?" and "I should simplify the setup to a standalone HTML file". Why Claude didn't think of a simpler reproducible case I don't know. It reminds me of how it adds new functions rather than mutate existing functions; it's like your relationship with simplicity is different when you are a superintelligence that can recall all of Wikipedia.

I haven't seen this kind of failure in simple cases from Claude many times, but it was on display as I spent an hour or two understanding the issue, every day, for five days. Every day Claude suggested the same fixes. Every day Claude suggested that I give up.

Tenacity is a Learned Trait

It turned out that the issue is very particular about page setup; you have to load a page while not using the default level of Safari's zoom. It took days to find a reproducible sequence, but once understood it's pretty easy to reproduce.

Claude, though, was behaving like it was discouraged and embarrassed of its failure to understand. I kept thinking about how this experience would be terrible for a more junior person because Claude was actively encouraging me to give up.

I stuck with the problem for days, because I've worked on long, hard problems many times before this sort of AI showed up. I remember there was no patience for bullshit on the BSD mailing lists, people could give hints or not but you always ended up hearing that the answer is in the source code, and you just need to accept that being Good at *nix administration meant putting in the effort, investigating your own issues, understanding how the tools all work. I've struggled.

Tenacity is an Earned Behaviour

I have 20 years of experience that told me that Claude wasn't helping me out, it was misleading to believe its assertions. I am sure that didn't have those instincts when I was in the first few years of my career.

I worry that our juniors are leaning on these tools. I worry that people are assuming the AI knows better than them, that they trust its output more than is warranted. I worry as I'm seeing people wasting the time of professional developers. You learn how to be strong by struggling, not by being told every problem should either be solved by one tool or is impossible.

I also worry that I sound old, that I don't have the mental acuity of Kent Beck seeing into the future, evaluating the past. I sound like Steinbeck noting that the problem was in part that the current generation of farmers did not love the land the way the old generation did. I'm looking at maladies of the soul, but I know the tractor's not going anywhere, that it's in my interest to use the tools. I just miss being surrounded by people who loved the dirt.

I remember seeing a junior who leaned on chatgpt to write Helm; we use Helm via Skaffold, and Claude is not great at undestanding what to do in Skaffold when when a helm chart doesn't expose a particular configuration knob to twiddle. Our junior seemed uncomfortable when when I told him the problem he needed to be be okay with being uncomfortable while he learned the tools. He didn't understand yet that being a Software Engineer is about interacting with an ecosystem as well as a programming language.

I've had the frustration of asking a question online and being told to go away and do more research. I've felt how hard it is to not know, and to be sure there was an answer and that the people who I thought knew the answer wouldn't tell me.

I'm a team lead. I'm building a consultancy, but I'm mostly a team lead right now at a normal company. If you're a lead, please keep an eye on this, please look out for your juniors and help them to work through AI's shortcomings. The tools will be better, they'll continue improving, but that won't sharpen anyone's person skills.

If you're a lead worried about this, I'd be happy to talk. I'm building a consultancy around exactly these problems: how teams learn, how they debug, how they stay sharp. Download my free Documentation e-book if you'd like to see how I think about things. Reach out, let's talk.