Meeting Claude

Okay. They slapped me on the wrist at work for saying in an email thread which CC’d my boss’ boss’ boss that I hadn’t (past tense!) used AI on a project. They apparently spent all of $10 for me to have a license. I was told I came across as “political”, which I couldn’t tell if they meant in the sense of me commenting on the political dimension of AI (I wasn’t; I can be much worse) or just that I seemed insubordinate in not using it (though, again, past tense). Anyway, aren’t I supposed to be flexible and open to new approaches or whatever? Fuck it. I had my $10. Why not see how well it could perform?

I installed Claude and started it off with a dumbass math problem. It got it. I figured I was being a little harsh so, after safely committing my repository so it wouldn’t totally wreck my shit, I started asking it honest questions about implementation that I would’ve otherwise researched on Google and Stack Overflow. Very computationally expensive, I know—with the single benefit that it had the ability to contextualize my questions with the code I had written and, by further contextualizing with its own memory bank of Stack Overflow shit, could reimplement what it remembered in a way specific to my asks. Problems arose with it offering incomplete or outdated solutions which, when pressed, it was quick to correct like “Oh, there’s a more modern library for this” or “Oh, I forgot to include this snippet” or “Oh, I fucked the timing on this async function.” I don’t think this saved me much time in hindsight except that I could “code” from my “search engine”, except I knew in the back of my head how expensive it was.

I tried giving it very specific tasks the next day. I knew what I wanted to accomplish, like adding material react components to my webpage, but wanted to see if Claude could do it for me any quicker. Since it was basically just reimplementing standard boilerplate code originating from somewhere in its memory, this at least saved me from wasting time and copying/renaming/recontexualizing shit myself—albeit, again, I don’t know if it was worth the cost of computation. After it would write some kinda dogshit ad hoc solutions which I had instructed it to implement once and then copy elsewhere, I thought it would be nice if I didn’t have to refactor it. So I gave it specific instructions for how to do so: restructure objects to invert how the code refers to properties (instead of a.x and b.x, do x.a and x.b); and generalize the component code into something that can be parameterized and called in multiple places. It glazed me for coming up with that and I was like yeah, that’s right AI, I’m your smart mommy. Was weird. That was probably the most useful case: as a souped-up search/replace function with the ability to perceive and manipulate code structures rather than just literal strings or regular expressions. That’s probably why software development is the only place where generative LLM use makes sense (besides fields which to general audiences may be surprisingly similar, like biochem): it’s a bunch of formal structures with commonly understood applications and best practices.

The biggest problem I saw, though, besides the ever-obscure actual cost of computation, is how confident Claude is in doing random guesswork bullshit. It’s terrible at CSS, maybe because it can’t fucking see shit, and especially so when the changes in question involve layers of material react fuckery (it’s nice-looking, but terrible to customize and make into your own, which no one seems to be interested in doing, hence being a blind spot for Claude). It’s fine for it not to know, but I wasted time prompting it over and over again to understand my ask and try more and more different approaches. (Admittedly, I didn't waste that time by mistake, although I was also curious to see how much of a pain it would be for “the user”, that is, myself.)

Again. It’s very “useful” as a fancy refactoring engine if you tell it exactly what you want, or in general as a search/replace or mass copy-paste tool, but what time it saves doesn’t feel like it’s worth the cost (especially not once the AI firms stop eating it on our behalf). It’d be neat if that tool existed without being hooked up into such terrible infrastructure, and I suspect it might be possible if an application understood the language and libraries it was working with, but LLMs as they are feel like a simultaneously cheapass and overkill approach to having such a thing. Too bad.