Mountains

Code is Sisyphean

Reflections on LLM usage

  • LLMs
  • I suppose this was inevitable.

    In the previous posts I made about LLMs, which I’ve condensed down to just three posts with audience focus, I spoke stridently and with great vigor against the usage of LLMs at all, essentially teetotalling.

    I’ve re-read it, and I still stand by everything I wrote, which is why it remains up. But I realize I’ve been a bit one-sided. In my programming practice I try to hold pragma over dogma, and on further reflection I came to realize that I was being a bit dogmatic in my resistance.

    I am definitely not an “AI drunkard”, but I would say that I am less a teetotaller now and more a “having a drink in moderation is probably ok.” I still have ethical misgivings about how we’ve introduced this technology, particularly the absurd greed-driven fervor that is spawning data-centers with little regard to the communities and various ecosystems. I still think that this technology should not belong in the hands of private companies, nor do I trust any of them to be responsible about its usage.

    I still stand by every comment I made about the implicit harm it can induce on the learning process of a budding junior in their journey to becoming a competent senior.

    HOWEVER.

    In February or March, around the time that Anthropic released Claude 4.6, I tried out using it with a coworker while we were pairing. Previously I had experimented a little bit with Opus 4.3 to mild success. It was neat but still felt unreliable and stunted. Opus 4.6, on the other hand… well. I get it now.

    Delegating to Claude

    One of my earlier tenets of using LLMs was to “treat it as a junior, not a senior.” Typically, this means delegating work to it and then reviewing / checking that work. Here is a short list of things I’ve done where I’ve delegated work to Claude:

    In all these cases, this is work I could have done myself but was able to turn it over more quickly by delegating it.

    Bug hunting

    My coworker and I both had a general idea of where the bug lay, so we felt confident in our ability to check Claude’s conclusions. We described the normal behavior, described the bug, described approximately the behavior we were expecting to see, and listed out the files of where it could start looking. It churned for 5-10 minutes and then came back with a specific analysis and proposed fix. Surprisingly, it also found an additional bug that we didn’t know about but that was related.

    Realistically, this probably would have taken us about an hour to do and I’m confident we would have figured it out. So there was a substantial time-saving there. The bug was also small enough, conceptually, that we could individually integrate it into our understanding of the application’s function.

    Data combining and permutating

    There is a tabletop board game I’ve been working on for a while. It involves a lot of traits/qualities that are permutated / combined in various ways, and it had a few holes. I gave Claude my seed list of what I had already, described the resultset I was aiming for, and then told it to fill out the list using that scheme.

    This case took 15-20 minutes, but it searched the Internet to research more terms / concepts related to the dataset and came back with some very informed opinions (with actual URLs for support) about how to structure this data. I recorded the findings to a static file for reference, then had it apply this to the dataset.

    The end result was very elaborate. This would have probably taken me considerably longer to generate this, and honestly I’m not sure I would have thought to look for some of the approaches it took. This was a very impressive win.

    Data-diving

    Normally for this kind of task I would use a tool like ripgrep (or just grep), and some regex, maybe. But sometimes the needle I’m looking for isn’t a simple string.

    There have been a few cases where I’ve given it tasks like “Find me every test in the test suite that uses an order with two or more items”, or “Look through my YML files and find me every top-level record key that has mixed-case” (that one could be regexed but I wanted to see if it could do it).

    I’ve also used it to do textual summaries. There are a few books I own and have read already, but also have the PDFs. By using the pdf skill, I can have Claude convert a PDF to text format, then have it create a summary index of the book, and then ask it questions about things I don’t remember and can’t find in the book’s physical index.

    I actually ended up doing this with several books I owned and creating expert skills for both programming and also game design – I am aware that the LLMs have been trained on all of these books already, but that ocean of knowledge is so vast the individual perspectives have been diluted. By concentrating a skill onto 5-7 books specifically, I can get concrete reference examples when asking for opinions.

    Two Exceptions

    In two cases, I have treated it as a subject-matter-expert:

    I have very little prior mobile development experience, so I’m breaking away from the original tenet in this case. I can however, at least read the generated code and leverage my experience in a way that an Engineering Manager might – knowing what questions to ask, how to phrase them, and what things to look for.

    I find it a little easier to learn new things if I can see examples that are familiar material so it’s more of a diagonal move upwards instead of an orthogonal one. As the apps have taken shape, I’ve been able to use this to build some competency about how an iOS or Unity app is structured, where files are located, and the general flow of working these apps.

    Ideating with Claude

    A coworker showed me how to use the /plan feature and I’ve found it to be very useful.

    When coding, this has been useful for both bughunting (see above) as well proposing it enhancements or greenfield ideas. In both cases, Claude CLI is capable of seeing the full app tree and its contents.

    I find myself pushing back a lot more on this, than with the more more surgical bug-fixing mentioned previously. Particularly in the way it generates code.

    Most of the generated code I’ve seen so far has been “technically correct” (the best kind of correct, I’m told), but has always felt too atomic / “contained”. By contrast, what I would like to see is code that more clearly integrates / synthesizes with the existing codebase.

    Using /plan does help to alleviate this somewhat, because I can get a better sense of what it’s proposing before it executes. But just to be safe, I always ensure I have a clean git status before beginning, so I can review the changes in detail. Sometimes I treat the Claude contribution as a spike and then go back and rewrite it myself, refactoring it to integrate better into the code, as I prefer it.

    Assisting with Claude (rather: being assisted by)

    There’s two ways I want to approach this:

    1. How I have it assist me, broadly speaking, via CLAUDE.md
    2. How I use it as an Assistive Technology, as a neurodivergent person

    CLAUDE.md

    Claude, specifically, looks for a file called CLAUDE.md. You can put one in the root of any of your applications. You can also put one in your home folder (~/.claude/CLAUDE.md).

    In both cases, it’s a document that describes, in plain english, what you’re trying to do and how you want Claude to behave.

    ~/.claude/CLAUDE.md

    I treat this one like a very broad “get to know me” directive. It describes the kinds of work I will be using Claude for, and any specifics about how I want it to help me.

    For my development work, I describe the kind of work I do (web development using HTML, CSS, and JS, as well as doing a lot of development in Ruby and Rails). I describe my coding practices and preferences in broad terms, as well as what level of specificity I would like it to have in its responses.

    For my game design work, I describe the types of games I often make, some general thoughts about my approach towards design, and also the document structure of my design documents. I have some ruby scripts I wrote up a while ago that I use to convert YML files to CSV files that I can ingest into dextrous.com.au to export cards from. Additionally it has designated areas for ideation docs, playtest logs, etc. All of this is described in the doc.

    Finally it has some general statements of the kind of attitude I want it to have (TL;DR - avoid sycophancy and be concise but with clarity).

    .../{repo}/CLAUDE.md

    In these files, typically for development projects, I will generally first summarize the project at the 10,000 foot level, then do subheadings with more specific details about implementation that it would need to know. I treat this as a sort of “onboarding” document.

    My best advice here, if you do this, is to create the file, write a summary of a few sentences about the project, and then spin up a Claude CLI session in that directory and give it this prompt:

    > Please read through @CLAUDE.md and the rest of this project and let me know what more you need me to clarify.
    

    Even better, do it in /plan mode so it can give you a Q&A and come up with all the edits it needs to make, and then it can write out the updated doc itself. (Proofread it!)

    As an Assistive Technology

    Other examples of “Assistive Technologies” would be things like screen-readers for vision impaired, subtitles for hearing impaired, curb-cuts for people in wheelchairs / mobility issues, etc.

    For me, the issues I have are cognitive. When I have challenging days, it feels like the desktop in my brain is very cluttered / has little free space. Sometimes this means it can be harder to lay out a lot of different moving parts in abstraction to synthesize the ideas together. I’ve found Claude to be very helpful with this, particularly in /plan mode – I can lay out a problem, reference files, give it additional contexts, and have it help me better visualize the whole problemspace more effectively.

    I can tell this is helpful on these days because I feel a sense of mental relief where normally it would be anxiety/frustration.

    In the past, the way I would approach this is typically through writing a lot more things down, drawing diagrams, etc. It helped, though with limitations. I can also take transcripts from calls and import those transcripts into the Claude dialogue and integrate those into the abstraction.

    Honestly, this relief alone is worth the $20 a month I’m paying.