git-dupe: A testament to the usefulness of agentic development

I made git-dupe to solve the problem where I have notes, skills, and agent instructions that are part of my personal workflow, so I do not want them committed to the public repo, but at the same time, I want them to be in the project at the locations that tools expect them to be at (e.g., AGENTS.md file locations matter on agent platforms). These files cannot be simply gitignored because I want versioning for them.

The solution that I came up with is versioning those agent files in another repo, say <private-repo> (this repo is private to me and holds the history for my agent files). The project code itself is open to the public in <public-repo>. I overlap the two repos to solve the problem (e.g., from the POV of someone working in the workspace, the agent files AND public files are BOTH there).

You can find the git-dupe repo here (public code, MIT License). The repo has the authoritative specs for the project and goes VERY deep on the technical side of things, if you are interested. This post will remain laid-back and will simply be me sharing my experiences and thoughts on git-dupe as the creator.

It’s all fun and games until you actually sit down and try and plan to implement a coherent version of the idea. There are many promising paths that you can take when starting off - many of them being landmines in disguise, and it took me many tries to realize git-dupe in its current form. I am not going to pretend that I somehow figured out the current CLI surface on my first try - it is the culmination of trying many different ideas and converging onto what it is now (and I am quite satisfied with where it is now).

I already knew, right off the bat, that I wanted a CLI surface similar to plain Git so that I didn’t have to spend time learning new vocabulary. Git was also already a proven product, and I did not want the majority of my efforts to be in reinventing the wheel from scratch. So that set the tone - make something somewhat similar to Git, and leverage the fact that Git already exists.

My very first attempt with this (this was my original personal workflow which I used but did not intend for anyone else to use) involved a required project folder structure (which ultimately ended up being very annoying). Essentially, the idea was I would have project/{private-dir/, public-dir/}, where private-dir/ and public-dir/ both were their own Git repos, for agent and public files, respectively. My idea was that work is always done in public-dir/, so agent files AND public files were both in public-dir/, and when a user does a versioning command like git dupe add, a filesystem move (e.g., on Linux this would be mv) would be used to first move agent files to private-dir/, then a normal git add would be done in public-dir/ on the public files that remain there, and then a normal git add would be done in private-dir/ on the agent files that are now there, and then finally, another filesystem mv would bring the agent files back into public-dir/ so that work can continue normally in the public-dir/, as if nothing ever happened. In essence, the only time private-dir/ is used is when a versioning operation is used, and outside of that, ALL files reside in public-dir/. This strategy worked because filesystem mv is nearly instant and allowed for normal Git calls in the corresponding directories so that not much wrapping had to be done. This strategy actually worked pretty well, but it was a pain to develop, and was inelegant because of some non-trivial edge cases and the fact that filesystem mv was being used as a hack. It also presented many difficult design decisions like “when should git dupe override native Git?” (for example, in my first version git dupe add functioned as a general add operation; git add essentially became useless if you were using git dupe; I also made git dupe status combine the output from the native Git from both repos). In summary, this version did its job, which is why I actually used it myself, but it was not coherent or elegant.

The first version served me as a personal tool for a long time. After a while (and after some conversations with friends of mine who are also involved in software development), I realized that there is actually a gap in development tooling for this, so I decided to try to make this a comprehensive product and make it available to the public. Of course, this resulted in me going through a few more iterations, sometimes with radically different ideas. These included keeping the actual files outside of the worktree and keeping symlinked versions of those files in the worktree instead, wrapping the whole entire Git process so that an operation always has a corresponding private operation, and finally, the idea that we are at today: separating native Git and git-dupe operations so that we can take advantage of Git primitives while guarding the few operations it actually makes sense to guard. This final checkpoint was what reduced a possibly gigantic project (300k+ LoC) to a medium-sized project (~60k LoC). Yes, 60k LoC still sounds pretty big, but when you consider that git-dupe is heavily correctness-oriented (~45k LoC for tests focusing on high-leverage paths in a demanding spec sheet), and the fact that it gives meaningful next-action steps to pretty much all practical scenarios that will be encountered, all while overriding the original native Git hints that would clash with it, 60k LoC is, in my opinion, justified.

This was a BIG project - perhaps the biggest one I’ve tackled to date. 60k LoC is by no means massive, but the weight of each line of code in git-dupe makes that figure understated. I would NOT have been able to carry this project out without the help of agents. Researching Git documentation on what native Git surface to use to ensure compatibility for all minor versions of Git from 2.43 to 3.0 (i.e., pinning down the surface that has historically been stable and will, in the foreseeable future, not likely have any breaking changes), writing hundreds of strong tests against the claims in the product specification to ensure that the promises hold, refactoring and boilerplate code, documentation and fact checking, behavior probing, researching which parts are compatible with macOS and maintaining the codebase so that it is easy to add support for macOS (yes, I had macOS support in mind from the very beginning), these things would’ve drowned me if I was tackling this project by myself; either my quality standards would not have been reached, or I would have spent years of my life bringing it to where it is now manually. Needless to say, if I was not involved in agentic development, I would never have run into the use case for git-dupe in the first place and git-dupe would not exist.