Skip to content
AIMUG
RSVP
← All talks

Mon, Oct 5, 202620:58Coding agentsAgents

The coding factory: canonical specs and sequential agents

Share this talkLinkedInX

Jake Cukjati

Jake Cukjati shows how specs, not prompts, let agents code for 23 hours straight.

Watch on YouTubeRecorded at AIMUG, Oct 5, 2026

Takeaways

  • You can get agents to work for hours, with a maximum run time of 23.8 hours.
  • Prompting into code is not good, but specs to actual work is the effective method.
  • Agents work sequentially one after the other with handoffs to communicate with each other.
  • The key part is like one agent at a time during implementation.
  • After review, we generate reference materials that feeds into the application so agents learn.

Resources

SpeakerJake CukjatiNomi HealthAll talks →

Transcript

212 lines

Captions generated automatically from the recording. They may contain errors. Suggest a fix

0:00Introduction to AI Agents

  1. 0:04Hey guys, so my name is Jake Cukjati.
  2. 0:07I've been pretty much interested in AI agents and Claude Code for the better part of a year.
  3. 0:15It's been a great discovery, and yeah, I mean, I'm just amazed by what you can do with it.
  4. 0:22So today I'm going to be talking about my, like, journey from Just taking agents and coding to creating workflows, and how… I, you know, get a coding factory going.
  5. 0:38So, if you're initially, like, starting out, maybe you're not getting alignment with your coding agents, and you're just telling them, go and create some code, you know, go do XYZ, whatever it may be.
  6. 0:54But, if you're not getting your agents to run for hours, and you're just sitting idle for, like, 30 minutes at a time.
  7. 1:03maybe you want to consider how you're actually, you know, performing work with your agents.
  8. 1:08How can you get them to go, you know, much longer, right?
  9. 1:12You don't want to be a bottleneck, and that's what I was trying to attempt to do.

1:18Specs Over Prompts

  1. 1:18So, are you, like, still prompting your agents to code?
  2. 1:22Because I just put them in a slot, and you just go after it.
  3. 1:26I can get agents to work for hours, and the most I've gotten is about one day, 20… 23.8 hours.
  4. 1:37Not quite.
  5. 1:38Okay.
  6. 1:39And you can see like here I have like over 112 agents working for 16 hours.
  7. 1:46Producing 60,000 lines of code. The previous one was 110.
  8. 1:53So, coding for prompting to code is not good, but specs to actual works.
  9. 2:00work.
  10. 2:01That's what you want to do.
  11. 2:03That's how I found a good medium.
  12. 2:07getting agents to do everything that I want to do.
  13. 2:11So I will spend a long amount of time making out specs.
  14. 2:16And then from the specs, I can just be out to work 40 specs, 20 specs, whatever amount of specs I'm gonna give the agents, and they'll just work all night long.
  15. 2:28And the good thing about the specs is it's canonical, so you have something to actually verify.
  16. 2:36I can't stress that enough. Like every single time that there's a bug, the agents figure out how it should work, just because it's like, oh, there's… there's the specs.
  17. 2:47Like, that's… that's what I'm gonna use to make it actually work.

2:52Dynamic Workflows

  1. 2:52So maybe you remember a couple months ago I said hey I have this open source project for CTL.
  2. 3:01That's pretty much obsolete now, because Claude Code came out with dynamic workflows a couple weeks later.
  3. 3:10And that's what I've been using ever since it's come out.
  4. 3:16You can replace that, like Agent SDK, or if you use Pi, or any other harness to have those notes.
  5. 3:23Same thing applies here.
  6. 3:25We're just calling the, TLIs from, like, shelters.
  7. 3:30So… It's not just… Like a school line, but… We're not just producing like one thing at a time.
  8. 3:41Or, you know, multiple cars in that case, when you think about code.
  9. 3:46this.
  10. 3:47It's, you know, it's a Kodi factory, and it… Works online badges.
  11. 3:53Or, you know, like, you can only have one worker at a time. I think… the mythical.
  12. 3:59Mammoth as a huge role.
  13. 4:03You can't, I don't use multiple agents working at once to get a job done.
  14. 4:09They work sequentially one after the other with handoffs. And that's how they do it.
  15. 4:15communicate with each other.

4:16The Coding Factory

  1. 4:16you so… There are multiple steps here.
  2. 4:21We have to start out with a great number of specs.
  3. 4:27gonna implement.
  4. 4:28And then we go.
  5. 4:29champion.
  6. 4:29meditation.
  7. 4:31After implementation, there's integration testing.
  8. 4:34Code red.
  9. 4:34use.
  10. 4:35huge And then, triage pretty much to, continue.
  11. 4:41With, Yeah, what have we learned from this text? What's loose? What wasn't fixed?
  12. 4:49What's, still open for our continued work?
  13. 4:53And then, you have to compound on top of that.
  14. 4:56So, I do want to stress that you spend a lot of time on code review, and with, evaluating my agents.
  15. 5:05Where I will study their prompts, if they are performing at the efficiency that I want them to, if they're sending too many tokens, too much context window size.
  16. 5:18I wanted to prevent all that.
  17. 5:21So the first, like, plugin that I have is Spectacular, where we can dish out our specs.

5:25Tools and Plugins

  1. 5:28There are multiple types of specs that, that it had taken control over.
  2. 5:34And basically, it claims how the application will be run, right?
  3. 5:41The great thing about this fence, like I said before, is it's all verifiable.
  4. 5:47And then I put this to Forge, which is the workflow, where the agents take it from the spec.
  5. 5:55To, planning the implementation.
  6. 5:58Where there's, implementation references, and that's a huge thing that gets built during the compounding, just before we start off with the next, spectrum.
  7. 6:10But yeah, the agents go through and test everything, implement it.
  8. 6:16Testing is only, like, concerning.
  9. 6:19With, unit testing here.
  10. 6:23The key part is like one agent at a time.
  11. 6:28And then, after we implement, I use Rigor, which is a skill plugin for.
  12. 6:35Test harnesses.
  13. 6:38So, the agents will verify that the specs are done, they'll have a checklist of what they want to implement for integration testing, and, integrate… yeah, do end-to-end testing that way.
  14. 6:54This works real well for back-end services.
  15. 6:58I'm not really good with UI, but yeah, back end, it's… It'll get you where you want to go.
  16. 7:06And like I said before, they have the specs.
  17. 7:09So if nothing fits the spec, if there's a bug, they'll catch it.
  18. 7:15So that's awesome.
  19. 7:16So I'm always in complete control because I just have to evaluate the specs themselves.
  20. 7:23This isn't running your agent on an infinite browser or anything like that.
  21. 7:30So yeah, I haven't spent a lot of time with code review.
  22. 7:36I want to make sure that the quality of the code, the intent, is good.
  23. 7:41You know, code architecture, infrastructure architecture.
  24. 7:45That's aligned with what I want to achieve.
  25. 7:48And then, I always like to ask questions.
  26. 7:51And I think… Findings in 18 of 18, that's just some random number that my agent found from a previous test run.
  27. 8:01So, I'm glad I pulled up there.
  28. 8:06Yeah, so compounding.

8:07Compounding and Triage

  1. 8:07So after review, we generate reference materials that goes and feeds into.
  2. 8:14the.
  3. 8:14application.
  4. 8:16That's pretty much how the agents learn.
  5. 8:19That's how I learn how the agents learn, and that's how they figure out.
  6. 8:27How the system works with these reference materials.
  7. 8:32What's the patterns organized twice?
  8. 8:36We want to solidify it.
  9. 8:38We want to keep on using those good patterns, and then the agents will maintain all that.
  10. 8:47So triage, just every single time I run a spec, my agents will create two items.
  11. 8:54If there's any, remaining.
  12. 8:56Like, to do work that we didn't cover, that's not covered in the specs themselves.
  13. 9:02So anybody a Warhammer 40k fan?
  14. 9:08So I run everything based off Warhammer 40K.
  15. 9:13And I actually use Jeff, which Colin, you gave me that inspiration. That's my Jeff.
  16. 9:21Product manager… It goes out and, Puts priority on all the tickets, and puts lockers as well.
  17. 9:35So yeah, this is, pretty much my tarot harness, my Agentic wheel.
  18. 9:40Creates the entire circle. So, every single run, it gets a little bit better.
  19. 9:46It's, yeah, it's pretty… It's hard to get started, but after a large amount of work.
  20. 9:54You can… You can start being real successful.
  21. 10:04Yeah, and the good thing, like, every… Every run, the prompts get way better, right?

10:14CLI and Context

  1. 10:14And another thing, maybe this will help Jeff, you were mentioning.
  2. 10:19you.
  3. 10:20Evals?
  4. 10:21Well, one thing I've noticed is if I create a CLI for my agents, and the CLI is the prompt engine.
  5. 10:29And it has a CAD machine in the COR. It's really, it's really helpful.
  6. 10:36That's how I can get the right context to the agent at the right time.
  7. 10:41and.
  8. 10:41The COI just says, "Hey, come back."
  9. 10:44You need to call me again.
  10. 10:46I think… This is another skill that I have that's really helpful.
  11. 10:55That's… that's pretty much it. But yeah, I just, I want to show this, and… Yeah, keep your sex as a canonical truth.
  12. 11:07Major sales into CLIs, and then compound everything.
  13. 11:14I'm gonna open source my, skills.
  14. 11:17Soon.
  15. 11:19Not yet, but once I.
  16. 11:23Get rigor working the way I want it to. Yeah.

11:30Q&A and Scaling

  1. 11:30What questions do we have for Jay?
  2. 11:33You're only doing one feature at a time instead of a bunch of spreaders. Yep.
  3. 11:39One, one inch at a time.
  4. 11:41using handouts sequentially.
  5. 11:44Or, JSON.
  6. 11:47Whatever the output.
  7. 11:48Started or start pressing back.
  8. 11:52Yeah, when I started with, 4CTL, because… And Profit said they weren't gonna allow you to use Agent SDK, for the subscription plan.
  9. 12:04I don't know if you remember that, but they did warn that they were going to prevent that from happening.
  10. 12:09Like, I knew at that time I could automate it.
  11. 12:13And the, you know, either calling the COI.
  12. 12:19Or calling Agent SDK was, like, the best way to do that.
  13. 12:24But they added dynamic workflows, and I was like, this is great.
  14. 12:28So, yeah, the first iteration for CTO, just had one agent, and the agent would run to, like, a million, you know, it would hit its contact, context window, compact, and keep on running.
  15. 12:44It, you know, so there wasn't a lot of efficiency there.
  16. 12:48So, like, if I show this.
  17. 12:53Here I have like all the agents. You can see like the The most an agent ever goes on this context window is about 330.
  18. 13:03That's, like, the highest context window, but I'm trying to optimize this as much as possible, so I have another agent that, you know, evaluates this.
  19. 13:13And we try to, converge on, like, 250… As our, our soft limit for contracts with those.
  20. 13:26Any other questions?
  21. 13:28Right.
  22. 13:29Julian on thrift.
  23. 13:32Make an entire set of database tables for me.
  24. 13:36It's changed the part so much. What would get rid of those same things?
  25. 13:40What would a great fabric?
  26. 13:42What would keep it from getting?
  27. 13:45Bill was slow.
  28. 13:48So yeah, nothing gets implemented until you run the coding engine.
  29. 13:54To run the coding engine, you need specs.
  30. 13:57And to create specs, You evaluate it with the agent.
  31. 14:02So, if the agent's like, hey, this is a gap.
  32. 14:06we need to fix this, then you're like, yeah, fix that, get rid of those cables, we don't need them anymore, throw them away. It's old stuff.
  33. 14:15And the best part is, like, while you're planning it, you didn't code a thing.
  34. 14:20Not one thing was coded.
  35. 14:23I mean, if you're… You know, doing, like, long number of sessions, like.
  36. 14:29You know, like, 40… Specs at once. Like, I can run for, like, 10, 15 sessions.
  37. 14:39Which is about, you know, like, 20… 20 hours of work.
  38. 14:45Zoom.
  39. 14:46change a spec, or would you create a new spec that seeds it, and then you seed it built in between?
  40. 14:52So… Yeah, as the… okay, so… There's a couple things on that.
  41. 14:58One is… The… Monorepos.
  42. 15:06And how your codebase changes over time, and how you handle domains, right?
  43. 15:12So, domain-driven design is very important.
  44. 15:16I don't have that problem right now, but I knew I'm gonna… hit it very soon, because my… the Copic side run is approaching 200,000 lines.
  45. 15:29And… I need more domains.
  46. 15:33So I have, like, 80 or 90 text files.
  47. 15:37And each one… So, each spec contains a topic of concern, which sets it apart from all the other specifications.
  48. 15:48And if you think about it like entry points from like, these are the entry points of this application.
  49. 15:56Each entry point is spent.
  50. 15:59can… Like, top down, it's basically like, you know, code, how it maps, and there's references, and.
  51. 16:07All that, right.
  52. 16:10So, at a certain time, the amount of context that an agent receives could be exhausting to itself.
  53. 16:19So… Building a new domain, you can separate your code.
  54. 16:25you can separate your specs and isolate it to that one new domain, right?
  55. 16:32So, like, for instance, I'm gonna have to split my, codebase to 3 or 4 domains, right? So if I do that equally, that's about 20 specs.
  56. 16:4320 specs, the agents can easily handle.
  57. 16:45Each spec is less than 1,000.
  58. 16:49Normally, it's around, like, 400 or 500 lines.
  59. 16:52So you have to be heavily involved.
  60. 16:55Texture. Oh, yeah.
  61. 16:57Yes.
  62. 16:58Yeah, everything gets architectured. I have… Skills for… design HTML files to show architecture, design, because we're going at such a fast velocity.
  63. 17:14Another thing is, like.
  64. 17:17computation efficiency.
  65. 17:19So I do a lot of, like, data pipelines and derived data, like, you know, for data analytics.
  66. 17:29So, Ronnie skewed as much as I can with… The limit of RAM that I have is what I try to maximize.
  67. 17:41Yeah.
  68. 17:44Other questions?
  69. 17:47Yeah, I'll ask a final question.
  70. 17:49So you mentioned with your specs that you make specs canonical.
  71. 17:54So as your different agents are improving the architecture, and they I assume updated a root spec.
  72. 18:03What's the mechanism that that triggers that canonical reference down to the existing code?
  73. 18:09Sure.
  74. 18:46Let's see…
  75. 18:53Maybe… let's do this one.
  76. 19:09Okay, and… Let's do…
  77. 19:20I think it's this one. Yeah, okay.
  78. 19:26So to keep my specs in line, I use a file. It's called a manifest.
  79. 19:35What that file is, is it gets generated based off of this one commit.
  80. 19:41Which contains all these specification files.
  81. 19:45These are… it's just the depth of all the specs themselves.
  82. 19:51It uses this one file to outline all those specs.
  83. 19:56Right.
  84. 19:57At the very top, it says what the commit hash is, and the agents will use this commit hash with this command, and then… to identify, like.
  85. 20:12All the specifications for that batch.
  86. 20:15This gets split up into multiple batches by a scheduler.
  87. 20:20Which.
  88. 20:24Should be able to show right here, implemented badges.
  89. 20:30And each one of these contains the hash of the canonical commit, the, the files that it uses, what it's blocked by, such and so forth.

Same night

15:01Corvic AI: graph analytics and dependency tracingJames Coffey · Graphs16:20Grokbot and Jev, a decision modelJoseph Fluckiger · AgentsTranscript14:25Harbor evals for coding agentsJeff Linwood · Evals25:21Mixture of models and semantic routingColin McNamara · Routing

Related talks

23:28Catallaxy: A Council of AI Personas Watching Your Coding AgentMax Marty · Coding agents19:49Under the Hood of a Coding Agent: The Loop, Tools and ReasoningJeff Linwood · Coding agents