chapters transcript notes
click any line to jump to that moment in the video
0:15 Hello everyone and welcome back to the Agentic Thinking podcast with Mike and Matthias. We're jumping back in. Matthias, you were gone for a week, welcome back. I'm glad you get made it back safely and you're . You are in the hottest week 0:25 ever where you live out in the UK. Absolutely. Yes, hopefully [laughter] I don't show, but it's pretty pretty wild here. Insane heatwave, 0:38 which is it's a bit boring talking to about the weather too much, but just 2 weeks ago we we had our heating on and we don't know how to cool down. 0:48 Yeah, [laughter] it's wild. Bad time to have a data center in the UK . You're going to have a hard time keeping those suckers cool. Indeed. Very true. But yeah, I've taken a break 1:01 last week, still catching up with everything. What's new in AI world from your point of view? Yeah, let's let's get into some news. we're going to talk about just some general things. I think 1:12 I want to talk a couple areas around, skills, some agent automation. Matthias, you've been running a monolithic session with some agents here recently. I want to definitely unpack what you're 1:23 doing and what you're discovering and how you're finding things. I'm finding a couple things inside the semantic model space that I'm I would to bring up as . in general, 1:34 let's talk let's maybe start with your news, Matthias. You you were gone for the week, but you didn't really stop working and Or did I? agents or agents 1:44 or did you? I don't know. Tell us the story here. What's going on? What what were you doing while you were traveling? . . yes, I'm super excited about this. This this is a 1:55 dream that's come true after months and months of heavy intense work to get to that point. , I've I've just been 2:05 away on vacation, and it's been a vacation, but I've also come back to having produced some really really substantial piece of software through 2:16 agents. much that one I've had probably the longest ever running 2:28 agent session. it was live and ongoing throughout 10 days, the entire time I was gone. Sure. and I just reviewed it on from my phone 2:41 occasionally and pretty much gave it a daily steer, you know, some inputs. Sometimes I had to tell it off a little bit, but [laughter] 2:52 I've had a couple of those days, too. the agent was insane. I pretty much gave it an initial prompt, 3:03 and the agent effectively built an agentic orchestration platform in a in an extreme 3:16 dogfooding way. Yes. the the the agent was tasked to to build this agentic platform and then to use whatever it's built to make 3:26 its its next respective iteration Wow. better and more productive. And that worked very very . It's It's almost to the point where it you 3:36 know, it was very self-reflective in the sense that it found Hm. on its own 3:46 gaps and potential to improvement. Yeah. And and it it then actively research those gaps, inspect them, and and dispatch 3:58 them to workers to fix them. , whilst I produced an original prompt with my vision, Sure. and whilst I gave 4:08 a daily steer, , with my thoughts and sometimes my , very specific directives, Yeah. 4:18 the agent ended up creating substantially more than I could ever have dreamed of. Wow. Just by having real-world experience with what it's to use this particular 4:29 platform to drive its objectives. And it's created stuff that I definitely never thought we would have, certainly not after 10 days. It's 4:41 created little tools and and CLI commands and all of that that I wouldn't even have dared to dream of cuz I I would have 4:51 considered them scope creep, but it's done it. and the the because I 5:01 very very explicitly prompted the agent to not do any work themselves, to delegate all substantial work to other agents, I was getting 5:13 along, , with a single session and staying within a million token context window. very very happy, but also there 5:24 is much to unpack still. One for me in terms of fully understanding what this thing is capable of. And secondly, I want to share this, and 5:34 I want to talk about it, and I want to do demos. , yes, I I I did I did a little post earlier today, but there's more coming. 5:44 let's go through some of the stats that you gave out in the post, cuz I think these are very phenomenal. And And again, I want to I want to give this context of these 5:54 Matthias, you're pushing on the edges, the very fringe of what can be produced with this. And I I don't I don't want to say this is maybe not our maybe not necessarily our North Star, but this is 6:05 a really important milestone of understanding. We have to be able to communicate why an agent, what is the good value that it adds here. , I think what you're exploring is 6:15 really pushing on the edges of what's possible and how it gets done. , let me let me go through the numbers here. The numbers, you said in your post here, and the post is also in the chat in case you wanted 6:25 to know about that as . 297 pull requests, incredible. 73 releases being shipped, going from version 2.1 up to 4.52, it's just 6:37 cranking out stuff, awesome. 160,000 lines of TypeScript created. you had some testing here. 85 to 6:47 86 ratio of test code to production code, test code coverage very high. And then a total of 900, this is insane, 6:57 963 GitHub issues and pull requests in just 10 days. And again, the part of this that boggles my mind, 7:07 these are impressive stats. A team of developers would need to be leveraging this to get this much work done in 10 days. In addition to this, you're on your phone 7:18 with what platform? What How are you talking back to the agents? what's what's the medium here? Is it a GitHub Copilot? Is it Which Which app are you using to communicate? . , 7:30 the the the the driving agent was Claude. One, because I love the the Claude remote feature, and the the how you can 7:40 interact with it on a phone. Way better than OpenClaw and Hermes at this point cuz it's they they've copied it verbatim. They didn't have that feature. Then they saw OpenClaw talking with you 7:50 Telegram and they said, "Oh, we're just going to mimic that make it just way better." And they did it a really good job polishing that up. Absolutely. Yeah, I was, , in the mountains having a quick look at what's going on in my chat 8:01 and just feeling good about it. Amazing. And secondly, because obviously you get a million token contacts winner, which makes a huge difference, . 8:11 That million tokens, , really changes thing compared to 260 or 400k or . but ultimately, I want 8:22 my own agent which by the way is called Hekaton to give me that same interface. , you know, using using Claude remote is is just an interim stage at that point 8:33 because I had it available, ? And it was when when this whole thing started, Hekaton didn't exist. , There's a thing , I'm I'm still using 8:43 that Claude session, but it's using Hekaton agents the whole time in the background and many of them in parallel as . And And and also the 8:53 other thing I did at some point I I I taught it to to use Slack. , I'm also using Slack to get 9:04 some more comprehensive status updates, every few hours, which is really neat because I can then I have nice history, ? And that that is retained. , it 9:15 doesn't depend on the lifetime of that particular Claude remote session. I can search through it. , yeah, but at this point Slack integration is 9:25 one way. I haven't set it up yet. , you can send messages back. Sure. Just Just giving the status outputs. for from concept, ? , this is a 9:37 project, Matias, that you're saying I need to be able to use multiple harnesses. . you're saying in here in the in here the the A Con platform that you've 9:49 designed here, it's agent orchestration and it's designed to run with many different agents Co-pilot, Codex, Cloud Code, or others. And , you're you're giving a declarative workflow. Here's what I want 9:59 to accomplish. . The agent can then pick up whatever platform it needs and use whatever harness that wants to accomplish the goal. Is that the the premise here? 10:10 The idea also is I really want to avoid the vendor lock-in. , obviously, , the Anthropic's in this world, they They want you to lock in. Yeah. 10:21 But, I don't, ? , , that's one of the drivers here, to be able to use all the goodness of all these absolute top-tier agent harnesses out there. Sure. Codex, Claude, Co-pilot, 10:35 Open Code, Pi, you name it, ? Anything that's out there, yeah. and but and obviously, , I've 10:45 worked with multiple of those harnesses a lot, , on various complex projects for many months . and 10:55 one of the limitations I've always run into is that I needed something more high-level. I needed something that sits on top of 11:05 them and is able to orchestrate those kinds of harnesses and have an oversight view. And that's precisely what A Con is. Yes. it's it it it is not trying to be yet another 11:18 harness. It is not trying to be yet another coding agent. I would be silly trying, , to to compete with that. to compete with Codex or Cloud Code. 11:28 But, what these guys need or what you need, if you want to be really productive and and want to have a solid workflow is something that 11:39 uses a a cloud code harness or a codex harness as a tool inside your workflow rather than as the only tool you're 11:49 working with and that's the big difference and Did you just Did you just build Kubernetes for harnesses? [laughter] I don't know. We'll have to see but 12:01 something which worked very very as you can tell at least from what I'm claiming here I haven't released anything yet but something which worked very for me was 12:12 I used copilot with GPT for implementation and codex with 12:22 obviously another GPT model for reviews to make sure and then obviously Claude to orchestrate the the whole thing pretty much had all three of 12:34 them in the mix making sure that I have I'm using different harnesses and in fact I asked some of the some of the jobs last week were to one do 12:46 some comparative analysis between different harnesses and different models and secondly to build the model eval feature it needed to make 12:58 that comparison into the tool itself. that's going to be released it's not just it's not just a coding agent orchestrator 13:08 or workflow tool you can also run evals with it if you want to know there's a lot of talk given that 13:20 top tier harnesses and and agents become much more expensive, ? Just look at what 13:30 happened to GitHub Copilot recently. There's a lot of talk about using alternative models and maybe using local models as opposed to 13:40 cloud models. , I wanted to I wanted to have a solid and standardized way to make assessments. 13:51 when which model it does the job compared to others and and when which particular harness works better than others. And , that's that's part of it. And , that yeah. Again, 14:05 I this is the last month of where we are have a free lunch in terms of Copilot. , I definitely asked the agent to to make heavy use of my 14:18 Copilot allowance. I'm counting the days that I've got left. I'm going to make sure I'm not any of those premium requests cuz they're never coming back. Or waste 14:30 them because they're not coming back [laughter] later. , use them up. Use them up. , yeah. That that was the idea. I'm just excited 14:40 because I wouldn't have made a bet that it would be successful and it would work out in such a short period of time. Amazing. This is the new world 14:51 that we're living in. A question came in here from LinkedIn that I do want to address here that I think is really relevant here. The question comes from how trustworthy do you find the output of agent work? And do you find spending 15:02 more time with validating and testing offsetting the time gained by the reduced development time? , , another or how effective do you find 15:12 agents validating their own work? . And I think this is the really the message of what you're trying to convey with this project particularly is 15:22 with [clears throat] a good system, a framework in place, you can ask the agent to build code, assess the code, test the code, and confirm that it's working as expected. 15:33 But, this is not done with a singular agent. I think that the mind shift for people here is to not focus solely on I'm just going to use Opus for everything. It's the idea that Opus or 15:45 ChatGPT or multiple models and or multiple agents do a better job at these different search of sorts of activities. And , today it might be reason 15:56 the system with Opus, build with Codex, create something with GPT, and then the combination of those different items you 16:06 that that's the way the world looks today, but in 6 months from , 3 months from , a month from , that story may shift. And , you need a declarative system that allows us to be 16:16 flexible. Is that a good way of maybe summarizing that? What What would you add to that? that, Matthias? , two things to say here. One is with this particular project, I was quite lucky 16:28 from a reliability point of view in the sense that the agent's task obviously was to build its own execution framework. Which means 16:40 every time a feature was ready, the agent then would use it for real work for its next iteration, ? , it it it maintained its own backlog. It con- continuously 16:52 added new items to the backlog. a new feature was shipped, a new release was made, and that new release was then immediately used to execute the next backlog items. And at that point, the 17:03 agent would have looked at whether any new problems occurred. If that if that happened, and that obviously did happen a lot, new issues were then immediately filed 17:14 and and fixed. , in this regard, first of all, , I didn't have the issue of, , first of all, I certainly did not have the issue of me having to test on behalf of 17:25 the agent because the agent did it, , themselves. , secondly, I did not have the issue of, , an agent being overly optimistic with 17:36 respect to passing certain developments because if if it had done that wrongly, it would have noticed within the next 20 minutes or ? , but I do agree, , there 17:48 definitely is a a wider, , 17:59 challenge, let's say, around what testing looks in an agentic world. and I'm not making this up. Yesterday, I sat down with my agent, , 18:10 to review some regressions we went through last week. Sure. were there were certain, , scenarios where fixes that had been 18:21 made in response to problems were forgotten about later and were undone, which really annoyed me. And , yesterday, I had a meta session about 18:31 that and we created a new behavioral, contract framework where, we said, "If you look at all 18:42 the meta testing, there there is a semantic difference between tests that, test certain internal 18:52 functions and features as opposed to tests that, , that, , verify behavior, , high-level functionality. And then, 19:02 and, , obviously, when it comes to unit tests, they can be very easily, , amended by an agent as part of their work, , when 19:13 they feel that, , they've committed code and the unit test suddenly doesn't work anymore, they would then be very much inclined to change the unit test. And that that 19:23 might introduce regressions quite easily, which is why yesterday we then designed a higher level testing harness that is 19:35 not codified, that is expressed through natural language and that is something which has to go 19:46 through a formal change process if any of those behaviors expressed in there need to be modified. And that Sorry, it's 19:57 it's quite theoretical. [laughter] Hopefully I can I can do a webinar about that at some point, but also that ultimately that was my solution and to 20:07 the question that was raised. It starts theoretical , but as we tease out the idea of the ideal state and then what happens with agents, there's this I think negotiation between the two worlds where 20:18 the theoretical becomes a bit more reality and you can build into what those systems look . , I do want to transition the topic here slightly. Want to move on a couple couple other items here that I think would be worthy 20:27 as we're going to get close to time here and I do want to wrap at the top of the hour. one other area I'd to really present here. This was entirely built with agents. congratulations, Matias. We have a 20:39 an entire website for agentic thinking. I want to introduce everyone here. We have done We're on episode 15 in only a couple weeks. We are going quick 20:49 here. There's lots of communication around agentic items and show elements. what we've done is I worked on some of the style and graphics of the website. We have agentic 21:00 thinking.show. , if you want to go look at the show, go find our episodes, our previous We have a dedicated site specifically for Agentic Thinking. We have our first 12 episodes up and on 21:11 the the show, the podcast currently. And the fun part about this was let me communicate how we built this. Sorry, you're going to say something. do 21:23 I'm super excited about this. looks great. and definitely tell people about the control K shortcut. Oh yes, I for [laughter] sure will. Is a real gem and 21:33 that's definitely something you you you want to be aware of. if you go to agenticthinking.com, you'll find or sorry, agenticthinking.show, I said it already 21:43 wrong. you'll find that there is our podcast, our episode and on the site you'll notice that there is a a little very bottom hand corner, there's 21:54 a little very small icon that says you can do a shortcut command. on this website, you can hit control K and you get a command line. The command line 22:04 shows up on the website and with that command line, you can search for any episodes. You can also change pages. You can do most things on 22:16 the website just using a command line. go to the latest episode, you can open the tweaks area where you can change the color and the style of the website. that's a a little hidden trick on the 22:26 website. Control K gets you into that command window. , let me let me just talk a little bit of briefly about what is this site it how do we build it? let me give you the workflow here. The workflow was 22:37 Matthias and I started talking about this show, we should do this. I went into Figma with my Figma MC MCP server and I gave it some general guidelines. Here's some images that I 22:47 want. I talked to my agent in VS Code and said, "Hey agent, I want you to open up the MCP server, go build me some wireframes." I have a Figma document opened up. 22:58 it built all of the graphics and artwork for a website page. And then from there I said, ", based on this information, rebuild me all the 23:08 components, all the little items, you know, the card, the the the list of shows, what a show would look ." And just generally gave it some features here. 23:18 And then it built out a whole bunch of screenshots and components for this. Again, this is I'm not building them directly in Figma. I just told my agent to go build it. Boom, it created them. 23:28 Then on a live episode of probably about 3 weeks ago, myself and Armando was talking and we said, "Let's Claude code design showed up." And this is Opus 4.7 just shows up. We get 23:39 into Claude design. Armando and I go on a show and we'd talk about, "Let's build a real website." And we said, "Hey, Claude design, go build me the Agentech Thinking 23:50 Podcast." And it built a full single-page HTML site with everything. I mean, the site you're looking at was entirely created verbatim from my 24:00 screenshots from Figma, go create this. And it made this beautiful site. And we added a couple features , "It'd be fun to do control K." , it made that feature all by itself. It used its 24:10 own control K. And then we said, "We want to search episodes. We want to refine a couple things." And the fun part I think is when you go to an episode, you'll see the entire transcript of everything Matias and I 24:20 are talking about on the episode. And when you click on the words on the -hand side, it skims to the video to the section where we said the words. 24:30 It skips ahead to the actual video segment and shows you what we said when we said it. anyways, we're very excited about this site. 24:40 It's all been built with agents, Agentech building this whole thing. And it was a little bit of steering from myself, but we have a full site. what's your initial reaction, Matias? This is the first 24:51 time you've been able to play with it and see the site. Absolutely love it. I love the fact I can change the color scheme and all of that. I haven't tested it on on mobile yet, but 25:02 I'll certainly do that once we're offline here. A year ago you wouldn't have thought that would be possible in 25:14 to be done by agents, ? and let's say let's just unpack a year ago. A year ago I would have wanted to build a static website. , this is all hosted on 25:24 GitHub pages. It's It's a GitHub pages that we have hosted. , typically I would have had to go learn how to do that, and I would have started with a template or an example, and I would have had to build 25:34 out what I want. This is highly customized. There's nothing template about this. It built everything from scratch. The style, the look, the feel, the the the how heavy code it looks, 25:45 the glitching, browsing the the episode index. Key terms and phrases and searching, all that is directly built into this tool. It just figured it all out. And 25:55 then with some general guidance, it gave me everything here. You'll even notice when you go to the I think this is funny. It's It's done such a good job of adding personality to it. When you 26:05 go to the episodes page, in the upper left-hand corner, you'll see there's a little dollar sign, and it has ls-la 26:16 episodes/ , it's it's putting commands it would be a terminal all over the site, which is just super fun and engaging to look at. Oh, yeah, ls list. , what's [laughter] what what are we doing here? , very 26:27 fun, and there's a lot of a fun little area here. it even has a little hosts page, and everything feels very appropriately and styled as if it was authentically built. Anyways, the 26:37 show.manifest.txt is available on the the the host page, and it's just fun. Anyways, we hope you enjoy this. we're currently only on the YouTube platform, 26:47 but we maybe we'll add more platforms as . we might do the full video on Spotify and push it other places. If you want to know about more things around agentic thinking and have additional 26:59 platforms you'd to see us on, let us know in the comments. Just let us know where you'd to see the podcast go. , currently we are only on YouTube. We're We're happy to go to other places if there's a need for it. 27:09 All . Just wanted to make that big announcement and show you the fun shortcut command to control K. All . Let's move on to another topic if you don't mind. If unless you have any final 27:20 comments, Matthias. , no. . Go ahead. let's jump over to our next topic here that I think is relevant. 27:30 . I have been following this for a little bit and I believe we just landed something that is, I think, impactful. I've been talking a lot recently on the 27:41 the Explicit Measures podcast, a little bit here on agentic thinking, which is we really need the Enterprise Semantics model. We've heard about ontology coming out from Microsoft. It's supposed to 27:52 help your agents get better context around how your business runs. What are the tables you have? Where does the Where does the data live and how do you interact with it? Snowflake, I maybe is spearheading 28:03 this a bit, but they have just come out with the OSI, the Open Semantic Interchange. And this is an area that I have felt Microsoft has a bit of a weakness in. We have 28:14 semantic models. They're very designed for the purpose-built items that we have. But when the semantic model gets really large, we struggle to give that data or that information to users, to end users. 28:25 And this new specification has just recently been finalized. It started being worked on, I think, earlier in the year. , it's been It's been around for a while, but they have got their first 28:35 finalization of the specification. And I'll put the links here in the as . It's on GitHub. , you can also see that it it's expanding the different partners that can use and build out this 28:46 interchange. , I'm very excited about this this system here and I just kind of wanted to get your reaction with TS. What's your thought around the open semantic model interchange? 28:58 the first thing that comes to mind is this is a little bit MCP service specification in the agentic world. only for 29:09 data artifacts and enterprise semantics. . I this idea a lot. I'm also [clears throat] thinking 29:19 here this is not very agentic in nature. However, it feels that in fact that it's written in YAML, human readable, 29:30 and a translation tool between what source are you where you coming from? How do you translate that semantics layer into something that can be ported to any other platform? I feel this is going to be very 29:41 useful for agents in general. , I feel this is maybe not a particular agentic experience the way they've written it currently, but I do feel this OSI standard 29:52 will be extremely useful for agents to be able to generalize what you what you build. And my my mental model here is when we talk about customers or date tables or how factual data relates, 30:04 I really want a system that doesn't matter whether or not I'm writing a SQL statement in a data warehouse in Fabric or I'm writing DAX statements in a semantic model. 30:14 At the end of the day, the resulting table from both systems should be the same numbers. I should be able to write a SQL statement and a DAX statement with dimensions and filters that get me the same answer. All of this is just 30:25 semantics to describe what information you want. And , I'm hoping that that this OSI standard will help us generalize our business logic into a 30:35 proper enterprise semantic model that we can throw at our agents. . I think this will be useful. . , you made me aware of OSI earlier. , 30:45 much more about it than I have. and I do, but I'm just looking at some of the published examples and what 30:55 is quite interesting AI context. you know, those are top-level properties which seem to exist at various levels inside the YAML 31:06 specification. And why wouldn't they exist, ? Noteworthy, but Point point the AI to why does this table exist? What's in here? How does this work? Yeah. Anyways, AI context 31:17 instructions, you've got AI context synonyms. is there anything else? Yeah, there we go. Fantastic. Which means this was designed and inspect, , from 31:28 from an from an AI native starting point. Which is precisely what we need nowadays. I would agree with that one 100%. All , with that being said, we've burned through a good 30-minute episode here. I don't think 31:40 there's anything else ready to wrap and close. Final thoughts here, Matthias? I'm going to bore you with much more episodes [laughter] on Picon 31:51 later on. But we've got Friday coming up, another demo day, We do. for this Friday, what's the demo for this Friday do you feel ? we are 32:02 going to drill into external skills, particularly for semantic modeling. what , how to find them, how 32:12 to bring them into your project, and what advantages you're getting from them. I love that. That's going to be a great demo. All , with that being said, thank you much for listening to the Agentic Thinking podcast. This is our talking episode about news, things 32:23 that are happening, what we're building . Just to give you some context on how we're pushing the edges of what AI and Agentic things are doing. Go check out the Agentic Thinking show. I'd love you to go check out the website 32:34 agenticthinking.show. Go check it out. See what you think. go play with the website. Go list listen to some older episodes. Let us know what you think. Let us know in the comments if you the site and if you have 32:44 any suggestions. That being said, thank you Matias. Thank you everyone for listening and we'll see you next time.