chapters transcript notes
click any line to jump to that moment in the video
0:15 Hello everyone and welcome back to Agentic Thinking with Mike and Matias. We are here again to talk about news that has happened in the past week. Again, everything [laughter] is moving fast. There's much stuff to talk about. 0:25 We've had no shortage of new topics. Hello, Matias. Welcome back. Yes. , great to be back here. Great to see you. how are things? Things are going . I feel the I 0:37 feel there's a bit of a a shift for me. Maybe emotionally, maybe it's sentiment-wise, but , I feel in the last couple weeks, I have really felt Microsoft's 0:49 direction shift investment. I don't know what you want what you want to call it, but I really feel there's a very pro- agent and agentic experience movement 1:00 and and I feel Microsoft is making the moves in a way that is aiding or I'm able to use better agents across my workflows. I'm hearing messaging from Microsoft 1:11 even leaders. I was just reading a post from Kim Manis today how she's incorporating an agent to summarize her emails and take customer feedback and give her a summary and she observed 1:22 which I think is something I'm observing as is agents aren't changing what I work on but they're greatly shifting how I do my process of working on things. I'm shifting the process of work. and 1:33 I think what I'm observing is Microsoft is embracing agents. It feels a more first class citizen and it feels the integration between Microsoft products and agents are 1:45 becoming much tighter. It's a it's a tighter coupling. I don't know if you're observing the same thing in your daily work as . one thing I noticed is there's 1:56 much high quality content coming from GitHub and around Copilot. Sure. , you know, obviously we talked about co-pilot a lot a month ago from a from a pricing 2:07 point of view because they switched their pricing model really significantly. , , tomorrow that will have been a month. , it upset lots of people. and somehow my 2:21 observation ever since has been that they've they've put out much more really high quality content 2:33 around co-pilot and around agentic experiences since I I don't know whether that's coincidence or whether this is just me being biased somehow but definitely enjoying 2:44 seeing what what's coming out of cop pilot and also to be fair I'm still using it I was I walked away from it for a bit initially but 2:56 turns out whilst it's it's not a complete free-for-all anymore you still get pretty significant usage out of it even 3:09 on a relatively medium-sized subscription . Yeah, I'm I'm finding also as I'm looking at the pricing for my team internally, ? I'm we are all GitHub co-pilot 3:21 for everything we've been doing up until this point and we've been felt the pricing was fair until they changed to the Azure AI credits. we're we're paying more to use the Azure AI credits. 3:31 we're seriously considering as a as a company looking at blending a little bit of anthropic claude with what GitHub copilot is offering. 3:41 our choice is becoming between the two. How do we balance the blend of those two different kinds of agents? I'm also looking very heavily at localized models. local models and 3:51 running models locally on computers or hardware that's specifically designed to serve models is becoming very attractive to me. the payback on buying the hardware and getting decent 4:03 models to run on local machines is not too far away. anyways, we're heavily exploring that as . We'll let as we as we develop and figure out more things here additionally. All , let's jump into 4:14 some of the news announcements. Let's take off the top of the list here. chat GPT 5.6 is out and this is another premier model, but 4:24 this is maybe taking a playbook from Anthropic a little bit. Matias, what's your what's the news on this one? Yeah, 5.6 been long awaited. particularly when when Fable 4:35 was launched the community was desperate to to figure out when open AI would would follow and it's 4:45 taken them about two months end of April was the last big release 5.5 what's interesting is 56 was released as a a trifold model if you 4:57 will there's not GPT 56 as one new model. There's a group of models, Soul, Terra, and Luna. 5:09 effectively mirroring the opposite haiku setup both in terms of pricing as as in terms of capabilities that Anthropic has. definitely see who 5:21 are the, , big competitors are. I don't know. I haven't had much chance yet to play with them. 5:31 Have you? No, I haven't haven't been able to look at those directly yet. would to get my hands on those. Again, time [laughter] where do I spend my time these days? It's there's many things to be 5:42 spending time researching, figuring things out. , haven't had the ability to go test it yet, but I'm very excited about it. There's even in the article that that I put it here in the chat window as . If you want to go look 5:52 down here, there's capabilities that are available to you. And you can see again they're running things against terminal bench. they have a terminal bench graph here showing that it's very highly rated on terminal bench as . High 6:02 scores are are ranking there. and then further down they have other really useful graphs and data that supports how intelligent the model is, 6:13 how good is it at reasoning, different exploits, which how susceptible is it to exploits or not. really good model designations there 6:23 as anyways, it looks it looks it's going to be a contender. Honestly, if I look at it, it looks it's going to be a contender with some of the top models that are out there. it seems it'll add a lot of value. yeah, very useful in that regard, 6:35 ? two things to add here. One, pricing has not changed. the the pricing for Soul, the the the Frontier flavor of it is exactly the same as 5.5, 6:47 which is different from comparing Oppus 4A to Fable 5, ? because obviously there was a really steep pricing increase there. the other thing which 6:57 is from a prag pragmatic point of view is much more important it's in limited preview . My understanding is that they're still negotiating with the US government 7:08 with respect to when and how this is being fully publicly available. chances are that you won't even be able 7:18 to get your hands on it quite yet. Is this going to be the new way of the world ? is every model going to need to go through a security review through the government to who 7:28 can get access? , as these things get continue to get more and more powerful, I don't know. It's going to be interesting to see how the slowdowns will continually the release speed will 7:40 have to slow down if we're going to continually have all these reviews with agencies and governments and organizations. It starts becoming an exclusivity thing. If your country doesn't have the technology in front or 7:51 the approvals to use these things, you got to either build your own model or you don't you don't get access. This is going to be interesting to see how this materializes moving forward around 8:01 nation states and how they handle these models and distribution of models I guess would be really interesting there as . good to know. I didn't know that part about the 5.6 soul model. 8:13 Maybe let's talk a little bit more about what we see in trending. Maybe this is a bit closer to Microsoft's side. the agent era has really taken things by storm. , if we talk about 8:24 Hermes or we talk about OpenClaw, Microsoft has their own version of an agent that's out there as . I forgot the name of it. The name is just escaping me , but they have 8:35 their own agent as . All these agents another claw flavor. Yeah, something claw, ? Yeah. I I can't remember. 8:45 It was open it wasn't open claw, but it was something that. Yeah. Yeah. But they they've renamed it. They rebranded it. I And and they had their own version of an agent in the Microsoft 8:55 ecosystem. And a lot of these agents that we see are very easily deployed on Linux. Linux seems to be a a top choice for a lot of these agents to appear. And it gives you access to the 9:05 file system and it can run commands and doing more things on the hardware that's there. , you're giving it an employee. Here's a computer. 9:15 We're going to let you do things on the computer, make files and such. This feels where we're going, but that's not always secure. I don't really want 9:25 to bring in a bunch of agents on my desktop machine and have them run there. there's this whole concept of sandboxing that's coming out. And , most providers on the internet are seeming to give some 9:35 sandboxing. There's some from Versell. We have some from Cloudflare. And Microsoft, it feels they're jumping a bit more on the bandwagon here with these WSL containers. Matias, maybe 9:46 let's talk about that one a bit. What do you think about this new container experience? . Yeah. No, this is definitely something I'm very excited about. and it's only just been released. 9:58 it was announced at build, but the the blog post for it is brand new from yesterday. WSL containers. , I personally spend a lot of time in WSL. 10:10 I, , , , that's totally up my street. And what they've released is a, , is a 10:20 pre-release version of WSL, which you can opt into. , and once you once you've upgraded WSL, , you get an additional CLI WSLC.exe, Exi which 10:33 allows you to run very lightweight containers which directly competes with heavy weights Docker and Potman. 10:46 the idea is you can run very lightweight containers on a Windows system the infrastructure 10:58 on top of the WSL infrastructure without having to install some some big third-party app that provides the 11:10 container runtime. and why is that relevant? , you talked about security and isolation 11:21 already. That's precisely why, , in an ideal world, you don't want to run your agent, , as a 11:32 potentially elevated process you know, amongst all other processes on your machine. You want to put them in a in a box, a box, ? 11:43 a sandbox or a container where you have much tighter control around file system access, network access and 11:55 where rather than having to use all sorts of complicated emulation techniques, you have a almost a physical limitation 12:07 within which that container runs, ? And that capability we've we've had this for a long time in terms of Docker Potman etc etc 12:20 but having something which is much more lightweight and directly built into WSL is really cool. Apparently, this also comes with a highly optimized 12:32 new file system, which is why you need to opt into a WSL pre-release, which substantially improves I/IO. big speed improvements. , the 12:43 other things you mentioned, sorry, I was say I really just want to comment on what you were saying here from sandboxing standpoint. , this is likely a very if you're a very heavy 12:53 non-developer, if if you're a PowerBI, if you're using fabric, if you're this whole concept of agents, sandboxing, put them on your machines, probably a bit foreign to you. I think 13:03 this is a great move to make this easier and more part of the Windows platform that you get out of the box. And I think a lot of the reason why this was difficult before was Windows didn't have 13:15 this level of support before. And I was downloading a Docker application and I was running building Docker containers and then virtualizing machines which took a lot of resources 13:25 and compute and I also liken this to be it feels a lot we're instead of giving an agent their own computer and I have a number of extra 13:35 computers on my computer around my office and I'm giving agents full computers on their own but they're much lighter weight computers and and letting the agent run on that. this doesn't this requires what if I 13:46 want three or four agents running what if I want more than one container running I feel this is a lot of the same pattern you would do when you give an employee in a computer you here's your computer here's the things 13:56 you can adjust here's the things you can edit here's that laptop and here's things you can't adjust there are certain network settings or things that are settings that are coming from corporate that are saying you can't 14:07 install these kinds of softwares you know we're only going to let you have these kinds of browsers and things that I think that's the level of control we're talking for agents as . Sorry, go ahead to your next point. 14:18 . you mentioned Azure and Versel and Cloudflare which is a an extension of this if you will, ? this is the same concept 14:29 I want to run my agent in a sandbox environment but this time I don't want to run it on my host machine I don't want to this is this is not an interactive experience but I want to 14:40 run this at scale with headless agents and with lots of asynchronous agents that run 14:50 where you have dozens or hundreds of them running in parallel. we've seen as at build as we've 15:00 seen Microsoft announced a sandbox service which builds on top of Azure container apps. 15:11 gives you a opinionated and and very highly simplified way of very quickly firing up new 15:25 containers in the cloud. very low latency with respect to startup time. they're meant to be shortlived. it 15:35 comes with an opinionated way of attaching an external file system to it etc etc. and other vendors, 15:45 Versel and Cloudflare in particular, you know, they they have similar offerings. in fact, they've had them for longer. Versel sandboxes comes 15:56 with a really nice TypeScript API by the way, Cloudflare Sandbox, very similar. this shows, , what the 16:06 what the industry and the market needs nowadays, ? Yes. , if you if you want to be able to delegate tasks to agents, you obviously need to 16:17 provide some runtime environment to your agent. If you then want to do that at scale and in an unattended way, it's got to run in the cloud. you 16:27 want to keep latency as low as possible. a a very simple and straightforward way to achieve something similar would be to 16:38 use GitHub actions but that one has the massive disadvantage of being very very heavy weight. , GitHub actions uses actual virtual machines and 16:50 there is there's a high cost there between you sending a task and it running because the 17:00 virtual machine needs to be available then it needs to be that it needs to be initialized it's running all sorts of processes that's that 17:11 really kills performance and and the the feedback loop that you're expecting here. super excited to see that stuff and super 17:22 excited obviously to have to see that there's choice whenever you have different vendors offering similar products it it 17:33 means because of competition obviously they're going to have a lot of pressure to deliver something deliver something. there we go. I want to maybe explore this. I've 17:44 heard a couple terms here. , one thing I want to cover back on our earlier part of the conversation was there is a Windows version of an agent. The Windows version of an agent, I believe, is called Scout. Scout is your 17:54 always on personal agent. , that was there's Open Call, there's Hermes, and there's Scout, which is a version of Microsoft's flavor. Again, they're participating in this agent space. 18:04 another thing, another term I've heard very recently and is under the firecracker elements. , have you heard about firecracker and this concept 18:14 of microVMs? Have you heard about this before? I've heard some people from Microsoft talk about this thing called I've definitely heard the concept of 18:24 microVMs, not the dirty firecracker. Yeah. Firecracker is apparently another flavor of being able to virtualize computers. They're very small virtual 18:34 machines. I think back to our point, we don't need these agents to have huge amounts of sub of operational systems. an agent just needs a command line and a couple it can run code, 18:44 ? , we don't need these VMs to have a lot of extra infrastructure on top of them supporting UI and a lot of other graphics things. It can just be on and running. And , there's this 18:54 other open source project called Firecracker. It's on GitHub and it it really it's it the whole mantra is secure and fast microVMs, ? Can you 19:04 stand up can you stand up a VM in a 100 milliseconds or less? That's I think this the stance or the story here around these microVMs on firecracker. 19:14 anyways, I find this very fascinating. Again, back to your points around this new world of containerizing things and building little subsystems. I was listening to a video on YouTube 19:26 yesterday, I think it was, and it starts talking about do I'm going to maybe propose a concept here. Do you think that the era of AI is making the local PC 19:40 relevant again? And maybe let me phrase out what his statement was. His statement was AI is demanding that we have more 19:50 capability on local machines doing inference locally, running file systems locally. Up until this point, most of our work has been pushing everything to 20:00 the cloud. Everything in the everything in a web browser, ? Put it in an Azure VM, a service that's in the cloud. Let them manage all the infrastructure and the networking and all these other pieces. This agent space for, , 20:13 privatization, having your data locally, running models locally, saving yourself some money running local models. It feels we're re-emphasizing the local PC. I'm just going to pause 20:24 there. What do you think about that comment? one thing I definitely see is there are two distinct flavors of agentic working, 20:37 ? . There there is the there's the interactive one, , where you are actively driving a session. . and then there is the 20:49 headless autonomous agents one and both of them are very important. autonomous agents I think 21:00 is a goal let's say that you know many people haven't quite achieved yet because obviously that requires much more thinking and engineering and setup 21:12 but ultimately I would argue running a large percentage of your agentic tasks using autonomous agents is what you would want because 21:23 that's the only way for you to free yourself, , from your own constraints in terms of being able to sit there in front of a computer screen and being able to react, ? yes, 21:35 we talked about a loop engineering last time, didn't we? Yes. which is, you know, which is exactly that where you say I'm creating an agent 21:47 that effectively prompts other agents to ensure that the sessions they 21:57 jointly run through are much more substantial and much more longived. You know, taking you as the human out of it. And in that respect 22:08 I would maybe challenge that statement a little bit and I would say yeah yes to to some degree a lot 22:18 of people are doing more work that's compute intensive on their machine because more people have been enabled to to do more interesting 22:29 computing due to agentic applications. But on the other hand, if you're looking at the long term, I would say there's definitely not a 22:39 massive move from from the cloud back to local machines. Quite the opposite. what we talked about earlier. all those cloud sandbox 22:50 environments are a very clear indication that this is where the industry is heading when it comes to 23:00 more significant largecale agentic applications. interesting. Yeah. I'm not sure where this is going to ultimately land us. I I I feel this mixed model approach is very useful for 23:11 organizations and I think as we look more towards this min maxing on usage of tokens. I think this is going to continue to evolve this why 23:22 work in containers how do we make agents run more efficiently? Do they need to live in the cloud? Should they live locally? What model are we using? Where are those models coming from? I think I think there's a lot of new pieces 23:33 in this architecture that's moving around here. And I think there's a lot of big players that we just stay in tune with and figure out what's going on. I'm looking at a hybrid approach, some 23:43 local, some cloud, and trying to figure out what that hybrid blend looks that it makes a lot of sense for our organization. I also I think also if you think about inference and how 23:53 things are working when you scale out to many many employees, I think smaller organizations are advantaged by AI. they can do a lot more with their looping and 24:04 agentic and experiences there. But when you get into large scale companies that have many more employees, does it make fiscal sense to is everyone running as efficiently as Matias is? Probably not. 24:15 how does how does that work? How does the efficiency story work moving forward? Love that. on on on that one, I would say, , if you're an enterprise or, , a company where the cost 24:28 of running agents is really significant and something you want to keep an eye on. And again, that's something we've covered, , many episodes before. there's a huge advantage to 24:39 not requiring or to having a larger proportion of your Agentic sessions run in some managed cloud 24:49 environment as opposed to employees local machines because ultimately you require some really good insights into what your employees are doing with 25:01 their agents. in terms of efficiency in terms of which models are they using how the token usage how 25:12 many sessions how successful are those sessions as a business and particularly as a as an engineering manager in a way as a CFO you 25:22 want to collect as much data as possible and you want to run some analytics over that to ensure that you're really spending potentially your hundreds of thousands of of dollars a 25:33 month on on model costs in in the best possible way. And if that's stuff that's tucked away on individual machines, you're going to have a hard 25:44 time getting those insights and you're more likely to waste a lot of money. Interesting. . Not sure I 100% agree with that 25:54 stance, but I do agree I do agree with the data portion. I I think there's in this agent world, we're generating a lot more a lot of extra data, a lot of new information that we 26:04 should be leveraging and using throughout to inform us how to build better solutions moving forward. Interesting. I didn't think of it that way. , awesome. what other topics do we have here? We have another topic 26:15 around we talked about chat GPT56, talked about some of the WSL containers. and I believe this is one I think is really fascinating. Matias, you found an article from GitHub copilot and GitHub 26:27 copilot is talking about evaluating the performance and efficiency of the GitHub copilot agentic harness. Super big title. Let's go through this one. I 26:37 believe this will be a good ending note here on our topic for today as we think about what's happening in the agentic space. Comparing the hardness of VS Code with 26:47 GitHub Copilot to other harnesses. This seems really important, ? And this is very much in the the spirit of what I mentioned before. I 26:57 I've seen a significant increase in terms of quality and quantity of of output around agentic experiences 27:09 from from GitHub over the past month or . what they've done and it's it's a very long and and detailed articles with lots of charts and and 27:20 tables and whatnot. Sure. They've looked at how does the copiloted CLI do as a harness 27:31 compared to other compared to native vendor harnesses in conjunction with top models GPT 27:41 and Opus and Sonnet. to be fair, , I'm a little skeptical when when I, you know, see those kinds of 27:56 data coming from the vendor itself, ? this is co the co-pilot team ships an article telling everyone 28:07 how great co-pilot is how [laughter] exactly how does their product compare to affect their competition of course it's amazing we wouldn't have written the article if it wasn't it doesn't make sense 28:17 if if they were coming out looking really badly they absolutely would not have published it but to I' I'd certainly prefer 28:27 to see that analysis from an independent a third party, let's put it that way, ? Sure. Yes. but the other thing that's really important here, and it's again it's it's 28:38 a theme we've certainly mentioned many times before. 28:48 your harness matters enormously with respect, , to the to to the outputs you're getting also with respect to to the cost. you can 29:00 you can run the same oppus or GPT model within very different harnesses. when they talk about native 29:11 vendor harnesses they mean running a claude model from within claude code and they mean running a GPT model from within the codeex 29:22 applications. But you can you can use those exact models with inside co-pilot and there are many other harnesses as 29:33 pi for instance open code deep agents many others yeah 29:43 look at it with a with a bit of grain of salt here what I'm missing and that's that's a bit sad. You 29:54 know, they're only comparing the top frontier models. I would quite to see the exact same 30:05 comparison with open weight models. we've mentioned Oh, yes. . that this is a massive trend . open weight models 30:15 getting to a point where they're able to compete really significantly with the latest frontier models. GLM52 in 30:25 particular is is one model that's had a a lot of very significant reviews lately. , many people 30:36 compare it to Oppus only that it comes at a fraction of the cost. this is where I would to see the the co-pilot team run 30:48 those kinds of comparisons. because then the the question is which part does the harness play, which part does the model play, and what's the 30:59 overall output here? I do want to give Microsoft some credit here. they're writing, you know, some articles around, , them being a very good competitor in a lot of these spaces. Halfway down the 31:10 article page, you talk about they open up with an an image here that comes from Terminal Bench, which I thought was really relevant here, which I I do the fact that they're including Terminal Bench's graphic here in their 31:22 documentation. that is to some degree a third party solution that's out there. They're doing some testing on these things. And the graph that I think is very relevant here is resolution rate versus cost per task, which I found to 31:34 be very fascinating. And in here they're doing the co-pilot CLI the co-pilot CLI on sonnet versus opus and then also comparing co-pilot CLI GPT versus codeex 31:46 and doing some comparisons there as . Again some of the results were inconclusive. I would argue maybe Opus 4.7 Claude 31:57 Code might be a little bit better in that area, but it definitely appears that Claude Copilot did a pretty decent job on set 4.6. six. , it seems some of the harnesses are, you 32:07 know, again, if you think about this puzzle that we're looking at, any harness, any model, what's the mix and match of the best solutions here? We're 32:18 just starting to get to the point of seeing real data and metrics come back out. ? If you're using chat GPT 5.5, makes sense. Go use the Copilot CLI. It seems it has a little bit 32:28 better performance there. If you're using Opus 4.7, seems a tossup. You could use the cloud code or co-pilot CLI. Either one seems a good a fair use case there. again, you 32:40 have to figure out what works for you. And even there's even a little bit of personal preference I think involved in this. when I talk to my engineers, they're , , it it feels to me 32:50 it's not quite as quantitative as I would , but they're , oh, it feels to me that when I'm using Sonnet, I'm getting these results out of C-Pilot. if I'm using the CLI, this agent seems this this large 33:01 language model seem to be a bit more performant. And we also have starting to form preferences around do we get the responses? Does that agent and harness combination work with how 33:11 we to design things? What we think about when we design? , I think there's a lot of negotiation that's still happening here around these spaces. , and it's still an evolving 33:21 space. But I would agree with you Matias h not having this for all model great these are useful and very helpful to look at the premier models. I want to see how this does on Quinn. What 33:32 does DeepS do here? M there's all these other new models that are coming out that are very specific for things. Kimmy is out there as . how does this all perform in 33:43 relation to those? What does that look between cloud code copilot and CLI or codeex for his way example? What does all that compare? I think that's stuff that we're going to be getting 33:54 as we get better at this and tests become more common in the space. We'll get better insights around this. Going back to your earlier comment, Matias. 34:05 If you give these other models access to your users in your organization, this feels all the data we were just talking about, ? Hey team of 34:15 people, go try these models. Go use these different harnesses. The team will land or fall into a pattern that they to use. They'll see what's useful there. But all of those usage, 34:28 every single prompt, request, command, token usage, that becomes fodder data that we could then come back and reanalyze for our organization. Does it 34:39 make sense to stick with copilot? Should we go on to codeex? Are our users finding better performance there? Are we able to get resolutions quicker? I think this is where your 34:50 data that you were saying describing earlier is extremely relevant and there I don't see any systems that are handling that part in in an organization turn on tools and 35:02 see what performs better and then start steering training and or education towards a particular tool because it works better for your organization. Yeah. , , we we both, you 35:12 know, have have a data analytics background. Yes. , that's where we've spent a lot of our career on. And this naturally this is something I'm very interested in again because it's 35:23 relevant economically. , I'm curious. I'm it's it's an it's it's a space I'm heavily invested in . looking at how can 35:38 you use data and insights to ultimately improve your your entire agentic setup. and and ultimately get 35:50 to a point where your your setup is the best possible you know the the most efficient one you can 36:00 have. but again there are many parameters here which harness are you using which model you're using which tools are you 36:12 providing? How are you preparing your workspaces with agent instructions? How do you prompt how do you react, , once you've had 36:25 agent responses? , loads of parameters here. and one thing we can definitely 36:35 hopefully agree on straight away, not there's no single best possible setup, for your company, for your team, for 36:45 your product. this the the best possible agentic setup for you is one that you need to arrive at. It's 36:55 one that you need to evolve over time. And I think this is only possible by collecting data and 37:06 observing drawing insights from it and then making improvements to your setup over time. agree with that one. Awesome. This is a 37:18 really interesting topic. Good conversation today. Neat things that are coming out . Microsoft's releasing chat GBT and OpenAI is releasing. very exciting time to be 37:28 in . and with that being said, I think we've talked about our main topics for today. We'll go ahead and wrap it there. Matias, as always, this is a wonderful time to kind 37:38 of unpack what's happening in this agentic space. I where we're going here. Let us know down in the comments or let us know on social media platforms what would you to see us demonstrate. this Friday we do 37:49 typically a screen share or showing you of what we're building and how we're creating things. last week we just started touching the surface of using the PowerBI skills for fabric and 38:00 working with PowerBI desktop to build some report and working with the PBI format. If you want us to continue doing that let us know in the comments. help steer what we talk about next to 38:10 help you learn more about agents and how that fits with fabric. Matias, thank you much. Great conversation today. Yeah, thank you. Talk to you soon.