chapters transcript notes
click any line to jump to that moment in the video
0:15 Man, AI is just good at music. Welcome back to Agentic Thinking with Mathias and Michael. hello everyone. Welcome back to the show. We're going to talk more about agent things, Agentics experiences, how we're building with agents. , and today our main topic is 0:27 going to be a lot more around GitHub and resolving these issues we've made in prior episodes. , before we get into that, let's just open up with a couple just news andor discussion points 0:39 here. Matias, you had a couple questions you wanted to run by me. do you fancy a bit of a quiz? You're you're going to put me on the spot here. I'll do the best I can. 0:49 Hopefully, I I answer here. [laughter] the thing is I've done a bit of a historical analysis around AI because my thinking was everything happened fast . you we 1:02 can very easily fully lose a sense of time in terms of when stuff has happened. I've done a bit of research which may turn into an article or 1:13 . Let's see. , guess , we've been talking about Frontier models and GPT55 and Office 47 and stuff that. 1:23 Sure. Guess when GPT2 was released to the public. Oo, GPT2. This is not 52. This is two. 1:34 Two. Wow. Major version two. . This feels eons ago. It feels it's been years. I'm gonna say four 2023 1:48 [snorts] late 2023 is maybe what I'll guess. Yeah, I I I thought you would go that way. 2019. Oh, wow. I was way too That was really 1:58 early. . And GPT3. Yeah, GPT3. Maybe a year later. 2021 2022. June 2020. 2:09 2020. . data. . wow it feels it's all happened in the last 12 2:19 months or some degree in terms of AI becoming consumer ready in the sense 2:29 consumer from a in terms of us how we're using coding agents and things that. Yes. in terms of is having such an impact on virtually 2:39 everything, ? In terms of how you're not you when you're not using agents, it feels you're doing something wrong or you're missing out. 2:49 Yes, true. But , this is all just the wow the the final peak of a 2:59 relatively long evolution, ? And you turns out you have to go back quite a few more years than expected to see, where these things have 3:09 come from and and how it's all started. there we go. Just a little I I I hoped it would come as a surprise and it did. thanks. You definitely 3:19 delivered. , when did when was when we started seeing just in general chat, , open AI or chat GPT or when you go to when was the 3:29 original release of the website? Cuz that was a that was quite a while ago. Maybe we can throw this quickly at one of our agents to go do a quick research for us . When 3:39 was chat GPT initially released? If chat GGBT2 was released in 2019, was this 2018 2017 is when the 3:49 original chat GPT website came out or is that later? November 22. the thing I can answer that quite easily because I did all the research. I was able to look it up 4:00 quite easily. Love it. JPT as a research preview to for consumers end of November 2020. . probably that's what threw me 4:10 off a little bit cuz I I think in the context you said chatgpt2 my context was thinking oh it's that's probably around the time when the website appeared and I don't think I started using it till 4:20 2023 or it was out there you maybe played with it not super useful and here we are it's it's everything that we're using and I think at the time that was the 4:30 first harness wrapped around a large language model and here we are with many many more harnesses and a lot more research around this this whole space is extremely fast. 4:40 memory seems the new exploration element. How do we give agents memory and recall things when they need and when they don't? This is a really neat space to be in 4:50 and also how do we host agents at scale. I think you you had an announcement you wanted to in that space. 5:01 Wonderful. Thank you for remembering that. I this is a perfect leadin. the Microsoft Foundry team and I and I follow Foundry pretty closely. That's a there's two ways I access 5:11 large language models. One of them is through Foundry. I use Foundry to deploy models and agents and then I use the APIs to go talk to them. That's one method. The other method I use models 5:22 and agents are through the GitHub co-pilot harness. that that's the whole experience that is GitHub copilot. I use that one. Those are my two main outputs of where I go get information from. 5:33 this in Foundry. They're introducing a new hosted agents in Foundry Service. You can have secure, scalable, and 5:43 compute built just for agents. , it sounds they're building these little microVMs. They're very small. They they spin up very quickly and 5:53 there's a file structure attached to them. There's production ready endpoints. There are multiple protocols out of the box. And Matias, I was just sharing with you a video from Matt PCO, 6:04 which I believe is he's he's really good. I love learning from this guy. He's good in agentic spaces, and he's talking about how he's building a little wrapper around spinning up a little tiny Docker 6:14 container and putting your little agent and harness in that Docker container to do a job and then turning off and getting rid of it when you're done because that's a secure you you need 6:24 sandboxing around these agents that you're going to productionalize and move. Because if I'm talking to an agent and it's making files and doing stuff on disk and Matias, you come talk to the same agent, 6:34 we don't necessarily want it seeing the information that I'm using or what you're using. You almost needing evolve there. , it makes separate files for each of our experiences as we interact with the 6:45 agent. And from my point of view, it makes complete sense, , in just a general evolution of things. what what you're talking about here with 6:55 Foundry agent service many other vendors offer similar things nowadays there's there's a lot of innovation happening in that space 7:06 and obviously if you want to if you don't want to be tied to to your computer with a chat interface where you give an agent instructions if instead you want agents to have more autonomy if 7:17 you want agents to run long horizon and tasks, you need to give them some other runtime environment 7:28 where they can run securely but also where they can run independently from a particular human user, ? And this is this is what those cloud hosted agent environments are all 7:40 about. and I'm pretty sure we're going to see a lot of both innovation and competition in that space. , , Microsoft is is is 7:51 where everyone else is with their Foundry agent service. I'm liking that we're seeing these announcements because this these are the things I'm looking at going, [snorts] I really my open call. That that's 8:02 a good system that I'm I'm enjoying working with. I having it available on on demand, but it lives in a VM on my machine or locally here on my computer. I I gave it a computer to have free reign around. it's 8:13 isolated in certain ways, but it it kind of can do what it needs to in that sandbox that I've given it. , I think that's a pattern. Me personally, I enjoy working with it that way. , , having 8:24 more systems and, , more sandboxes or more of these agents that can spin up and do different tasks and things makes a lot of sense here. , in addition to this, another 8:34 announcement that came out of from Microsoft Azure Foundry , was talking about the introduction of toolboxes in Foundry. This is another topic. I I kind of think this as you build agents in a 8:45 sandbox, you're going to want to give it access to different tools. , Entra ID, cloud services, GitHub, Microsoft Teams. There's there's tools you're going to 8:55 want to build to integrate what what does the agent need to know how to do and how can you securely attach that agent to doing things this is this is what I wanted when agents came up to 9:06 beginning when when Microsoft started putting co-pilot in fabric. I was , "Yeah, I can ask it questions, but I wanted to do things. I wanted to edit my notebook. I wanted to make an item. I wanted to take action on things inside 9:18 my fabric environment." And , this to me feels they're really giving the capability of discovery of tools, handling tools, giving tools, the ability to have that inside this context 9:29 of these agents in these sandboxes makes a lot of sense as . It's going to increase the capability greatly here in the near term, ? , if you think about it, it's it's it's not much the availability 9:39 of tools. It's for at least I think the real technical challenge is how do you authenticate those tools? 9:49 how do you provide credentials in in the agent environment they can access third parties either on on on your behalf or on behalf of a 10:00 dedicated agent identity you created, ? And securely providing those kinds of secrets or or logging into a service in such a way that 10:11 an agent environment can then use your login that's much harder from a technical point of view. And I think this is what things those toolbox services are all about. 10:22 And I think what we talked a little bit on Tuesday which was around the whole metering and measuring and and logging that goes along with this what is your agent doing? How many tokens did it use? Is there more 10:32 efficient ways of running these agents? Part of using Foundry and what I one of the things I am finding immensely valuable there is there's guard rails but there's also a lot of monitoring and logging that goes along 10:43 with those agents. we can we can kind of retroactively look at what the agent did, what it built and from there we can determine what's going on. anyways that's our news and topic items 10:54 that the articles and what we talked about are in the description below. , if you want to go check that out in the below description, they're also in the chat window. , if you're listening along and you want to go look at the 11:03 chat, that also has the links there as if you want to go read up more on hosted agents and foundry toolbox or tools for for agents. , that being 11:13 said, Matias, take us away. Let's let's talk unpack what we're doing here for demo pieces today. And feel free to share your screen. I just realized we totally forgot the screen share today. That's . I I 11:23 knew there was something. . Sorry. , here we go. I'm just doing that . While Matias is doing that, we're going to we're going to take another dive into working with our Agentic model here. 11:34 this is on GitHub. And let me turn that screen on there. There we go. There we go. . And this is going to be us working through our repo. This is a repo that 11:45 you've built out. I will make sure I put the link in here in the description and in the chat window as . This is us building changes with the semantic models all around GitHub. All , take it away. 11:55 Yes, there we go. Cool. yeah, public GitHub repo under the fabric tools or we are going through this repo live on on air, ? I'm 12:07 pretty much continuing from where we left off a week ago. just a reminder where it lives and if you joined us last week that we were in VS Code 12:19 back then. Let me make my screen a little bigger this. Hopefully you can see. and I've got a 12:29 particular branch checked out. this branch was created by copilot when we ran a cloud hosted copilot. again this is another example 12:42 of autonomous agents running in in a dedicated cloud environment rather than on your machine. and we asked it 12:52 to pull down the timill sources for a fabric hosted semantic model. we also asked it to use the PowerBI 13:03 modeling MCP server to inspect that model and to propose some improvements that could be done from a modeling point of view. , , , I'm 13:17 back in my VS Code environment, pretty much where we were a week ago. And I just wanted to show, , that if inside the the chat window, which is , 13:28 your primary AI interface in VS Code, , we also have a view of previous sessions. And it turns out, , this one is exactly the same session 13:40 that, , we ran through a week ago. your machine and your user profile retains a a full lock of those 13:50 sessions and you can come back to them. I can click on here and it opens the entire session which also allows me if I wanted to to continue 14:01 with that exact same context. that's pretty neat. and it's a nice way of picking up where you 14:13 left off. and that will be a really nice reminder for us to see what did we do a week ago. I said this is this was my 14:25 prompt here. the co-pilot cloud agent created this markdown file which listed some summary stats of about the semantic 14:36 model and then it identified a number of worthwhile improvements. In fact it got up to 18 of them. I ask my 14:49 chat my co-pilot in VS Code go through that file identify everything that is severe enough it's 14:59 worthwhile us investing time into I said anything with severity of medium or higher and then create a GitHub issue explaining in detail 15:09 what to do and that's what we ended up with and down here we have a nice summary of seven issues 15:19 that were created. today why don't we continue and have it why why don't we have it 15:32 work on on any of those? sure. we could randomly pick one or we can ask our AI friend to select 15:44 what would be the most impactful from a usability point of view. Why don't we do that? . I that idea. I'm going to I'm going to continue in that exact same 15:55 chat because again I already have the full context here and also because I don't have an issue with context window at that point. . this is a very 16:05 important one. Generally, , I advise people to create fresh chat sessions as frequently as possible because it clears your context window. 16:15 It makes your agentic experience much more efficient in terms of how good the LLM can be. but it's a judgment call. In this 16:25 particular case, the session is very small. It doesn't eat very much into my context window. And it turns out it's quite beneficial for me to keep that context in. ? it's it's not 16:36 A or B. Sometimes it's somewhere in between. and in fact remember down here you can see I can I can 16:46 hover over and I can see that I'm using 6% of my context window. that is a very low number. 16:58 . I'm going to say of those newly created issues 17:08 which ones should I address first from the point of view of user 17:22 impact. Let's see what it makes of that. Cool. and whilst it's thinking we can just have a look at our model selector here because if you joined 17:34 us on Tuesday we talked a lot about what happened with GitHub co-pilot announcements and GitHub pricing changes which are quite drastic and we 17:46 can already see a a very specific change here which is look at that 17:57 when we ran this a week ago, Oppus 47 came at a 7.5x multiplier. it's a 15x multiplier. The 7.5x which was 18:07 already substantially higher than anything they ever had before, leaving aside the 30x for Oppus Fast. the 7.5x was explicitly introduced as 18:19 introductory. there we go. Looks we're out of the introductory phase . didn't take more than two weeks. Speaking of how long it takes things to happen, that didn't take long 18:29 at all. [laughter] . One thing, by the way, it's very interesting. Something you get a lot when you ask a agent to create plans 18:39 for you. it's quite weird. They oftenimes associate implementation times with plans. Have you seen that as ? And it's always weird in the 18:51 sense that if an agent were to do that, it would definitely not take them two weeks. It might take them four, four hours, ? if a human were to do that, how do , ? , 19:01 one human may not be able to do it at all. Another one may do it in in six, eight or 10 weeks. , it's very strange. I don't know why they would even bother producing those kinds of 19:13 estimates. I've had the exact same experience and I've even debated with my agent a couple times. Hey, thanks for this estimate. They're , "Oh yeah, this would take about three months to build." I'm , ", I don't think . How 19:24 long would it take you the agent to build it?" "Oh yeah, two hours thing." I've I've I've negotiated with the agent a bit on how to work through things, but yeah, I have seen 19:34 the exact same experience. Oh, looks we got a response back. Yeah, there we go. , we've got something. , . it's talking about stale customer age. I don't 19:46 exactly know what that was about. It says customer table exposes age. if this is calculating ETL time rather than the val become stale between 19:56 refreshes. it's suggesting calculated column. I think age is probably calculated as part 20:06 of the M here. let's leave that because it's very much tied to how that particular model is set up. I always display folders. 20:18 that is something which is immediately visible to modelers. I would also argue measure descriptions 20:28 missing measure descriptions. There we go. five and six look those would be pretty substantial changes to the enduser value ad. Absolutely. why don't we go with 20:38 those? and how should we go about that? we know we've got GitHub issues created. 20:52 there are various ways we could go about that , ? we could create a distinct working 21:03 branch or feature branch for each of them. and for each issue we for each issue we could then create a separate pull request and all of that 21:14 which is definitely the thing to do when we're talking about really substantial changes given that these ones here are quite trivial. I want to shortcut a little bit if 21:24 that's . I'd say [clears throat] can we do no display folders and measure descriptions all in the same pull request? Absolutely. what I what I would say 21:34 please tackle who how do you two different numbering schemes here. . . four display 21:45 folders and six three measure descriptions together. 21:59 create a new local branch first before committing any changes. 22:11 when you commit changes, make sure you reference the 22:26 related GitHub issue. that is a that is an that's a nice way for us to still link it to that issue because when 22:37 when you put a hash number token in into a a git commit message GitHub will automatically show it as linked 22:48 it will it it will stay in that view but we don't have to go through the extra hurdle of of having separate pull requests and things 22:58 that. Sure. Yeah, exactly. all . , let's just see how it's how it's going to go. , obviously I've asked it to do some 23:10 git operations. I asked it to not make any changes in my current branch. I asked it to fix the issues 23:23 and also commit them individually. which by the way is a very important engineering practice when it comes to committing changes you always 23:33 want to group them logically I generally use predefined skills for my git commits and this is a this is a number one rule I've got in 23:44 there making because otherwise it's very difficult for you to trace back changes through complex code 23:54 bases all good to know. whilst we go through that let me show you a few other nuggets. for 24:08 instance here this is very cool. and it's probably not something many people have seen. You can open the chat debug view. 24:18 Yes I've played with it. Yes. This is pretty cool. That one is insane with respect to how much visibility you get in to your agent 24:31 operations. It turns out that everything that's going on in this chat is traced over here with respect to the 24:43 individual tool calls with respect to the specific invocations of large language models, which parameters they 24:54 get, what do they send back. , you know, this is a debug window in the sense that everything is shown to you. and it's 25:04 one of the one of the key advantages from my point of view for using VS Code for any AI sessions 25:15 because you have that extreme level of visibility here. I want to call out one just little quick item here while you're talking about this one. Also notice here in this 25:25 window that you're looking at if maybe if you can zoom in to one of the little GitHub copilot icons. You have a little the zoom it thing. I I want to highlight here. Notice the Yeah. Yeah. Just on one 25:35 of those. Notice the output that it's giving to you. it's it's one, it's just doing two things here. It's telling you how many milliseconds and there's also a ratio of here how many tokens per second that it's running 25:47 at. if I read this correctly, this is in Let me make it a little bigger. Yeah. 31,000 tokens per second is what it's able to rip out when it's thinking 25:59 about these things. I find this to be very fascinating because when you if you if you do any what I've been experimenting with is what happens 26:09 if I want to run a local model on my computer how many tokens per second could I produce and maybe I don't want to use GitHub copilot maybe I want to run something else on a different machine or a piece of hardware 26:20 the tokens per second is telling you how fast it's able to pull back those tokens and pull them back to you. it's a it's a velocity thing. 26:30 [clears throat] And I find that interesting to me. The other thing which is also something that is revealed that way quite neatly you you will see down here I've selected cloth sonnet for 26:41 this particular session. Yes. however when you go through your debug window over here you'll see that a number of calls have been made to other 26:53 models. For instance this one ? Yes. This this was made to GPT41. this is showing the real power of those tiny models. , 27:05 they're generally called nano or mini models. , they're extremely fast, extremely low latency. you wouldn't use them for any complex stuff, 27:15 but they're very good at, for instance, generating a session title. , this one up here, this is generated using a a a low latency GPT mini 27:28 model. when you create your first prompt, I never named that session. and no deterministic code could name 27:38 that. This is something that came from a reading about it. And over here, correct? over here you can see how things that 27:51 was produced behind the scenes. here is another one which is very very cool. I'm not sure whether 28:01 people are and I'm just talking because my agent is still going. . [laughter] That's good. Let's keep going to the tool. This is great. This is all what we're see. You're just going into the harness. this is just how the harness is working and what you're 28:12 seeing behind the scenes here. That's great. another one which folks may not be familiar with is the concept of a system prompt, ? , a a prompt is 28:22 generally what you think of of what you are typing into your chat, ? , for instance, when I just said you know, tackle number four, number three, 28:32 that's a user prompt. , it turns out that the harness you're using always has a an invisible system prompt 28:43 which it adds to all of your context to ground the large language model and give it foundational 28:54 instructions and behaviors and dos and don't do without you necessarily seeing that . for instance a system prompt would instruct the 29:05 large language model how to do tool cording and things that. Turns out VS Code with the chat debug window gives you full insights into what the respective system prompt is for 29:17 not just for your main conversation but even for small things this one here which here 29:29 this is something which VS Code chat created on behind the scenes silently. without you even knowing and it says 29:41 generate 10 unique progress messages. Turns out this is the stuff that shows up in the chat. here for instance, ? Reasoning this one, 29:52 that's one of those progress messages. the chat shows that whilst the LLM is working, ? and to give you the impression that there's 30:03 some progress happening and even that stuff is not hardcoded. Even that stuff is coming undeterministically 30:16 from in this case DPT 40 mini. There we go. and that obviously is not something you are 30:28 explicitly charged for. It's sort of part of what you're getting in this case as part of your 30:38 co-pilot subscription, but it's all stuff that that goes on behind the scenes. In fact, this one even took two seconds, which is quite substantial, ? only to produce 30:50 arguably some some silly messages that have no business value, ? But [laughter] there we go. Sure. let's see how this is going. 31:00 you can see here, ? there's a lot happening. for instance, we're reading files. we're replacing 31:12 we're changing files. here that's an example, ? . the replace in string in file is a very very common coding 31:24 agent tool. every single coding agent harness has that and it has three parameters what's the file where you want 31:35 to replace something? What's the old string and what's the new string? ? And we can see here this is my number of customers measure which had 31:46 a format string and the lineage tag and it wants to replace it with that exact same measure definition but additionally the display folder and it's determined the display folder named 31:57 counts. ? this is how coding agents and and coding agent harnesses work behind the scenes. there we go. cool. We can stop 32:08 exploring because we've got some stuff here. let's see whether to review before I look into the chat, let's see whether it's done the main 32:19 foundational things I asked it for. first of all, I asked it to create a new feature branch, ? And when I look down here, that's a win because clearly this is a new branch, ? 32:30 That's not the branch we were on previously. done. Very good. Secondly, it has already committed its changes, ? Because when I go into my source control tab here, I can see no 32:44 pending changes. when I look down here in my gitlens extension, I can see I've got interesting only one commit. 32:54 looks it failed me here because I explicitly said I wanted to break the two distinct changes into separate 33:05 commits they can all be associated with those respective GitHub issues. ? Looks that did not happen. , what what are we getting here? add display folders and 33:16 descriptions to all measure. . . , a small fail. , which means I would if I wanted to do that again, I would have to be much more explicit in 33:26 in saying what I want rather than thinking that it can infer. Bit disappointed with that because we're using Sonnet 46 here, ? , it's a it's a it's a top tier model. Should 33:38 have been pretty good at figuring that one out. But then again, this is cosmetics. you could argue let's see what has done in terms of sure work 33:48 the model what are what were we after we were after descriptions and display folders should be pretty straightforward in 33:58 terms of figuring out why do we have a change in modeled timo all very good cool this is yeah that's one of those Microsoft annotations 34:14 it's noise if you want. in this case because I've got the Microsoft Kindle VS Code extension in place it just decided it added itself to the 34:25 Pro Tooling annotation. Fine. I can live with that. that looks good. we've got the number of stores 34:37 measure count of distinct stores in the store table visible in the current filter context. Interesting. that is a very 34:47 explicit and very technical description. that's something which explains exactly what there is. 34:58 probably not something you would want to expose to a business user but your modeler would certainly appreciate that, ? Yes. but then again also I I I 35:08 never gave it instructions with respect to how I wanted my descriptions to be . I just said put descriptions there. I never added any 35:20 context to my prompt in terms of you know what was the expected audience for instance. Sure. I'll also maybe additionally note here as . , one 35:30 thing I'm finding a lot of value is when your your audience of who is consuming this semantic model matters. I think you you alluded there very briefly, Matias, around your modeler will this, but 35:41 a business user may not have the technical chops to be , what are we really talking about here? Distinct count of what do we what do we really mean? , I'd also [clears throat] agree if you're giving 35:51 me things to modelers to consume and build reports with or an heavy analysts, I found it's a really good practice to even say, I want you to take 36:01 the formula and put the formula inside the measure as text. Because if you're only consuming the model in a build only space, ? You're only building 36:11 against you're not able to edit the model. I find there's a fair bit of weakness of how does this model works? what table or what column does this measure operate on? 36:22 recently I was just working in a a measures table. There's a whole bunch of measures. , those measures could apply to any number of fact tables in the model and there was four, five, six fact tables. How do I know which fact table this 36:33 measure is operating on? as an end user, as one who's trying to build reports on top of this model, these are to your point pieces or details we would have 36:43 recommended back to the agent. Hey, these are the pieces of information I care about to make this description richer for that type of consumer, 36:53 ? And just reflecting on this particular example once more, ? , this is precisely the feedback I would expect to get from a junior 37:05 developer in the sense that someone who is has had a few years presumably using DAX and and and semantic models and who 37:17 knows technically, , what a particular DAX function does. And if if I then ask them put a 37:28 description on this measure for me, they may go down this route and and be very descriptive in terms of technically what's going on here, ? Yes. Without fully appreciating that 37:40 what I didn't say, but what I implied was put a description here that makes this make sense to a business user. you 37:52 know someone who lives in Excel and wants to understand what that particular measure does as part of the semantic model in in in in that 38:02 sense we're we're getting a junior developer result here on this particular one let's go through a few 38:13 more examples to see lot of here we've got counts counts display folder sales sales slashquantity 38:23 display folder sales slashamount sales amount sales amount sales amount sales amount it 38:34 decided to create a folder hierarchy it's calling out measures in the descriptions look it's saying it's using using same period lat this 38:44 is these are very technical descriptions of what's going on in here I'm seeing distinct count oh look, that one is talking about inactive relationships. Total quality sold filtered by delivery. 38:55 Yes. Uses the inactive relationship between that's that's interesting what it's what it's doing there. It's calling out relationships in these measures as . That's cool. the reason why I'm pointing this out 39:06 is that your mindset when it comes to prompting and instructing your agent should be in many cases the same mindset that you would use when instructing a 39:17 junior developer. , don't imply too much. Be very explicit and you know what the context is in 39:27 within which a task is given. but then also you've got to come in and review and critique and send feedback 39:37 back if you think maybe we shouldn't have gone for those kinds of very technical descriptions. . sure. And , , I'm sure we're going 39:48 to see lots more of the same. Going through the other ones here, counts. , . . , interestingly, , 39:59 we don't have display folders here. just descriptions. Yeah. And then here we have some folders. . , also, , not 40:13 sure whether , just having a display photo called Counts Everywhere is super fit for purpose here, but there we go. , 40:24 that's that. , , it was not a an immediate win, I would say, . 40:34 let's look at what we're getting over here and let's look how it's justified what it's done. , it's saying it's created this branch. 40:44 That's great. It created this commit. oh, . Apparently, it's got a closes message here. I couldn't see that because as you can see the commit 40:55 message itself is big that it is truncated and this particular view does not allow me to scroll in it. 41:05 . but it's also problematic because when you put closes hash three or closes hash4 in and 41:17 that commit gets pushed it will close that issue immediately which means we've lost the ability to do a review here and to explicitly 41:29 decide when that issue should be closed. the there should merely have been a reference to hash three or hash 4 rather than a closing 41:40 instruction. That's something which we would want to do explicitly after review. . I want to I want to maybe make a comment 41:51 here. I'm finding in our workflow here, Matias, this is this is a little bit of how I'm discovering things as . I want to unpack this a little bit with you while we're while we're in line physically doing the work 42:01 . We're making a couple comments here I wish it would have done this. There's a couple things here. There's some other items here. I just want to maybe throw the idea 42:12 towards you and we'll see where this comes in in the in the rest of the session today. We're doing all these changes to this model and we're finding the prompt that we use 42:22 to get going here was helpful. U and and we'll finish off this process here. This is a moment where I'm looking at this going maybe this should be a skill 42:32 things that we're talking about adding descriptions maybe doing folders this might be a skill that we want to think about in the future that's saying is this body of work making 42:42 measures by description and measures by folders is this an activity that I potentially would want to reuse between myself between you other team members in 42:52 the future a lot of what I do with agents and again I want to really emphasize a term here that I'm trying to coin which is called agent creators or the creator agent. ? I using 43:04 agents to create things, build stuff, build tooling, build systems that I can reuse over and over again. And as I look at this, there is a a level of 43:14 reasoning that has to happen around the measures and what folders we're building. But this is a great opportunity to say, let's finish the process here. Maybe we'll come back to the at the end of this is 43:25 reflect. Hey agent, reflect on what we just did. Describe this session. What did we just accomplish by closing these tickets? And let it summarize what it thinks it did against the activity. 43:37 And then we have the opportunity here to shape a little bit more of what that was. Hey, you made closes three and closes four. I didn't really want you to do that. 43:47 I want you to tag the issues without closing them. ? , we can do kind of minor tweaks here. And then once you refine the process in talking with the 43:59 agent, then you say, "Great, I'd to make a skill around folders and measure names." And all this work that we're doing, ? I don't want to prompt 44:09 this again. I don't want to do the same work prompting the second time we do this and , ", shoot. It did it again. We've got closes out there again." another thing that I'm learning is I the I've built a skill 44:20 called reflect or reflection. And I use that sky invoke that skill and say in my session that I'm working with an agent I'll say reflect. And then what 44:30 it does is it reflects on what we built. It outputs here's what I learned from this session. And then usually I make another one that says make a skill based on this this information that you just learned. we can 44:41 recall this. maybe we'll let's hang on to that one to the very end. But I feel this is a good moment to interject this idea of while we're doing this think about in your process what would you want to reuse? because this is 44:52 a good candidate for skill building. . Totally agree. Absolutely. and also remember how we started this whole thing by me saying create a new branch, 45:02 ? Yeah. This was precisely to whenever you create branches there I look at it as a safe experiment. you 45:12 know creating a branch means you're creating a new clone of your project in a very lightweight way and you may or may not proceed with it. 45:23 having this branch where we've identified quite a few things that we don't really , why don't we use this branch as an opportunity to think about how do we need to provide how 45:34 can we provide more context moving forward for modeling operations that next time round we get the result first thing and don't have 45:44 to go back and forth and skills are the way to do that I'm in inclined to not even push that branch because I'm not happy with what 45:55 we got here. I I want to keep that as a as a local learning experience. . And I want to I want to use this for us to identify 46:08 we need more context in this particular project. that for 46:18 the modeling operations on that particular model we get better outputs moving forward and that includes how we are 46:29 creating commit messages that includes particularly how we think about descriptions on model items and 46:39 we don't want to have to repeat all those instructions over and over again. you already used the correct term here. Skills happens to be 46:49 exactly the artifact that will help us with that. A skill is effectively a prompt shortcut. That's how I think 47:01 about it. . A a skill is a a a bunch of context, a bunch of a written up stuff that you want to be 47:13 included when you u send a prompt, but you don't want to have to repeat it every single time. And , sure, why don't we take that one on next 47:26 time we look at that project because I think we spent quite a lot of time on this already today. And why don't we focus entirely on what skills are and 47:36 how you use other people's skills, how you create your own skills, how you refine skills, and how we use skills exactly as you described before. 47:48 that when we let agents work on our project, we get much higher quality outputs first time round. 47:58 one one quick comment here in the chat which I really love that's coming on here. , the user is asking or commenting on. I use the rubber duck feature to review my plans before 48:10 and after my commits. Have you have you used the rubber ducky feature when the when the agent says I'm going to I'm going to do a rubber duck here and and talk to itself about what it's doing. 48:24 I I know the concept, but is is is there an actual feature called that? I'm not sure if it's exactly a feature, but my engineer has also seen the agent start talking about doing a 48:34 rubber duck. It's it talks to itself as in a rubber duck fe. I'm not sure if it's an explicit button that's in the UI, but , , telling your agent, , proceed using , you 48:45 know, add do rubber ducking or or talk to a rubber, , do the rubber duck feature while you're thinking through this. Adding that additional context allows it to rationalize to 48:55 itself, hey, I'm going to write down my assumptions, look at my assumptions, and then go back and react to them. , that rubber ducking thing is, , again, I didn't really understand what it was when my developer told me about this. 49:05 what is what are you talking about? It's really you're in the bathtub and you have some ideas you want to talk out and you put the little rubber duck on the edge of the bathtub and you're , ", rubber duck, let 49:14 me talk let me let me just talk it out loud with you. Let me think through with you the idea." And usually the act of speaking things out loud, organizing your thoughts, trying to think about how you communicate something to another 49:25 person, in this case the rubber duck, it allows you to unpack, oh that doesn't make sense or I should probably rethink this or maybe I should phrase this differently it can do a better 49:35 job later on. this idea of rubber ducking is a really neat feature here as concept that's can be used with agents. Absolutely. . Very good. All . 49:46 yeah. , I've got loads of ideas how we can possibly proceed from here, but not sure we want to go. Yeah, next time. Next time, exactly. 49:57 Let's I think next time would be a great one. Let's land on some skills. Let's take what we've learned here and go back into this conversation. We'll maybe pump out a couple skills or a skill that's going to do more of what we want and 50:06 then we probably should leverage that skill, try again, and see if it can then update better. let's throw ray branch and maybe this is 50:16 another point or just an idea that we want to maybe lay out here as . A lot of times when we used to build code we tried to build the code every time because it took a lot of time to get 50:26 there to do the effort. What you saw happen just is while we were discussing and talking about other things the agent was automatically building and working on things. I think one of the again I'm 50:38 going back to this creator agent experience or this creator a agent idea if it's creating things it's not as expensive as we would think it wasn't a lot of time we don't have to be 50:48 committed we don't have to be invested in the code that's being built because we can just create another branch try again create another branch try again and the 50:58 act of doing this is refining the skill refining that prompt that way in the future we get better more accurate results results. , a lot of times I used to be writing with an agent 51:10 or building things with an agent. I get to a result and I'd be enamored with that result. I would continue talking to the agent trying to fix what it didn't do the first time. And 51:21 I'm finding more is I'm saying that prompt didn't work. The output it resolved didn't quite meet my needs. I'm going back to get a new branch and start over again. and I'm doing more start 51:32 overs because it's less risk for me to just do that increase the knowledge and the prompt and get a better output result and then I'm not spending I'll find if I continue iterating on I 51:43 don't this measure I don't that you're spending a lot more time getting to nitpicky all the little details as it as opposed to if you just wrote a better prompt up front it 51:53 would have given you much better output at the end. Is that a you resonate with this Matias as ? Totally. Totally . And yeah, I 52:04 think one thing we're really missing in terms of agentic development experiences is to make it much easier for you to do 52:16 parallel experimentation of sending the same prompt to different models or different coding agents and then making a choice which one you want to go with. ? This is something which would 52:27 require a fair bit of manual coordination and setup nowadays, but it's something I would really want to be a a a primary part of my DX my 52:39 developer experience when I work agentically. let's see if that happens or who comes up with a 52:49 tool that. But ultimately, you know, if you think about what do you need to be better, what do you need to produce better outputs here, it's that 53:01 stuff, ? don't feel too precious about the outputs you get from a particular prompt. That's a good way of putting it. 53:14 throw out the same prompt multiple times and then select which output you want to go with and then maybe feedback to your agent of 53:25 those three choices I number two why don't you reflect about that that moving forward better what my preferences are yes agree 53:35 what I would propose for next time we we keep a copy of this exact same prompt which arguably is is very small 53:45 and unspecific. we prepare that particular project a bit better with context and skills. We send 53:56 the same prompt and then we look what we get back. And I it because I think that that that continuous improvement cycle is precisely what you need to know nowadays 54:08 if you want to be a good agentic developer. I would totally agree with that. , this sounds a great already. We're teeing up a really good episode for next week. next week on Friday we'll do another one of these again. we'll do 54:19 this this elaborate we'll continue on our development cycle here. this is real practical learning for people who are building and using things agentically on top of PowerBI and fabric 54:29 already. we hope you enjoyed this episode. Matias, thanks for the demonstration today. This was awesome. Really what you're doing here. debug mode. Awesome. By the way, that's cool seeing that come out. How much 54:39 information is being generated behind the scenes. Super cool. Thank you all much and we'll see you next time. Bye.