chapters transcript notes
click any line to jump to that moment in the video
0:15 Hello everyone and welcome back to another Friday episode of a Agenetic thinking. With me I have my good friend Santosh from Microsoft. He's a PM inside the Microsoft ecosystem building 0:26 lots of a genetic tools and solutions for us to use every day in our agents. Santosh, welcome back to the show. I'm happy to have you back here. Let's 0:36 talk about these skills. What what are we building? What's coming out from Microsoft? I'm extremely excited about this. Thanks for having me on the show here. I hope you enjoyed build. 0:47 Yes, very much . Awesome. Nice to see you again and thanks for having me on the show. we're excited to share some updates. And yes, skills for fabric. 0:58 [clears throat] Skills for fabric is something that we as part of fabric team made it available for the open source community as part of FabCon. This was one of the 1:09 keynote announcements. And the excitement has been huge. The adoption has been really really interesting and vital. Lots of customer use cases. 1:19 the repository has been growing. More skills have been added. to introduce what skills are, think of it more 1:30 everyone is going to be start using agentic engineering. It could be any tool of your choice. Could be cloud, it could be GitHub CLI, it could be Codex, cursor or any what have you. 1:42 In this case, you don't want to start your data engineering project or your end-to-end analytics framework or a medallion architecture or 1:53 monitor your existing data stack through just a prompt. You You've got You've got to want to have all the best practices baked in, all the 2:04 authentication handled. . Everything should be governed and sandboxed. And that's all packaged as part of skills for Fabric. . It's It's It's available as a plugin. 2:15 All you have to do is just download the plugin. And as we keep shipping updates, Bogdan added this functionality as part of the initial build. , Bogdan 2:25 Christian started this project. I had the opportunity to collaborate and contribute towards the authoring and consumption for the data engineering 2:35 scenarios. And all the other teams have come together and they've contributed to all the workloads in Fabric. , we you have Event House, you have data 2:46 flows, you have SQL data warehouse, monitoring and operation management skills. You also have admin capabilities. , it 2:56 covers all the workloads inside Fabric. , you would need If you wanted to get started on an agentic engineering on Fabric, 3:08 all the tools and skills would be the best tool for you to start from any agentic CLI tool of your choice. let let me give you a little more context for some of our users who are 3:18 not quite familiar with skills. I We talk about skills a lot and maybe people have heard this before or other organizations talking about what skills are. , a lot of the conference at Build, I did 3:28 a couple table talks around specifically what is a harness? How does How do you identify what the harness is? Harness is software that wraps around the agent, the large language model. Skills are a 3:39 way of customizing and giving agents more information about how to identify and group and build better toolings around this. , , Santosh, you have a great slide up here. It's 3:49 talking about how how a skill can help an agent move through this. And I love this animation here because there's user intent. I want to build a certain thing. And in order for my user intent, my 4:00 question, what I want to create, a lake house, a test, a report, whatever the things are, Yes. you have all these different areas where you can start to step into and then skills are a a customization, a 4:12 plugin to help the large language model give you consistent, repeatable results, but skills are designed for your workflow, how you to work. It gives 4:23 you the ability to highly customize these harnesses with what you want. And Think of these as instructions, ? Exactly what you mentioned. 4:33 Sure. You have a bunch of agents and subagents to go work on tasks, work towards a goal, ? , 4:43 you want them to be grounded on all the intent, the harnesses and controls and context 4:53 that you would want to share. And you would also want to make sure that it is highly optimized Yes. and it is consistent and it's repeatable, ? you don't want to 5:04 keep repeating all of this as part of your prompt or every every message or every interaction that you have with your 5:14 agent. That's that's the whole intent where you package all of these things into a skill or a plugin . it that it scales across every 5:27 session that you have with your with your CLI. I want to I want to talk specifically, Sint how some of the plugins or the skills that you created directly, I believe. if I look back 5:38 here on some of the commit history on this repo. all these skills exist in a GitHub repo. I've already included the link here in the chat window as . It also is in the description. , it's 5:48 microsoft/skillsforfabric with hyphens in there, where you can go access all these skills and use them with your agents. And Santosh, you've got a ton of stuff in here. There's a 5:58 There's a a number of commits that were made when you were making commits to things. I see one here around let's see here. check updates. 6:08 End-to-end medallion architecture. Lakehouse event authoring. I saw a lot of , [laughter] were these made by you or the team? how how do these get in here? check updates 6:19 Bogdan added this. , the reason why we wanted this was think of it this way. , we keep pushing updates every every week. . Yeah. 6:29 It's changing all the time. the problem that every user has with skills is the skills get out outdated because the parent plugin could be 6:39 refreshed. , check updates what it does is it does a weekly check on your last session. it checks in if 6:49 your if the skills repository and the skill plugins are up to date. About to the report. And if not, it automatically refreshes that you get all the latest and greatest updates baked in. , 7:00 say for example, runtime 2.0 got released last week as part of build. and we have exciting new updates on native execution engine. , we are adding some best practices based on 7:11 that. All those things flow into the next update that gets published as part of skills. , that's how it works. how do you unpack this one a little bit, ? You're , you're saying you've made updates to the APIs for the 7:24 optimized runtime engine. And , you can provide skills to say, "Hey, look at this notebook. What could I do to to optimize this based on what the notebook is doing?" Cuz this is 7:34 This is verbatim going back to do I need to write these tables down to the lakehouse in a read or optimized format or not? Is my cluster set up correctly or not? Those are Those are 7:44 all business rules that because you're Microsoft. You've been building these things with customers. You understand what needs to be optimized and what doesn't. you're able to 7:54 package that in that knowledge into a skill and then we just run the skill. Hey, check out the skill. Tell me how to use it. Put it against my notebooks. Oh, this is really exciting. I this a lot. 8:04 stays stays updated. , say for example, I'll you give you an example, ? In runtime 2.0, there were tons of 8:14 updates that were shipped as part of price performance. And say for example, one key update is we have enabled something called as 8:24 an optimized scale down mechanism or we have made deletion vectors Sure. default or driver mode based snapshot acceleration default. , in those 8:35 cases, you don't have to your skill in the next update of skills, you will know these things. Your system will automatically fine-tuned and say it's going to be 8:45 switching to runtime 2.0. It has all this context. it's going to be able to make decisions based on that. If it's designing a pipeline, it's going to be able to make decisions based on that. that's the whole intent. And 8:56 just to point to this slide, that's the whole We just want to make it as easy for users that they can get these updates and they can have a 9:07 more seamless agenting experience because there's many releases happening. There's many updates that we are shipping. It's hard to keep track of. , all the users would have to do in this 9:17 case, they install the plugin. . Say I'm starting a a data engineering workflow. . It does Maybe I'm starting a new project. It could be a green field one or if I'm going to be 9:27 managing my existing one or optimizing an existing workspace. They just describe what they need in plain language, in natural language. And and they specify the scenario. They 9:39 don't even have to choose the skill. They could if they want to, but they they could just specify it as a a single shot blunt question. And the AI their model and their agent is 9:50 going to be able to choose which are the skills that it could chain for achieving this task, ? It breaks it down authenticates since it's going to Let's take an example of 10:01 GitHub CLI. That's been my favorite tool of choice. I I just specify this given that I'm already authenticated on Azure AD. It handles my login. It's going to 10:12 make sure that all the actions that it's doing it's within the access boundary that I have permissions on. And then it's going to be going and creating fabric resources, managing 10:23 them, monitoring them, or fine-tuning a job. , I could say, "Hey, this job is running slower." Or maybe it was running properly for the last three 10:33 three months. I see some issues in the last week. Can you go explore and tell me what is what has changed?" , I could just offload it. It's going to run as a background task and it's going to 10:43 give me a summary, ", this has been the problem. Do you want me to go fix it?" Maybe there's a data skew, ? That's how I've seen people use it. That's how I use it in some of the cases when I run into issues. And I think 10:56 many of our enterprise customers have been have been using this since we announced and it's been pretty exciting. Awesome. What else can we share about some skills here? You have a couple more 11:06 slides here I think for us. Yes, I can again I just want to unpack what it is and then I would want to show you a demo. , again the skills what you get out of them 11:16 first, you get all the enterprise fundamentals. It knows what a fabric what authentication is what Azure AD account that is going to be logging 11:27 or it's going to be using. It manages the secret scanning. There's no It's not going to have any one thing that as these agents are good at is on-demand code generation, 11:39 ? Yes. the skills we have made sure that it's super lightweight. the reason we wanted to be open source is that the community can contribute because I'm pretty sure you'll be running into many scenarios where 11:50 you're building solutions for the customers. if you find an interesting one, do do contribute and that it could be used for other customers as . 12:01 And the next one is the check updates one. This is this is a skill that Bogdan had added. Yeah. it automatically keeps updating. And the next one is it's going to be tool agnostic, ? 12:11 It's going to you're going to be able to use it in any tool of your choice. it could be VS Code, it could be WinSCP, it could be Cloud. and all you have to do is just do a plug-in update 12:21 and update the skills files. That's it. Awesome. Amazing. in terms of collection, I think we briefly touched upon this. . It has SQL DW authoring. it's 12:32 broken down in authoring and consumption. . Spark authoring is SQL DW authoring. it it knows and it chooses based on the intent and the scenarios that you 12:43 describe, but you also have the choice you can choose a specific skill that you want to assign a task to. And it has agents in terms of check updates, end-to-end medallion architecture. you have a data 12:54 engineering agent, you have an admin agent. There are there's also a new agent that has been added. It's for app dev. And there are tons of other capabilities in terms of 13:06 skills within these in terms of say monitoring. and I got a chance to also add a few migration skills. And Jenny on my 13:18 team has added a lot more resources which makes and expands it to complex migration scenarios. there's been a ton of excitement around that as . 13:30 I would I would strongly recommend to recommend you and the community to try this out. And And if you run into issues, I would say for example, say for example you find a 13:41 bug or a or a gap in a or you're you're saying, "Hey, you need a specific feature area that needs to be covered." Do raise an issue on the repo. 13:51 The team will monitor Sure. and we I monitor, and we we hope to ship those updates pretty soon. Or feel free to contribute as . We want you all to contribute to 14:01 this. I'm going to put here in the chat window as . If you have an idea about a skill, ? I'll I'll keep put the issues page here as . And as I go back to the skills themselves, again 14:12 looking through the list of skills that you have here, there's some that I'm going to call out here directly because I can, and this is our show. We can do every we want here. Santosh, I've got a great one here. There's one called Databricks migration. 14:23 And if you read the description, it's taking notebooks that you would have had in Databricks and migrated them over to Fabric and running them over there. There's another skill about migrations 14:34 around a Synapse migration. , you have Synapse, and you want to modernize your Synapse environment, bring the pipeline over, bring the notebooks over to Fabric, the Synapse migration will 14:44 look at this. And this one is quite interesting because from what I gather in my initial runs on the Synapse migration, it's going to even do some translations 14:54 for you. It's going to help you migrate the notebook. There's the MS Spark utilities translating that into a proper namespace that would let you use it 15:04 inside Fabric. again, there's some syntax differences between how Synapse runs and how Fabric runs. Exactly. Yes. And this skill is going to help identify 15:14 what those differences are, replace them, fix them, and help you migrate. Awesome. Very, very nice. , these are great skills. I love all these other 15:25 items here. And there's 16:21 I do have my agents, but I want to make sure that I see good results. , Yeah. 16:46 is. I know you you've been using this, but would love for your community and also your viewers to test it out and share some more feedback. Yeah, go ahead. Let's Let's see a quick 16:56 demo of what you're going to use for some skills. And do let me know if you're if you're able to audio. At Zava, moving from manual 17:06 data handling to an AI-native customer analytics platform. Today, I'm using the new skills from Microsoft Fabric open-source plugin in 17:24 First first let's look at the Santosh, it's a little hard to hear. Those skills, they're coming in. It's the audio video is really slow. 17:35 It allow co- co- [snorts] [sighs] Hey Santosh, I think we're losing a 17:45 little bit more of our connection here. it's getting a little bit delayed here on our side. Must be a a weak connection. I'm going to cut it there, Santosh. we're going to come back maybe have to load the video 17:55 separately, but it doesn't look it's coming through too clear across the video screen share. 18:06 All , I think we've lost the connection here. I'm you to pull Santosh off of here. I'm going to say that is a quick summary of skills on top of Microsoft Fabric. You want to go check out the skills for 18:16 Fabric repo. the idea those skills are there for you. You can then see them directly inside the the skills area. The link for the Microsoft skills repo 18:26 is down here in the description as . make sure you go check out that link for the Microsoft Fabric skills on GitHub. The issues is also listed there 18:36 as . if you don't mind, you can also go check out those skills directly. And if you have any issues or ideas or things you want to build that are not new, make sure you go make 18:46 a comment on the issues area. Let Santosh and the team from Microsoft know what you want to improve, how you want to make it better. That being said, thank you all very much for taking us through an adventure 18:58 around here from Fabric skills. We hope you've enjoyed some of these skills, and we really appreciate Santosh for jumping in and doing a a demo and talking a little bit about the the layout of 19:09 skills and how they work. We're going to continue to explore more of these topics around skills directly on the Agentic Thinking podcast. , stay tuned. Next week, we're going to keep going further 19:19 with using skills and other Agentic experiences directly inside Microsoft Fabric and inside VS Code. Thank you all much, and we appreciate you, and we'll see you next time.