FABRIK Frenzy Good afternoon and welcome to FABRIK Frenzy. With me today we got our great and experienced colleague within the Power Platform, data platform and everything, Morten Søndergaard. My name is Oliver, I'm commercial responsible for our BI division in FellowMindDenmark. Morten, welcome to his, Morten is joining us from our Funen office from Odense. So welcome Morten. Do you read me? Perfect. And hi and hi to everyone watching. And thank you for having me on the FABRIK Frenzy broadcast. Perfect. I think we might have a connection problem, but I, let's try again here. At least we can hear you. So that's the most important thing, I think. So, yeah, if you could, I don't know if you're able to share your screen, because I think that Morten would like to show us the latest update about the Microsoft FABRIK, right? Yeah, exactly. Are you not seeing my screen or is it... We do see your screen, so everything is good. It's not in presentation mode, but I see the slides and it's perfectly, very beautiful. Alright, very good. So, yeah, at this broadcast I'm gonna present the newest updates from Microsoft FABRIK. And last week Microsoft build took place and a lot of new announcements from FABRIK was announced. The theme was pretty much a lot of AI agents, something called the agentic web. And I'm gonna talk a little bit about some enhancements to co-pilot. But other than that, I'm not gonna touch that much into the AI part. So I'm gonna present a few of the news that I think is exciting. Some of them brand new preview features. Some of them enhancements to existing features. And then we're gonna deep dive into one of them. So first off, I'm gonna talk about co-pilot enhanced capabilities. And what I mean with this is just that now co-pilot is available in almost every fabric experience. So co-pilot is already an integral part of everyday life as a developer. But the new thing is now that you can use co-pilot within, for example, a notebook or a data flow. And co-pilot will have your context as part of its answers. And exactly that makes your life much more easy when developing using co-pilot. Then we have materialized Lakeview, which is a public preview feature. As of yesterday, and also I checked this morning, it was still not in preview. So everything we're gonna talk about regarding this is what I could read from Microsoft's homepage and the examples they have provided. But it looks very promising. You can define your business logic. You can define data quality constraints, rules. You can do orchestration and monitoring all in the same place, which is the materialized Lakeview. Then I'm gonna talk a little bit about some shortcut enhancements and it's called shortcut transformations. Where you can in your shortcut, you can use AI to do some transformations of your data. like sentiment analysis or summarizing text documents. And you can also use the UI to turn your CSV files in the file part of your Lakehouse into Delta tables without writing any code. Or doing anything other than creating the shortcut. And lastly, I'm gonna talk about and present a demo of the translitical task flow as Microsoft has named it. And this enables us to do write back from power BI. And a lot of other cool features because behind the scenes of a translitical task flow is actually. also a pretty new feature called a user data function. So with regards to this, a write back function from power BI to the various sources. And would you elaborate a bit on that? Is that based on our targeted. budgeted budgeting forecasting or when would you think that would be relevant? Yeah, definitely. It would be highly relevant in any kind of planning situations like forecasting budgeting. Yeah, if you are doing time registration or. or any, any scenario where you would need to write back to your data source. Um, but especially yeah, in, in financial planning, it would be very interesting. Now that, uh, we can use this translitical task flows. Okay. That's interesting. It's, I think it's going to be very beneficial for a lot of, uh, different customers, not only the, the, the larger ones. So, uh, interesting. Yeah. Uh, yeah. And then I would like to say, or. Uh, I would like to, to show that there's a new fabric roadmap tool, so you can actually now go to the fabric roadmap tool and see which new features are planned for, for preview and, uh, which features that all have already been released for, for trial. Um, so I definitely encourage everybody watching to, to look at this. Uh, there is a lot of other new features which I'm not talking about today, but they are visible in this roadmap tool. All right. So let's talk a little bit about co pilot. So now co pilot is pretty much everywhere. Uh, and in a fabric perspective, we can use co pilot inline in notebooks. You can use it for developing data flows. You can use it in data factory experience for understanding complex, uh, pipelines. Uh, and co pilot is, is really nice. And especially when you can use it or in it, it understands your context, because one of the pains, at least for me, when I use co pilot, uh, is that I have to explain. The context for it for co pilot to give me a good answer. Um, and when co pilot is already embedded or you, um, accessible, for example, in a notebook, it will have the context of your lake house and your columns and your data and the primary keys and all of that. Uh, and then it will be able to give you a much more qualified answer. And this help us, uh, not lose momentum when we are coding, uh, because we have to, you know, give a long elaborate, uh, uh, writing to to co pilot for it to give a qualified answer. And now the copilot is actually available in all, in all fabric skews, not just the F64 and upwards. everybody can try this out, but a word of caution, of course, because using co pilot does consume capacity. Um, and if you are on a small capacity, uh, I think you could burn your capacity quite, quite fast. If, especially if you're more than one person using this feature. So you should have a clear strategy before just. Enabling it in the admin portal of fabric. All right, but let's, um, let's see it in action. How copilot can help us when we are developing notebooks, data flows pipelines, and how this context awareness actually works. So I have present, uh, prepare a little demo here where I am in my notebook and I'm turning off. Copilot inline completions. Starting up a session. And now I'm telling copilot to refresh its context. And then it will have my lake house with all the tables, uh, and all the columns in its context. And, uh, then I can start in natural language telling copilot what I want. So here I'm just telling it to, to group some, uh, some data from multiple tables. And it will give me a query. And. It figures out on its own, which columns I need, which, uh, keys it needs to do the giant. I don't have to tell it that because it has it in its context. And I can fire off the query, see that it produces. And then I can choose to accept or discard the query. And of course, every time you use copilot, not just, uh, in fabric, but everywhere you should, uh, look. You should sanity check the code because sometimes it doesn't give you the exact thing you want. For example, here, it gives me a limit 10 in my, uh, select query, but I want to save everything to my lake house. So this query is, uh, producing a table and saving it back to my lake house. So we can just take it for good. Uh, every time we presented a result from copilot. Jumping to a data flow, we can do the same thing. We can use copilot to, uh, to summarize or describe a data flow. I know for myself that especially when I am. Uh, handed the data flow that I haven't constructed myself. It can be pretty difficult to, uh, digest what is actually happening, but now we can use copilot to, to do the heavy work for you. And you can also use it for example, for documenting your solution. And this is a pretty, this is also a new feature. You can, you can use copilot to create a custom column. So if you don't like writing in code, now copilot will just do it for you. Um, and it's all about, you know, you know what you want to do. Maybe you don't know what, how to do it. Uh, so you can ask copilot for a little bit of help. And lastly, in a, uh, in a data factory pipeline. Again, you can use copilot to summarize the pipeline. Could also be nice if it's a more complex pipeline than this. You can also use it to. To add, uh, pipeline activities to edit pipeline pipeline activities. Um, and just in, in general, make your life much, much easier. Yeah. So this is, this is a copilot context awareness in fabric in almost all fabric, um, experiences and artifacts. Um, but again, word of caution, it does use, uh, capacity. So you need to be aware of that. That's very interesting and kind of a groundbreaking, so to speak. I mean, with regards to this, uh, capacity use, are you able to make some sort of an estimation before you start out? Um, before you start, um, yeah, using copilot. I mean, it could be great for, I think a lot of, uh, customers to know more about the, the potential use of, uh, capacity. So, um, well, I, there might be some price, some calculators out there, which can calculate, uh, an estimate. Uh, but other than that, I mean, um, the price of using copilot is actually, uh, published on Microsoft homepage. So I think it's something like per 750 words, you are priced X amount of capacity, um, both for the, for the input prompt and the output, uh, response. So yeah, you might be able to estimate how this would impact your, your capacity. In other words, would you, with the customer be able to make some sort of a calculation telling the customer if their current capacity is sufficient in the future using copilot or how would you approach it? Yeah, that would be possible. And, and if you can't make a calculation based on, uh, the price of using copilot, you can always make a small, uh, POC or a test to see, okay, now we use copilot for one day. How much of our capacity have we, uh, consumed? Is this a representative of how much we're gonna use it and then take, uh, take a decision based on that. Perfect. Interesting. All right. Jumping to the next preview feature, the materialized Lake view. Um, so this materialized Lake view lets you build a declarative data pipeline using spark SQL. So it's, it's pretty much like defining a, a view in a ordinary SQL statement. Um, but now there's also built in support for data quality rules. Um, so this is a, the materialized big Lake view comes with automatic monitoring of your view. And it. Also generates a report of, um, how your view has refreshed and updated. And if there has been any violation of your data quality rules. Um, and it's, it's intended for, for orchestrating queries across your source tables and other materialized views. Uh, and essentially to make. Um, and I think that's the multi-stage Lake house architecture much more, um, a much less complex. Um, and much less complex because the materialized Lake view can handle all the orchestration and monitoring and. and all of that, that we had to build around, uh, our business logic, uh, so that developers can focus on business logic. Um. Um. Yeah. Yeah. And if you use the materialized Lake view in all the, uh, stages in your Lake house architecture. So you have materialized Lake views all through Bronx silver and, and the gold stage. Um, then fabric will figure out itself the dependencies between these views. and views so when new data is arriving, it will schedule or it will refresh the, uh, the views in the correct order. Um. Also the view, uh, is utilizing the dental exchange data feed, so it will only. Update new or change data and it will also only refresh if new data is available. Um. Yeah. And you're able to define these data quality constraints. constraints within your view. Uh, so you can define, for example, what should happen if, uh, one of my constraints, uh, is violated. For example, if I get null values where I didn't suspect it or negative values for a column that does not, it's not meant to have negative values. Should it drop the record? Should it just flag the record? Should it fail the update? Um, and you get all of these results, uh, from your materialized Lake view. refreshes in a lock where you can see exactly which violation or rich data quality constraint was violated and how many rows were processed and all of that. Um. So it comes with a lot of features out of the box. I would say a lot of things that we would otherwise have to, to code ourselves. Um, and in the future, uh, there will also be support for pie spark and incremental refreshes. and this is then the. The quality report that is automatically created for you when you use the materialized Lake view. And, and as I said, it hasn't been released for public preview yet, at least not when I checked last time, but, uh, I'm pretty excited to try out the feature because I think that it looks, it looks really promising. Um, perhaps we can. Uh, we can, uh, we can, uh, set up the, uh, set up, uh, pipelines and data data platforms much easier using these materialized Lake views. Uh, and especially when we also use the new shortcut transformation feature. So. Shortcuts in, in one Lake, uh, uh, is a nice feature. So shortcuts for those who are not aware of it, uh, objects in one Lake that points to other storage locations, and that can be other, uh, artifacts in one Lake, but it can also be external data locations like, uh, Azure data Lake. Um. Um, and the key thing about it is that you are not copying data to one Lake. You're merely just creating a link to that data. Um, and now with the shortcut transformations, you can also transform that data as it is. Well, not being copied, but in the link between the data source, uh, and, and the target. Right. So you can use, um, AI to do some sentiment analysis, or you can summarize your documents and data, and you can also use the, um, the UI in the, in the Lake house. Um, to create a shortcut from your table section to your file section. And then you will then, uh, fabric, um, will figure out, okay, you have CSV files in your file section. Would you like to convert those into Delta tables? Um, and then it does it for you. Um, and then. Every time there is a new CSV file. Available in your file section, they will automatically be part of the Delta table. Right. So when new files arrive, your Delta table is, is, uh, updated. And if you have these materialized views, which are constantly looking at your Delta table, then they will also be updated when new data arrives. So. This could potentially mean that, uh, you with a very little effort could have, uh, could have scheduling or orchestration of your data pipeline without even setting up, uh, triggers or schedule refreshes and stuff like that. Um, yeah, and you can see again out of the box, you can see logs of how many rows are written and, and you errors, warnings, et cetera. All right. The last thing I want to talk about and go a little bit deeper into is the translitical task flow. Because have you ever wanted to write back from power BI to your data source? Well, yes. Uh, I hear a lot of customers asking for this functionality. Uh, the only thing they ask for more is how can we get this data in Excel? Um, and it was actually also possible before translitical task flows to write back from power BI to your data source. You could, for example, use a power app or you could buy a third party tool, which enabled this, but now we have a much more native way of doing it without having to pay power apps licenses or buying a third party tool. Right. And it's called translitical task flows. They use behind the scenes, something called user data functions, which is also a new feature in fabric. Um, and these user data functions is actually what is doing the heavy lifting. We just, we just, uh, in our reports, we actually just creating a button button that fires off this data function. And, uh, if you want to try it out, it's something you have to turn on under the preview features in your power BI desktop. And you also have to enable user data functions in the admin portal, but let's take a look at, uh, how these translitical task flows actually work. So I am gonna go to, and I'm just checking. Can you still see my screen? Can you see that I changed to, and I trade and I changed to, um, my fabric frenzy work space in fabric. No, we, we do not see anything. You do not see that. All right. Two seconds. Let me just share this window. What about now? Yeah. Perfect. Great. All right. So I have prepared a power BI report. And as you mentioned at the beginning of the broadcast, Oliver, when I mentioned translitical, twice flow and right back, um, one of the key, uh, areas when you can use this is actually financial planning and budgeting. And, uh, in this case, we are going to do some budgeting. So in this report, we have some departments and some budgets, and we need to, to, to revise or edit those budgets and even do some commenting on each of the records. So in my report, I have a, uh, I have two user data functions attached to buttons and those user data functions need some input. And the input they need are the, the filter context from my report. Um, so what I want to do is that I want to. Do some budgeting for the customer service. I want to change the budget in the budget category salaries for February, right? So now I fill it down to this exact record here in my database. Then. I can choose to update the comment. So. I, I would most likely do that after I change the budget amount and I can choose to, to change the budget. So let's first change the budget amount. I'm just going to write 5,000. and click this button, submit budget, submit budget. And when I click this button, it's going to fire off the user data function in the background. And. Yeah, it says request submitted the action completed successfully. All right. Let's also update the comment. So. Uh, we can say. We are. Getting a student worker. In February. And that's why they need to increase the budget. That and submit the comment. All right. And this report is actually in direct query. So. If everything was working correctly, we should see that the budget amount changed and the comment changed. I did have some issues also. Uh, playing with this beforehand. I suspect that it might be the sequel endpoint that is a bit slow, but it doesn't matter. We can always just go to the sequel database and. See it for ourself. Right. So. Before I forget. So it's the customer service salaries. Right. So let's just check that it actually did write back to our, to our database. So it's updating the budget and I'm just gonna have to find out which department key. key. The customer service has its nine. Right. So we can say where. Department key is equal to nine. And. We also have the budget. Post is equal to. And I already forgot it. Uh, it is salaries of course. Yeah. Salaries. Right. So here you can see the amount changed to 5000 and the comment we're getting a student. Uh, blah, blah, blah. Yeah. And, um, for some reason, the, the direct query isn't working, but, um, it is writing back to our SQL database. And what is actually enabling this is these user data function. Let me just show you quickly how it looks. It's. Uh, two functions written in Python and the connection to the SQL database is managed by the function itself. So I don't have to have any connection strings, uh, any authentication keys or tokens, anything like that. It's all handled. By the function itself. Um, so I really think that this translitical task flow is, it's gonna change some things. I mean, it was possible beforehand to, uh, to, uh, to do right back from power BI with some hacks, I would say, but now it's much more native and, um, yeah, it opens up a lot of possibilities. That is, that's truly amazing. I mean, uh, with regards to the, um, to the other. And updates, I think this translitical task flow is, is what surprised me the most. I would say it's, uh, it's very interesting. I think that the, the, the market is gonna react quite positively, um, to this. So, uh, uh, we are running almost out of time, Morten, and I think that your camera might be, uh, taking a day off, but, um, I was wondering what, what did you see as the most, um, um, extreme new, uh, feature that has, um, surprised you the most? Well, I would say it's definitely these, uh, this translitical task flow and the user data functions, because it's almost, it's almost limitless what you can do with it. Because you're actually just programming in Python and it's, you're not limited to just writing back to your data source. You can also incorporate, um, approval flows, or you can, uh, you can do all sorts of things because you can just code it. But, but the cool thing is that it's now also native in power BI to fire off these functions, just using a button. Uh, so that's definitely what surprised me the most. And, well, it's just something that has been, uh, has been, uh, cried about in the market for so long. So now we finally have a, a great solution for writing back. I definitely agree. So, uh, as we are about to run out of time, Morten, I would like to thank you so much for your presentation and the demos. Uh, that was, uh, that was, uh, very interesting. So, um, yeah, besides from that, I would like to say, uh, thank you to, uh, to all the listeners and, um, yeah, have a nice evening. And thank you so much, Morten. Goodbye. Thank you. Goodbye. Bye. Bye.