hi and welcome to fabric frenzy the October version where we'll talk about all the new things that have happened in Microsoft Fabric and in powerbi since well last time last month so at feline we'll do we're doing this uh this fabric freny once per month where we talk about news we Spotlight into specific features and sometimes more than that and today we have what I believe is a very interesting Spotlight because we're looking into the new experience in fabric we're looking into Data activator which was released for Public public preview here in October so for anyone new tuning in who may or may not know exactly what kind of size fabric is fabric is a culmination a combination of all the tools we already know and love from Microsoft in one platform so we have data Factory we have the synapse capabilities we have powerbi and then finally one L as the um one tool that that unites the rest and now we also have finally the data activator experience so in fabric we are able to do every everything from getting our data to Preparing that data in our tools to using that data and storing that data so it's really a full stack data analytics but also data engineering experience where we can do every everything around data in one place it's very nice so what's in the back of news for this month so in October we now have the data activator we got the release of the fabric road map we got the online data modeling editing cap capabilities in powerbi the new and fresh look for the homepage and icons in the powerbi service and finally the release of the semantic link what all of these are and how we can use them is what we will spend the next few minutes about so let's start with the first one we got a new user interface so a fresh new pack of icons um with different colors that has kind of a different meaning so you can have you have something like yellow uh represent ing reporting have something like magenta red representing real time but also the usage of data in real time has some blue which is uh storage and and purple again uh uh storage and data sets and such and green Compu and transformation so Microsoft didn't actually release a proper explanation of what these colors meant but saying that they put them into natural uh um categories and then we can decide for ourself what exactly they're supposed to mean but um it's really cool I mean they removed colors people were not happy about that and then reintroduced them in a new way so well that there's that but besides that they've also uh re redesigned the main page when you open powerbi so as you can see here on the left side of the screen um I'll just make it a bit larger here there is a new page with a homepage you can open reports there's a list and quick access to foral reports um what they actually have done here is aligned the experience much more with what we already see with other Microsoft Tools in an office so this will be much similar to what we see in Word Excel office and so forth so whether we like this or not I don't know if that really matters the the key is here that for the end users using the program um everything should feel much more familiar going forward then we also have another news we have online data modeling editing capability updates so we could already do some editing in the uh online data models but now we can do a little bit more we can configure and edit the relationships of our models you can see there's a whole new list where we can get a quick overview of all the different relationships using some Sleek icons to to make it very clear very fast what kind of relationships they are so we can clearly what is one to one one to many relationships many to many relationships as well as the filtering direction of those relationships so it's a very neat overview of our model we also have the possibility here to add filters to specific um specific relationships and of course we can also edit them change them within the model so we now have a possibility to change our relationships directly in our data model online without having to open the powerbi desktop cool Edition then we also have the semantic link for powerbi uh in Python and Spark so with the semantic link we can now connect to our powerbi data sets we can query explore and read our data and we can read our measures directly from within those data sets in our code so that means if we're programming in Python we can actually use this new Senpai uh python Library which will let us connect to a data set and fetch data from it even from the measures within that data set so I think this is a really cool Edition that will help us with uh probably a bunch of new use cases even even uh a lot of them that I don't have the imagination to to come up with right now but some of the use cases I've seen so far and and had considered so far is of course being able to query and explore the data of powerbi data sets and measures could help by enabling applications to use this data um right out of out of the box we could also use uh use it to validate the data in our powerbi and do this programmatically from within python or even from a um a a spark notebook in fabric we could also utilize this to do some sort of live write back to powerbi data sets so imagine that we're quiring the data set we find out some new insights uh some something we want to enrich our data with we could technically now write this back into our lake house and if we're using direct Lake Connection in our data set that data would be reflected on would be updated in the data set immediately so technically we can now read data from our data set and write changes back into the same data set all within python code uh so very Dynamic and very programmatic and finally we can automatically detect relationships we can join the data we can get a 360 overview of our data set from code which could help us with documenting our our um data sets but could also help us with detecting data quality issues so we could do things like query Dimensions query facts making sure that when we're grouping by those Dimensions that the values are within some certain range or so forth the possibilities are endless is basically what I'm trying to say um and to sum it up we can now use our powerbi data set in Python code really neat very flexible feature then we also have a bunch of non-released features but a new released road map for Microsoft fabric so in October Microsoft released the road map for fabric which had uh news and road maps for each individual experience so we have more than we'll be able to cover in this session today but luckily this is all this is just a road map so the features are not here so we will be able to cover them going forward whenever they are released but just to sum up some of them we have uh news within admin governance one Lake uh powerbi synapse and data Factory and each of these subcategories will have it its own road map with so many new features so if we start with the admin governance one Lake and data warehousing there are a few interesting um features here I would like to point out um there are there is a lot and and there's yeah of course more than we can cover but around all of them there are two things that that seems to be in most of the experiences new features one is git integration and another is is integration with the deployment pipelines and for me this is one of the notable uh notable releases to keep track of when we will finally be able to develop our full solution in Fabric and have it version controlled in git but also be be able to use the deployment pipeline uh to make sure that we have a proper segregation of our development test and production environment and when that has been enabled by all the experience experiences is for me the time where fabric can really shine and become um a unified data platform besides from that we also have if you look at the list we also have have um API support for a lot of them and we have specifically for um one leg we have the security model that is incoming very very very important feature but it's set for second quarter 2024 um so I'm looking forward to that but but if we're looking at very SEC setups with with high secure security requirements right now maybe um it's better to push off those plan for 2024 and finally for the for the warehousing there's a very uh interesting feature here which will let us save our SQL queries our SQL queries as views or even make the result of the query turn into a table so that is something that I'm very interested to see how will be implemented and it sounds like a a very um neat shortcut for analytical uh data analyst to create actual p lines based on just being able to do SQL queries more than that we also have news on the data engineering and the data and science part there are so many here that is worth mentioning that I will have to pick out just some of them um schema support for Lakehouse should make us able to do better categorization of our objects co-pilot integration and notebooks of course will let us become so much quicker at developing and writing code in in um in notebooks but also in particular notebooks in app is something that I'm looking forward to being able to not only have powerbi elements in our apps and fabric but also be able to now add notebooks for um bunch of use cases and finally we had also the semantic link We al already talked about that but we also have data rangler improvements to the data science experience finally we have also so many news for data Factory um I in particular am very excited to hear about some of the features that are coming to the data flow Gen 2 which is the easierto use version of data ingestion in fabric where we will see support for incremental refresh we will see a fast copy support it's a it's a way to actually copy data similar to what we do in pipelines but with the data flow UI and we will see enhancements to how we out put the data as well as git integration so having all that in first quarter of 2024 is something that Mak makes me very very excited about the future and fabric and then finally for realtime analytics the longest list of all features um and there is a lot well we have things like co-pilot integration we will have uh event stream now being able to to to put its data directly into Data activator we have integration with notebooks so we can use our real-time data directly in our notebooks as well as getting Delta support for the kql database meaning that realtime analytics will suddenly be part of the whole unified data format that is the pet and Delta format used in all the other experiences of fabric so it seems like we have a really good list here and when we have all these capabilities fabric is uh much much more mature and most of them are if you noticed are scheduled for this quarter so fourth quarter of of this year and and the rest is and yeah and the rest of them is scheduled for first quarter next year and second quarter the year after of course there may come new features that are not on the list yet for the years going forward but those are what we can look forward to right now and one of them was the data activator being able to create alerts and um action based on our data so speaking of data activator let's look into our Spotlight of the day which is data activator so with data activator we should be able to we will be able to use our data in a completely new way so in powerbi we use our data by looking at it by analyzing it by studying it by learning from it data activator is something completely different data activator will let us take an action on that data and it will let us do that in real time so the moment something is wrong or something is fantastic or something that we should actually do something about we can trigger an action and it does it with a completely new UI it does it as a completely new experience in fabric using the data that we already have in fabric so we can connect it to a powerbi report and use uh a specific data point from there we can have it connect to our realtime streaming uh pipelines from the realtime analytics experience and use the data from there and in in in in this way it will become the the the tool that lets us energize and use our data much faster and and much more systematically than we may otherwise have been able to so let's let's dig into it let's see how it works so on the surface dates activator uh and we we will also go through this in a demo so if this screen is a little bit small don't worry you will see how this works we will be able to select our data and View at a glance how this is moving over time we will be able to detect patterns in that data and find those outliers that we want to base our actions on and finally we'll be able to set up an action so we can act on the data we can currently we can send emails we can send teams messages and I believe we can also do some power automate uh triggers here but let's let's look into this in uh in the service and finally if we set all this up the result of this will now be that I can trigger a message in this case an email to be sent to a recipient about one of our data points suddenly not living up to one of our requirements so we can take very quick and very fast action on uh on our data let's take a look wait uh yeah so this is the fabric user interface I just went here and created a new data Act activate the object so this object here is called a reflex I then connected this to some real time demo data here this is a simulation of some sort of package delivery service where we have uh packages being shipped we have packages being um let's zoom out a bit here so you can see the actual data oh there you go so you have here packages being shipped packages in transer and packages being delivered and for each of them there are a bunch of uh data points here so what I can do is I can pick these data points and make some analysis on it so I can go here and say well I'm interested in packages in transit so let's imagine I want to uh I want to study and find out how we can um study and find out how we can sure that our packages which has maybe Seafood in it does not go above a certain temperature range this is very very um near it's very real time use case application I mean making sure that food doesn't spoil or go bad or become a health risk so the first thing I need to do is I need to pick my object so in this case the unique identifier of my D data that I want to track and in this case this are this these are my packages so my package ID is kind of my key here I want to create an object based on that I can click it and click create an object I can also go here and say I want to assign this data to a new object and do this more manually so I could say I want to create a object here called the package and I want to define the key column to be the package ID and could even assign some properties to it right away if I want to do that and this case I'm interested in the Special Care property and the temperature the Special Care is the property that defines if this is seafood or not and temperature is the temperature over time I can then save and here and go to the design mode and you see for me it already created the actual event here which is the package being moved in transit in in real time so we can see each and every little piece of data point here up in the graph so every now and then we get a new timestamp of something new happened to this package I can expand this to show more than 24 hours if I want to do that or less um and then it created automatically for me also these properties which are like uh Dimensions so we have here the temperature property and the Special Care property and that's fine now I have the data that I want to work with but I want to set up now an actual rules so that we can start acting on the data I do this by creating a new trigger and in the new trigger menu there will three new views appears so we have the select view we have the detect view beneath and then finally the ACT View and we need to go through all of these three steps to set up our um or data activator reflects workflow first of all I pick what's interesting to me I can pick here an existing property Special Care temperature that's not the object I care about I care about about the package and Transit um actually I care about the temperature so I could also pick this from the property and as you can see here we actually can track now some of these objects uh and we can track how the temperature moves over time so we can see for some of them when they start going in transit the temperature will naturally rise makes sense they've been cooled down and it will go up and up and up to a certain limit that's cool but maybe we have a limit up here where they cannot go above a certain point this is what I do in detection so I can now say I don't want this um I don't want this temperature to ever become greater than let's just say 15 130 and that gives us a few bullet points here and that's cool but but this is for all packages I actually also need to add a filter here for Special Care it doesn't need to be a number it needs to be here it needs to be equal seafood and now it's filtering to only packages with seafood and you can see here there is one package which consistently has been over this value for a few data points now this view can be a little bit confusing but it's actually split up into two parts so the first part will show show you uh which of your data points that will be triggered or that will satisfy this criteria uh but it's only from our chosen population it's only from our chosen example segments so up here we actually can choose how many examples we want to want to view at the same time so I could expand this list you can see up here if I expand it with a few more it will actually update and add more graphs more more data points to my um to my overview here same thing happens down here and you see suddenly there's a another one popping up with triggers that it's above this certain temperature range that's nice if I know which kind of samples that I want to use at as my Baseline so let's let's say we actually had a case in our company where some delivery was spoiled because it had too high of a temperature we can actually go and find this delivery in our data then and use it as our um example uh data in here and then I could analyze it in here and find okay but it actually went over this limit a lot of times so let's set a rule around this time so we can both explore the data and find out where our limits and uh and detection criteria should be based on and we can also set them up and see then the result of of doing so so that was the first View and the second view in the bottom isn't limited to these example data points or the example packages so those are for all our data points so this is the actual result of if we set up this Rule now it is telling us that we will get around four warnings per hour going forward maybe that is a bit much maybe I'm not interested in that maybe I'm more interested in in getting a warning if it's been above this uh this limit for more than uh three times in a day or three times in a few hours something like that well we can go and limit this and now hopefully get a lot that was a lot fewer data points so maybe just two oh well that didn't work so we'll go back to triggering this each time but that's also fine we have a capability here to to dis distinguish between do we wanted to trigger every time time or do we want to detect a pattern and then trigger at a given interval so we could have could have done that but apparently this data didn't have any any double double triggers once we set up this and we are aware of how many triggers we're going to spawn by doing this we can finally go up and set up the action what should happen all these times per hour when this happens we could do something like an email we could set up a teams message and there's a custom action here which is grade out which will let us do maybe something else so for now we could set up an email we can trigger who needs to receive this what should be the subject the headline optional message could be test or something like that could even include some of the data from our actual um from our actual data down here so maybe I want to include that it's Seafood I want to include the temperature I can do that and then finally I can save this probably should rename it to something like seafood not C Food exceeding uh 130° something like that I now have a trigger I have some properties and I have my event I can now save this it's already saved and click Start and will actually start running so I'll I'll I'll skip clicking this button to to prevent myself from getting spammed with all these triggers but I can click on the send me a test alert and it will also without starting it without setting it all in motion you can actually click and see how it looks and by then it will successfully execute this test action uh based on the available data cool so that was that was Data activator um I think this is one of the very interesting additions to Microsoft Fabric and the whole realtime analytics part of Microsoft Fabric and I think going moving forward we will have an easier time identifying patterns in real-time data but also generating actions based on it using this tool in data activator that was what I had for you today I hope you enjoyed it learned something are happy and Keen to tune in again next month and uh besides that I just wish you a good rest of October and November and see you again in November thank