Alle Broadcasts
Fabric Frenzy #15: Opdateringer fra FabCon i Las Vegas
59 views
Unleash the boundless potential of Microsoft Fabric: Your monthly source for cutting-edge news and limitless opportunities.
There is a wealth of new opportunities for data analysis and insight in Microsoft Fabric, and new features are constantly being added making it quite a challenge to stay updated and stay organized.
We want you to be fully updated with the coolest options in Microsoft Fabric. That's why once a month we give you an update on the new features and tips and tricks on how to take full advantage of Microsoft Fabric.
Register for the upcoming sessions. You are also very welcome to send us suggestions for features and functions that we should take a closer look at and take up in the next Fabric Frenzy session.
Agenda
Overview of the most important new features in Microsoft Fabric
Tips and tricks to make better use of new as well as old features
Concrete examples of cool business applications
View transcript
Good afternoon and welcome to Fabrik Frenzi, our monthly broadcast about the news and updates from Fabrik. And well, as you can see, I'm not Matthias Halker. Matthias is not able to join todays due to his parental leave. But today we are very lucky that we actually got our co-worker from Finland, Juho, who's able to give us the latest updates. from Fabcon in Las Vegas. And as a truly connected company, I'm very honored and I'm very pleased and glad that you're able to join us, Juho. So welcome, Juho, from Finland. Thank you. Thank you very much, Oliver, for your kind works. I'm Juho Jääskeläinen, I'm from Feloman Finland, working as a fabric product owner here and a senior consultant. So should we dive in, you think? Yeah, I think so. Okay, so this time in Las Vegas, there was 100 and new three new fabric features, quite a lot of things, but not maybe that much shiny things, at least compared to previous times. So to sum it up, overall, this time it was more focused on maturity, slowly maturing the platform for those enterprise applications. And then there was also some things with the pricing. But let's take a deep dive. Yeah, because 103 new features, that's a lot. I mean, I think that you have, it could be wise to take the most interesting, yeah. Yeah, I have picked only the most interesting ones for you all. And also don't hesitate to ask questions, we can pause any time to answer those. Perfect. Also with regards to the questions, I see the questions from the audience. If you think something needs to be elaborated, or you want us to dig into other things, just write the questions and I'll see them right away. Perfect. Thanks, Oliver. So the first topic, environmental handling. So the first thing we got here is this new feature variable library. And what this actually enables us is to have this global parameters between your dev test and production environment. So now you have a product ready way to handle all of these. And they also come up when you do deployment. So this really helps up your deployment. So this really helps up your deployment pipelines. And then, which also was missing was that before, your copy data pipelines didn't work with deployment. But now with this new variable library, you can also do that. Then there's other CI, CI, CD improvements. At the moment shortcuts are now supported with lake houses. And the tables are to come. But remember, here is one thing that not all patterns are supported. Only certain kinds. are not all patterns are supported with the same. And then notebooks, notebooks, variable library support is also coming to preview. And actually, it's already enrolled in in certain regions. So you might want to check that out as well. And then we also are going to get this fabric CLI tool, which makes us a command line interface to manage your fabric. So this is really good. If you want to set up like custom deployment automation and things like that, or those really deep things. Then we got some news also on the data management. There's this new feature called item tags, which enables your product data products to be classified according to whatever is your business need. For example, if you want to categorize your lake houses or semantic models with for example, business lines. Now you have a great tool to handle that. And then those can be seen on the UI and they can be queried by those stacks. And this really helps helps the organization to keep up with the data. Then also the one one like catalog, which contains all the data you have access to in fabric can now search not only tables, but it can also go into the tables to see what columns are available and also metrics if you're looking for the semantic model. So this this helps to look into your stuff really well. And then my favorite feature at least. So now there is one leg security. This came into private preview and you need to enroll to get this, but it will be coming to a public preview. I think in the next quarter. And what this does is that previously all your access and security was behind all the every object had its own kind of schema. Lake house had its own SQL had had whatever SQL always has had power BI had it. And then if you did something with spark spark didn't care at all. But now with one leg security, there is one place where you set up your security rules. Is it role level security or column level security? One place and and all the items, everything will behave with with this. So this is a really, really big step forward where where managed management of data security becomes a lot easier. Then other security things. There is no more support for the V net. And then also event stream finally got an entry ID authentication. So you don't have to work with those as I ask tokens anymore. And then there's key while support for short shortcuts and service principle for grace, creating those. Then something I'm deeply passionate about, which is that the developer economics. So both data warehouse sequel and play house are now. How would I say they they work better together before you have to make the decision if you want to start your workflows with lake houses and vice part or with a sequel and warehouse. And then you were very married to that decision for the rest of your journey. But now, no matter what you choose, it's more easy to mix and match things. So now notebooks can read and also write to warehouses, which is a excellent addition. Then there is better autocomplete for notebooks and warehouses that's already enrolled. And I think this was a good improvement to the old ways. And then this was not a FAPCON news, but I must say this, that there is now a VS code extension called fabric data engineer, and this supports most of the development needs. So if you really like that night mode on your desktop when you develop and check it out. Then there is changes on the pricing scheme on how you pay for your fabric. So now this is maybe the biggest new new thing or the most shy nesting of this release wave is that AI skills, co-pilots and all the large language model stuff are now available in all the capacities. So this was previously only for the F64 and above, which was quite expensive. But now everyone gets to play with this. And I have actually already tried it out. And at least the SQL endpoint one seems to really know well the dynamics data model, at least on the CRM side. One thing to note here is that this eat up your capacity. So please put on your alerts from the capacity management. Then a new pricing model for Spark jobs. So you can now purchase fabric so that there's a specific Spark pricing for you. And when you enable this, this means that when you run your notebooks or Spark jobs, all the consumption goes to that Spark jobs. So this would really might give you a good amount of savings in these cases where you are running only ETL, let's say once a night, it's really heavy, and you just want to pay it as you go. This might really bring you savings. But then what also they came up with this is that there is this max usage ceiling. So you don't have to worry if someone makes a silly cross join and tries to make a silly rose or something like that. So you can sleep your nights well. This is kind of a game changer, I think. I think it's kind of interesting that you're not able to buy the large and the very expensive capacity in order to get the AI and co-pilot abilities. So this is kind of a surprise to me. And I think the market will react positively to this. So, yeah. Were you expecting it? They are nice, I must say. Yeah, yeah. Interesting. Interesting. Then there is this new feature. feature this came bit before the FAP gone, but still there is this item called AI agent. And what this now brings us is that on top of your SQL endpoint or then on top of your semantic model, you can set up this AI agent and give it a specific instruction on, for example, how which which tables contain what kind of data and what kind of viewpoints on that data should you have. And then you can talk to it and it will know your data all along. And here's a quick screenshot on how this works. And this is also now this was when it announced it was behind up 64. But now this is also available in it or is slowly enrolled to be available with with all all capacities. Then there's also a lot of other stuff not categorized in a sense, but. But now, now you can trigger a data pipeline when a new file is created in one leg previously this required an event stream and that take to can have for minimum. So maybe some cost savings here and so so nice like lot pattern loaded whenever you get new stuff. That's nice. Then high concurrency notebooks within the pipelines is not generally available. and then they also gave out this predefined spark optimization configurations. That's a long word, but. So if someone has ever looked into spark there is like. Maybe 200 settings that you can tweak a bit tweak around go find that opt in thing for you. But now what Microsoft has done is they have given out this kind of templates with a very like elaborate naming that what's it for and what's it not for that you can try. So if that might help to speed up your etl's and then they also added a support for getting your data from Excel. Then on the SQL side on the warehouse there is no audit logs available for for the warehouse. Which might have some use cases, especially with audit trail applications. Then one one like also shiny thing is that there are now this AI functions in the notebooks. So if you have some kind of data text data or whatnot, you can now summarize classify extract information from it translate that text fix it cramely or generate response based on it. So it's really easy to use we actually did a short demo on this and and creating this kind of like stuff where we took a form classified entries was it positive was it not positive. It took me like five minutes to create which is really fast. And that's all here is the summary of it. Yeah, great. Well, which of the other ones the others did surprise you the most. I think that. I think that. Out of the scale of the spark was like, I think that was some maybe the biggest surprise I was not expecting it and and to be frank, we have like. Just a lot of time, I think to figure out what's the optimal way to buy out capacity and now. Now we need to do that work again. So how would that affect the medium or small customer? I mean, for larger customers, it's kind of a no big deal. But how do you think it will the midsize customers will react to this? Yeah, well, well, let's say that you had a you have a like really, really heavy initial load phase. And you and for that you need to get like F 16 after 32 something like 2k a month just for that job. but now you could just pay it as you go. You don't have to have it up and running just so you have it available when you need it. You can just know pay as you go and whenever you need it. So that I think that that's really interesting approach. I see. I see. So also with regards to the AI, the co-pilot things that are now able or comes from with all capacities. Do you think that Microsoft is moving in another direction here that they are trying to go with with with some sort of ability to to serve the masses? It's not only the the big companies. So what do you think? Yeah, I think it's a it could be a game changer. Exactly. And I think it's more going to this direction that the AI is is for everyone. Yeah, exactly. Those with the big big pockets. Yeah. So we're not only demonstrating the data by giving the data to everybody in the organization, but also to to all organization with the abilities. So it's a bright and interesting future, I think. So, great. So what else? What else you what else you would like? What do you think? What is what's also surprised you? I think that one like securities. I was hoping Microsoft would do something like that. and I am I am very much pleasantly surprised that they made this because managing all of those different kinds of security details is is a lot of work. And now that can be streamlined. And it's a lot, a lot easier and very nice. Perfect. Perfect. So with regards to prediction and AI and machine learning and all that, what are you what are you hoping for in the future? What are you looking for with regards to fabric and features? I think it. Let's be a bit honest here. I think currently most companies, most entities are doing their analytics are always looking in the mirror. They are looking what are what did it look yesterday? But maybe slowly with all of these capabilities, we start to move our analytics to like looking forward. What would what would be with it being in half a year in the next quarter and so on. So I think that's a really big change in the like whole dynamic how things work. But but good thing, at least for me as a data engineer is that for all for all of these AI things to work, they really need that good basement. The quality of the data needs to be good and then they can they can be basically as good as the data is. So exactly exactly. That's kind of an important prerequisite. But I think, as you mentioned, it's looking bright and I think the maturity level from from the basic and the average, at least our customers, that has increased a lot. So I think the market, so to speak, is ready for for the AI now and in a whole new way. So so I'm I'm very positive. And I and I as you hope that the AI will be a larger part of fabric and the the abilities for the smaller companies also. Yeah, exactly. Yeah. So I don't know if you got any any other things that you would like to mention, but the interesting updates, I think. I think I recommend everyone to look into the Microsoft official documentation and see if there's anything interesting, because this is a very short list on my favorite stuff that came out. Yeah, perfect, perfect, you hope. But then I would I don't see that there isn't any questions yet that I need to answer. So I think that I would like to say thank you very much for joining you hope it has been a pleasure. And I think this connected company thing that we have we've done Denmark, Finland and the other countries is I'm very proud of it. So thank you very much, Joe, and to the rest of you have a nice evening. Thank you.