Advancing Fabric - What is Microsoft Fabric?

hello hello thanks for joining me once again for another Microsoft build Book of news style update that's what we normally do whenever Microsoft build is happening however I'm going to put that off we're going to talk about the book of news next week because a book of news covers every technology touched and announced by Microsoft in the build conference and it goes through going we've got this preview this preview this preview this has been updated this is now generally available and really really useful to get that massive view of everything Microsoft's working on but honestly if we did that it would be dominated by one big bit of news and that is this thing called Microsoft fabric Microsoft fabric has finally been announced we've got the green light uh to talk about it for the very first time and essentially it's a new tool kind of kind of a new tool um and it's being announced for public preview the day at the build conference now Whenever there is this kind of announcement coming from Microsoft it's always the Fanfare and lots of big massive vision statements and lots of marketing speak to be fairly Frank so I thought I'd do a quick follow-up video just to say look what is Microsoft fabric what is it how does it work should you care about it are you going to use it are you going to suddenly start using it without realizing that you'd actually sign up to it what are you probably talk about these things shouldn't we so that's what this video is all about to give you this quick primer for what is fabric how does it fit into the existing world where does synapse fit in where does power bi fit in what are the what are we even talking about just to get you started on this Microsoft fabric Journey if you're new around here don't forget to like And subscribe you picked a doozy of a video it's a big old announcement so let us know what you think down in the comments let us know how you're getting on and yeah let's go from there let's go and see stolen a couple of slides from build so we can talk about what's actually happening here we can talk about some of the things so number one thing that has been announced fabric Microsoft fabric now there's a picture that's gone around a lot so it is this it is this one holistic view of what is inside this new tool called fabric the data platform for the era of AI okay well let's before we start talking about that what is it it's a new SAS platform that encapsulates all the different types of compute workloads you had in synapse with all the existing sassification the nice ease of use that you get in power bi we'll stop that's it all the lake housing sparkly sequally bits of synapse brought into the same SAS platform as power bi that is what Microsoft fabric is if you're already a user of the premium version of power bi you'll just see some of these features off fabric just get enabled and get turned on you will see that working so fabric is a SAS platform that encapsulates power bi plus a load of stuff that you would have seen in synapse and other areas of azure okay so that is your takeaway what is this thing that's one of this now the way they're talking about it is just trying to use familiar Concepts that have been really successful in the rest of Microsoft to try and anchor it with you so the way they talk about fabric is the way they talk about Microsoft Office so if you take office and you've got PowerPoint you've got word you've got Excel you've got a variety of very strong brand products and you know in your head if you're trying to work out some numbers in a spreadsheet well that's Excel it's not PowerPoint they're taking the same idea and trying to apply it to fabric so inside this umbrella term a Microsoft fabric you've got power bi power bi is an existing tool you know if you're doing data views you go into Power bi that is an existing tool one of the pillars within this particular Suite of tools and what they're saying is actually a lot of the workloads and synapse a lot of the way they put setups together there's so much difference in terms of how you could use different things it was very confusing so it's an evolution of synapse to make it easier to make it satisfied to make it work the same way as power bi but all these different things are just akin to having word and publisher PowerPoint each other office Tools in that Suite now like Microsoft Word versus PowerPoint versus publisher some of them are fairly Niche and you wouldn't use for everyday-to-day things other things you'll use constantly and that's what's going to be inside fabric these different compute workloads these different types of activity which have a different name and they come from different areas but that's what fabric is all underpinned by this thing called one Lake so before we get into any of those other boxes along the top let's talk a bit about one Lake because it's it's an interesting concept so the way they're putting this across is well it's like one drive everyone knows OneDrive you've got teams and you've got a SharePoint of all that stuff and everyone just on their machine will goes into their OneDrive and they have all the files that they need uh and essentially Microsoft argument is why can't we just do that for Lakes it's just cloud storage so if you're going to be storing some data and SharePoint stuff that you need to work on you put that in one drive but if you want to have some data if you want to put some data somewhere so that other people can access it you can work with it or you put that in your one link think about them the same way now how we do that currently is well we wouldn't have one Lake we'd have many many many different Azure data Lake stores so we go into the Azure portal we deploy some of your data legs we then connect to it via blob storage Explorer or via other browser and it's not as easy to work with as it is to work with OneDrive I'll hands up I completely agree with it um and that's the mission statement for one leg so as soon as you turn fabric on yeah as well okay well here's your Lake then yeah you need to do security in there you need to decide who can access which different bits of data all of that stuff it's not absolutely relevant but making it as seamless to connect and to get files into and out of as OneDrive sure great mission statement makes entire sense so sitting underneath this Suite of tools inside fabric is one one Lake which is yeah makes sense uh there's loads of other bits and pieces of one leg we'll do other videos around each of these different workloads because honestly there's so much to talk about in these different areas but getting that concept right going okay well there's a centralized Lake there's a centralized really easy to use really easy to access really managed like you don't need to worry about spinning up many many different ADLs Gen 2 stores theoretically inside fabric um and then the different workloads well let's take a bunch of these workloads and kind of talk about where they came from so I mean doing a quick run through your data Factory is your data Factory now it's an evolution of synapse integration pipelines and the OG data Factory tool um alongside some other ideas with power aquarium some interesting stuff in there you've got data engineering now that's essentially the synapse Spark engine but just with a load more functionality and a lot more really really cool stuff put inside it should be a lot more up-to-date it should be a lot more hugged in should talk to everything a lot nicer you've got enough data science now that's partly some of the synapse uh data science stuff partly some of the Azure male stuff we're going to be seeing various different data science workload those science activities inside this box data warehousing is an interesting one that is our t-sql engine again we still need the ability to write SQL and run store procs and have various different transactional workloads there's loads of stuff that we need to do inside a data warehouse and we have that compute workload and we'll talk a bit more about that in a second um about real-time analytics in the form of data Explorer the custom engine so we're still seeing that ability to Big Time series log diving analytics you got power bi in there there's a thing called Data activator which is the new thing that we can't even really talk about right now so there's much more stuff coming in there I'm sure they're going to be even newer logos and icons and things coming into here where we'll see what that makes up so to try and put a bit of an anchor in people's minds when you're looking at this going okay so some of it's existing stuff some of it's not existing stuff how does this actually what um so if we have a big old platform that we have built using a mixture of azure some standard Azure portal bits some synapse bits and we've got power bi and then we're trying to say well how does this all translate into that new the F world the fabric World below I mean so we'll data Factory and integration pipeline just turns into this new data Factory workload within Microsoft fabric your various different data Lakes become one data Lake managed in the same way as one drive with all that power and functionality your Azure ml plus your synapse email comes that's data science workload you can see these are direct translations you can see they're not thrown away lots of stuff they've done before they've just taken those existing tools and evolved them to fit into the software as a service platform so there's some parts that coming from plane Azure and loads of parts that are coming from synapse itself so this is a controversial one so we're no longer gonna see two different sides of the t-sql fence so currently in setups analytics you see dedicated SQL pools which has the big MPP engine and you have to turn it on and turn it off manually you decide how many compute engines it has or computer nodes it has and it's quite chunky it's very big it's very powerful but you've also got the Polaris built serverless engine and you just write a query and it just works with very black box and there's some things you can't do in it now what we're actually saying is these two workloads just coming back together into a single uh serverless well single SQL workload notice we're not saying servers anymore we're not saying dedicated then let's just set up a better warehousing it's the T SQL engine it works one way not two ways and it does everything inside of it so again it can be interesting diving into the depths of well what does work what doesn't work what how do you take your existing workloads how do you take an existing dedicated SQL pool workload on synapse and just bring it straight in one of the whole plans is it should be a real simple migration path we've got spark so spark pools you've got incidents just become synapse data engineering so when we're calling talking about some data engineering I know we're talking about the Spark engine all the good stuff going in there all the compatibility with various other spark engines all built in and again I mentioned we've got the custo data Explorer engine coming as this synapse real-time Analytics but then there's a direct link from most of the different workloads we're seeing into what the evolved form of it is inside Microsoft fabric I mean I've included power bi in there but honestly power bi isn't really going anywhere power bi is still power bi it's just now it suddenly has a load of siblings sitting in the same family whereas I'd have a lone wolf before now this all sounds like a fantastic story right and there's one of the marketing elements that you keep seeing is oh this is a unified data platform a unified analytics platform and it's well notice the same argument made for synapse when it was first released oh we've unified everything it's all in one workspace uh and then they couldn't necessarily talk to each other in the same format you you had a dysfunctional family of tools sometimes where one thing wanted you to work in a certain way and other tools in the same synapse family wouldn't want you to work in the same way so I can I get it if people are listening to all of this and being slightly jaded and going well we've heard all this before but there's a shining Beacon of Hope that actage makes this story make a lot more sense so when we're looking at all of these different workloads we're saying well okay I've got some data that I've stored as part of my my synapse Warehouse I've got some data installed as part of my lake house coming from the data engineering site we've now got a concept of a lake house and we can save tables as a lake house a little bit like how we had Lake databases inside of synapse but just making a lot more sense and much more plugged in you can have your custom databases you're gonna have data sets uh built in some of the other low Colony side things in your power bi side of things so we've got lots of different data stores and this is one of the places where we had a problem in synapse because in synapse the warehouse would be in dedicated single pools and agenda SQL table it's only accessible when dedicate is turned on your lake house stuff is just in maybe in Delta format maybe in parquet maybe in something else just everything stored in different languages the fantastic news inside fabric I think is Delta everything is stored in Delta and they everyone insists on calling it Delta parquet just data it's Delta format but absolutely everything is stored as Delta so if you bring something some data in via data Factory you can land it as a Delta table if you're right a bit of spark in data engineering Yolanda is a data table if you write some t-sql inside the warehouse engine and you land it as a SQL table it's stored in the lake as Delta now that is a fantastic story that is saying yeah you've got lots of different routes to get at your data loads of different options for how people can query your data but the data is held in a single consistent standard format can land your data via the warehouse and read it by a spark you can land your data by a spark and read it by the warehouse you can read all of these Delta tables directly in power bi without needing to lift and shift it into a model don't need to import it you can just read the Delta directly now that is huge that is a massive massive statement going right there's just a single copy of my data stored in a consistent coherent format in any of the berries compute engines can read it that is great kind of exciting really really cool so I can see there's going to be some people watching this going ah yeah I've just finished implementing a whole synapse platform and now the message is essentially coming cool synapse was really good but actually we've done it better and that's gonna be the story the story is essentially synapse isn't being turned off if you currently have synapse do not worry you can keep running that platform that is absolutely fine but you're not going to see a huge investment of features and of work coming from Microsoft they're going to be putting everything into this fabric product I tried to make it as easy to migrate as possible that's going to be the expectation of the next year or two if you are building out a huge thin apps like house you're gonna at some point move it to be a fabric Lakehouse and that's going to be a hard build to swallow for some but it's gonna makes life so much easier for a lot of people if you're using any other tools if you're not using synapse spark to do a lot of data processing you might still actually land your data into the warm lake and then serve it to power bi from there because it's a really slick end-to-end Journey I'm gonna see a lot of people who just using power bi at that moment log in and suddenly see all of this extra power at their fingertips that they can actually just start using so it makes a huge amount of sense as a product as a Direction as a Microsoft stick in their flag in the ground and going right we understand things perhaps weren't quite right in the past everything is Delta everything is consistent coherent everything is SAS platform note there is no Azure here I am not in the Azure portal to use any of these workloads if I'm writing T SQL if I'm writing spark if I'm doing data science I'm logging into the fabric portal the same way I log into Power bi I'm not going into the Azure data portal what is your portal to build out a data platform that can then surface anything this is all software as a service that's a very very very different direction for anything we've seen from Microsoft on the data platform silent events part of it driven by the sheer success of power bi as a way of working as a platform as a as a cult of people absolutely loving the product and the way it works and how easy it is to actually use so it makes sense to take all those other things which are seen as a specialist area this this deep area of expertise to try and Stitch together products into a data platform to just say well actually why don't we just make it really easy and go I would like a Microsoft fabric please cool and then that's built it has a one Lake and I go well I'll build a bit of a lake house I'll build a bit of a warehouse I'll publish it all tied together into a power bi report without having to worry about import modes and then start giving that report to people it's such a powerful message so yeah I thought it worth just spending a little moment aside from everything else going on in the big book of news to just say this is big and you'll see in the rest of build there are deep dive sessions going into engineering Warehouse data science one Lake all these other areas but hopefully that helps to get that kind of just Eagle eye view of going what is this thing it's essentially the SAS power bi platform but extended into a suite of products called fabric of which power bi is just one product and then a load of things coming from the rest of azure and synapse added as new different products or workloads within fabric to allow you to do data engineering via spark did a warehousing via t-sql moving data around data integration via data Factory and power query loads and loads loads of stuff in there so that's what fabric is it's a new family of tools as a software as a service that is very very firmly setting the direction for where Microsoft are going in terms of data and analytics in the cloud and it's pretty cool to see there's a hell of a lot in there we've got lots and lots of videos coming over the next few weeks just to try and help you understand everything that comes in this one gigantic box don't forget to like And subscribe and we'll see you for the next video cheers

Channel: Advancing Analytics

Views: 28,962

Keywords:

Id: LkjRy2HRrks

Length: 19min 27sec (1167 seconds)

Published: Wed May 24 2023