Dataverse: What is it and why should I use it? - MKE Brew City Power Platform

[Music] all right good morning everyone uh just gonna do a quick audio check here this is mike carlo from power bi tips with me i have anthony and mr steve schilling uh as well uh anthony good morning uh you wanna introduce our guest speaker today yeah it's uh one of those bright and squirrely mondays for me so uh really happy to be here and we have steve schilling so uh with that uh you know steve we're gonna we're gonna be going over the topic uh and and this is one that you and i have talked quite a bit about with dataverse and database for teams and how that kind of plays in the broader ecosystem when you're building applications and so this would be really exciting to go through this and mike and i actually here before you jump down what we're talking about this too is uh this is this area where you know there's so much potential in this space and we're really excited to be here and learn more about it so with that uh the topic what is what it is and what it uh why should you use it so that's gonna be uh an awesome topic thanks steve for volunteering sure okay little intro here milwaukee power platform user group uh you know this is something that we've we've put together a little bit over a year ago now and uh we've had a ton of fun in this space so continue to follow uh the event or i guess the the milwaukee power platform user group through linkedin and then with that we have just quick introductions we'll do a presentation and uh steve always has his demos lined up and at the end if there's any questions we'll make sure that we wrap those up and uh incorporate that into the presentation all right with that a little bit about myself uh my name is anthony escobedo the milwaukee power platform uh user group founder i had a lot of fun in this space saw that there was a need and uh you know i guess we i was a part of the sharepoint user group prior to that and now we have a more specialized uh kind of user group here that we can take advantage of i'm a technical architect at concurrency and i've been in uh technology for eight going on actually i think it's nine now i just gotta update my slides but at the end of the day uh you know continuing talking about traveling and that we we do that through our experiences and happy to have steve here follow me on linkedin twitter and i have my personal website with some blogs and some other information about myself again just really excited to be here and learn learn more about this topic from steve thanks anthony uh my name is steve schilling i've been i'm senior manager over productivity and i've been in the industry the it industry for like a million years now um but uh you know more recently like the last 18 i guess now in the uh the office sharepoint collaboration space uh you know starting out back as far as sharepoint 2003 and much more recently working uh you know primarily in teams and and power platform um dataverse powerapps portals which i have um you know shameless plug i did a uh a presentation a few months ago here on the at the user group on that yeah not his first time looking forward to talking about uh dataverse today and uh you know some of the things that have come up you know for me over the past you know year or so are you know things like you know large sharepoint lists you know on premises when people uh you know get a large list and they don't want to don't want to deal with it they just up the threshold and then when they go to sharepoint online they find out that oh well that's not a movable target anymore so um dataverse actually kind of comes in handy in some of those situations where there are a lot of you know sharepoint lists that are large and and can be dealt with that way so um that's kind of what got me into you know talking about it today is that you know i don't hear enough of people talking about it and why they should use it and and uh how they can use it both uh you know natively entered teams yeah i think one of those things too is that you know steve over our conversations the the fun solutions that you put together in the last you know year or so specifically in the data the dataverse side or the database for teams or portals and just having lived in this space now for you know for a while and and being able to now give some of that feedback back to the the community here so this is really exciting and we also have uh we also have mike here as well so we'll get we'll give him the stage here to before we jump over yeah i'm very excited for this topic i mean as a small business owner this resonates very well with me just in general uh you know steve just particularly i'm a guy who runs a lot of my business in sharepoint lists like and so you know at some point in time things are going to get larger and it's going to get to a point where i'm going to need to start thinking about what's the next more sustainable part of the solution as i grow my internal solutions and i'm working with clients right now business departments business units same kind of scenario right we've got a lot of data we're comfortable we're moving away from excel sheets we're kind of doing the sharepoint listing and there's probably like another grow up path here away from the lists and more into a dataverse environment so really excited to see how this is going to move forward i'm just going to be the producer kind of in the behind the scenes here i'll i'll catch questions across the social media areas and i'll feed them over to steve over to you steve okay let's see here um okay sorry my mouse just went a little bit nuts on here no worries um so here's our agenda you know what is dataverse why should i use it and how is the team's dataverse different that's one of the things that that comes up is uh well if i can do all this stuff in teams you know and dataverse is there then why do i need to you know pay the extra money for dataverse and we'll talk about some of those uh limitations of teams davers here in a bit but uh um let's uh you know kind of get right into it here and uh um talk about uh you know the power platform this is the old slide for the power platform you know where we have our you know four basic products and then kind of our uh our layer underneath and then uh the new one basically swaps out common data services with uh um data verbs and uh dataverses had many names um it's had let's see common data services uh i think there was some talk about calling it dataflex at one point um i don't think that ever uh became a real thing um there's uh what xr xrm i think is the uh the old dynamics version of it um so a lot of different names um when you get into the team space there they called it oakdale before it went live um so that's one of the things that we'll talk about is uh kind of some of the the naming and terminology and you know we think about things like um the there we go the terminology that uh comes along with uh with dataverse and the updates to the terminology it's like well i've never used it before so why do i need to know this old stuff um one of the reasons is that uh when you see a lot of um the documentation the blog posts the solutions um they're gonna refer back to some of the old you know dynamics terminology cbs terminology so you kind of need to know what you're you know comparing apples to apples so when you see in maybe an old cds post it's still relevant it might say entities and fields and records um where when you go into your um your dataverse environment you're going to see tables columns rows um and then this one here the the yes no versus the two options i still think it's technically a two options because you can change yes no to be whatever you want to be it can be on off it can be yes no it can be true false whatever so the two options was actually more relevant i digress a little bit um let's take a look at the at the dataverse itself um what is the dataverks it is a system uh a database if you will that allows you to securely store and manage data used in apps it's not just a pass-through um so it's not like a connector it's actually you can store the data right in your dataverse you can collect it there you can import it in dataverse you know full dataverse you can export and then your apps all have a single place to uh to connect into that data um which also means that you can kind of filter it out sort it out organize it secure it so that it's safe for your uh your users to use um it doesn't have to be just your enterprise it team using this data you can if you do it right set it up in such a way that your your power users citizen developers whatever term you're using for them can connect into this data and use it where you know in the past it was like well you know we're not going to give our uh our you know back-end databases you know whether it be db2 sql oracle whatever it is you know directly to our end users without putting something in front of it well dataverse can be that something in front of it and it can go up pretty quickly so um so data versus how powerapps and office dynamics all use you know the uh the business data that you have um stores data in standard and custom tables and we'll talk about standard custom tables here a little bit with rich metadata you can build the powerapps apps workflows chat bots things like that that can be can go against the data that's stored in dataverse and then we can add business rules validations and uh security and the security is actually kind of a kind of a nice deal with dataverse and we'll talk about that a little bit too yeah here's the uh kind of the microsoft diagram for what dataverse actually is um it starts out it's basically dataverse is an api um the tables the views the forms the things like that that you create in the dataverse really you know end up getting consumed by your applications um so it's uh what they call api first um everything is there and then you've got you know powerapps flows teams um anybody know what this icon is here it's a little uh kind of sideways looking paint thing here that's visual studio so um you know even custom apps can you know reach in and grab data from dataverse so um that's huge for everything else yeah it is i really like the idea of vm so this is this is the principle of like low code no code but yet there's still like a rich layer under the hood that does have pro code type things in there so i love the fact that they're having this approach of here's a bunch of uis to help you make it easy by the way everything's still built on robust code you can go modify and go deeper when you can't quite get everything down the ui oh absolutely and uh you know when whenever we would go back to you know predecessors to something like dataverse or you know even sharepoint things like that things like access you know you get to a point in access where it's like well i can use access for most things but then i get to a point where it's you know above my head you know i need to you know add in multi-user capabilities or you know something along those lines and then you have to bring in your uh your professional developers exactly and something like access wasn't that wasn't really thought of all that well or was you know kind of more like the fringe cases with dataverse all that is built in you know from the get-go so um awesome your professional developer features are all just right there for you so when you do get to that point you can just hand it over to somebody they don't have to you know reinvent the wheel and rebuild from scratch and just take what you've got add to it and get you on your way love it very cool stuff yep so um so in addition to that database you know when you talk about you know a database we think of you know tables rows you know views maybe some stored procedures functions things like that um and then you know security is there but it's not as as robust as say like an application layer security well in dataverse you have all of that you have you know your authentication authorization is there um auditing is there and then we start getting into kind of that logic layer which you know there's any dbas on the phone i apologize um in dataverse it gets mixed so uh we can put business rules on on tables and fields and things like that so um which i'm pretty sure that under the covers you know if you were to look at the code of microsoft i'm sure that the layers are separated but as far as you know we the users are concerned it is just one you know sas system that that we can incorporate our business logic and our data all in in one spot um from a data tool we've got uh you know modeling reporting there's some visuals and dashboards things like that that you can do right in dataverse um then there's also the uh the power bi another thing down here on the bottom in the in this data section here is this thing called common data model and we'll uh we'll kind of talk about that a little bit but uh um instead of uh open source uh tables for doing a lot of you know common activities you know so it's like um you know account and contact and you know tables like that that they've set up that kind of represent a lot of normal business um the storage uh storage is you know relational databases which is uh which is nice um you can store files and blobs in there um log files uh semi-structured data you know data lake stuff you know where i don't know if i can come up with a better explanation of what a data lake is but it's kind of like that closet with all your junk in it um that's not really organized but it's all there and you can kind of pull it out and use it as needed that's sort of what a data lake is and uh dataverse will handle those types of scenarios and then integration is another thing that's uh that's big in the dataverse as it is today um there's eventing so event to event of event to a service bus eventual web hook um so you can trigger off external processes based on something that happens in dataverse um there's exports to uh you know sql data lakes um and then of course there's power automate that can go over the top of all of this yeah i think the advantage here that we're seeing with dataverse is that it's something that is already already in use you don't even know that you're using it sometimes so like like you mentioned earlier uh you know with there's uh power power apps power automate uh power push you in power bi all that good stuff that that's out there and a lot of times when you're you're already using this even in sharepoint and things of that nature it's it's something that you spin up and like on the back end you're using power automate and then those power automates are stored in dataverse and it's just it's like this ecosystem that just already lives in this space that you start to just use and then you you go out there and you intentionally use it you're like oh i've actually already used some of these these bits and pieces and one thing that's not on this uh this diagram that really you know deserves some time as well and it could actually probably be its own topic is uh the idea of solutions you know being able to pick up these uh these customizations and things that you've built and move them from you know one environment to another so that you can do you know some alm processes you can have you know dev test and and prod environments that you can that you can create in uh in your power platform where you can move these solutions through and test them know update them things like that so i think it's the ability for like a citizen developer you know some maker that's out there in the world that that has a tenant they're at their organization they streamline you know some small legacy task and then they're able to just tack on all these extra things that we just talked about without any additional effort it's just built into every single app every single solution or or power automate it's just the ability to be able to extend it to any of these features you know with with little to no effort is is the i think that the big win that we get with using the power platform and all that it has to offer oh sure okay um now jen's kofad i hope i got his name right um i'm giving him credit for this uh took that uh that last diagram we just looked at and kind of scaled it down for the dataverse for teams so you notice that there's a whole bunch of missing things here um things that are missing are like the security auditing uh the entire logic tab which would uh ultimately end up being built into your your apps instead of the the data um reporting is removed from the the data there's no uh visualizations in teams storage log files there's no really good logging in in the teams database and then once you get over into integration um pretty much all you've got left is power automate none of that eventing stuff is there for teams and then there's you know a few other limitations too as far as like uh storage and things like that but uh we'll kind of talk about those differences a little bit later but i did like this diagram that uh that he put together for this and jen's co-fought he's got a blog out there at this ii.dk sure you can full credit for that he saved me a whole bunch of time i didn't have to mess around with that image which is definitely not my strength anyway so um there we go um the common data model this is that uh um that schema and there is a uh a github out there that has all of the uh the schemas for this and json and you'll see that there are these core set of tables here account activity contact currency email so forth um and these are you know a series of you know standard tables that come with every uh dataverse that you uh that you set up you'll see when when we create a a new dataverse you'll see just a whole bunch of tables that just show up and maybe you know what they're there for maybe you don't uh um probably they'll probably come into play at some point um there are other you know common data model schemas that you can add to your dataverse um based on what your license is if you've got dynamics and things like that you can bring in you know sales and service and finance and and all these kinds of things that can help you uh you know build applications based on a more standard schema and steve i've already uh dropped a link for the github common data model schema in the chat windows as well so i've already got that out there so check the chat uh the common data model schema is actually a well-documented structure that's available to anyone to go look at and review what data makes sense for your team and your business this is something that microsoft makes sense to look at this because uh from a from a microsoft standpoint right they are interacting with hundreds of customers you know and and there's these are the common patterns they're finding across industries and companies that hey if you know i'm a manufacturer manufacturers care about these kinds of columns and this kind of data so this is a good place to start thinking about your company and i think a lot of times i when i work with people or any companies around this they're like well our company our business is unique yeah to a degree right and so there's a lot of this this concept here if you can conform your internal data to what the standard is you can get a lot of bang for your buck right out of the gate by using this kind of common format and schema here yeah to and to kind of you know build on top of that yes every company is unique um but you know when you think about it only about the last 10 percent of it is unique yes so when you've got a data database and the common data model like this you can take that you know common data model schema and extend it yes so you know when we get down into here we've got these standard tables that you know commonly used concepts and activities um that would be like your account content customer um you know any of your activities like you know emails and and uh letters and things like that that you send out campaigns marketing campaigns that kind of stuff and then you can extend to you know make your business unique without having to you know start from scratch and uh um you know rebuild the 90 that's already there so um very cool stuff there and it's got um you know it's already built out with the uh the the standard stuff that you know ninety percent of people are gonna want and use um but then you have the ability to extend those with custom columns and relationships so anything that uh that you think is uh relevant to your business you can add it and you can also distinguish what is custom from what is standard so i'll kind of show that here in in the demo custom tables if you've got something that doesn't fit in the the common data model um great add a table um you can add tables you can add as many as you want you can add relationships to those and set up your own schema within a schema you can set up your business rules and validation um calculated columns um you know things like uh uh the old uh in like in the sharepoint world you would have a lot of you know either sharepoint designer or you know power automate flows that uh would uh do things like well if a column you know is set to this then i need to go and update you know this column with today's date or something like that you can do that not just at the uh at the application layer um but you can do it at that data layer so that it's consistent across all of your applications and then we've got another thing here that's actually very very powerful as well important synchronize you can pull in your data from external sources using the connectors like uh like you would see in power power automating power apps and use power query to kind of tweak that and manipulate that around to get it where you want to be so uh you know very powerful stuff as far as the import synchronize you don't need to use like ssis or you know other you know heavy duty developer tools to to bring that data in um sophisticated security model providing row level security and models hierarchy and also um i don't have listed on here but uh field level security so say like on a customer record you've got you know customer name address uh you know all the goodies but then it gets to you know stored credit card number you can you know secure that down to where almost nobody would be able to see that or and then you can also calculate a column that would be like the last four digits um so really cool stuff like that and it's all built in which i remember back in the day we if we wanted to do something like that we had to we had a lot of work to do to kind of build something where we could implement a field level security that's a quick example of a business rule here it's a a gui designer and it would be you know something like uh um on the account if the uh if you check the credit limit if it's over you know probably a thousand or a million dollars who knows um then you set business required to um you know basically you want a vp to approve it you can set up these uh these kinds of simple um you know set a column um you know clear column and let's see what else can we do here um we can you know validate data show error messages if it uh is outside of a range and then if you're using model driven apps you can also uh you know show or hide columns um that would be like on your your forms and views uh enable and disable columns so that uh if something doesn't you know match a uh a rule you can maybe hide show enable disable and then there's a uh um you know some business intelligence stuff that can also be done in in the model driven apps that you wouldn't be able to do in canvas abs but all that stuff is also still at the uh at the data layer so um a lot of fun stuff there so something interesting i want to point out here is you're talking about dataverse but yet i see here on the screenshot that you're showing you're actually showing like a power apps in the upper left hand corner on the ribbon bar here so this is uh is this powerapps or is this dataverse or is it the same how do you how do you describe it as the clients because to me this is an area where i think microsoft could use a little bit better messaging around where i'm at and what am i looking at here so maybe you can explain for us how am i looking at like in your world your lens on this how do you see this fitting with the way they've note they've made the messaging here right so when we uh when we go into the powerapps maker um and you know when you see this and probably uh in the in the context of like the microsoft documentation this uh a screenshot like this would make better sense but you're going to be in the powerapps maker portal which is going to um put you in this uh you know where your your top bar here is going to say powerapps and then you'll be in the dataverse and the tables and then you navigate into a table and there will be business rules and when you set up the business rule um that's where kind of this the rest of this uh screen comes from so you would be in the powerapps makers portal could do this cool excellent thank you okay um why would we uh you know want to do anything with uh with dataverse well it works with uh any kind of data um relational non-relational file image data lake you can you can pretty much bring in whatever you want to it works with any type of app because you know everything is uh is rest api um you can uh use c sharp apps you can use javascript or java javascript um you can use uh you know anything you've got mobile apps uh web apps uh teams you uh can all use databars analytics there is some built-in analytics in in the dataverse you can create some charts you can create uh some simple dashboards based on the data that's there uh plus you've also got power bi that has a connector right into the dataverse that can you know pull out the data and and do whatever you can do in power bi which is you know just about everything um i still um you get uh whenever i go into power bi all the stuff that i never knew that i could do i'm always finding something new every day so and that's just gonna continue i'm sure um there's uh you know the security features of it you know how it's integrated into azure id for authentication and authorization so that i mean only people that you want in there can be in there and then once you get beyond that then there's some role-based security you can create roles you can create custom roles to uh to your application so that uh maybe you know managers have a different level of access than you know non-managers and customer service reps might have different access than sales reps um so you can set up those roles and and adjust your security accordingly plus then there's also the the row level and column level aspects of it as well which is something that you generally don't have um like in a sql database i don't know role bait or row level is uh probably pretty difficult to do in just like a native sql or oracle database okay there's compliance that the microsoft trust center which has you know just scads and scads of documents in there and if you've got some kind of a compliance issue odds are you'll probably find find it address their things like socks and hipaa and that kind of stuff um here's one that uh that's kind of a good and a bad um the limits you know there's entitlement limits that are basically based on your licenses and and seats and things like that you get you know so many operations per day granted to you and then there's you know the service protection limits you know the throttling that happens and you know it's probably one of the things that's most complained about but it's also a feature because it uh keeps the service consistent you know those service protection limits are there to you know ensure that you know somebody's you know big giant migration doesn't you know bring your system to a crawl so while it you know when when you're the one that wants to do that big giant migration it's a nuisance when somebody else wants to do it and they shut them down it's kind of a blessing uh backups there are you know the two types of backups in the database there's the system backups which happen you know as kind of microsoft disaster recovery and automatic all that good stuff but you can also do manual backups yourself you can uh you know create a backup say like if you've got a production environment and you want to you know import some new uh a new solution or an updated solution you can back that up manually before you do it and then restore it if you need to steve one one thing with the backups is how how easy it was to do that too and i think it takes like a backup like every couple hours you know by the system but like you said if you're making major changes you just click a button and the manual backup's done within a few minutes it's very seamless i was very surprised uh when when i did that for the first time uh you know it was it was pleasantly easy and and then it showed up right away that hey there was a manual backup that was taken and then uh you know with that also you don't need to uh um if you're if your dataverse database is big um you don't need to find storage for that um a lot of times you know when you've got like a sql database and somebody wants a one-off backup it's like oh geez i gotta go and find you know another you know 100 gigabytes to back up this database and that might take me a couple of days to to find a spot to put backup well okay i don't want to deal with that so um dataverse it's in you know the data center there and i uh i'm sure that there's a big operation of of handling storage and making sure there's a but there's plenty available but uh you know to us it looks just like magic i mean you want to take a bit of backup of a uh 100 gigabyte database it's like it just happens so yeah click a button you click a button and it and it goes know depending on how much is at your fingertips as an admin um you start to look at you know like the the reasons why you said you know why dataverse um and it's just being an admin and then you know looking at a browser screen and then being able to do all these things with the click of a button now imagine having to do all of this you know if it was a sql database or whatever it might be but it's an on-prem kind of database that you have to take advantage of and i mean to be able to to achieve what we're seeing on the screen here with something that's on prem it just it takes so much more work to be able to to take advantage of these features and it's as simple as you know like backups i mean there's programs that do that and but you have to maintain those programs here it just it works and even when you talk about backups for you know other cloud services like uh like sharepoint i mean um i don't know people that are that are doing regular backups on sharepoint online um sharepoint on-prem it's you know there's you know it's kind of a cumbersome process to actually fully back up your your sharepoint on-prem um but in the dataverse you know you can like i said you can kind of take a backup of these things before you make big updates and uh um if it doesn't go right you've got to back out plan um but it's always been one of the things that people talk about when they're doing changes to sharepoint it's like well we got to do change control so you know always on one of the uh the change control questions it's um well what's your backup plan well let's see once we start you know uh putting stuff into sharepoint online we got to keep going until it works because you know you there's the undo button just you know isn't there really terribly easily um well with dataverse there's an undo button and you can you know confidently say well we'll take a backup of the thing and if we have to back it out we will restore and that uh there's like three buttons on that screen so it's like you just i mean you either backing it up or you're you're restoring so i mean and you're clicking a button to do either so there's the three buttons that you get yep um you know and another thing with uh with dataverse is uh um i didn't put on on the list here is uh your applications it's kind of it's a it's a really nice you know midpoint of you know kind of that low code and full you know fidelity code like you would have with a web application and sql server um and you know that kind of a an architecture where once you get past the simple sharepoint list library scenario and you need something a little bit more you know dataverse can definitely fit into that spot very easily and you can bring in your sharepoint list data and your file data and you know consolidate it there and move your application to the next level yeah they both support like the export and import from excel and all that good stuff so there's there's ways that you can do that that make it a little bit easier even with power query too yeah power query actually will uh you know make that whole import export process quite a bit easier you can uh you know do some mapping and things like that in there it's it's it's nice super powerful yeah it's it's something that i think is is opening you know the the power platform people like yourself and myself you know opening our eyes to that but that's where you know we have power bi folks like mike and steve who uh you know work in that space quite a bit and you know i have so much to learn in that space because you know power query is just it's one of those like hidden gems to the power platform people but power bi well like power platform as in you know higher level but when you talk about power bi i mean it's something that they use very frequently so i i'm very excited to dive deeper into that at some point this is this is a really unique area for us particularly everything i'm used to is i look for data sources for power bi and and now looking more to like what can be a data source for power apps and having a single mail managed solution like this makes a lot of sense to me i think this this is making my life a lot easier uh and then you know when i have these one off or edge cases of how do we get data into our power bi reports this is a great platform where we can start saying okay let's in a very quick way in a similar fashion like how i would build power bay reports quickly get data into a place where it's regulated it's measured i can have business rules on it and it makes my effort downstream consuming the data in power bi much much easier oh for sure because a lot of that in not only that but it's uh when you bring it into dataverse you can you know bring in dataverse from maybe a source system say there's a an erp or something like that and then um you can augment it using uh you know powerapps power automate and then report on it from uh from power bi so you've really got the uh the whole thing just right there and then with the uh the way that it can sync that data it can you know bring in the data from uh from the source system as it needs to and you know keep separate from the uh the data that's been you know augmented by by the users you know off to the side it's very cool stuff now we bring in this whole uh you know dataverse for teams idea and uh you know the uh as we saw in the uh the diagram before the uh the dataverse for teams does have some limitations over the full-fledged dataverse um otherwise nobody would ever pay for old dataverse they would just host everything in teams and and call it good but dataverse for teams is low code platform built into microsoft teams um the benefits that uh include you know the support to build apps flows chat bots all within teams you can use the uh the whole idea of you know tables and views and and things like that with uh that are there with date of earth um you get uh a little bit different security model with uh with teams it's uh it's aligned with the teams so you get like the uh you know the group owners and group members group visitors um those are your roles in dataverse for teams as opposed to um the uh there's a a series of different roles for um full fidelity dataverse um this one is probably one of the biggest reasons to use databurst for teams it's included in most of your team's licenses i say most because i think it's pretty much all but as soon as i say that then somebody will come back and say well there's this uh you know oddball freebie teams you know that uh you know comes if you buy a uh you know a chromebook or something like that that like oh well i didn't think of that particular scheme um but most of your team's licenses i know that all of your e-licenses will uh will include dataverse for teams so um and the other big thing about that is that it's not just there for your internal users you know that wouldn't need like an extra license your guests so if you have external people that you invite to the team they can use the apps in in teams that you've created in dataverse um where you know a typical model driven app or um you know canvas amp that you've created just in native apps those people need a license they would need you know like a b2b uh type of a scenario where you license them to to use those apps in the database for teams you don't need to do that um you can uh support in a in a database for teams you can support a million rows which uh when you start talking about large lists and in sharepoint and things like that you can you know large list in sharepoint i think technically will support more than that but you always tend to run into that five thousand item view threshold um so you know using a a million rows and dataverse is uh quite a bit more efficient than all of the uh the messing around that you've got to do to deal with large lists and sharepoint online and powerapps and of course um you can always upgrade it to a full database so somebody comes along and says well geez you know we uh we built this thing in teams we want to uh you know we're using a whole bunch of stuff and now we want to uh add into it uh um you know a web hook we want to uh you know be able to call out to a web hook save you know open up a service now ticket or something like that well you want to start doing stuff like that we need full dataverse and rather than having to start over from scratch and rebuild it you can upgrade it and just continue on so very cool stuff there um a few of the limitations we've kind of talked to a couple of them here um and this is not a topic that we've talked about yet but uh um mobile offline um if you have the the powerapps app and you've got an app on there that uh that's running um you can set up some of these dataverse tables to be mobile offline meaning that they'll bring down a copy of some of the data and you can work with it offline if you don't have a connection teams doesn't have any of that most of the standard tables are not available for data first for teams so all that common data model stuff that's that's so cool um you really don't get that you'll get like the user table um and maybe the organization table and i'll show you what those are here in a minute but you'll see a big difference between an empty uh dataverse versus an empty dataverse for teams professional data if i can talk professional developer features um limited only to data visualization there's no api access or plug-ins um so yeah the uh dataverse for teams really is a team solution it's not a uh it's not intended to be a replacement for the dataverse if you want to do all that stuff i'm sure that if you called microsoft and said geez you know we want to have you know these external applications integrate with our dataverse for teams they'd be like hmm well you need to upgrade that to a regular database and then you can do all that stuff and then they'll be like well but then we got to pay for it well yeah that's yeah you're getting more stuff so you got to pay for it but um so that's uh that might be a kind of a sticking point as to use whether you use data versus teams or full dataverse is if you need to access it from like outside applications yeah i think i feel like you don't really know what you're missing until you get in there and you're inside data for teams and you don't get to take advantage of the things that already built uh that's when you start to see like okay i did save some actual time there otherwise it's you know when you're in theaters for teams you're starting from from scratch but it's that also that grow up story too right i mean so the idea here is like get started with what you're doing in teams and then it's an easy one-button press to promote it into full dataverse so it's kind of like that okay you don't need everything to get going just start figuring stuff out and then once you hit a hurdle at that point in time you can say what is the real business value of what we're producing here and then yeah you can decide yeah this makes sense let's push it all the way let's and that's the thing the grow up story of this is really interesting to me yep exactly and uh so that's uh that is a that's a great point you know you get into uh you set it up in teams um any of your your team's users can do this and uh then you can uh if they get to that wall you know where they need uh you know some additional features or or support or something like that they upgrade it um they run out of space you know there's a two gig limit on on the data that's in there um for teams they can upgrade it and then they've got i think uh for some reason i was thinking that the limit is four terabytes um of a dataverse database which is huge i mean four terabytes i remember when uh when one terabyte was you know just considered science fiction you i know even think the starship enterprise had a gigabyte of storage but um yeah i think about you know uh when you go way back and you know we talk about like well you know two gigabytes you know four gigabytes terabyte and it's like yeah there was a time when uh nobody even thought about that now it's like well it's not gonna be enough there was a time when we talked about in megabytes like there's a whole world of like how many megabytes does your computer have now like that's not even yeah things are crazy these days my first uh my first computer was an ibm pc uh had two floppy disk drives careful now you might you might date yourself a little bit here but i had 256 kilobytes of ram whoa you're talking in kilobytes oh my gosh and my uh my uh my current laptop has you know 64 gig and three terabytes of storage yeah laptop laptop this is things you drag around with you like i know your your first computer wasn't something you were dragging around anywhere oh that wasn't weighing 100 pounds that's like a box made out of the case was made out of steel oh my gosh wow i mean it didn't withstand you know like a nuclear blast i think but still it's still running all the internal revenue service stuff that's all that's all they run right there right all the government programs are also running these old boxes nice um so another couple things here that uh um you know aren't in teams um model driven apps are not supported in teams um and neither is the component library so when you're uh you know creating apps and you want to uh either bring in a component library or something like that share it it's not available in uh in teams um and some of the security features like customer roles because it is integrated with teams you have to kind of use the the group roles that are there those aren't there activity logging auditing um field level security uh not there in dataverse for teams and most integration features as we saw in the uh the that diagram that the gens put out pretty much all of your integration is wiped out except for power automate which um in most cases is enough so um you know there's a lot of a lot of stuff happening out there that's just on power automate you don't need any of the uh the automatic eventing and things like that so um just because there's these limitations uh there's still a lot of stuff there so we have like 10 minutes for your for your demos yet but one question that i i was thinking is your business rules is that something that you can take advantage of with database for team um i don't believe so i think you're right there's another question here from linkedin if i can throw in there another one here as well is adam's asking what are some of the decision-making factors when comparing whether an organization would use a dataverse or a power bi data flow or if they can just both use power query online so i think the question the nature of the question here is like we have data in like why would i want to use dataverse versus maybe like a powerbait dataflow that sounds like a sink one question right there and i can kind of talk speak to the power bi dataflow side of things and then maybe steve you can fill in there a bit and then um so let me just i'm gonna just touch on the power bi question and then the data verse side of that so power bi and data flows is really connecting to a data source and shaping that data and bringing it and preparing it in a way that would be ready to go for your power bi report so technically you can't edit data in a data flow uh you're basically connecting to something else that has the editable data and you're you're bringing it forward you're gonna do some transforms to it so if you just want to load data from a system and bring it to power bi data flows might be a good option dataverse my understanding is similar in nature but it actually has a whole data structure behind the scenes where it's a bit more around we've got tables we've got relationships we've got like a sql server that's kind of managed for you and holding this data for you and now you have the ability of going to insert update edit records all that kind of stuff and do all these automatic backup things so data versus like another level deeper saying this is a live data source that could be operational meaning you're running your daily business on it but also it's a an area where you can report right out of and then maybe steve did how did i expect did i catch it there is that kind of where you would think of the breakpoint yeah that uh the biggest thing there is that when you bring that stuff in um via your power bi you're bringing it in and you're transforming it it's kind of more of a read-only correct read-only space yes when you bring it into dataverse you're bringing it in and then it's you're bringing it in kind of that same way i mean you do the same types of transforms and and connections to power query that you would in power bi sure but you're bringing it into a structure that is read write so answers the updates like you said um are all relevant there now if i had to put my my my big thinking hat on here on this one dataverse is just like a managed sql database right there's there's just a lot i mean there may be other voodoo things that are going on behind the scenes i don't know of them i'm not aware of the back end of the infrastructure on the other side of things but to me this feels very much like hey microsoft's going to just make sure that there's a sql database available to me they're going to figure out how to run it on whatever machines that they're going to run it on i don't really care they're only looking at the data storage and so that's why you have this rich layer of like i can edit stuff i can change data because it's technically running on like that sql level infrastructure is that kind of your perception too steve and how you see that okay yes it uh the data it gets stored in uh in sql databases as far as i know um and then the rest of dataverse is that you have data access and uh um and some business logic layer on top of that so um you know when you think about it as a as a database it's a uh it's a sql database with a whole lot of steroids um you know because you've got you know a lot of your business layer you've got any validation your logging your visualizations things like that are all you know poured into that as well okay all right let's uh take a couple minutes here for a uh for a quick demo on this um when you go into you know your make.powerapps.com so you go into you know office and you click your uh your powerapps and you come in here and you get to the uh the default environment in powerapps which when you come into the default environment you click on tables and there's nothing here and you're going to be tempted to create to click that create database uh don't um let's uh let's not do that in the default environment um what i tell people to do is i tell people to create a separate environment um because this default environment has opened everybody as as we you know have probably learned in a past uh experience with uh with powerapps and there's no way to keep people out of this without unlicensing them so we create a new environment and i just created a just a trial environment here um this is empty and so when i come in here to this data um which in some tenants it will actually show up as a dataverse i don't know why this one doesn't but uh um you'll see tables choices which is kind of an interesting concept but uh i don't go too deep into that data flows for getting data in and out um azure synapse link connections connectors gateways and the gateways would be for things like on-prem data so forth um so we come into the the tables and we will see a whole lot of standard tables we've got like these uh these common data model tables like account address appointment attachment so forth and then we've got up here we can look at the custom tables manage tables which manage tables is uh if you're familiar all with solutions um maybe that should be a topic for us to we could talk about solutions in databurst at some point um there are solutions that you can get from microsoft third parties um your own developers that can be either managed or unmanaged managed solutions are basically like a bundle of packages that they don't want to mess around with too much um so any tables that would come along with something like that would show up as managed and then you've got your awl here which would kind of show the whole the whole mess of everything um now i've got a teams in uh environment but what you'll find is that you won't see that in here i've got my default environment and i've got my regular dataverse environments but if i do go into teams and the only way to create a team's environment a database for teams environment is to actually go in open up the powerapps app and then once you you know try to create an app in a team it'll create the environment for you automatically there's no you know to the admin center or anything like that there's no way to create a team environment um which i thought was kind of weird but i guess i can understand why too so when i go into the build part of my powerapps app it'll show me all of the teams that have an environment so i have this soc team which is uh you know one of contoso's contoso's uh default teams and they should have an environment there and if i go to the power platform admin center i'll see that there's an environment of types of microsoft teams but i don't see it in power apps so here's something that uh that is also kind of a nugget if you want to see those environments you go to flow instead of powerapps and then you can see basically the uh the databurst environment don't know why that is yeah i always thought they were very very much the same because you can do a lot of the same thing you should be you should be able to i mean you come in here and you think all right well i've got an environment here and i've got an environment here and they should actually show the same thing but what you're saying is that in power automate that it's a different experience that you get when you're looking at the dataverse environments inside of powerautomate it's different than what you would see instead of powerapps right you'll see your team's environments and power automate but not our apps so i'm sure there's a reason behind it i'm kind of curious what that is now yeah i i can't see there being a reason because i would have expected them to kind of use the same code but i guess not um so we get to uh we get here to tables and uh um i'll just you know quick try to show the new table interface i'll uh create one of um this is actually one thing that the dbas probably like on this um where it gives you a display name and a plural name so a lot of times you know if you've got uh say you want to create a a table of of meters um best practice is to go singular with the table name and then you know have a plural elsewhere but um this can kind of help people you know keep people in line with that um you give it a primary column name um and the other thing that it'll do is you can have a display name and a space is in it and it will automatically do that and of course it doesn't like one of those things that you start to dive into like the the natural flow of how you create sql tables columns and all that good stuff it feels very familiar when or even sharepoint columns things like that or uh excel columns it feels very much the same it's very intuitive it has that pop out that that makes it simple to digest yup and it takes a little bit to um provision a table because it's gotta you know create a whole bunch of stuff for you um and i think i've already got one here in teams just a couple minute warning here yep i'll kind of give you look how funny is that if i go into my tables in uh empower automate it bounces me back over to powerapps because that's where the uh the table head varies see that this this to me this is the part where i think people get this is my complaint i'm like i get that it's all one system and it's all kind of playing together but like the branding to me is like you're kind of like mixing me up here i i click on one app i'm expecting to go to a summerslam the same place just name the doggone thing dataverse just just rename this stupid thing clean up your branding microsoft so i created a table and this was uh you know created in you know teams um and you'll see that uh there are both standard columns which you would expect to see created by created on um import sequence and modified by all that good stuff and then there's these other ones these custom that are the ones that i created so i filter down my list here to custom the only things that i really care about are you know the transmitter id utility type um name of the meter and then whether or not it's active but in order to do all of the other cool stuff that they verse does it needs to know all of this other stuff here so um you know the uh the time the version number all this stuff and this stuff is all available to you um which is uh which is kind of nice and you think about well how many times have i had to dig in and get a system column um it drops all that stuff in there for you um over in in the uh the dataverse you have pretty much the same types of things you know the the same standard columns um one big difference between these two interfaces is this stuff across here so you've got your relationships where you can relate it to another table business rules views forms dashboards charts keys alternate keys what that one is commands and then data over here in the powerapps version you've got columns relationships and views so no business rules no keys no visualizations none of that stuff so that's one you know limitation there um it almost makes it like more digestible by end users too because like if you're brand new to this and you start a database for teams you look at all those tabs and all these things that are going on you're like wow there's a lot of things that i can do here and where do i start but then you look at it in database for teams they boil it down to that you know it's cool those core concepts for you which make it a little bit more digestible yep so okay um so that's kind of just a quick look at uh at what the uh you know the dataverse is versus the dataverse for teams um when you come in here you can create you know your apps and in in a regular dataverse you can create canvas model driven or portal apps which we talked about once before when you're in teams when you go into the build tab you can do a new and you can do an app or a chat bot but then there's also when you click the see all you get kind of the advanced view of things in which case you can go in and you can uh um you see your tables and create them there so you don't actually have to bounce out to flow or power apps to see that stuff that's just kind of a backdoor view so i can come in here and i can you know get to the same edit screen for my tables right in teams um one thing you'll also probably notice about teams that uh is annoying to me anyway um probably isn't for everybody else but um the powerapps app doesn't support the dark mode which that's why it's looks weird here okay um what did we have for uh for questions and did you get them answered or i think we've got one here at the very end um questions uh this is from raphael off of youtube um any sql versus t sequel i think is one of the questions he's kind of this so this is going to be kind of like a rattled list here but it's like there's a lot of little questions here um can you use any sql versus t-sql inside using this environment let's start there there's a couple other questions i'll try and throw together here too um inside this environment as far as being able to query out the data um it's going to be pretty much a power query so you can use uh well not so much in teams but uh in powerapps you can use dataflows which can you know push data in or out and it uses you know power query and connectors so pretty much you're lifting like you're selecting a couple columns and kind of lifting the whole table kind of what you're you're basically doing there you're not gonna do a whole bunch of like direct sql queries against that information you wouldn't do direct sequels again yeah sequel queries against this perfect and then um storing blobs versus azure blob storage so is everything on the back end here i mean this is all inside the sql server they're not using blob storage that to me sounds like a source um and you know under once we get under that layer of dataverse we don't know what microsoft's doing with that's true yeah we really don't my gut my gut feel is hopefully this feels like a lot like uh sql serverless right so where you have so i'm i'm familiar with like the data architecture of synapse and then using sql serverless to touch data somewhere else and the cheap way of doing it is put your csv files or your small file directly inside a blob storage a data lake data lake gen 2 essentially and then read the information directly with sql server so there may be some kind of like flavor of that going on here where there's like a sql server that kind of you know an mpp sql server that's sitting in the middle here and then there's like a storage account underneath the hood that you don't see or have access to but you it's basically reading and writing from storage there well it actually gets a little better than that even i'll see i'm not even close [Laughter] um what you can do and uh let's see where did i see that we're going into the weeds on this one yeah you can create a uh a file data type um or a i should say a column with the data type of file interesting and then uh there is a uh um and i might need to actually enable some stuff in my environment but uh um there was an option to uh actually have that file stored in sharepoint oh interesting okay if you use just a file like this yep maybe i'm using a different or the wrong data type wrong data type exactly but uh um or maybe it was image maybe not um but yeah the underneath the covers i'm sure that the microsoft is you know doing that uh that whole you know blob storage optimization stuff but uh yeah yeah there was a uh it seems like a trend that they're they've been pushing and or promoting like if you look at like architectures that people are using more now it's this idea of separate storage from compute and you can store things at lower cost and not accessing them all the time but then they're doing a you know a better job of modularizing making a module that is compute layer that could either be and now they're looking like it's sql and spark potentially too so i really like this new architecture that they're driving people towards and i think it really makes sense in cloud spaces or spouts cloud technologies at this point yeah i mean last time you know i had to deal with any of that kind of storage stuff it was well you know you keep your uh your transactional stuff on your faster disks and then you've got the old spinning discs in the uh in the back room that can yeah you know store blobs exactly um i'm sure they're doing the same thing only they probably you know don't have a whole lot of spinning discs and things like that anymore but true um but it's i'm sure it's the same concept okay we're gonna use the old uh the old flash storage versus the new flash storage exactly another one here another question around is and this is one i'm gonna i'm gonna skip around here just a little bit raphael also asks um most importantly can i export my table to excel so the table's there i need to get it out to excel can we do that absolutely and that is a uh that is a great feature i mean every tool is not a real tool until you can export to excel i mean that's that is that is my gauge for like is this is this a real tool i can use in the in the company can i export to sell yeah checkbox okay that's my first requirement yeah is this a microsoft tool or not this is yup and uh what you do and i'll do this again you go into your table and you've got you can edit your data in excel ah nice and ooh actually this is even better it opens it and then it's just kind of creating like a real-time connection to the data so that way when you hit save it kind of you can kind of publish it back in it totally is oh my gosh i love this stuff this is the best okay this is a great example this may this is going to make your whole uh hour worthwhile right yeah this is the cost of entry right here your your price to to play here has been worth by this one feature alone [Laughter] if we would have led with this everybody would have bailed out we would have doubled the audience well we've got is we opened it up and you know this is my ball and it's saved as a local uh excel sheet yeah but uh um then of course uh indication here and you know you've got a little oh look at this okay cool that you're signing too and then um of course uh and sometimes you'll even do that just to create the framework of what you're going to work with too if you're going to be importing certain things or working with data there's going to be certain ways you're going to do it through this and this has been a lifesaver i like this idea this is this is pretty slick i didn't know all this existed in excel for you but uh but yeah you sign in and then hopefully it doesn't want to try to automatically sign in um i should have and so this is the idea of making sure you have the right account access it takes you in yeah and then it'll then link into you that table so oh i love this this is this is very similar to what i see in like analyze and excel but it's again read-only version of this not not actually like being able to edit data like here i do this with um azure devops there's another connector that i use very similarly where it creates a connection for me i can extract everything out of azure devops and then push everything back into azure devops by building out and that makes things so much faster for me there's a small things that just make sense to do it in that space right i mean totally everything's gonna you know it's not a one-stop shop but excel is very versatile it's very flat well it's a very business-centric tool right i mean that's that's what we're very comfortable with when i you know add a couple numbers together i need to sum something like that's what i'm dropping into like that's like a basic of business 101 so to have something like this available to us still and integrate it with these deeper tools that are this is where you should store the data it makes total sense and then the other thing about this is it's more than just um you know publishing it's a uh it's a you know full tool here you can uh you can go to here and you can it'll bring in your lookups so you pick one oh my goodness and then it's uh you know your choices oh wow and then when you you click uh you can do whatever you want in here you can save it um like i said this is saved locally yeah like in my downloads folder yep um and then i can uh you know publish the data back and of course um i did not know this changes the read only fields but is this is this also based in like i mean this can work for the teams dataverse and the regular dataverse correct this does work for the teams uh dataverse and uh but one thing about this that i found is that this connector does not seem to want to authenticate at all if you put it like in a teams tab but you can excel works in inside teams embedding this as a tab inside teams maybe not the best idea yep that makes sense i think that makes sense hopefully that's going to be something that that gets fixed because um because it's super relevant that would hold us back from using like uh um sharepoint lists you know or the one of the big features of using sharepoint list is that ability to throw it into a tab yep be able to do some bulk stuff you can just see all the data right there boom change it and you know everything else downstream automatically updates yeah totally yeah if we could get this into a teams tab where people can use this in the teams tab then you know a lot of that reason for using sharepoint lists goes away oh i like it a lot i'm a big fan i'm a big fan of making tables in excel that's a huge one uh the last one i think here is a question around well there's kind of two more questions left one would be around just um query folding and using power query so just to be clear um in dataverse and dataverse for teams both of those will allow you to use power query correct they will um with uh with dataverse you can you know use your data flows and your power query going in and out on in teams it's just it's just one way so when you uh when you do a data flow in and you start doing your power query into the dataverse um yeah it's just you'll be transforming data just the one way anthony you wanna take the last question here yeah so there's one more question from raphael around uh licensing and pricing so this is the question that i always have around this stuff is what do you do there yep uh from a licensing perspective what's the actual question this way so i think it's just around you know what is the licensing and pricing scene for app for end users so you know how you know what does it cost me to run and there's like we talked about two flavors power apps and then we have power apps for teams um just you know it sounds like we talked about teens licensing a little bit it's kind of included with your e-licenses um but yeah what are you licensed zombie licenses even some i think some app licenses which i was kind of curious i don't know if you know anything about that steve but i haven't looked too deep into that i know that um i feel like the ability is there that you're able to use it even with some f licenses yeah for for teams uh uh it's going to be in most of those teams licenses yeah or for full dataverse i believe you need like a per app or perlis per per user yeah per month per user on either of those so it's like a very much power apps licensing type model where we'd say yes her app per user usage kind of stuff okay yep go ahead uh that's gonna allow you to use all those premium uh features whether it's uh on-prem gateways uh third-party connectors uh you know custom connectors uh these advanced features that we're seeing model driven apps i mean that premium licensing skill really gives you it opens the door to really the majority of you know the enterprise type of thing that you're going to be taking advantage of so once you start to get that personal productivity uh you know microsoft teams and and those types of things can do really really good things and even when you're in you know a group setting or a uh you know in a department setting you can usually get by with that but when you look at enterprise type uh solutions that's we're going to want to look at deeper security auditing uh you know there's more of the application lifecycle management and things of that nature cool yeah so so long story short the dataverse for teams is included in your o365 license um full data versus now okay perfect good clean break point right there at least you know that's the first time i've ever heard a good licensing break point for for microsoft you know what it costs and all that kind of stuff that's a totally different story not gonna ask that question yet because i know that's another whole session you could probably just do little data first licensing pricing and that would just be a whole another session right there all the things going on not only that but uh whenever you start talking about pricing it's like whatever you say it's gonna be wrong because everybody uh you know based on their volume and oh that's true yeah it changes yeah well with that i think we're we're at a good point anthony you want to give us a wrap up here you got a couple slides to wrap us up uh just just a simple thank you screen and i appreciate everybody coming uh this is going to be one of those things that we're going to be able to look back to uh over time that you know dataverse first dataverse for teams you know what is it why would you use it a very exciting topic thank you again steve and actually in just under an hour actually just over a half hour i suppose uh that we have another uh we're gonna have another conversation around github and have that you know and what that looks like for the power platform so again it's gonna be another good conversation steve thank you for coming back your second time we look forward to having you a third i know that you were looking to sign up pretty soon here so yeah we'll keep you all rolling maybe we can maybe we can talk about solutions and uh you know some of this uh this this deeper dive stuff into this database thank you so much appreciate it yeah thanks for breaking this down awesome guys well thank you very much that was another user group session in the books we'll get it out there on the social media things we uh if you are following along please uh go find us on linkedin we are power bi tips on linkedin as well so feel free to register there we have a whole bunch of events coming up um we have events all week this week we also do as power bi tips we also do the explicit measure podcast for anything around power bi related things anthony runs all of our power apps stuff so anthony is one is the user group leader for anything powerapps and code related things and that's we're gonna have another session here in about uh 12 uh 12 noon central standard time we're gonna do another session just around github 101. so if you're familiar with github or you want to hang around for that one as well go grab some lunch take a little break we'll be back in about 30 minutes thank you all very much we'll see you around thanks do you think [Music] [Applause] [Music] you

Channel: Power BI Tips

Views: 196

Keywords:

Id: QYj5wU-_o50

Length: 81min 46sec (4906 seconds)

Published: Mon Dec 13 2021