Using the Scanner API in Power BI - Chicago & MKE Power BI User Group

do hello and welcome to uh power bi tips the power bi user group with chicago and tommy's doing a presentation tonight so welcome everyone to the user group milwaukee um uh places downtown a few venues still people are a little um things may be opening up in the spring which would probably be better than our like are having our first meeting in february but uh hope sounds good okay so this chicago milwaukee user group cool we're all good we're good now sound audio has happened yeah we've been on here for 30 minutes already so for those who are chicago based chicago-based hopefully in the spring we'll we'll have a venue we have been reaching out to people which is really exciting we're getting close to meeting again having pizza having jimmy john's it's all great so but i think i think the very first the first meeting i think i think we all need to get together or do a movie i think that's a great idea do a big machine lose a hackathon i like it i think that'd be good and then i think we talked about this that what we should do is milwaukee and chicago go against each other and then whoever wins there will do like a baseball game at the venue like either milwaukee or like wrigley well i'm pretty sure that you'll have to come to milwaukee then look it's a bet a little a little more expensive than steak though but uh tommy likes losing beds i [Laughter] you know i have yet i think i do owe you but that's for another day but yeah so hopefully i think that's a great idea to do a milwaukee chicago joint meeting so that would be absolutely wonderful there's a lot of things today um anything from the milwaukee side mike nothing not a whole lot going on this month um i'm actually out the last part of the month so this will probably be our user group so this will be our our user group for this month um we want to update our our audience this is this is the content we're going to put out probably not going to be anything extra or special so thank you all for attending we appreciate it if you guys are hanging out here from milwaukee thanks for joining us i think you're going to be finding a lot of good value from this session i think tommy's got some really good stuff on some amazing tricks up his sleeve for what's going on tonight so i'm looking forward to it yeah so in just case you guys don't know my name is tommy puglia i lead the chicago power bi user group mike um hey i'll let you introduce yourself i think exactly hey i'm mike i run uh power bi tips with seth bauer and we also run the milwaukee user group and i help out with a number of other user groups across the us just trying to help create content and push content out there's a lot of great speakers and we're here to kind of help promote that and get people out there speaking about power bi all right yep and perfect let's go and don't forget at the end we will be joining a team's link that will be sent out at the end true we're going to hang out and have a happy hour we'll have a happy hour so grab your drinks you can start hey you guys can start drinking now so there's zombies i'm gonna wait tommy's not about to demo and drink we're gonna keep him sober for now don't demo and dr all right so let's let's get the screen as always and great timing too for this meeting um we have the november release of power bi yeah and some this is a major update mike there's a lot of good stuff going on here this is a huge one so let's get we'll jump to the screen yeah we're there we're good we got it so a few things just to note here that i i want to just kind of uh point out actually let's do this guy we have the new formatting page and bookmarks and buttons this is absolutely huge uh just if you guys have not seen the microsoft blog this is one of those game changers here when it comes to um the design and what a lot of people's pain points are again it's just come out mike have you played with it at all the the formatting pane or the bookmarks navigators let's say both yeah so i just did a video actually on youtube i explained i i was so enthralled with the page navigation and bookmark navigators i thought i played around with it a little bit i read the blog i kind of i said oh my goodness this is going to save me so many clicks this is going to save me so much time i've spent so much i've spent so many hours wasting time building bookmarks and page navigation this was huge so i did a like a quick little 17 minute video just explaining how they work how to build them and then just a couple little stylizing options you can apply with them uh to make you know so i made a a page navigation element and then showed you how to hide a page versus not hide a page and show that or not and then i also did a quick little uh toggling between two different kinds of visuals like a bar chart and a line chart where you actually have tabs that represent the different type of the different chart types so you can click on the tab and it will show the report and then you click a different tab and it shows a different one so man way way easier so many less elements um and it saves me a ton of work so i really like those two features yeah and um your favorite i'm sure by far is the scorecard visual oh yeah so for those of you who are still like my favorite um so for those the major updates on goals and scorecards and also the scorecard visual is now here as well uh a few other things that we have that we'll kind of go into uh let's get out of here and then just like i said goal level permissions power automate integration i i'm so excited about this and i think this will be another user group and kind of being the the the disciple for goals and just what that can do for a business so that's going to be a user group meeting um in the future not enough talk about that especially especially in this space so really cool things here but let's dive in let's just show what it is just get everyone kind of pumped for it so download the latest updates and let's just kind of show we'll do our demo of course it's baseball and let's just show again the formatting pane so what we have on our that is not the right one here we go so this is what we're doing with the formatting pane where you can see actually i want to choose here this is what the new formatting pane looks like and it's a little different where you can see here actually i'm going to exit out of this guy cool a little different it's a lot of different nowadays it's a lot of different it's a little bit more than a lot a little bit it's it's there's a lot of things changing here so yeah so you can see kind of here that the there this is on toggles and we'll kind of break out and we'll see basically there's no more scroll a scroll within a scroll which we've talked about before this is pretty cool um and yeah dan no love for the pie chart updates because mike would turn this uh the user group off so i can't talk about i'm legally obligated not allowed to talk about pie charts without mike turning off the user group meeting you can rotate it now and then let's see if i have the buttons in here i thought i had this one had the buttons oh i think we have the scorecard visual in here as well there we go that's what the scorecard visual looks like so you can actually see and really cool it looks like your formatting it looks good because based on based on your color formatting you can i can actually go in here choose this guy and i can actually choose if you can select it here we go you can see the what's to connect the scorecard i can change the colors why can't i do this in the service i don't know but really cool and then let's say i want to create the bookmarks and again let's say let's create on page two and if we wanted this to actually nothing here don't worry about that so let's create a button so all we have to do we'll go to our insert why do i keep doing that for it all right control one shape control one i know i wanna here we go navigator and then you can see there's the bookmark navigator and page navigator yeah so let's choose the page navigator here boom done already built boom done all my bookmarks ready to go which i'm really trying to not to give everyone a headache but you can see here what you can also do which i love about this look one of them's already auto filled so it's a no selected selected it knows it's selected it's so good yeah the amount of uh so here's how you have to build this if you want to show is selected this just gives you is selected by default it's like a it's it's saving the state of the clicking of the of the bar which i think this is a huge feature set here because in order to do this you would have to have a hidden data table that was behind the scenes and that hidden data table would have to change the formatting color for you of what item has been selected and then you have to add that to your bookmarks it was a whole long exercise on how to get that thing working but now with a single item called selected it just works it's so much better look at the font color i can change the font color to whatever i want i can do it's got so let's choose the selected here to ah that's a picture okay so i guess yeah it's gonna be get used to doing the design but let's say i want to choose this to blue now a phil tommy my guess here is if you're going to put a picture behind that fill my guess is you can use a gif behind this yes you can svg or an svg so if you use an svg it'll be it'll be rendered nice and crisp but if you use a a gif i think it will animate i think i don't need to do that i think you can animate your buttons yeah tommy that's gonna waste another couple hours of your time figuring that one out yeah so alex is already uh trolling he's like this is not the scanner api yes this is baseball data so we'll get to that but this is new this is i think this bookmark feature is to to mike's words a game changer yeah um but we're we're we're really excited about this so anyways and then finally let's talk about quickly i like i said i'm trying to garner everyone's interest with the scorecard so what we can do now with goals and i'll go here let's choose my baseball scorecard and what i can do here is i can actually one actually column settings hide a column so like it's that let's say i don't need the progress one or the due date due date might not be super important i can hide that so i'm just going to move this here i want to show and hide i'm going to say notes and due dates are done i can move this around actually i got to go back yeah so column settings i can move this guy so let's see progress is huge obviously that's going to be an owner not so much so now i can get the scorecard exactly how i want it and what like i said what what scorecards can be and where they're going is absolutely incredible we're not going to do the automate integration yet but just look into scorecards if you haven't yet so anyways this is interesting to me i think this whole development cycle that we saw here was something brand new that we've never seen before we saw the goals show up as a part of the application a part of the web application and then we saw the visual show up that produced the goals which i think is a very interesting development cycle to have them build a full visual in the service first and then give you that visual after the fact inside a local local power bi file so that i find that really interesting so a fun one i have personally i actually have one for uh cycling data that i do so uh thanks to people like strava you can see i've been trending this all year since goals have kind of been out and um it's been able to trend and actually just show me and i have automated status rules so i'm not close to where i should be for for biking but i can act i've been using this to actually see am i how far behind you are yeah i get it why i shouldn't be eating donuts and actually going with dang bike yeah exactly yeah exactly but um the goals now are automated and now i can actually connect this to other services like send me a notification going get off your butt time to go go on the bike yeah time to go ride but all this is like i said just something that i've just been using personally it's cool but it's this is something that i am so excited about and again what i think you can do for a report and integrate when you get a new request in try to say is there kpis what are you trying to aim for build a report into it it looks great in in a power bi app too so um yeah just want to kind of get that uh the november updates there's obvious there's a lot more november updates so if this is kind of like one of those power bi goals or the goals is a premium only feature i'll just say i'm going to catch that question right there because goals is premium so tom thanks for the question uh the goals are premium only feature yeah so this november update feels one of those like you're trying to thank people at the oscars like i can't remember everybody but thank you so this is this is uh this is one of the more major i think updates for the year for power bi so check out the blog uh november blog but let's gonna dive in then to the meat of today and what i think obviously people hear um i'm really excited about this and personally i think this is something that hasn't been talked about enough especially because of what it has and i so i'll consider this under my underrated features uh in the power bi world right now just because of the content available so let's go through our agenda then mike do you have anything added before we kind of jump in or no we're off we're good to go keep going keep cruising this is good let's go alex guys is super excited to see the scanner api he's like he's chomping at the bit so alex thank you very much we appreciate it man i i got expectations now you got expectations do it good i know now now now the pressure is on um so let's talk about the first before we get into the demo we're going to talk about some major what we're going to say use cases and why do it why have the scanner api and we're going to run through actually doing it some of it will kind of speed through but a lot of it we're going to actually show you there's been a lot of interest on this but again something we're really excited about so let's talk about the objective um and what is the scanner api didn't know why i was going to do that animation we'll take that out on the recording so the scanner api is not b but let's actually kind of go through what the scanner api is that guy definitely got deleted today so anyways um the scanner api in power bi is something they've been working on for the last nine months but they have really kind of built and put it on steroids not to use a baseball analogy but before is just be able to kind of scan your server see what's available but what it can do now and what it can pull is basically anything that's related to your tenant metadata so uh it can be grab your tables your data sets your reports in the entire tenant so regard regardless if you um are an admin or if you have access to a workspace it's going to pull everything in your tenant from your workspace to the data set all the way down to the measures what they've released a few months ago was the ability not just to pull in your measures but the actual expressions in them so you can actually say hey i have total sales is in 15 of my data sets is everyone using the right calculation not only that it can also pull the power query you can actually pull the queries for those particular data sets actually say hey is people use are people using the right things how many columns so all this is absolutely integral for any admin or anyone in power bi because that's what we've dealt with um for millennia so to speak since the dawn of power bi one of the biggest issues for admins was to try to rein everything in because i can see what's in my workspaces and you can try to do gold data sets and gold workspaces and try to manage it that way but sometimes if everyone has pro if you don't have uh you know if you have premium per user before they have the ability to create you know only these people can have a workspace it was very hard to rain what's going on and even if you have a lot of workspaces that you own or are an admin of it can be very difficult to what go in each day and check if people are creating new reports so that ability has not been available in the past or and if it was it was a very basic and also very difficult so that's what the scanner api is and let's kind of go um let's talk about the importance in the use cases of it so the biggest part of this is we're going to talk about is the governance so this is again being able to really um focus on your ability to manage what content is being created who's creating it um when is it being created because even by the date you can actually say how many data sets have been uh created over the last few months and then who's creating them so you can rein in those wild west people let's say you have data analysts in your departments you don't have an internal team well you can give people ability to say we'll give you access to publish a report but once a quarter you have to come to a bi meeting to learn here are our themes here's the best practices here's the standards of our company and if you don't adhere to it we will disallow you from publishing to a workspace and again all this can be in a sense managed or monitored in using the scanner api data quality uh being able to make sure we you know every time we use a sales table or we're looking at sales it should come from this particular sql table well don't check the data sets that have sales in it you can use search it's a uh it pulls out um in different data sets in this case is json but we all know you can work with power bi and json so you can actually pull and do your searches to say is everyone everything using the right content exactly um so this is some people may ask someone actually actually mike just said about what's the difference between this and purview well for first off this is free completely free so i mean her view so yeah what is it doing right purview behind the scenes is actually taking like a data catalog of all your assets and it goes beyond just what's in the power bi data model it goes all the way back into like sql server what tables are there what views you have so perfu can do a lot more of this we'll call it a scanner it scans your entire environment and kind of tells you where all the data is and preview also comes with the ability to auto detect what kind of data it observes in each of the data columns so when purview kind of goes through and scans your data sets it's saying oh i see there's some pii data here because these are names of people and these are companies and so here's phone numbers and this looks like credit card numbers so the the purview application has the ability of detecting your data and that way you can kind of search at a high level across all your assets and say where do i have names where do i have companies listed and it'll try and resolve to you all the databases or things that you have from an asset level and so the downside of this is her view starts at around and again i think it depends roughly but i think if you do with the pay-as-you-go model without a lot of scanning i think you start around 400 a month and for scanning without scanning that's just just turning it on turning the the knob go right that's your price point you're starting at around 400 uh a month not the best price point i think at least for smaller organizations but i definitely think this is more valuable when you get to a place where we've got a number a team working on top of data integrity and making figuring things out so as the data library will grow in sizes and naturally will it would make sense to turn on something like purview on this what tommy's talking about today is as long as you have the right permissions it doesn't require premium correct tommy you can just go hit the api with a service account this is for users this is yeah you it well it's an app registration okay so it's not even based on the user you don't have to even technically well you don't have to be an admin to refresh it but there's other things that are administrative okay yeah good so and the other part too is with purview not to say purview is bad but purview you're stuck in that interface in that ecosystem right you can't access the data in purview in another system with this you can really connect to this in any way so um and again part of that being free and part of this is geared towards power bi so um part of that that's absolutely huge so let's say let's talk about then we know we understand the use case from a governance or just to monitor i mean i since the scanner api has come out and since we built basically we're able to build a solution this report i protect for myself uh my organization i look at this every day i just to make sure everything's looking right people are creating the right reports hey someone created a report for this well we already have a report for that so maybe i need to contact that person or okay this is a new report maybe we should promote this so a big thing we do at our organization is we do every every tuesday we do office hours where people can come in and they can join between an hour every once a month we do a power hour which is basically a promotions we talk about new reports you know trying to spread the word so i'm i'm constantly looking at this to say hey is there a report here that's relevant for someone on my team or is there something here that maybe shouldn't be in that workspace or let me make sure that reports meeting the best practices so and then that's a conversation going hey did you use the theme you know make sure to turn this off in the app because we don't want other people looking at this so there are so many use cases here that i think for a lot of you who are in that kind of admin real pro space this is a an integral part of i think what we should be doing so let's talk about setting up the scanner api so it's an api which everyone loves because you know uh every one of us is a coder not really so it can be a little difficult so there's a few things that we're going to do is there's some pre-req pre-reqs that are vital that you get off the bat um this this caused me a lot of to use an italian word agita in the beginning which means you know acid in the stomach a lot of anxiety in the beginning just because if you don't get this right in the beginning it's not gonna work and microsoft makes a lot of details on this so there's the power bi docs and then you even have to read the azure docs which are always you know simple and straightforward so being a little sarcastic but obviously just make sure you get this side right from the the get-go then we're going to download the github repo um their thanks goes out to two people um uh fairy bowman and rui romano for both the solution the initial solution to create this in a flow and then uh the idea of the inspiration for the template so uh fairy bowman and i don't believe what country he's based out of but i do want to make sure i give him the credit and they find him on linkedin and thank him but when the scanner api first came out before all the additional metadata uh he set up this flow that we're gonna that again is really kind of inspired by and we're gonna just all we're gonna have to do is once we get our prereqs configure the flow of some variables set up the automation and let it run part of the github repo has the power bi template file that all we have to do is create it update it with your theme and it's good to go so let's talk prereqs because this is again super important um there is the we're not using our authentication this is very important to kind of understand this has nothing to do with uh there's a there's a bad joke there but this has nothing to do with you so to speak this has everything to do with the app that you're creating the access the scanning is all within the service principle uh app that we're creating or the azure ad app so as when we create it make sure there are no admin consent required permissions on it honestly it doesn't need any permissions at this stage in azure so don't enable anything create the app make sure there's no admin consent i even make sure there's no user consent at this stage because at this point you don't need it uh the reason why it because obviously a lot of you're going that sounds safe um but they're secure we're creating a security group in azure and all we have to do is add the app to it and then we're just going to go into power bi admin settings and make sure it's part of a re the power bi api access and once we do that we're ready to run so how do we get the repo so i'm going to put this in the chat because this should be available yep i'll snag it tommy too and put it in the okay youtube chat stuff it should be right here so we'll pull this guy and let's kind of run into our demo so for those who want to follow along all we're gonna have to do here is we're gonna run into this repo here and we're just gonna download it so let's go let's go and download if we're gonna just download the zip and we're good to go so let's start then first with our prereqs just like i said to get this right and i'll kind of show what some of the uh registration looks like also there's a blog out here if you want to we'll put the blog out here too this is another step-by-step as well that kind of goes through everything with the app registration this also includes all the links for the documentation so just to get the service principle right creating the azure 80 app and then allow the service principles so let's go to microsoft azure and what we're going to do is i'm going to go into my active directory and we're going to go to app registrations i want to show you the one that we already have and basically to show uh what we uh what's already available so you can see here that we have an app registration we created in the app and again we're going to go making sure that the api permissions here you if you see this it's not going to work so i just want to make that important this is a wrong if if any of this is here you are not going to have uh um and you're talking about the power bi service 22 section right is that what you're talking about yeah the uh yeah yeah so what we really want is the uh pool i believe it's this one this is this is tommy checking things out and seeing how it worked and how it didn't work yeah you're not oh so it didn't work this is that's exactly okay got it for your benefit for your benefit so all you have to do is just create a an azure and again if you don't have the ability to do this um ask your it it's like can you create an app registration we don't need any api permissions and add it to a security group and you can be part of the security group too so let's say we're gonna use a pool uh this scanner and again as we create it it's a basic kind of setup uh all we're gonna do from here you go back to your uh your active directory we'll go to our groups and we'll basically go in here and we'll see we have a pbi scanner here you'll see here the pull your scanners here and that's the only thing in that security group simple uh at this point all we've done is we've followed the instructions for to create the app registration right and we've added to the security group also vital that i want to make sure that's under uh from what you need because you're going to need the a few things the tenant id the client id and the secret so all we're gonna have to go is our properties you're gonna we're gonna copy and paste here our application id which is our client id and then all we're gonna want here also is our tenant id so uh you get the tenant id from just going to active directory got our 10 node id so object id is also our client id so what else do we need we need a create a secret uh for those who have not dealt with azure or at least active directory for like yours uh truly once you create the secret you're only going to see it once so make sure that you copy it the first time store it somewhere secure but what we're going to do is we're just going to go to our i believe it's no polar scanner and actually no we want to go to not let's go back we'll go to our app registrations and we're going to go to certifications and secrets and you can see here that we're going to create a client secret call whatever you like and once we create this it's going to create it for us a client secret copy that somewhere important not the secret id the secret value um i have had people on github uh there's a few discussions you'll see on on our repo where people are like it didn't work i use the skit you know the id it's like you might you have to use the secret value so if you do it wrong the first time again no worries because you can create another secret secret value so what have we done at this point we have created our app we've added it to a security group we've made sure that that app has no permissions or admin consented permissions uh especially the power bi if it's microsoft graph it's fine but we've made sure that there's no admin or in my case like i said i once it happened um there's no permissions consent in consent permissions for power bi within azure portal and it's in the security group that we've created and we've grabbed the client id which is the object id the tenant id and the secret value at this point now we are good from the azure side so we have everything we need from the azure portal and all we have to do is jump into power bi remember the name of the secret group as well so let's jump into power and let me know mike i know that you're following along too so kind of for the sake of everyone else if i'm jumping too fast or do we want to uh how are we doing i'm comfortable in this space i'm able to kind of follow along here so yeah keep going um we'll see how people follow along i don't see any messages yet in the chat yeah so we're good to go i'm not going to show any i'm happy to show anything if if anyone is um wants to see something real quick just like we would do live uh exactly just add on the on the chat if you just want to see something again just for your uh for your own sake so we're happy to kind of uh backtrack if needed yeah just so make sure you make a comment and we'll try and recover and get you yeah perfect so seeing people are good again uh and let's kind of jump in and let's go to power bi so i'm in power bi and i'm in this amazing thing called scorecards and what we're gonna do is we're going to at this point at this point i need to be a power bi admin so this is important i want to see dark mode um at this point i need a power um i need to be an admin of power bi or at least that permission obviously with the azure portal you need to have those permissions as well but at this point let's say you you don't have this ability in azure portal a lot of people in honestly a lot of people shouldn't it really should be your it team so if you don't have the ability to go in the azure portal you can literally just send them the instructions on the power bi the power bi dot tips blog say here's what we need let me know in that setup and tell me that you know here's uh and we'll be good to go but i still need to be a power bi admin or somebody does we're going to go into our admin portal we're going to go to our tenant settings and we're going to scroll down there's an area called api permissions we're going to go here and it is all the way at the admin api services so it's this guy here in case you haven't seen um obviously you can search for it as well but you'll notice there's three here that we want to do and we're going to actually enable for each of these because there's the initial service principle to use read-only power bi admin apis at this point you go hey that makes sense we just that's what we're doing so that's the first one but if you want that enhanced a admin apis and this is our first one you'll notice that we have a few others here you also have to do some of this in per view too if you're setting up per view so if you've done purview this should be a little similar but you want to enable you want to add that security group that we've created which that app registration's in to these three settings enhanced admin api responses service principles to use read-only admin apis and the final one is enhance admin api responses with dax and mash up expressions this one's not essential this one's actually for creating um writing dax and getting a result back yeah but the the yeah and one thing i wasn't putting out here just from a best practice standpoint it's always best practice to make security groups and try and add these security groups here because then you have a centralized place to manage all those properties back in azure so that's kind of what right that's just the general rule of thumb um when you're building these kind of things or you're providing access to things in groups right really highly recommend doing things with service groups or service principal groups as much as you can right absolutely and and no mike you're you're spot on so at this point and again you oh the blog kind of outlines this as well well we have created we've done everything in azure and i think i i will say this for the fifth time just because i know once you enable it for some for a lot of people if you initially enable something and even disable it there's still some there's a bug or an issue where it doesn't work where you have to create another app there are no admin consent required especially with power bi in the in this service principle app um we've added to the security group we've got our client id tenant id and secret value using a security key then we've added it that security group to power bi admin antenna level settings with the enhanced api responses at this point we're ready to go we're ready to get that github repo and begin to get this set up this is the easy part now so i downloaded the zip file i'm gonna go here i'm gonna go to the zip file open this guy up i'm gonna extract all get this guy going and there's going to be a folder here there's going to be two folders there's going to be the power bi automate and power bi or power power power bi automate power auto main power bi the power bi is the template file that we'll use later power automate has the two flows that we're going to import into flow so let's go in i'm going to go and now into [Music] i'm going to go into flow in my tenant and all we're going to do is now you'll notice i have a few here just testing out but we're just going to go to the import button so this is not a solution this is the other thing as well um we did not create a solution for this these are two individual flows um just because solutions if i think if there's a use case we can create a solution but we just have two individual flows here that we'll do so all we're going to do is we're going to choose this import in the top right again to here let this refresh and we're going to upload so we're going to go into our downloads again what we extracted power automate and you'll see there's two a refresh daily or the api flow call we're going to do the api flow call this is the one that again please give a lot of credit to fairy billman for all the work that he did with you'll see how intense this flow is if you have just dabbled in flow and you wondered how complex can a power automate flow be this is a great use case so let's go in and again these are both zip files it's going to start uploading and as we wait for this to upload if you have not used flow enough yet there are huge things for this so kevin arnold asks what licensing do you need for power automate um they did just change no they changed it for power apps there is a pacific power automate license that you need i think you there's a lot of connections you can do for free but the good ones obviously you got to pay for so and those are the premium license so all we're going to do here is a few things uh if you've never uploaded a flow let's go ahead mike so tell me i've got a question about that one um yeah this flow is using the one that we're uploading right now um this one does have is this a premium one that we're doing right now because it's an api call honestly i i i don't think so i don't think you need premium for this okay good district just double checking for the seventh question on there we'll see unless i should check my billing for azure though i don't have premium on mine so i will watch it as it goes through my tenant and i'll let you know if it turns on premium on my own so i'll share point yeah so i think these are all standard connections good i think if you have like an e3 or e5 you should be fine okay so um for this good to know we'll find out the all together we'll find out we're gonna yeah uh so a few things if you've never created a flow or if you never uploaded a flow actually if you never use the flow check out the power bi tips uh power platform user group where they talk about things such as powerapp apps and power automate there's the there's the plug so there's a few things to uploading the flow that we want to talk about here uh you'll see the first one is import options and related resources just important to know um the import options are going to be certain really the flow itself for the most part we're going to create as new we're not going to overwrite anything especially in this case and then the other thing is related resources these are our connections other things that might be a power app or if especially if you're doing solutions other things that are connected to the flow in this case the sharepoint notifications those are our connections which i believe are all standard so everyone here as long as you have an e3 or e5 you should be fine so let's kind of dive in we're going to go and we got to go through all these options create as new select during import and we're going to click on this little wrench say create as new and i'm going to call this scanner and then these next two and again you have to do this before you can import it and if you've used flow you can just use an existing connection like i already i we use flow so i have sharepoint connection so i'm going to do here selector import it should find within what i already have my two connections here so i'm just going to say yeah i just use that now i could create a new one so if you've never used flow at this point what happens is you have to create a new you have to create a connection source right because if you don't have a connection to sharepoint this will not work for you so make sure at this point sharepoint library if you have a bi one or uh something an organization one internal team one uh that's what we're gonna use technically you could configure this to be your onedrive i don't recommend this because this tenant level of data in the same way that you should share your power bi uh files and somewhere where people can access it and not only on your own machine same idea here if we needed to create new we just create new pop-up comes out and we'll go through a connection we don't need to do that so i'm just gonna save this one and i'll say notifications pretty easy this is honestly something you can delete later because i just get a notification on my phone when uh the scan is complete because i'm a nerd like that so and that's awesome it's pretty simple and at this point we're just going to go through and we're going to import now at this point what you want to do is identify the sharepoint library that you want your flow these files to live so for a lot of people um they sync their sharepoint libraries on their computer i think that's a great idea at this point what i'm going to do real quick tommy before you move on yes absolutely anthony uh has a question around are there any special requirements or permissions required for the sharepoint folder that you're linking to no the only the only real permission is if you can edit that sharepoint library document just make sure that so okay that's all you need yep so in this case what i'm going to do is i have here which is i believe my one uh employer consulting i have this bicoe i'm going to go in here and i'm going to say i have this p i created p by admin and i have meta files now at this point you'll see this this flow gets a little uh you'll see okay we'll just go through so and i'm going to keep all these files in this meta files here right so p by admin meta files and at this point what you'd want to do is copy the folder name and copy the link to sharepoint how do you do that if you're on your computer all you have to do is i can actually just right click and show more options actually i want to do here i'm going to right click on the sharepoint folder i'm going to show more options and i'm going to say view online because that's part of onedrive so that will open this guy here and you'll see that basically all i need is this part in my sharepoint library i need the sites the sharepoint sites and the bicoe that's all i need copy that to notepad one onenote wherever you want write it down but that's what we need and we just need to know the folder that we're going to save it in so hey looks like it's all good to go the import yeah so tommy one question about that one so i'm i'm i had the sharepoint connection in mind uh didn't have the connection the notification connection is that i had to create a new one for that one do i look for a power automate that is notification is that what i look for it's a connection called notification yeah so the connection is called notifications got it it's yeah it's literally notification to get a notification on your phone if you have the app if you don't have the app you will never have to use this so but technically i probably should remove actually release four i'm going to just remove it and say if you want a notification here's how to do it because it's a it's unnecessary to the overall solution gotcha and makes total sense yeah but it's fun to have anyways so if everyone's with us yeah i think so this all right so let's kind of let's open the flow so this should be saved on my flows but i can just go here i'm going to open flows and what i'm going to do here is i'm going to get this guy here and we're going to edit our flow so what you're going to notice at this point as we wait on this beautiful computer tools is we got we got a bunch of stuff here now you'll notice this is a complex flow and again there's a lot here that we're going to kind of dive into and we're just going to get right we don't have to edit anything here everything that you see at this point we don't have to touch and you'll also notice the names of the variables because these are variables that we're going to use later in the flow we don't have to touch it uh you'll see like uh do not edit there will be places where it says do not edit and there'll be places to say entered here hey look i see string tenant which is the tenant id the client id and that client secret value i also see sharepoint library and the folder location so these are all things we've already copied and pasted our tenant id our client id and client secret all from the azure portal the sharepoint library url and the path of the folder so let's look at some of these again just to see what they look like you see the folder location and again it's it's usually gonna it's usually gonna follow the shared documents and then the next part is gonna be the the rest of your file you can get that just from your windows explorer share documents forward slash p by admin slash meta files good to go now the next one and i don't think i have this in here that i will recommend is create another folder and i should have said this before so i was having problems at one point with the trigger working and the way this trigger works is we wanted to refresh daily and again you have to obviously do this push um or to refresh my data set so at this point what i wanted to do is uh you don't necessarily have to i recommend this create another folder in the same p by admin called meta trigger we're going to have that because what we want to do is we want to trigger our data set to refresh every time a new ap every time a new file is created in our meta files again i'm i'm sure there's a better way for those who are more flow advanced don't hate me but um this is kind of the way we're going to do it also as we're wait just a real quick thing i wanted to showcase to my carlo here is if you look in the bottom left of the corner i have updated all my slides to now have the new power bi logo it looks absolutely incredible and great doesn't it i think microsoft would be very pleased with your tommy and that mvp of the year that is my adhd for the day guys if you haven't seen it already so at this point again we can see the the other things in here we're not we're not touching if it says do not edit we're not touching so all the things we're obviously editing is the tenant id the client id that secret value the sharepoint library the folder location and the folder location trigger now honestly at this point if you've inputted that this point this flow's done right so what let's kind of just walk through real quick what this flow's doing and we'll kind of dive into um the next part of this and we'll get this running so it's basically again this is part of the brilliance of this is the scanner api is pulling based on a single workspace it's pulling everything in here uh what fairy bowman is doing is basically looking and scanning looking at all your workspaces storing those as a variable and then doing a loop to basically say okay look at each workspace and putting that all in one json file um i've tried to recreate this myself but again this is pretty incredible stuff do not touch anything else at this point we are done the only thing we have to do is the variables all right so we have the flow now let's upload the next one and the next one is basically our push i would recommend also to keep the um your scanner api open the that first flow we created because we're going to need this url all the way at the top of the flow that we've just imported there's a trigger that is the when an http request is received um and there's a url here we need to copy that url because we're actually going to extract parts of it uh so just copy that store it again in somewhere but that's at the very top of the first flow we imported so let's go in also make sure you enable the new flow in some cases it may not be enabled you just have to turn it on turn it on by going to your more commands and you'll see turn on so at this point we're going to import the next flow so real quick yeah i'm catching up here just a bit here stuff on a question here you said the variable for the sharepoint library you start at the name of the sharepoint library and then kind of reiterate out after yes so you'll see it should look like this it should be your oh site sites and then just whatever that sharepoint library name is so basically the site name back to the beginning the beginning of it exactly perfect exactly that makes sense so if it's you know companyxyd.share forward slash slides forward slash sharepoint library name and then the folder location right behind that one just make sure we got that one right there you start from the name of the site so that you're basically doing the sharepoint um now you're into the shared documents section so now you're doing shared documents and then the um the is that the folder you're actually going into so it's a shared document so then there's a folder you're dropping it into right right right so shared documents will always be there so always include shared documents yep um and then the other part and then again the other part here is that meta trigger which should be very similar it's just that same one but meta trigger so trying to follow along quickly here so i can get closer to where you're at no problem and like i said for those if anyone is not following along if we're going too quick please let me know and we can kind of we can backtrack so we want to make sure that everyone is able to follow along especially mike who's incredibly slow at all these things so i guess i don't understand exactly what's happening there on the scan the shared documents you're doing shared documents but is is your domain where you're putting these things for the trigger and the meta files are those like if you go to the url domain name when i look at mine i'm seeing just form slash all items when i go into folders that are there it's not showing me that the the folder path on those things where so just kind of curious can you just navigate i'm just cuz the the curious thing i'm questioning is in one of those items can you just go to where those live on sharepoint just show me the sharepoint site yes for those maybe that'll help me see so i have absolutely so we should have this boi this is the same one oh interesting right this is got it so i see that you have shared documents forms slash all items in the url but what you have down below here is something a little bit different so okay that makes sense just because we're the way it pulls because the way flow wants to receive it is not going to be that url like obviously all mumble jumble yeah okay so it's going to be different than what you're going to see on the sharepoint site url which is what i was anticipating that's where i got tripped up i think right got it all right i'm going to see if i can match your stuff based on what and then what did the did you actually add in the tr the meta trigger file yet or is that kind of the next step it's i i created that folder as well and it's in this it lives in that pbi admin folder so i i have p by admin and there should be two folders there meta files and meta trigger gotcha all right i'm gonna or name it i'm gonna try and i'm gonna try and name it as close as i can there so you can name it at this point you can name it really whatever you want yeah make sense yeah yeah tommysucksthetraining.com or yeah so or no no i'm just making sure it's way too fast yeah well this is why we record these things so we can slow down a bit so that's that's totally fine just making sure that we're getting everyone through there so yep cool keep cruising so yeah we should be good to go so now we're gonna import the second flow import same way we're on my flows and now they're going to use that second flow and we'll wait and again let's go to upload should remember that same location we're going to call this refresh daily or we're gonna select the refresh daily one again there's a scanner api role call that we already selected and we're gonna choose gosh i love zoom it sometimes uh refresh daily because this is going to be the thing that runs every morning and it's going to trigger our first one so we'll upload here that notification one is going to be a bit of a hiccup i think i would recommend if you're doing these i think that pulling out the notification might be a good idea just initially to get people working without having to notify people yeah i agree and we'll definitely i think we'll definitely do that so at this point what we're going to do is you'll see this ppi scanner refresh we're creating as new if this is our first time there's no other connection we have to worry about so we'll say create as new i'll go daily refresh scanner save it as something we'll click save we're all and then you can see now we're good to go click on import and let's go we'll import this now again it's already ready to go remember this point too i should have this url open uh the scanner api now here's what i want to do here so let's open our flow the new the second flow our refresh daily and you'll notice there's three variables here and we're like i said we're extracting the url from our first flows trigger so we're going to move if everyone should have at this point two flows open our scanner api the long call with our variables and our refresh our daily refresh uh trigger if you guys have that ready to go all we're gonna have to do this point is basically put input these variables so what we're gonna do and again if uh can you run the flow to create any history files i love that you asked that anthony i'll show you exactly that so let's do this we're gonna go back to our first flow and we're gonna copy this http get url the trigger action what we're going to do is i'm going to just copy and paste this to a word and you're going to basically notice there's a lot here we're going to go to format and all we're going to do is we're just basically going to copy and we're going to extract three parts of this so i'm going to show basically exactly here you'll notice there's a first part last part in modified sense and the reason why we're doing this is because rather than for those who are asking hey why don't you just set the tr the flow to be a trigger right uh or reoccurrence rather than having the second flow well two reasons one there's additional setting that the power bi scanner needs that we're doing in this flow and two try if anyone has tried to remove a trigger uh in an existing flow and try to input with a new one best of luck to you so the first part here is what we're going to do is i'm going to actually go to an existing one just to show this first part is everything before the modified sense so it's going to be everything here okay so it's going to be if you're actually looking at that url from the this is remember from our first trigger it's going to be everything before the for it's forward slash modified sense and everything before it that's our first part we need that and we're going to copy and paste this into the uri first part or url first part and you can notice here when you copy and paste the trigger the look kind of i'll show designs to see mike's but i'm glad mike you're following along because i know the confusing face our scanner if this is i'm getting lost i didn't i didn't catch where i had the text notepad i must have missed that step there where you grab that url where did that come from this is coming from our first flow the the our refresh and this is the trigger yes is this is this just the url of the first flow no no no no no no so if you go i'm a flow newbie i don't do a lot of flows no problem this is perfect because i i bet a lot of people are so go to your first flow yep and you'll notice the trigger the http request is received the variable is copy different yeah the trigger trigger that's what i missed okay and we're gonna copy and paste and what i'm gonna do is i'm gonna have these three things you'll see here this is all i'm gonna need this is our first part yep and i'll i'll do the drawing this is the first part got it and this is our sec this is the last part got it so now we're going to go to our beef our daily refresh i'm going to you're just going to copy and paste that in here into here and then you're going to um do the last part and that's everything after the question mark and the parameter the url parameter so it's question mark everything after makes sense now here's our fun part because why we're doing this this way is the way microsoft flows work is they work or excuse me the way the scanner api works is you need to say hey when are you pulling the flow sense all are you want to pull everything of from all time or do you want to use you know your data and a cache that already scanned and plus anything new obviously your first time you're going to want to do all um so and it doesn't really unless you're doing this five times a day if you're doing it once a day you can always do the all setting and you'll be really generally fine um but just want to like put there the default should be all and if it's not just say all in there and you'll see there's a comment there that says all for everything meaning pull all time and even if we already had an existing scan pull it again in case anything changed uh or put a date time in in a certain format the um pto or i don't remember the exact name of it but tmz i think yep that's like that's like a global time zone that looks like that yeah the global times so and it has to be exactly in that format or it won't work yeah yeah and the max you can is is a scanner like hey just pull everything since you know uh 15 days ago but it's got and at this point if you wanted to add in a modified sense you can add an existing uh action here and you say pull now convert the value and put that as your string and as the value there and obviously make sure it's the way it is this way so alex dupler is actually asking a really good question he goes is this setting gonna break someone with a tenant with like a hundred thousand pvi seats yeah so that's that's actually alex that's a great question so if you have a lot right the first time you do it you have to do all because you don't have any other existing scan if you've never scanned before the first time you do all if you have a ver if you are enterprise i don't know someone who works at microsoft or a big company yes a big company um at this point what you'd want to do is something where you basically go in say add an action and you can basically go into built in date time current time and then you would pull in another one that would format the value to be in that tmz format sure and then you basically pull that into that step your value here yeah that current time so and again but it has to be in the right format so if you are someone who works at a very large company the first time all because it's since your last scan not since the data's changed since your last scan so it's going to pull if you say since yesterday it's going to look at that last scan plus anything new because it's going to remember again the existing data so that's that's a great question um but it's pretty relatively easy to add that modified sense in here so is this the modified sense is basically going back in time and grabbing a time stamp it's saying hey i'm going to go back to this date and time and say grab me anything that's changing as of this date until right now when i run the api that's my latest scan right that's it that makes sense because every time you do a scan it remembers that the data in there makes sense so it's saying yeah so if you pull something 15 days ago yeah exactly exactly so because it will remember you know anything that hasn't changed really so uh and the scan will also i want to make a note 2 the scan will look for anything new if you had a data set it's already scanned but something has been modified it will pull that in even if you haven't modified since cool but the first time do all and then for those with a very large enterprise a data set um [Music] yeah put in that scanner refresh so at this point hey we're ready to go make sure both flows are on make sure that you set the settings all right and all we're gonna do at this point is um we're gonna really again it should run every day but all we have to do is just let it run let's go to flows now which one are you running you're running both of them or if they run one no so you can't run you can't run um you can't run the uh the http request one gotcha you can alright so so all i have to do is click run yep your flow and once that's done the scanner api one will start running i'm glad you did a good write-up on this tommy this this is this is one that deserves a nice long step-by-step write-up there's a lot of little moving parts here that um make sense but this is right this is really cool stuff and i man i like the flood the idea of the flow is able to pull this api information oh yeah yeah but again if you hey if you want to have some fun look at the scope that fairy did i always want to have some fun always right count me in for a good fun time in power bi yeah oh this is power yeah so at this point the flow should run if you check the the scanner api now i have been like i said been doing some additional things it mine's running mine had some timeouts yours shouldn't yours should be generally fine um so yeah um mike how's yours doing i have just it said i had a little error where i said my flow was off so i had to click a little button that says turn it on and so once i turn it on it says your flow has successfully run to moderate modif uh monitor it words go to the flow's run page so i'm actually watching it run right now okay and we'll see how long it gets until it fails because i did something wrong probably how long is it at right now um oh it it got through the initial steps and i failed at the http call other requests so i probably did something wrong in my get request there okay let's i think for the sake of time let's kind of take a look at your other oh it says error uh workflow trigger is not enabled could not execute the other workflow oh you got to turn the other flow on too see there simply simple issue again you know look at that let's turn it on let's enable it so click on the other flow uh let me switch on uh yeah click on the other flow here um where's the turn oh turn on there's a little power button got it it's powered on now now i'm going back to the scanner refresh and now we'll hit run on that one and we'll hit run flow i don't want to say i'm slightly sweating right now but i am slightly sweating right now we'll see how good the timer is how much did mike pay attention looking at it now it went a little bit longer but i still failed it api is not accessible for this application so i must not have added the right permissions to something so i think that's i think that's a michael problem i don't think that's wrong with your flow so far so i think we're good okay so keep cruising on your end but don't look at me so and i would like to hear from other people again we did we we did go through some of this a little quicker than i think uh just for the sake of time but like i said it is recorded in the blog step by step but if anyone has been following along at this point you should be successful um but let me know and also we'll have a happy hour later yeah we can do it for me uh i'm happy to work with anyone message me on linkedin if you are having issues there's been a few things that we've have fixed um over the past but uh we should be good to go so at this point let's say we had our successful run right we're good to go we've created the file what it did it created a json file and again for those listening let me know if you had a failed or success so let's go into our little guy here and there it's going to be the template file it's interesting i'm i've been getting to have issues in one tenant like in my test tenant but my other tenant i have had no issues in the same exact flow so figure that one out yeah so in the power bi folder there is i want to pull this up again just to have uh open this template file and all we're going to do now at this point it's important to have that uh we're going to the sharepoint url again and the meta files folder so we're going to say power bi tips this is going to open uh power by desktop oh we can input water here so i'll just go here i'll say import a template i wonder if anyone imports it that way if anyone just opens it go into power bi import this guy and it's going to ask you for two parameters it's going to ask you for the sharepoint url and it's going to ask you for the last part of the folder now of course the desktop is opening on the other one but that's okay so let's see so we'll see here sharepoint folder and the folder filter and this is the deepest subfolder so what i'm going to do here is i'm going to put in my file and i'm going to put in meta files now at this point you'll see once you input if i can move this that's always fun we're trying to move a folder over here we go this is the i i'm trying to turn off the multitasking windows 11 at least the when it likes to snap to things so oh yes because it will freeze the heck out of it you can see i'm slowly bringing this over windows 11 has its good parts there are right now a number of negatives still i'm just not quite ready to jump all in yet right so let's let's show this again so we'll go to transform data our parameters you'll see our folder and our the deepest subfolder that we had and that's this is the meta files and let's say refresh now and we're going to apply our changes and once we do this if again if as long as the flow ran this is gonna work like a charm so um if you need to connect to the first time that sharepoint folder you have to do so so i and really in good news i was able to i think i copied something wrong or pasted something in wrong on my flows so i recopied and pasted them in and now it looks like it's running on my end so it wasn't you tommy it was me well i you let me know so we'll see what happens when it rolls through here i'll i'll keep you up these the first time can be a little slow okay because because you're all pulling all the data so that would make sense yeah you're pulling a lot more than you normally would yeah that would make sense why it's it's a little slower there yeah cool so no one has said it's failed which either means that no one's following along because i went too quick or it's working just great so let's go with it's working great we'll go with that yeah so you see all the data's loading in now and again at this point you have the part that i love about this and i really am a proponent of the power platform because at this point once you publish this you're done because it's always going to run once a day and uh you have the data everything is configured now so so i'll say this to me the flow that i pull imported in i think in the one that's triggering the daily flow pieces yeah that one looks like there's a setting there for like the trigger timing it looks like it's set up for one month right now just the way i imported it i didn't change it but i think that's the full setting so i just want to make people aware it will it looks like just double check them and maybe tommy you want to point out now this is loaded you may want to point out where do you change the frequency of the triggers to run in the flow yeah oh that's a that's a great point oh um thank you mike so the reoccurrence you can set up for as quick or as long as you like so you see the scanner api the recurrence the blue one so yeah see right here here's a little one yeah our trigger once that trigger so once per day and you can change that but if you take the drop down option on the day level and uh i believe mine set up going in the way i imported it just as is so check it on your own but when you see that you can click on the drop down frequency and then change that from day i'm assuming there's like a week in there too right day weekend year yeah day week so once a week once a day don't do once an hour a minute if you want to break if you want to get it email you very quickly don't do that so yeah so the reoccurrence if you want to do once a week you can again depending on your company depending on your settings generally one day's been fine for us middle-sized company um you know 250 reports minimum sure once a day works so cool all right so let's go we have the report now so um if we're live at this point i say raise your hand if you're you're with me so far so mike how's that flow doing you are it worked it's still it's thinking it looks like it's doing that nothing's failing so it's it's grabbing all the stuff so it may time out it may time out at some point but we'll see what happens yeah so like i said for some reason um if people have time out of issues let me know because i have been seeing a few more so but i think we should we should be good but if unless let's at this point dive into the report yeah for sure mike please interrupt me if it works or doesn't work i'll let you know because like i said the most updated version should have fixed the timeout error the one um but some people have had an issue some people have been generally fine sure so we have our report here and you'll notice here we have another quick thing on the top right that's a power automated trigger just how to add it but what we can see here is i can actually see in this tenant view and you like the power bi theme is our workspaces and our data sets and this is phenomenal so this is just the cusp of it if i want to look at basically what's been created i have a pivot table here and if i just wanted to go in it's like you know what just show me everything based on the created date so and i've actually created a personal bookmark for this where i actually see when is the last data set that's been created okay i have this baseball data set created on 11.3 i can see there's five tables i can see the number of users awesome i can also search based on the datasets who's configured by here i can actually this is going to show me the email address of who configured that data set who created that data set and i can also see the data sets by the workspace so let's go here what i also have is data source by the data type and uh this was a little configuration in the in the power query where we kind of because with sql and etc but you see well i have how many are based on the web how many bases are on a file an extension sql server or the azure data lake storage and what i can do in here is i can actually dive in and see what servers or what databases so let's say there's the database that someone shouldn't be using i can find out here in a second so really powerful stuff so let's kind of dive in um the sample that we showed before that baseball db that we showed so what i'm going to do is i was going to click on this and we created a bookmark in here see the details for the baseball db and we have just some general information here we're going to see basically a few things here the table names i can see basically what tables we have here i can see the columns the calculated columns and the measures and what i can do here is pretty simple i can actually let's say i want to see okay how are people calculating the bad inquiry because let's say you're the mlb everyone should be using the same bad inquiry if i just click on batting table here you'll notice the bottom left is actually going to pull in well it should pull in the query it's oh there we go it pulls in the actual the entire query for me that i can actually see the source to filter what i'm doing i can see the source is github because i'm actually pulling from a github repo um baseball databank would actually kind of keeps up the most up-to-date baseball data and i can see basically select the row the batting's db i can basically see all the the expression here so what i can actually do here is again i can filter and if i choose the table it's going to show me again it's all filtered i can see what measures are in there okay at batch for home runs i'll select here and it's actually showing me the dax query so i can see for this data set what measures are in what tables and are they using the right information so let's go back i can also see if it's in a dedicated capacity i can see the data set id sometimes you need to use that if you're pulling in the refresh data but it kind of gives you that general information let's go back here and again i can go back and create the workspace so i can see basically in my high level view of what's in here so what i'm going to do here is i'm going to click on data sets data sets is just another high level view and again you see what tables are in there seeing again like data sets with more than one reports so that's actually showing me hey uh van this van are still puglia actually has two reports in here and is actually helping me understand like how many data sets are connected to the same report which again is a huge uh um solution especially when we're using our gold data sets and then i can look at the tables and i can actually see again part of the expressions i can see all the k like the the table name it's in so i have this some of my uh in a use case where we don't want to use measure tables anymore we used to but now we don't want to some of my reports still use measure tables so i can actually see what's doing here and i can actually see what the data type is i can see this is a calculated column or calculated measure i can see which ones is just a column and then i can see the actual uh expression here i like the little icon that they they put in there for each one of those that's nice that's from that's from ruby romano yeah nice so i gotta go rui thanks rui so and you can see that are we using the right expression and again i can click here select the data set name um i personally like this particular one because we use this for our selected values so if it's filtered show this value just like on our drill through pages so and again i can see what tables are in here as well i can also see what measure uh how many calcul like what uh tables and how many measures so where the uh table is coming from and then one of the more powerful one is the data sources so i can actually see here basically where the data source is coming from it's like okay uh we have a lot coming from file let me see where that's coming from so i'll click here like okay i have this sandbox data housing sales prices you know what that shouldn't who's using a local file that's that's a conversation or if you're using your your databases it would show you this as well where you're going you know what you know that's the wrong database that's the dev data set so we need to use the prod one just to make sure everyone's using the right things the last one we have is users where we can actually see basically what users have access to what reports and this will give you that full view where i can say okay for tom uh say michael perez i can see these are all the reports that he has access to so this is the and this is actually something that's been a conversation microsoft of the blog is the user artifacts parameter in the scanner api so that's what this part is using and finally our details just our tables of if we want to see it just kind of all that additional information our ids um just kind of if we need to extract anything as well but really from the summary page we have that ability to kind of go in and see all the information on a high level of what's going on in our in our data set i can see how many workspaces how many data sets how many reports uh and again even data flows as well and again this scans everything so how did your flow do it's getting hung up on the main flow so looks like everything's triggering uh there is a do while loop where it's getting into determine the amount of brake uh uh batches of 100 and write the variable batch account and then it's doing this do while until is it still running yes it is okay it's called one minute or so it could be because this is your first time running yeah it i'm not sure it says the input is batch count of one and the output is batch count of one i don't know because it's going if you have basically it's looking at if there's the way it was configured is if there's yeah so if you have more than one it's going to patinate it so to speak gotcha okay because you get all the data right exactly gotcha um cool but yeah so and this is the again this data set as long as you have the json file because if you wanted to you could pull through all your json this makes it a lot easier i also i'll encourage the community too if you had any examples i'd love to see other contributions here but this is a report that we use uh in our weekly meetings to see what's going on in our tenant again if someone else is creating something obviously here this is my own personal tenant i'm not uh uh i don't think this internal meeting but this is kind of showing what's going on when things were created how many tables i see uh one of these has you know you can flag this stuff like hey why are there 18 tables do you need 18 tables in a data set i can hear seth in my head saying that's too many why are you so many tables in there yeah um but i can you know and begin to really dive into what's going on on my tenant again i can see what this log analytics workspace has 22 tables do i need that that's something this is helping that first part to identify where can we optimize in that governance and again gives that ability to see the data sources as well once you have those flows configured this runs every day once a week but you'll always have the data and you always also have here the last time the scanner was refreshed so everything should be kind of and again pre-baked ready to go sure so at that point let's kind of just dive in real quick to what we did and um we'll kind of jump on the happy hour i like it so we we did the flow we uh imported the power automate flow we did our prereqs we set up the scanner flow in the daily run flow and basically what we did from here is we got the power bi template report we put our parameters in we have our summary pages our data set pages and we are good to go at that point so really that's all i have um again i'm happy to kind of run through the report more so does the scanner api give you the workspace users as well it does not um i yeah well we need to do q a yeah yeah is there any other questions that go in so um go ahead and throw in if you have any questions that you have here at the end about the report about the process things around the flow itself uh feel free to ask them here i think it would be appropriate to do that now before we jump over to happy hour just so that way in case someone has a question on the video after the fact we can capture those comments so anyone else have a question feel free to answer ask your question don't see any other questions well tommy thank you for a great demo i think that was superb came out really well i think uh you know despite my best tries i get close i got really close so it's probably something it's not aired out yet like i said the first time the flow can run very slowly yeah okay excellent um oh we're getting a few questions yeah yeah a couple questions coming in uh will it be pop oh so we already asked it it will be possible to get the uh does the scanner api give you the certified status i think it does and um honestly i haven't looked at that that's a great question that's actually a really good question like i'm thinking here that's like certified data sets right yeah whether it's promoted or certified i imagine it does um but like i said actually let's let's see the uh let's look in our data sets here okay endorsement details so yeah i think it actually does so let's go in here let's choose our data set here looking around so this is cool so it's a record so i gotta extract it so you can see here the endorsement details yes that's probably another table but yes it does so it looks like there's something in there for you they just kind of parse it out yeah which i didn't do yeah it's okay so it's there tommy why are you doing everything come on but that'll be before um so and then we'll be carlos asks will will be possible to get the users per app yeah um this has been a point of contention for people the app details is different it kind of exists but it kind of doesn't it's so it's a little it's a little um it's a little odd so really at this point it's just workspace and who has access to the data set so not necessarily not not not as straightforward as the workspaces good question good yeah do we know what the road map looks like they've really done a huge amount with this i think between the the user artifacts the metadata and the expressions along with the um the ability to write a dax query like i'm sure there's a way i can send a you know take that dax query there wrap it in and evaluate and push it and power automate within the report um but that's for another another another user group but i'm sure that's possible but no i don't think there's anything on the roadmap right now because they've done a lot i'm going to i'm going to kind of speculate here a bit because i'm again i'm feeling like this is a precursor to things that you would want to see inside purview so while the scanner api is probably trying to do a just a very large dump of information my gut feel here and portions of this is the only reason why these things are starting to show up as one is for admin control type things but there's probably a lot of other things behind the scenes going on where microsoft is going to integrate this api into other things and so this may be part of this scanning activity that happens with purview and it seems to fit really well with what you'd want to scan out of power bi data sets and data models that pervu would want to have visibility to so i think this is going to be really interesting where i think i mean this scanner api is doing and getting correctly from right or wrong tommy this is more about the data models this is data model centric we're not having yeah the api doesn't kind of spit out like here's the 10 reports that use this model and here's the fields that are used in these reports it does so it does yeah so you can actually see here if i want to i can actually dive in and i can see batting i see here's the data sets and then what i can do is what is that is that the report though or the data set no so let's let me show you so this is it's the oh i see what you're saying so it doesn't say the like if you had an additional stuff on the report correct yeah it will tell you if there's more than one report okay yep um yeah but yeah steve steven gottward is going so like there's there's this kind of like lineage traceability thing metadata so like there's like there's now like this is the data model metadata where there's a data set attached and we're getting all the things that are attached to that data set but there's like one more step deeper than that data set level is okay now that we go down to the data set now i need to know what is on the report page and what is in the visual and i don't think that's there yet but if i had to read the tea leaves here on on what's going on in you know the world here my guess is there's another api coming that says hey for this data set give give me all the things that go along with this data set so you could loop through another part of this that would say for each data set for each report give me all the other individual visuals in that report and then return that back to me because you could also do that stuff as well right right cool tommy you already beat me too i just ah you got it out there i had to do it bitly because it's too long it's too long so we just sent the happy hour so i'll let you i'll let you say that you're better oh it's fine so we're no no no no please please we're talking about we're done we're done with the session today this was like the training portion um getting into this particular thing around scanner api so thank you thank you for following along with us uh thank you for your great questions and engagement we're gonna cut from this meeting and we'll wrap this one up i'll hang out here for a minute or two and tommy will hop over to the happy hour go grab a beer go grab a drink go grab some hot water or another coffee if you don't drink that's fine too we're just gonna hang out we're gonna spit ball for like another 30 minutes or so just about power bi things and then we'll call it a wrap and then we'll call it done for the evening so thank you all very much for attending i hope you found some great value from this and we'll see you in happy hour thanks guys [Music] [Music] [Applause] [Music] you

Channel: Power BI Tips

Views: 487

Keywords:

Id: tvLJsTgt6Y0

Length: 91min 28sec (5488 seconds)

Published: Wed Nov 17 2021