CICD Magic with Power BI and Microsoft Fabric

[Music] [Music] [Music] [Music] [Music] [Music] [Music] [Music] [Music] [Music] oh oh how's it going great great how are you Chris how are you ruie hello I'm fine thank you excellent excellent excellent excited to have everybody here today um uh we are gonna be going deep into one of my favorite topics and uh Christina I I know this is a personal uh favor of yours as well yes oh yeah I mean r hats off I mean I have absolutely loved all the work you guys have done I'm so excited to dig into this um and talk about it further right I am so pumped yeah go ahead I want to say I think before we kick off Chris um I'm just gonna quick we're already getting some questions in the que so just really quick want to go through um some of our rules on our chat so first of all um preface any questions you have with a Q um if you have that Q colon that's going to help us see that it is a question we should queue up for ruy or for each other to discuss um the second one is going to be please do not repeat any questions don't spam the chat we will see it we will get to it don't worry um we don't want to kick anyone out of this awesome conversation so please respect the rules and we'll be good to go awesome awesome awesome and we have a question of the day um and I I I I really like this one because I think it's very poent Christina you want to hit it yeah have you used CI integration with powerbi yep so if you have leave that in the comments we want to know what is the understanding of people in side of the chat so as we do craft this conversation with ruie we've got good uh insights on where you're coming from okay but before we do that ruie can you introduce yourself we know who you are but maybe not everyone knows who the great ruie is so yeah sure uh my name is Ru I'm uh I'm a product manager on the parbi team and I I focus mostly on uh professional developer experiences uh with parbi and and with in fabric but my focus goes to the to the parbi workload uh so I'm talking about things like Source control uh code formats because those those are also important by having a proper uh format for Source control and code development and yes I know that the current uh formats are not the best we are changing that we can talk a little bit about that uh and um so basically experien is that most of them to be honest they were already available before powerbi and we kind of remove them or we simplify them and and and maybe that maybe no I I do believe that what what make uh that was a good decision that made powerbi as and the idea the focus on my team is to actually bring back those those experiences uh using using our tool so you don't you don't have to to use an external tool uh just to get Source control that's something that is uh important and and a uh basic experience for any professional working within a team well that's fantastic now would you like to do a background question to to let people get to know you a little better better um but ruie can you explain the journey from five-year-old ruie who wanted to be I don't know a teacher or a firefighter or a Goku uh how did you go from a 5-year-old ruie to a principal program manager at Microsoft ruy how did that happen yeah actually Goku is my child hero I love dragon ball I'm I'm from the Dragon Ball generation actually um and uh so uh I don't know let's start with I always were passionate about uh about computer science and computers um so since I remember myself there was always a computer or some something in the in the house uh then I I did all my work life in as consultant uh most of my work life as consultant more than 15 years I worked at at a company called Dev scope uh where I went from a junior consultant to uh to the bi and data manager uh where I had an amazing team uh it was an amazing experience uh and then I got the opportunity and and and I I remembered like five years ago thinking hey what could be the next step for me what I could do and and my dream was always working work to work on the partyi team and even better working on the parbi team focus on the features that I I was really passionate about I the other day a colleague sent me a a picture uh for like eight years ago where I was doing a presentation that it was called powerbi developers and at the time powerbi for developers and at the time was using the rest apis to uh to to deploy stuff to uh to embed to uh uh basically was telling the story of what was available for the developers at that time so I was always very passionate about this area uh and I joined the parv team uh initially at a cat team where was amazing so working in a team of superstars and um just sharing uh just be able to work with them and work with the most strategic customers in the world and then I got the opportunity to uh to join um to be a feature pm and uh and especially on the on on an area that I'm really passionate about and uh and yeah and and this is where I where I am right now I'm really happy to uh to be able to drive these features and uh connect with customers learn from them uh and build features that can impact their lives and uh as especially as a from the developer Persona that is fantastic and I have to say ruie I I I owe a personal debt to you uh my projects and a lot of the work that I've done over the last uh several years have been heavily based upon articles documents guides or templates that you've created out there so thank you very much for all your hard work we really appreciate it so thank you thank you but with that let's um before we dive in we like to do a community sip this is our lunch and learn so grab your beverage of choice and uh if you happen to have one R We we forgot to prep you on this one but uh we're gonna be doing our Sip and we'll dive into it all right let's go let's go all right let's take a look let's see here and uh so uh I don't know if we want to do this in more like a open Q&A and just starting with a few questions or maybe I can start with a few a quick intro on what what are the the parv developer experiences and what is parv developer mode and and get integration and maybe we can stop there and then just open the floor and uh and yeah if there are more questions on cicd we can we can jump into there into that uh if not uh like the new formats we can we can uh I don't know let's go with the flow to sounds good sounds like a plan sir okay so uh so basically let's start with um what is the powerbi developer mode is say the parbi developer mode experience so this is all about making uh uh parbi parbi desktop and our modeling tools to work with a instead of working with a binary file to to work with a with a folder representation with files with editable text and because it's edit text this can unblock things like Source control and uh and we call the pp developer mode but in reality the first feature that we are shipping as in this umbrella of developer mode is the save as part project that we shipped in June uh last year that allows you to save as a PVP and this is what you see here on the on the on the right on the right side you get a folder with two folders in it one folder for the semantic model one folder for the reaper by the way we are changing the name uh soon uh it will be semantic model so today it's still data set uh and this will uh unblock things like uh uh just git just because it's it's text textual format you can have a local git repo and I'm not even talking about Cod development and Asia devops just locally this has an immense value uh because uh this will allow you to track to backup and to recover all your development so this will allow you to for example uh I want to recover the last the version from last week and this is the type of stuff that if you don't do you will wish you have done it when you have a problem right uh and uh and the only thing that you need to do is save as a PVP initialize a git repo and use a tool like vs Cod to initialize that get git repo and you're are set and and uh and then every change that you do every time you open powerp Desktop and you you hit save and you make changes and you hit save it will save the those uh files to that folder and it will be tracked by git so you can look at git as like a database of files H and you can stop here so if you want if if this is where you are if you only care about backing up and having like a a track of history of all your developments you can stop here now but you can make this a lot more interesting especially if you work in a team uh uh and with with multiple part developers and maybe today you are using one drive to share the work between them because it's a binary file and that's to be honest to share a binary file yes one Drive is probably the best tool out there uh but that's not that's not a good experience for COD development because uh you might overwrite each other work so you need to use tools like aure devox or G up uh where you uh you basically uh create a repo in one of those git remotes like Asia devops and you synchronize with a push and pull of your local G repo to that AER DeVos and once you do that now you can enable code development you can have multiple developers working on the same project one of them working on the report another working on the on the data set uh maybe two developers working on the on the data set uh and uh and uh if there are no conflict between them because they are constantly sinking to that Asia devop um instance uh no problem if there are if there is a conf let's say uh two developers working on the same measure they made a change to the the some change to the Dax code uh then that is a conflict and then it will be up to get and aure devop to solve that conflict so you have a a tool that will help you whenever there's a conflict to to solve that uh and of course the final part uh and I see this as a a circle uh that can that goes in two directions you can publish and you can connect that Asia devops ribo at the moment only Asia devops uh so you can use gab for code development that's fine but if you want to automatically publish the fabric uh you need to use aure aure devops today and yes gab will come in the future um so uh but today with devops you connect the workspace to uh uh one of those G repos in asure devops and you have a b directional uh synchronization so and by B directional it means that everything that you have in the repo will be synchronized to the workspace but if you make changes in the workspace those will also be synchronized to the aure to the to the to that and you can go back you can you can do the inverse cycle so you can uh make a change in the workspace like create a report or make a change in the semantic model using web modeling uh and uh and commit that change to a devops and and then locally in desktop um pull those changes and open it open them back in desktop so this means that you can and we want to make this experience better and better to the Future you can work in the desktop tool or you can work in the service you can and you can work on both so maybe today you are not in your laptop uh and you want to you need to quickly do a create a new measure you can do that in web modeling you do a commit so you have the the metadata source of Truth and tomorrow you continue that work in desktop and I think that's amazing uh because not only uh allows the the developers and the users to work where they are more comfortable with um and uh and uh I I I I used to say that it's no longer whenever you have a g integration it's no longer a bad practice to do changes in the service because in the end everything will be synchronized to the to the Asia B so that's why this has like the two arrows between the between the between them uh so one more time what what we what we all allow you to do is just save as a RBI project and that's it maybe in the future we might have a developer ribbon something more more fancy but right now is just the save as type uh and when you save that it's just a a new option that you see in the in the save as you also need to enable this is still in public preview so you need to enable a a preview feature uh you get this this folder with all those files and we can get in details about what what what is on those files but there are documentation for that so I can share the links in the to to the slides in the end but if you search for parbi developer mode uh or parbi project uh you will you will get to the documentation uh and there are there are an explanation of what each file means but the most important files are the model. bin and this is today which is a team so that represents the semantic model so all the tables all the measures everything about your semantic model and and that is and the other one is the report. Jason that has that holds the definition of the entire report like all the pages all the visuals and it's a single Json file uh that uh that uh I know it's not the best format for Source control and we are actively working on that and uh uh we should have some news really soon um and uh and and the same thing for the for the semantic model that we are going to switch to Tindle uh the Tabler model definition language uh that it's a much more readable and editable file format that will make the source control and code development experience a lot better um so um uh Donald parishes is commenting how much he loves pbip and it's looking forward to Tim Dill do you have a a a date where we could uh we could be expecting that is that something uh that we could see soon okay yeah uh I don't want to I don't want to commit to dates and then have have this on record but let me say it's really really really really really soon okay okay it's not that soon it's really really really soon okay and um yeah there might be some regressions that will want maybe instead of being really really really soon it will be just really soon but uh um you will see that to be clear we want it working and if you find errors exactly don't roll it out like until it's it's it's all good right like happy to wait a month you know or or whatever so that that is uh fine is only good as an ice cream flavor Ryan it's not something we want in our code yes yes yes and uh uh Daniel oder says he likes how ru's Bim file uses a tabular editor icon so that is fantastic we should definitely do that yeah it's it's the only tool that lets me me open the The Beam file directly right so y yeah if you're not using tabular editor you should be using tabular editor and I've done live streams on it I've done videos on it um Daniel we'll have to have you on again to do it again because we not more knowledge has to be spread about the the glories of tabular editor it's uh um I was surprised I did I did a survey how many people were using tabular editor like over 90% didn't know what it was so we got to do more education there um but but we've got we've got another question in the queue but I've got one for for you um uh so one of the big things I'm frequently tasked with is is I'm constantly like asking people like hey there's something that needs to be done and rolled out today uh is this something that you could get through yeah we'll get it done by the end of the day and um if I'm using devops and I I've got that I I've got everyone using devops can I then see all their as they start to commit changes and stuff inside the devops log a get log so I can actually track that this work is being done and I don't have to like babysit my developers is that something I can do I'm sorry I I I think I'm having some some network issues so uh oh are you okay can you hear us yeah I can hear you okay so you was saying that you want to track the the the Easter devop so that that was the last part I heard yep yeah can can I see if I'm if I'm a lead and I say okay uh or let's reverse it let's say Christina comes and says to Chris Chris the uh the laad isical Chris who's always uh overc committing and underd delivering uh Christina says Chris can you please deliver on this by end of the day and i' say of course I will Christina I'd be happy to can Christina watch my commit inside devops and like actually track the changes to know that Chris is not deliver not working on it and is not going to delivered by the end of the day to say like hey uh this is not in place is that something we could do so and I'm not sure if I if I completely understood the question but the thing is as soon as you have G and uh and uh and you have AIA devops which also has a g repo in inside of it you can every every change that you do you can track those changes right so you can uh so every change that you do it's a commit and you can track those those commits uh though that does not mean that uh you can track the work that the developer is doing right now because git is a a local offline uh repository that it gets sync to the to the to as devop so ke uh now another thing that is very interesting and uh we can talk about that is the the part of the pool requests where I do my work and then I do a pool request and let's say uh you commit that and the pull request could be let's say a new report so you create a feature bran and normally the work the flow is like this you create a feature Branch where you that you name okay new report uh you work on that new report you make all those changes you can do that those those you can make that development either in the service or in desktop and when you are done you are you want to push that new report into into the into the final environment so let's say uh production uh and uh but you are still working on your isolated branch and uh talking about branches let's go here so that I'm talking about something like this you have the main branch where let's say is you have one report here and then you want to create a new one so you create a feature Branch you do all your work and then you commit you want to you want want to commit those changes into the main brand so everyone can have access to it and uh and to do the way to do that is through a pull request and uh you you issue that pull request and the pull request might uh uh uh might have uh uh might run or trigger a bunch of validations uh like tests on your development and it can even have an approval workflow like say Christina is the gets a notification saying hey Chris did this work uh do you want to review it uh maybe ask uh uh review the code or maybe just review if the continuous integration build roles have failed uh or or or gets or or were successful and if everything is is okay then uh Christina uh approves that pull request and the approval of that pull request will make the code to be merged into the main uh into the main brands that could mean triggering another pipeline that will deploy into production or it could mean uh because the main branch is connected to fabric kit that the the the the the new feature will be automatically synchronized into uh into the workspace uh I don't know if this if this is what what you were looking for as a way to track things but um but that's that's exactly what I was talking about yes so Christina has told me it's my job so I have to go create a feature Branch because that's our our process flow and then I have to do my work and then I have to deploy my work if Christina hasn't seen I haven't done it then she knows that Chris is slacking as as normal so and then I can go and be like Chris you you need to seek approval for these things yeah now with that Ry you mentioned like a an automatic syncing to the fabric workspaces is that something new that's happening because I know right now you have to click a button in the work space okay yeah it's uh it's uh so right now you're right so it's uh you need to to go into the workspace and manually click a button but uh also very soon uh there will be apis that you can call to force that syn um and uh and with those apis you can call them and uh basically through AER develops you call the API that whenever the code is synchronized into uh is committed into main you can call the API that will force the synchronization uh into into the workspace oh that's going to be awesome and I'm assuming the vice versa is also going to happen where if changes are made in the workspace we could call an API to have it sync back again exactly exactly so you have it's a it's suppos you'll get two sets of apis one for the for the commits and the and the and and the pool and and the other for the push as well awesome okay and it today we have we have to go into service and hit sync is that how that works one button yeah one button but uh I'm sure R you've probably heard this a lot but I mean customers have thousands of workspaces right so that one button click a thousand times can be pretty painful um so we appreciate all you're doing to make that actually that that's outside of my team because the thing to understand about fabric integration and yes today uh it's mostly parbi content that you see there I believe that it's also notebooks it's in lake houses although the lake houses are limited it's just the shell of the lake house not actually the tables but the fabricate integration is not only for parbi it's actually a platform feature that every workload that exists today in fabric uh should or will integrate with with the with the fabric in integration um so um it's not just just for pobi Al although most of the items today are for are for powerbi are powerbi items actually we're just you know we're just selfish R we just want everything for powerbi for ourselves you know um what can we say me too me too I I fight for that but Cody's got a great question here about thin reports does it make sense for a thin report to be as a pbip um it doesn't or it doesn't appear to have a folder data set which which makes sense but also it doesn't have a get ignore what's the best practices when it comes to thin reports in powerbi and for our viewers who might not know real quick just a thin report is going to be something that the main data set lives in the powerbi service and you're connecting to that in from another powerbi report so sorry go ahead yeah so uh so today when you when you save a so you open part desktop you you live connect to an existing data set uh in you have a thin report right so you don't have a data set in there if you save that as a PVP you will only see the report folder because there is no uh semantic model so and uh and and the difference is will be that in the there is a special file actually a very important file let me go there sorry I do have like this is like my deck where all everything around the developer mod I'm always jumping uh so this this file is really really important because this file not only is an entry point into parbi desktop Because by the way when you save as a PVP you get uh one folder for the data set one folder for the report but that's not does not mean that you cannot have more reports and more data sets in the same folder you can uh because the what we actually wna uh we want to have the same mental model that we have in the service where you can have in the in one workspace you can have multiple data sets and multiple reports uh but when you save as a from a pvx you only have one H and uh and we generate that sales pvip that by the way is probably the least important file of the whole thing the reason why it exists it's because uh we save to a folder and you need to have like a in that folder an entry point to the to the to forbi desktop again to reopen that pbip uh so if you look into the pbip content you will see that it points to the report folder now another way to open a a a report a pbip report a PB a PBP format uh to open in desktop is instead of opening the PVP you can open the PV so the PV is also an entry point into PBI desktop that by the way if you connect a a a fabric workspace to git and then you clone that git you will notice that you don't have any pvip why it's because let's imagine that you have 10 reports and five data sets in that workspace uh then you would get like five or I don't know uh you will get multiple pbip files that would be a mess so uh to open back the the report in desktop you should open uh the PV file and another very very important uh um attribute of the PV is that it's going to it it holds the connection to the semantic model to the data set I don't know if you noticed but I'm always mixing data set and semantic model I'm also training myself to to try to use a sematic model as much as I can so we're trying the same thing Ry it is a task many many years many years yeah uh and uh and actually it's curious because before powerbi it was semantic model the multi-dimensional thing it was a model it was never a data set and then in powerbi we start calling it you we train ourselves to call it data set and now we are and I think it was a very good decision renaming the semantic model because that's what it is um now the the the the the definition. PV holds the reference to the data set uh and it has two types of references it can be a bath reference that's the default so whenever you have the works the data set and the report in the same workspace it will always be a byth so in here you can see that uh this report is referencing the the data set folder in the same location or it can be a by connection so if you save a PBX that is a live connect report as a PVP you will get a by connection so this is also a good way if you don't know how to create that by connection you can just save it as a pbip and you will see the the the by connection connection created for you and you will notice that a by connection is basically just a connection string right it's a connection string to the and using the xmla connection string to the to the workspace uh another thing important to mention here is that by default every report when you save as a pbip we can say that it's a thin report because it's it's disconnected right even on the bath if you look into the bipad this this report can live without without the data set or or it can even point to another data set that can be on the same folder this is just a reference we actually when you save as a pvip I can say that it's already disconnected from the from the data set it it holds a reference not the same thing with the PBX because in the pvx it's that it's the everything is in that file so you cannot if you want to switch to a to a uh a live connect you actually need to uh today you don't have any any UI feature to do that you need to let's say delete all the queries and then connect to the to the to the data set uh with the with the pvip if you want to switch between a live connect let's say a a a an import uh um or a local data set you just need to switch the file so I I even see some customers doing things like uh let's say they don't want to refresh the data set locally and they and and they have everything in the same workspace so you can so during your development you you temporarily switch to a live connect so you go to the definition PBR and you put the bu connection there you do your all your work because you don't want to refresh your data set locally and then just before the commit you put back the bipad and uh and everything will be fine uh so what I'm saying is just by changing this file you can switch the behavior of the report if it's if it's a a connected report connecting to a data set in a service or if it is connecting to a or if it will load the data set locally and uh and even allow you to edit that data set if you want oh that would save so much development time not having to refresh the report locally especially if you've got huge models yeah and you just you're not doing any model changes you're just doing visuals wow yeah that's that's awesome I was I'm taking notes on the side here so yeah yeah like clip this one on YouTube come back to it um uh because and I guess this would this is where uh we could make a hot swap because I I've had problems with people who are pointing to analysis Services then they want to point to a data set like that could be problem problematic because you got to drop the analysis Services then put you know like youd have to scrape it out but this would be just a change in that URL in the PBR file right Ry yes although uh the current experience uh today the the connection to uh if you want to connect to a asan analysis services or local analysis Services is exactly as you said so you go to the connection string and you you you change the connection string to uh to uh to point to the analysis services but we want to change that behavior and we want to change that behavior because we want to have one more time the same mental model that we have in the service and in the service if you have a a partv report that is connected to analys Services you notice that it creates a semantic model for you like we call it a stub data set and uh and we do that because that that that it can have some Gateway connections or it can have different CRS all of that now what we want to do and what we will do on the pbip is if you have a a pbip connected to analysis Services you will see that it will create a data set folder the same same way that it does in the service but that data set folder is just a connection string you can go there and you change the connection string there but it will still be a byth reference in the report okay okay uh and and the idea is is to make it make it simple at least on the on the on the conceptual and the mental model that okay it's the same thing in the service so if you if you look into a folder that you clone after uh U the G integration and and you clone that repo the folders that you see should map to the to the items that you have in the workspace okay okay I I think that makes good sense um uh a qu before we go on to my next question and I have a more difficult one for you ruy uh and so I'll prep you uh in fact I'm gonna prep you with my question but then we're gonna ask no we're not gonna do that I'll screw that M that'll be too hard let's just start with poeg wants to know is pbit going to become obsolete if so thank you know or no yeah that's that's a good question uh not at all not at all okay one thing I don't know if I if I I don't know if I mentioned that in the beginning so my focus is on the on the developer experience that's where my as a PM I'm I'm a I'm I'm a fighting for the Persona that is a developer but we have other personas and to be honest those other personas they are I believe that they they are what made us so successful right so we don't we cannot and and if you think uh let's say a business analyst that is coming for PBI for the first time working on the pvx and on the pbit is much more accessible to them and it makes a lot more sense than working in the folder so let's say for example I want to share I'm a business analyst and I want to share my work with you uh it's much easier it's just one file I can put it in one drive and send you the link other than say a PVP even even zipping that PVP and sending it over to you and you need to unpack it and and extract it so it doesn't make any sense uh so the PBX and the pbit it will never go away it will be the default format and the most popular format uh in in the powerbi development uh the pbip is an alternative whenever let's say you are in a in a stage where even for a professional working with a PB maybe if you want to do something simple and I want to share that analysis with other one other other developer maybe that doesn't does not make any sense to save as a PBP now this the the point where where it makes a lot of sense to think on the pbip is I'm working on a team definitely the PBP will be much better uh I'm a uh I'm this is a corporate report or a very important report and data set and I'm going to to have a multiple iterations of the velopment definitely the pvip makes a lot of sense because uh I can I can just have that tracking history that I I was just talking I talk in the beginning uh I can easily roll back uh into previous versions uh and uh and that's a huge advantage of course that this this this also means that going into the pvip world you need to learn you need you need you need to be comfortable with things like gits you need to understand what G you need to understand how to work with kit uh and taking a step further you need to understand like um g remotes like a pull a push uh how to uh you need to understand things like branching and on branching I I I I would say that especially if you are the most important thing is uh there are many first of all you need to understand what is a branch right like a branch is an olated branch of development uh but there are many strategies out there uh these are four or maybe the most popular uh uh the top four strategies uh and I and it's difficult sometimes I get this question oh what what's the branching strategy I should use and it depends it depends because it depends on the on the team on the on the on the maturity of that team uh it depends on the what the organization is using maybe for other projects so um you should choose one and maybe if you are starting you should choose the simplest one maybe that the trunk based development where you only have one branch and and and later on evolve into something like feature branching where uh you have uh every time you work on a feature you create a an isolated branch and you work on that feature branch and then you commit into main uh and as you start to evolve in and your maybe your projects start to become more complex you you uh you bring more people to the team you might think on things like the Microsoft release flow or the git flow uh where you have things like release branches and uh and feature branches and uh and um and uh uh and and and and and pipelines that trigger when whenever you create those branches uh I can do a demo of something like that um but uh but it it the thing is you you I I uh especially if you are starting maybe you should just start with Git and just maybe not even have a branch just do commits and and and have the tracking history and as you start to evolve and you start to learn the basics then evolve to things like okay I need to learn about branching and I need to learn uh how to use that and apply that strategy within my team uh now what I can tell you and I can promise everyone it it it pays off okay it's one of those things that uh it really it really saves a t of time uh it gives you uh much more reliability into your developments and one thing that I'm actually very passionate about uh is the quality testing uh that is a tutorial uh I can share the link uh but there is a tutorial that is on the parbi um on the parbi project documentation let me show it really quick there it is so and this tutorial it's you can follow it it's a step by step Tut and uh and the thing is and believe me this is this this is something that can save like a ton of work a ton of time especially on the on the most expensive members on the team the senior people uh just just doing validations even for the senior senior people on on every development that you do H and running validations like okay are you following the naming convention uh are you uh um are you uh creating the measures with a with a with the right uh standards uh does your measure has as a format string um do you have a column that is hidden it's not us it anywhere uh but it's uh it's still in the model uh and it's just there spending maybe it's it's one it's a column that is consuming 20 30% of the memory uh so you can have those types of rules and uh and uh and whenever there is a development and whenever there is there is the will or uh after I finish my development I want to publish that to the main branch I create that pool request I was talking about and the pool request will start a build pipeline that will run uh uh validation rules over the semantic model and the report and and by the way it's using two uh open source tools the tablet editor uh that runs the BPA rules uh from Michael kovalski from the cat team and and it does exactly the same thing for the report that will run the the it's another tool called PBI inspector that has also a um a standard set of rules that runs over the report and does things like uh if you are if you have a theme that this this is my favorite rule by the way you have a theme in your report but you are using static colors maybe the developer did not know and and put some static colors that should break the build uh and the developer will get a a a meaningful error in the end saying oh you cannot publish domain because it failed on rule one two and three and the developer maybe even even even if it is a junior developer can understand what's going on and fix The Code by itself uh and uh and save precious hours from the team lead or the or the the senior developers to validate and even teach that that that jit developer so this can save uh hundreds of hours to be honest and having that in your development process so even like catching it going from Dev to uat means it fails right there so you know right at the lowest cost opportunity you can get those checks and tests place I that was the question I was going to prep you for so R exactly for example on that um can you guys hear me yeah yep we can hear you uh exactly on that uh that the the scenario I was talking about of the column that is hidden and it's just there because the developer thought that oh I might need that later that's that's a very bad principle because once that thing hits into production then you are not sure if anyone that is consuming that data set actually is using that column and then removing that column is a breaking change so if you can detect and even if it just puts some thought into it hey do I really need that column no I don't need that column let's just remove it if I need that column I will add add the column later um and and and and the beauty of of of this is all those rules and these amazing uh tools they uh they are extensible so you can build your own rules uh for example when I uh when I was in Consulting uh I put a lot of effort uh with within my team to implement what we called like the quality bar for any development that Dev scope did uh we put a lot of effort in those rules and because we knew that when we hire someone new or someone from another team came to the project instantly they will get some Quality Bar uh standards because the rules will fail they cannot push into production of course there are exceptions sometimes and and the T and on the pull request you can even go there and say okay I'm okay with this let's let's uh let's let's go further but as you say you detect those things early on uh before they reach into production and you can you can define a a quality bar into all your developments and and also consistency right so like like naming convention so you can make sure that all your semantic models have the naming conven that you want them to have or your customer wants to have for example maybe the customer wants everything in lower case no with this nothing will pass if it's not lowercase it will fail on the on the on the build uh so beautiful and and the thing that we did with the pvip by the way this was all everything was already available two years ago or three years ago this is not nothing new the thing that we made it possible with the PB is just more accessible because now you don't need to unpack the PV or you don't need to connect to the it's it's the tablet editor can can connect to the model beim file and run the rules and the same thing for the PBI inspector uh so we just make it more accessible and easier than uh more easier than than ever to run this type of stuff and uh uh if you look into the tutorial you will see that is basically the only thing that need to do is copy a yaml code put it on your repo and that's it all all your every time the next time you do a commit or a pull request that that that that those rules will run and it they will validate your development that's just awesome now uh now that we have our pvip files unpacked and we have access to that and um can our data engineering team members put in or do you have sample code or examples of uh data valid or data engineering teams validating that the changes they're making don't create breaking changes for reports or if they are breaking changes that we create a separate pipe to get uh new PB pbip files that have those modifications in them so uh you see what I'm saying yeah I see I see where you're going um let's say if everything is in in uh in git or in devops then you have access to the to all the metadata so and you can you can say that okay if I'm changing this column or if I'm changing this measure uh you can find that measure and that column or that table in all the reports and uh and uh uh and also make sure that the reports will get updated now the problem is when you have reports and Excel workbooks for example using people tables that are referencing those um those uh those columns and those measures and uh and for that situation the the this is not something that the the the metadata will solve it can make it easier for you to to detect those dependencies and um we can even do some uh some like say vs code extension if you open the the all those folders and uh uh detect that for you but it will still be a problem in the service and the solution for that problem is things like P viiew that will uh DET that will scan uh and uh and it will give you the impact if you change a specific column on the on the semantic model what type of impact it will have on all the reports wow all right fair enough I'm just looking to automate uh you know because it's great when uh if fabric is get getting us to the point where everyone can wear every hat but in some organizations there's a separation of Duties right and those data Engineers come in and they've got a job to do they get their request and they get it done it'd be fantastic if we had uh and I I would imagine we could now build out a a deployment pipeline test regression test oh yeah to make sure that there's no breaking changes just not something there no yeah that that that's for sure uh and you you you also have another new weapon that we can talk about which is the apis uh something that it's not uh uh broadly talked about but uh but they are available and and and uh and one of those apis is let me move to that slide not a problem good product managers and or program managers and Architects always have slides at disposal that they can quickly throw up to talk about that each so one of the apis is called the get item definition API and the get item definition API lets you and it's the first time that we Shi this API in December and uh and we never had an API like this before because you you you you had an API the export API and the export API exported the pvx and sometimes that export API didn't even work because the the the data set was modified with xmla and then the download does not work this is different because this API let you connect to a report and get the definition and only the definition of the report and guess what what's that definition the definition is using the PVP format so with this API you can also uh do some uh uh uh let's say for the reports in the service in Theory you if you want to understand the impact of a change on a semantic model or a new column or a rename on a column and you want to detect the impact on all the reports you can do first detect the impact the impact on the reports that you have in the source code so because you already have the definition uh and that could be done by a tool or it could be done by a find and replace so you open the entire folder in vs code and you just search for that column and you know and you can even change that because another thing that we allow you to do in the pvip is making changes to the metadata and and then let you open it back in in desktop now the get item definition also allows you to do that in the service because you can uh get the definition of the reports that exist in the service so you could in theory Loop all the workspaces that you have access to and uh and get the definition of the reports and understand if that if first if they are connecting to the data set that you are changing second uh if they are using any column or any measure that uh that you are also changing wow so now it is possible little bit of effort along it but it's something that could be done you but that's good foral yeah yep yep yep um there was a question we had oh where did it go ah data flows where uh is there any plan about including data flows as support items for git integration so uh I would say definitely I don't know the timelines for that one more time the fabricate integration the goal is any workload any item that you have in the workspace should have a git representation and it should be exportable to git and imported from git uh now I'm not I'm not aware about the timelines on the data flow team uh to bring that uh compatibility into into fabricate integration uh but uh but yeah one more time the the the the goal is uh every every item to be to be able to be exported to get we we'll say by June and then we'll be wrong by either know too fast or too late Chris we can't that I don't work for Microsoft so I can say things like that yeah you can say you can say can say next month yeah yeah yeah sure tomorrow sure I'll be there by next month yeah we're not making a commitments here what I what I can tell you is if it's not there uh there there is a plan now it's it's it's it's a matter of priority and each workload team because there are within fabric you have multiple teams you have the pobi team we have the data integration team uh and uh and it's uh it's it's also important to get that feedback and also uh for the for each each team to consider that and prioritize it versus other amazing new stuff that that that they are building um and um but the goal in the end is every single workload should have that that K that K integration yay oh I can't wait yeah I I'm gonna say I just saw a video this morning uh or is a little YouTube short and absolutely loved it they were describing something called the M the Michelangelo effect do you know what that is have you heard of that I heard of that but I I I don't recall okay so uh Michelangelo a famous sculptor would look at a block of marble and he would chip away at it to unlock the Masterpiece that was within and the Michelangelo effect applies to how couples will um see the masterpiece that's within the other person and will gladly work on the other person and take Improvement tips from the other person to become that Masterpiece that uh that that is within them and that's how great couples become great couples that's so sweet I love that I think there is a lot of the Michelangelo effect going on with powerp because we are talking with people who love the the the tools and the the love the the the piece of marble that we're working with which is our our data ecosystem and we're just carving away at it together like unlocking the Masterpiece that's within within it so uh ruie thank you so much for for coming here and unlocking how you are chipping away at that Masterpiece that is devops and cicd uh within Microsoft fabric so before we end is there anything you would like uh the people are watching this live live stream to do uh when it comes to the work that you're doing how can we help you so uh just keep the feedback coming so every time you have something uh that it's missing in the product please log it in the public ideas promote it uh broadcast it get get the votes it's something it's really really important for us as as BMS to understand okay what what what is missing what it's more priority uh and and to set the right priorities so for example I can tell you that uh we shipped bbip in uh in June and until then there there was some a lot of bug fixes but there is nothing measure so the next big thing is the the Tindle and the the next big thing after that is the the report the new report format why it's because that was the main dissatisfaction uh factor that we found from customers saying hey but uh it's still one file if I'm working with multiple developers I cannot work and the same thing for the report hey I cannot even if I have two developers working on the same report I cannot do merge I need to choose between one and the other and that's what we are prioritizing all and putting all the investment uh on the team that is working on the on the on the pbip uh a lot allot other things could happen like let's say the git window uh something that some customers asking like okay I want to be able just in Des up to commit and and and and uh and view the history but maybe yeah you do have a workaround for that and that's using BS code or whatever other gtool and uh uh there are other other features that maybe uh should be prioritized before that so one the fabric ideas is that is is is something that we look all the time to uh to understand and and and balance those priorities so please uh always do that and uh yeah whenever and feel free to share them with me and Linkedin just tag me I will follow I'll follow up for sure awesome awesome well ruie Christina thank you both so very much for joining us today and and sharing insights on cicd that's something that we just need more and more people to be aware of and exposed to to the amazing things we can do uh if you're at all interested in learning python with Microsoft fabric on Friday at 7:30 Susan baz is gonna be on our live stream we're gonna be talking about learning python with Microsoft fabric so that should be a good one L she's awesome so definitely tune in guys I like her almost as much as you Ry no I'm just Kidd no but thank you again for coming on this has been awesome it was awesome always glad to to talk and chat about this stuff uh and um thank thank you for having me awesome see you next time thanks everybody peace byebye Baker Tilly digital combin strategic industry insight and advanced technical expertise to uncover and solve your digital transformation challenges if you're interested in learning more check out our website at Baker tilly.com digital [Music] shh

Channel: KratosBI

Views: 2,394

Keywords: power bi, microsoft fabric, power bi desktop, business intelligence, power bi service, power bi tutorial, power bi for beginners, power bi best practices, power bi interview questions, microsoft fabric demo, microsoft fabric power bi

Id: G7xTW_76eT4

Length: 70min 0sec (4200 seconds)

Published: Thu Feb 08 2024