WSUS Maintenance and SQL Optimization for ConfigMgr with Q&A

very cool all right guys what do you say are you ready to jump in that's a dangerous decision to give either of us a microphone just now i'll blame justin then that seems reasonable that seems reasonable so i guess one thing yeah for sure uh one thing to note here right so yeah we do work for patch my pc but this isn't necessarily a patch my pc specific uh session or anything this is just wsus and updates and maintenance and all that kind of fun stuff so we just happened to deal with patching a lot what do you mean maintenance i thought we don't touch wsus that's right we don't all right so uh yeah kind of as cody was saying here this is going to be a little bit of a kind of controlling the chaos of ws section here um so a little bit about us first um my name is jordan bensing i am a microsoft mvp in the ems space i work a lot with software updates as i mentioned we both work for patch my pc uh i'm from michigan a little bit of information here um i'm pretty sure we can probably make sure people get the slide deck and i promise uh this won't be death by powerpoint we're going to jump over to demos we'd just like to have something so that we can actually give you something with some information in it to take away to kind of flip back to and i i have some reasonable ish commons of video games ramen and and dogs or something other i'm up to four dogs now yeah so um cody mathis uh you can uh usually find me doing all kinds of silly things uh over the pandemic with keyboards it's about a hobby i've kind of picked up and it's infected a few of my co-workers i'm from michigan as well jordan and i are actually about an hour from each other uh and uh yeah oh my system center dudes that is not my blog no not but that's sorry i copy pasted bad yeah no no no problem at all so that's not my blog but that's an awesome website uh so that's totally not that one the other one that you do the other thing the other is one thing yeah the assist man squad is just mad squad i typed wrong yeah no problem didn't want to be yeah not me but yeah so let's start to start with the the lie you've all been told right um so over the years kind of in the in the past um when configure manager and the software update point in wsus first came all together back in even back in the 2007 and probably even earlier than that days one of the big things that was always pushed on to us configuration manager admins or towards the idea is you set up your software update point you got your update it's all configured um and then you just kind of you you ignored it right um and that was it you didn't want to touch it you just you wanted to leave it alone you got all the green lights you saw all your green lights and your slight components and you said we're good we're never going to touch it ever again we don't care what happens we just don't touch it if it does our solution is we're just gonna burn it to the ground and build a new one it's fine right treat it the same way as if you found a spider um and that's not 100 true anymore at all um in fact it's it's it's really actually kind of important that you you do do some maintenance now um and so some of the big things though is what does that mean what does maintenance what does maintenance mean um so there's kind of a couple of different kinds and types of maintenance you can deal with the first big one here is this is that at the end of the day it's it is a it is a database right it's a database of a bunch of xml files more or less right there's some other things in there but realistically wss updates and all that is a bunch of xml data that all lives inside of a database for people and in order for clients to actually be able to access that data successfully we need to have some type of way to effectively scan against that so we need to configure either some sql maintenance some windows internal database maintenance maybe some iis stuff and then of course we need to clean up the old updates which kind of brings us around to a question of but why so go ahead cody yeah for the why right the the perspective i try to take on this is that at the end of the day wsus is a windows app it's an iis app and it is going to be subject to all of the pitfalls that any scaling app is going to have right you're going to have performance degradation you're going to have resource allocation issues and the issue creeped up over time with wsus there was a finite and smaller number of updates that even existed right and then as time went on more operating systems more products more things are supported uh it just keeps growing and it starts to become a bigger issue right so so the the old prescriptive idea that you don't touch it stops applying when the actual use of the product has changed i think that's kind of for me the big why yeah and i think it's kind of something that's interesting to note too is if you look at some of the history of of w some kind of how it got built um we were recently doing some uh testing where we tried to break it um on purpose for a reason we'll talk a little more about that later but if you go back and you look as recent as windows xp which i know xp isn't exactly recent right but you can see how naming standards even and the structure of the data that lived in wsus has changed and shifted over time some of the updates that we saw when we were testing this because we were trying to see if we could sync every single xp update known to forever out there um and they were like weird like security update for i don't remember was like it was just called security update that was the type like okay that's that's super helpful as to what that is right and so as naming standards have advanced as the technologies have advanced to the different operating systems as we've gotten the different features of windows 10 and how those things have gone on more and more data has been stacked into and more and more features and functionalities have been stacked into something that's been essentially considered feature complete since what 2007 i think i want to say ish um i mean i think really the only major core change since then has been anything to do with the windows 10 features right that have been added on for how the encryption of uh content goes through so that technology and all of those pieces and all that information everything like that was all very much so geared and sized and scaled to that so with that in mind uh let's go ahead and jump out of powerpoint here i'm gonna change screen shares real quick unless you have an overwhelming desire to do so not immediately i think you've got it pulled up okay hold this up real quick we'll jump over here and talk a couple maintenance things real quick remote desktop all right so first things first in terms of maintenance if you if you haven't done these already um the first one that really just to take a quick look at and see is we come over here into site configuration if we go to sites um the configuration manager team recognized that there was there's a big need for maintenance one of the big places that they were getting a lot of pain in still was in the software update space and so one of the things that got added both as a result of some community push as well as just some some internal things this is we have this wonderful set of these three maintenance check boxes right here um and i i highly recommend that you go out and if you don't have these checked today that you should go out and look at checking them um one of the things that we we noticed recently and cody i don't know if you want to talk about this a little bit since you're the one that was was testing this is just how absurdly good this maintenance has gotten from a simple check it and forget it perspective yeah absolutely so we mentioned a little bit earlier we were trying to break wsus right so i wanted to recreate the idea that i have a lot of updates in wsus and my syncs are failing time me out and so i had wsus i made sure that the indexes did not exist because that actually speeds things up and makes it pretty effective uh and then i went in and i i had gotten up to about 20 000 updates in the council so relatively significant amount right trying to represent historically maintaining a w such for a long time and so i was hoping and expecting i run a sink and then i check these clean up options and and what i've seen we've seen it a lot with customers i've seen it in my previous environments it times out it fails to do the cleanup and that's because there are the the stored procedures that actually get ran they have timeouts right there's built-in timeouts and wsus to to prevent catastrophic failure or database locks etc and so those timeouts were occurring but configuration manager in more recent versions has gotten very smart and that cleanup actually will pick back up and retry so i was able to blow away something like 14 000 updates over like i think it took maybe over an hour or so um but it just kept churning and it blew away like 14 000 superseded uh or declined and expired updates that you know according to my maintenance settings uh with no issues at all and then it added the indexes and everything was happy so that did not used to be the case those even when those check boxes burst were introduced they did still cause problems they would time out but it seems like they've been iterative and continuing to improve them yeah so that's been something that's been really good and really helpful for pretty much everybody just around the board across across the across the gambit on that one um so that's one step that you can do that's that's pretty simple and is is pretty easy uh to get things set up and started off right off right off the bat but what about everything else now i've got sql management studio open here behind the scenes so you can probably guess where this is going so we talked a little bit about this idea of indexes and removing things from the sql database and this that the other thing but none of the check boxes that are here necessarily flat out address things like sql health the actual health of the database so one of the things that's very common again in the configuration manager world is set up sql for configuration manager and leave sql alone don't touch it don't look at it don't don't poke it with a stick don't don't anything to it right and again we kind of come back around that idea of well but maybe um there's some things that we should do so there's some pretty simple setup tasks and we're actually going to go through here and set them up if you've never heard of it before it's this article by a guy named ola holland so actually got here and what makes this script or the concept of maintenance important so if you note that checkbox it says add the indexes it does not say reindex the database right there's a very very important difference you can add indexes all day but until you run an indexing operation those indexes aren't going to be populated right so you can kind of think of an index as is like a hash table or a mapping a key a key value pair more or less there's a little bit more to it but that's what it boils down to right it says hey when you're looking for a go there and you it helps you do it quickly but the the index itself is just it's the it's the idea it's like do this now when you create the indexes when you run an index of a database it populates those they do actually take up space in the database so that checkbox is only half of the struggle here yep you can think of it kind of like a library system essentially if you've ever been to a library and you've gone and you look for things by the dewey decimal system or the library of congress or whatever like that right the index is the process of creating the record of where that book is supposed to be in the library right but then you have to have an order to that that system right they need to be in alphabetical order or something so when you go to the card catalog if they're at the front if anybody here remembers card catalogs from the public library back forever ago and we could go outside um you'd walk up you'd pull it out and you'd look in alphabetical order through the card index you'd find it it would tell you what shelf or what location that's exactly what this is well the problem is over time basically gremlins come in the middle of the night grab the box of index cards and just throw them up into the air and then shove them all back in the box whatever order they want and then no one can find anything um it's the same concept as what happens here so what what hollingen then does is actually helps rebuild those when they reach a certain percentage of being out of order essentially and there's a couple of different ways that you can implement it um one of the easiest ways to do it does require a little bit of uh editing after the fact if you do it but is if you just come in here copy this and over here and first things first let's go ahead and start the sql server agent because that's important to have running if we want to actually see any maintenance tasks get created i neglected to do that it happens you know it's fine when you have snapshots of the vms in perfect condition you never need maintenance that's right let me just go ahead and uh paste this right here this is basically i'm not going to go through and try to explain line by line by this because this was written by somebody that's much smarter at sql than i am but the general concept is it goes through all of what are called the user databases in here which means it's going to hit all the databases on the server including things like my config manager server and my sdb as well so it's actually going to help improve performance there as well um and basically you just copy paste it in here and click new query and then you're done ish when i say your done ish is that you still have to come into the job activity monitor down here and if we refresh it actually jobs sorry come on did it did it not run did i not click the button i apparently didn't click the button i apparently clicked new query and not execute i'm blind hit refresh there we go and we can see it's completed and it's now created some wonderful little jobs for us here including things like doing an integrity check on the database doing things like reindexing it now the one thing that we still have to do here is we need to come in here and specify a schedule on how often we would like to have that run and once that runs if i go ahead and actually just go ahead and start it i'm just going to manually run it start job it step that run away from us yeah so we should actually be able to go look at the indexes actually we can go see them that they got created on a couple tables that are supposed to be oh yeah sure you have the tables written down therefore uh you know i don't but i could go find it real quick and we can ramble between now and then okay i'm gonna go find the log file real quick for it redoing and rebuilding indexes while you do that then um so real quick we're gonna go and jump on over here into our friendly neighborhood sql install i think it's actually installed on see if i remember correctly on this server yeah what's that but no what are you looking for i was just trying to remember which drive sql was installed on this server um i found it so in a second here assuming that i have the right instance we should actually see some logs show up we can go and check there we go does it log in certainly do it's supposed to right there oh there it is yeah so if we go down here and we go ahead and open that you can't open it in a text file but that's not very fun uh but what we can actually do instead is if we can find the open with button choose another app wordpad yeah wordpad absolutely that's the app i want to open it how did you know i'm actually going to choose to open it with my favorite log viewer ever and hopefully it's near and dear to everybody's hearts in here good old tm trace what could possibly go wrong and there we go so you can actually see here is it's going through and as you can see i apparently had some database indexes that may have been a little bit fragmented just a small bit um and if you actually go down through here you can actually go and see which database it was where and and why it got indexed or why it was picked up for what its fragmentation level was and we can actually see why it got cleaned up how it got cleaned up in the state that was left in we can actually go to the two tables all right cool and we can go through and we could go through and look at all the different niches it's doing but the end of the day what that does for us is it increases the speed at which an update can be found so that when the deletion process is happening it can actually find the record and then execute the call to delete the record which overall just makes your life better for both your clients and your server go ahead and pull it up if you want to chat while i'm going and finding that table yeah so um with this right what's interesting about these indexes is initially the uh some admins in the community were just really tired of seeing the you know processing old updates and wss maintenance just uh time out all the time and so they investigated it and they found out like hey if i create this index things get better and and it eventually it kind of made its way through and microsoft implemented they they sanctioned like hey you can use this index we will support it right and then it made it all the way up to okay the config man team is going to implement this in the product so this came from the community they found these in this index that would be useful and made it all the way up which is pretty cool and and there you go here they actually are here is the indexes they take up space right yep those are it and so they do take up space actually um but uh yeah it's it's neat they pretty much map uh something in table a to something in table b and uh you know they speed things up quite a bit it's at least thanks to know they exist yep yeah clear the filter out here real quick yeah it's gonna look pretty much the same all right so that's pretty cool and and neat and awesome for sql but what if you didn't install wsus on sql can we just check these magical check boxes and everything will still be a okay it depends depends right so there's a couple of special scenarios around and about these particular check boxes here that make life a little bit better a little bit maybe not better and that has to do mostly with the fact of if the database is remote or if it's local to the box if the database is remote from the box these commands won't actually process across the network to that matter will not or or if it's a wave if it's a remote or if it is across the network it will not process them for it i will for sql though as long as it has permissions yeah as long as there's permissions it will for sql but not for win right it needs to be written across the network and it will not process for these so that's just something to keep in back your head if that is the case mr cody has already put this here and we do have these links actually uh in the slide deck that's the big reason why we even bother with the slide deck um is is that i'm going to go ahead and grab that real quick and bring that up here this is that it addresses what you can do on top of that so for these there's some additional special things that you might need to do in order to get that maintenance to work and here's actually where you can call out what others are for that um it's just something that's kind of good to know and to keep in the back of your head for that um yeah so but what it boils down to right is access you know and you'll actually see in the log files if you have a remote with you you will see that it failed to connect right it could not not authorized access denied and it can't connect to the remote wit instance uh so you'll you'll know when you've got those check boxes checked and you dig into like wsync mgr log you'll see the failures yep yeah so i think we dropped three different links in there uh so we've got three links it's there's all kinds of information on maintenance uh it's kind of funny something we said we were never supposed to do we now have a uh insane amount of information about the things we're supposed to do and uh and it touches on a lot of interesting things you know one of them is is also capacity limits uh which we'll start to to talk about that a little bit maybe not specifically around capacity but uh ways to approach it when you do need more than one ws and some of the gotchas yep i think which kind of brings us to one kind of last little thing here around that is is that and one of those things one of the things that does get referenced that frequently gets forgotten about is that at the end of the day all this traffic and everything for how the clients are then accessing that data that lives in the database is all being done via a website it's all being done via is if we actually hop on over here real quick we can take a quick look-see at an application pool setting real quick here um and this is another thing that commonly kind of gets glossed over and one of the best practices chunks in there will actually have some configuration guidance on this now something to be aware of as you are flipping through here is is make sure that you take a moment and read some of those settings in particular the ones around this one the the limit percentage as well as the private memory limit recommendations for it um depending on situations scenarios and things like that sometimes the right answer is is zero uh sometimes the right answer is there really is no best practice and sometimes the answer is do the thing that's in the documents for it all depends on what's happening in your environment and what other scenarios you have and we'll cover a little bit of the who's and what's and whys and hows of those and some things we can do about that a little bit later on when we kind of get into that multiple software update points or different things like that that we can do around things like that in the future so yeah for sure and and you know like he's had said that this is like something it's something better than nothing here you know the the default iis settings will likely trip over themselves in good time in in wsus world if you've got a couple hundred clients it might never become an issue they start to scale up thousands of clients tens of thousands you're going to start seeing a problem uh and then you're going to start seeing all kinds of scan failures and then you're not patching all of a sudden and you think you're patching so it's just really really important to be mindful of this it is a windows app it runs on iis it runs on sql those things need to work well yep so i'm just gonna change venue again here real quick and snap that and uh this shirt back out all right so we will of course we've got all those links right here so don't worry if you didn't happen to copy paste them out of the chat so it'll be there and available to you uh so let's talk about the many faces of wsus speaking of the uh all the different types of options and things it's all wbs at the end of the day it's all software update packages at the end of the day it's all stp files turtles all the way down as somebody on our team likes to say um so what is that what does that mean right well what are the configuration options you'll notice that we use this word of sql wid and sql express again here right well those those are the common ones so we kind of just glossed over here but there's a whole bunch of other different possible outcomes and depending on what your organization's needs are they might affect that and what's kind of interesting about this is these become more important to understand because they sound super great and super easy to implement when you're just doing normal updates but there's some other important gotchas when you get into third party patching like what we we do a lot of what we see a lot of when we're talking to different people out there um so the first one here is is the idea of sharing a sql instance uh so at the end of the day you can actually have ws right the the app itself sit and share you can have two instances of ws sharing the same sql instance okay um and each of those will act as their own website clients will come to that website and they'll come back and it will use that same sql instance you can also use the same content storage behind the scenes so that means you're only downloading the content effectively once and everybody is referencing the same location you're not duplicating or doing anything weird or strange like that and creating any oddities out across the scenes um and there's a couple of other things that you can do along that you can also have hierarchies internally right one of the check boxes that's actually inside of the configuration manager console i'll show when we go back over there you technically can sync from three potential sources when you're for your software update point inside of configuration manager you can tell it to sync from microsoft you can tell it to sync from nothing and you can tell it to sync from some other internal w server as well which gets into the concept of how you can create a software a hierarchy of wsu servers internally maybe you have some devices that only patch with wsus instead of patching with configuration manager and wsus and things of that nature um yeah there's a lot of different ways there's a lot of different ones and the big thing that's kind of important here right is is there's not going to be any one specific size fits all for pretty much any organization at all um i think jason sandy's is probably the one that is most famous for saying the fact that there are no best practices right best practices is what works best for your organization or something of the sort right um and it's it's extremely extremely true right i can sit down i could tell you back when i worked at an organization with 150 000 you know servers i'd tell you that yeah i had 42 software updates and you points and you'd look at me like i'm an absolute madman and then i would tell you that i had fractional t lines in some locations and that i had secondary sites those locations so when we bought or sold that entire entity we just cut off that particular limb so that we didn't have to worry about trying to disentangle the infrastructure we just cut off that secondary and all components to it and oh all of a sudden it makes sense that's why you're doing that for that of certain number of clients with that certain number of things so there isn't going to be a one size fits all type scenario necessarily for anyone yeah for sure and and so there's some considerations right when we start to to scale up you know we're talking about multiple subs we're talking about shared sql instances we're talking about shared content storage and there's a couple things that can kind of happen to that that kind of leads us into our next slide here right where we're talking about multiple subs how what kind of chaos can this cause you know what what issues can happen and what things work well why do you do it what's the point and and so the big thing to keep in mind when you when you have multiple software update points it if you have a shared database then it's in your it's the best case scenario regarding if one fails uh because when a client runs a windows update scan which the sccm agent it will kind of kick off and zombify and use the windows update agent it goes and it scans against wsus and as long as there is happy healthy scanned data results and it says yeah that's valid and then i'm okay with it it's a delta scan right it's a smaller scan it's a smaller pull down in bandwidth it's a smaller uh network process and memory usage on the client and on the server and for what one client fails it's not a big deal like if you're if your windows update scan fails on your endpoint you don't care no one knows uh there's no impact but but the problem is when lots of them fail then it's a domino effect and then you eventually end up with a crippled wsus server or you end up failing over to other servers right so something we mentioned here is that a sub failover is a hard failover so you can configure multiple subs in configman and one of the things that happens is when a certain set of error codes happens the sub your client says okay you know what i'm going to go look for somebody else you're not very helpful and it goes and it gets another software update point now a lot of the components in configuration manager are boundary controlled and when a resource is not available in your boundary right you can't use it unless there's fallback configuring etc but with a sub um you know you can you can continue to use that sup no matter where you are so if you fail over and then you go to tijuana uh and you're still on that sub that's and who knows where you're hard tied to that sub right and so you have an interest in keeping your wsus happy because if you start triggering full scans and if you have a large catalog all of a sudden you've got x number of clients downloading a 300 megabyte catalog from tijuana or to tijuana right and you're gonna start making some people very angry and and so that's that's one of the considerations that you have to be mindful of right there's there's more to it than just firing up software update points and assigning them to boundaries it doesn't always work the way you think it does and one of the big commonalities on this one or one of the big ones where this happens right is this is when we see the nuke and pave approach right it's almost become a an internet meme at this point of the idea of like ah wss isn't working i'll just nuke and pave it right um well if it's the only software update point in the hierarchy that might be fine um it might not be fine right depends on what you have configured it might not be fine in the fact that all of a sudden it's down for an hour and a half or so and those clients were able to sit in an idle state for a long enough period that they said no we're gonna go to microsoft and start downloading whatever the heck we feel like at this point right um or if there's no group policy restricting them or no other things restricting them befriending them uh or it could be that they um maybe there's multiple software update points and this cody was saying all of a sudden they start looking for somebody else to talk to um and i think one of the big ones i think there was a i think was ee2 wasn't it was the firewall error code it was missing for a long time in wmi there was a big thing about this where if they failed over was is that trying to get them to fail back there was a missing error code for firewall errors essentially so you couldn't even get them to fail back over to other things to get them to go back where they were supposed to so if you only have one sup might be fine might not be fine you have multiple subs it probably won't be fine because that's probably gonna go talk to somebody else um and when that happens you can start to end up with these interesting phenomenon where things start to just get pinned all over the place uh so one of the big things i know we mentioned earlier is this is that it's all sdp files all the way you know all the way to the bottom it's all and it's all xml it's all wsus everything is all you know things like that one of the other things that people forget about a lot of the time is that it's also all wmi um and one of the big things with more and more companies and more and more people adopting these ideas of building these small virtualized thin clients um that people will hop on and use whether it's azure whatever virtual machines or a hyper-v set of virtual machines or citrix or it doesn't matter right um they have these small very thinly provisioned machines or they update or whatever is that that data is all on wmi so when these scans start happening the real world impact i know we're just saying oh these scans these candy scans well the real world impact of that is it starts pegging the drives on the hypervisors because all of a sudden 500 clients all at the same time on the same that are all sitting on the same disk all need to start a scan or 200 i'll need to start a scale whatever i'll need to start a scan um and so it pegs the hypervisor and as a result the host then gets hit well if that storage pool is also what wsus happens to be sitting on for a server because it's all in the same data center or whatever like that all of a sudden now the wsf server is getting hit as well while it's trying to redownload a catalog or rebuild the information that he used to have and it it can't do it so it just blows up so a couple of the ways you can get around that are one of the common solutions that a lot of that we typically recommend if you're going to do something like that right is throwing a firewall rule in place to allow the server to actually survive so that it stops getting hammered with iis requests one and the clients will give up for a short period of time as opposed to actually trying to continue on with the scanner continue on with their download and things like that um yeah so just something to be very cognizant of as you were kind of as you're working with things yeah for sure and and you know with with these multiple subs and having shared content right so um a shared software update point in a properly configured state right shared database and then sharing the content uh now content is not it's not as important right in a configman managed sub the microsoft updates don't use the content directory except for euler's if an update has a eula if you've ever gone through and done some of like the deployment packages and configuring updates you probably saw a checkbox that says like oh download and accept ulos or what should i do right there are actually updates in config or in wsus they have a ula associated with them that ula has to be downloaded and if it's not downloaded you cannot actually deploy the patch so it's not uncommon to see people with a broken content directory uh for various reasons and then they actually have not deploying those patches that had a eula so it's it's something to be mindful of uh you'll see it in your log files if you pay attention that it's failing to download the ula but with that said right if you have properly configured this up with shared content shared database it's your best case scenario in a failover because yes there is still a little bit of traffic and processing that happens if i fail over from sub a to sub b it causes a delta scan this is something that's in the docs regarding failover there is still a delta scan so it'll do a failover and it triggers like okay i need to go and check right does that scan but it's a lot less intense than a full scan especially because if one of your clients failed over there's a good chance a lot of your clients are about to fail over so uh if you have a failover happen and it's not shared content it's not shared sas db you're going to see that domino effect potentially start to happen our whole site thousands of clients all of a sudden hit a new server and start doing full scans and you've got no iis maintenance in place and no indexes in place you're going to see w says die and so you've got to be kind of aware of this cognizant of what's going on you've got to have preventative maintenance in place to prevent this from happening uh and i think one thing too is there have been times where uh you know you may want to deliberately fail over a sub right and i forget when they introduced it it was a while back though for sure you can manually fail over a sub there's a right click option um if we end up on a console at some point during this presentation we can show it you can right click and it's one of the client actions you know switch to next software update point yep cool so uh let's go on here so what it's one interesting thing that you uh brought up there so one of the uh big things to kind of note when we're talking about with these shared subs right is how how can we avoid configuration drift right because one of the things that cody kept bringing up there is is making sure that we've always got you know these properly configurations we keep saying this that magical world of properly configured and proper configuration right well there's four things we're gonna kind of show these here um the first one here is kind of interesting because it's becoming more and more common and i'm actually gonna flip over into a demo screen here real quick instead while we're kind of talking about the rest of these here i just want to see the list here um before we go into these is the wonderful world of proxies go ahead and uh change some screen share here again and this is actually a pretty recent one that we've started to see a ongoing trend in lately um that started to pop up more and more um in the past couple of months here so a lot of the times what we're starting to see organizations will do if you come into configuration manager here go to administration go to servers and some roles and go down here and you find your server and go to google's fashion software update point we right click look at properties is you'll note that you have this proxy and account settings down here okay now it's not checkboxed here it's not configured anything here because of what's configured on the primary site server at the primary site isn't set up to allow it so this isn't actually going to let me check any checkboxes here now what's important to note here is is these settings here are per software update point role in the environment now if you have two software update points that are sharing the same database one of them is always considered the primary between the two of them okay one of them will always be the one that's doing the talking out to microsoft one the other one will always be wanting to do the behind the scenes you know just servicing updated the client ready to take over okay if you only have this setting configured on one of the two of them and the what the primary goes down during that time frame the proxy settings will get nuked and the content will stop properly downloading for the time until they switch back i love the art you've created at this point the the drawing all around it right it's just kind of you know it's very important to the point here the circling of it um so and it's kind of weird because everything will look like it's working except for the fact that content can't download everything else in ws will 100 look like it's working um you'll see good syncs uh or you'll see one good sync one bad sync one good sync one bad sync back and forth as it fails back and forth between the two of them depending on what ports are and are not allowed when you're going through the proxy right um because it depends right if you if you allow going to maybe say microsoft but not the third party stuff right you might see a kind of good sync but not a really good sync where it's not fully getting all the content the sync to microsoft might work but you can't download the rest of the content so it's really important to make sure that if you if you're using them you're checking them and you're setting the proxy settings to be the same across all of the different components and everywhere out there otherwise you can end up in a weird situation for them for that um which brings us kind of to another relatively important one which is the idea of certificates and there's a very similar one here in the same world of certificates so i know this one's very near and dear to your heart cody with the number of hours that you spent troubleshooting it the first time you saw it oh yeah yeah this was a a massive failure to read logs on our parts though uh which was kind of funny and in hindsight but uh there is a really fun scenario in in this one specific to third-party patching uh no matter the catalog right just third-party patching in general so there's there's two main certificates that you're gonna have when you're dealing with third-party or sorry wsus the main one is going to be the iis cert right if you're configured for ssl you're going to have a server officer it's bound to iis and you know your clients have to at least trust it they don't need to necessarily off against it but they have to trust it and that's fine but the other component is the code signing certificate right and i don't think we have them configured here at this time uh it's super quick to do it but you know we need a code signing certificate now that code signing certificate is it's that trust right it's our it's our trust that says hey your environment or at least something you trust signed this file so you can trust it you can use it and there's some reg settings to tell clients if it's self signed etc there's some details to it all but at the end of the day it implies trust and that trust though is more for more than just on the client side it actually is also necessary on the server side so what we saw in customers environments was this weird anomaly right they would they'd come to us they'd say hey all my updates are getting deleted i can't download them they're gone everything's just gone it doesn't make any sense and and we noticed a commonality pretty quickly okay you have multiple software update points that's kind of strange but uh that it's getting deleted it doesn't make sense but let's dig in and so we would see we pulled up procman we really start to dig in and what we saw was one of the software update points was deleting the file it would just come through and say hey gone no i don't want you just blow it away and the root cause what ended up eventually being was um the the second the other server it did not have it did not trust the code signing certificate and and this actually can happen in a like a fully configman managed environment too right so if your secondary or other wsus servers don't trust the code signing certificate and they get past the baton for hash validation they will delete the file that's how this happens so there are a bunch of roles that uh wsus has there's a handful of them i guess not a bunch and those roles actually there's a health check every five minutes that the windows uh the w service does it runs it actually runs the stored procedure in the database every five minutes by default and one of those things is the role really that handles hash validation and so if you're a primary wsus say you've got a shared sense shared stop uh sdb shared content and then your primary maybe it's offline for five minutes during patching it fails over to the secondary server that server becomes responsible for hash validation you don't trust the cert guess what it's going to come back through the next time you publish any update maybe it's dell hp lenovo it doesn't matter what it is and it's going to delete it it's going to blow it away it's going to be gone and and so the solution right is to just make sure you have consistently you need uh consistently configured servers you know you need to actually have your code settings trusted now it's common to not run into this because you you have that checkbox set enable third party updates and then all of a sudden configman's magically managing your certificate and then you happen to have your other wsus server in a collection that's targeted right it says oh yeah okay it's got the enable third party updates client settings it's going to get that code sending certificate it's going to trust it and then everything works but on the flip side we see this because there's plenty of customers that you know don't target their servers with the client policy that says enable third party updates so justin jordan justin's chatting he's throwing me off uh jordan has this article pulled up and we detailed this pretty extensively right we show you some of those queries you can pull up to go and look at uh what is the nlb front end master server right so you used to be able to and you still can just don't do it you can configure ws in a network load balance way uh it's not really any any way shape or form needed uh but though that that language is still in the database that nlb front end master is still in use and so when that nlb front and master moves you don't trust it bad things happen yep and all this just goes back to the whole like you have to make sure that everything everything matches all the time if you're going to do multiple subs sharing the content and sharing the information from that perspective yeah for sure um i think we've got maybe time for for one more thing do we want to show if we have if we're allowed to have a few more minutes here i think we can show one more thing we can show one last little gotcha that's kind of interesting that started i want to say it was in server 2016 or 2019 um that started being a thing which is the instance that you've said you know what enough's enough i'm done i'm deleting wsus and getting rid of it uh i want to rebuild it for forever um i'm gonna revert a snapshot here real quick um yeah i didn't even know this was a thing he's like yeah that bug i don't i don't know what you're talking about and then he showed me the first line actually pretty neat problematic basically if you if you didn't know starting in the server 2016 2019 there's been a recurring theme in our conversation here of it's all such and such well it's all powershell now um so things like server manager and server role when you're installing roles when you want to install a new role and everything like that it's it's all powershell at the end of the day um if you actually go into windows admin center if you go and you look at those different pieces and parts it's it's all just running a configured pre-configured powershell script with variables behind the scenes um so we actually go over here and give me one second to reshare out now this snapshot is done applying window it's thinking about its life all right so i have a a server here you'll notice it's the same server that we were just on um and if we go into server manager [Music] you spend time breaking things it's actually sometimes very difficult difficult to break something in a very specific way yep um but yeah so if you uh take a quick look here you can see that i've clearly had enough i've deleted wsus and i would now like to restart because we were talking about the scenario earlier you know where maybe you just have had enough and something's really broken and so reinstalling it is the way forward for whatever reason right um so to that end let's go ahead and get wss reinstalled see it's it's gone is still here but nothing else is let's go ahead and click add a feature we'll go next go next go next scroll down to the bottom and check windows server update services and sure you can have all the features go next and next next uncheck with check sql because you know why not and uh go ahead and do what is it j i think it's j on the sky yeah j w s us teams go away that seems reasonable now you'll notice that the next button is grayed out here right and up here it says specify an existing database to install it okay please check the connection to validate the server connection well the default is is this the language on this was a little different than 16. just click check connection hey successfully connected to the server everything's great you can now click the next button next install it's going to go ahead and it's going to go and install ws everything seems a-okay and we're just going to kind of continue along here for a second and there's a few kind of weird issues like this that we that we run into right um and and some of them some of them affect only third party updating some of them wsus in general uh and it's it's kind of interesting you know uh with with w sus there's a finite number of things that can go wrong i'd say we've we've really pinpointed quite a few of them over the over the years now and which makes it kind of kind of interesting we get very kind of like we perk up when we see something new uh so i thought we saw all the problems you know you've broken w's in a new way i'm amazed i give you applause i've been graced by a dog i mentioned i have four dogs i just had my uh youngest small furry creature up here down here hello apparently she wants to say hi this is the youngest of the shadow gremlins the german shepherd who's trying to eat my face hello would you like to sing all right so it's just finished installing so we're going to say close on that so if we click on the wonderful notifications here we'll notice that we have this post thing here which if you've ever worked with w6 at all at some point or another you've without a doubt run into the post install actions or the post util tool that exists within configuration manager or not within configuration within wsus and the tools folder i'm going to go ahead and click on the launch post install task because clearly i need to do so to finish configuring it and now it's very angry with me so if you actually go here this will actually if we click in on this we actually go look at the flags and go to the task details it'll give us a log file we can go look at this log log file down here oh we'll notice between go look at c users app data temp and this one i'll i'll just edit this with notepad it's fine right so what we'll notice here is this is that it's missing the value of the instance name a required configuration value was not found in the system when it actually goes to actually do the post install action so it's not able to successfully configure it so how do we fix it at this point well the way to fix it is with our good friend powershell in much the same way that you would go about fixing just about any of the other things for that overhead powershell heading over into program files heading over into uh update update services and then tools we'll find our good friend wssutel or at least my good friend w says youtube space and help and we'll find that we can go ahead and take a look at the post install commands for it which is this right here because effectively that's all it's running at the end of the day is this running wssutil.exe space host install and if we want we can actually do help post install to get the entirety of it at which point it will then give us the different options now what's important to know about this here is this is when you run a post install command it's effectively running some reinstallation pieces here and it will make changes to the registry it will make changes to things inside of the database and everything like that as it's going through this there's another interesting feature involving iis and some other things like that that can happen depending on how you do it when you do it when you move content this that the other thing with how network shares are as well but anyways all that being said if we just put in good old post install and then if we were to put in our sql server's instance name and the content path we would get to go to where we want so if i just do sql underscore instance underscore name yes fqdn matters that's another one where if you don't use fqdn it will also do the same thing the same behavior if you don't use fqdn in that field if you just use normal um the short name it also does the same this same behavior and then of course if we go over to contender equals j colon slash and this will then go ahead and actually run the post install the way we wanted it to now if we've gone over here into the registry we might be able to still catch it before it actually makes the changes here um but if you actually go into the registry and if you look down in here in the wonderful world of microsoft update services server and setup product caught it before it happened you'll note that even though i checked the sql option when i was going through there because i didn't click i just hit check name i didn't actually put in a name it actually typed in the with variable option instead which is the other half of what the post install component is not working so this is now going to work it will go through here and it will fix that and in a moment or two here we will actually see that finish up but i think at that point that's pretty much uh kind of sort of the end of what we had to show um so if there's any questions about any weird things in wsets that people have seen or anything like that our time i think we left a little bit of time left i think we've got a few minutes left here um any questions that anybody has or weird things like that curiosity things it's tough you don't find a lot of people that have interest in wss what are with the exception to the rules here all right so we got a question from john tracy is there a way to import catalog updates into ws without using ie through the wsus mmc console example i recently had to import the net framework 4.8 upgrade for windows server 2019 that one is catalog exclusive interesting i want to say you can use powershell to import updates there's a it's part of the command i think it could be wrong though i feel like it would be one of the dot-net i feel like you need to make an object for it though yeah yeah yeah i've not tried john to be honest with you i've used i've done exactly what you just said uh and i guess you could probably figure out something with scott as well yeah if you really needed to really make your own yeah you could always use scott i mean so uh matthew with your question uh you don't want to edit it manually and the reason you don't want to edit it manually is because some of those fields also get updated in the database as well and if you have a mismatch between the database and registry that's why you end up getting you get various errors there's a there's a bunch of areas that can occur when content doesn't match between the two of them um or you know the like if the instance isn't correct so generally you'd want to use the the wsus util to perform those changes uh that's going to be the best case scenario now we did try to make the server manager kind of post install tasks run by editing the registry but that's not even what it references uh it generates like an xml file that it stores that you'd have to go manipulate um so yeah yeah yes if you enable yes it will oh man for sure is it although we're just tried to break wsus by enabling all the drivers and did yes we did so enable driver updates and w says will the windows update clients scan against all of those drivers what kind of impact would that be and will the client scan them if they are not approved so yes so here's something really important to know this isn't even just to do with drivers in general every single update in wsus that is not declined or gone right deleted so if it's not declined it including if it's superseded so superseded updates 2 will be scanned against so the larger your wsus catalog the the worse it's going to be for your clients and that includes right these these dal and lenovo catalogs we did a v smug andrew jimenez and uh i think justin where did a v smug presentation on adele drivers did a presentation on third party update catalogs so the dell drivers in you know in particular you can go view the applicability rules for them they have very complex they'll have 20 or 30 wmi queries on a single update and and what that ends up you'll see uh the wmi service on your endpoints get pinned you'll see 99 cpu usage when your client is just churning through 5 000 dell updates evaluating applicability rules right so it's not to say don't use them it's just that you need to use them carefully and then make sure you have a well-maintained environment right you need to have that only the necessary updates that are going to be out there and then you need a well-performing iis configuration so definitely you need to minimize the number of updates you have they're going to scan against everything approved and not approved in the configman world doesn't matter we don't care right and and that's some of the don't touch w subscribe right don't approve updates don't not approve updates don't create computer groupings right you don't need to do that stuff in ws and that's still true right don't touch ws in like don't do that but like there are other things that need to be done and and it will impact your performance and you know jake alludes to the the v3 catalog so that the catalogs for these uh drivers have gotten better they've not you they started categorizing the the updates right so and the way they do categories for the the drivers is by model generally or sub model so you can say okay i want the whatever lenovo t480p drivers and it it helps so uh matthew has a question do you recommend declining update objects in windows 10 servicing updates that aren't used i recommend declining basically anything that you're not going to use just in general if you're not going to use it you should probably decline it something actually i was going to call out that cody started to mention there was is the fact of what updates do or don't get scanned against right um so config manager wsus have mostly the same language when it comes to how it classifies an update and what its state is from a readiness perspective to go out okay there's the blue arrow which is the metadata only the green which is the hey i have full content i'm ready to deploy there's the yellow star which means the update is superseded there's the black x which means the update has been expired and then there and then there is the red one which means i'm missing content wsus uses very similar ones but there's a couple of key differentiations there the expired indicator right does not necessarily unless you've configured the certain options for it does not necessarily mean that it is declined it simply means that config manager isn't going to pull the data across into the ws instance anymore from that perspective okay in seven days it will tombstone the record out essentially as long as those updates are still classified as superseded and exist inside of the database and actually cody i think you found this recently when you were testing something um you saw that saw it happen you actually saw it happen was where if the update is superseded but is older than the month timeline so i'll actually pull it up here real quick um if it is older than the timeline but is in a superseded state the update will not be pulled in but it will still exist in wsus and get scanned against so that's the the important differentiation is it still gets scanned against even if you're not necessarily pulling it over into config manager yeah this was an interesting one so i i seen the setting i generally knew what it did but this one caught me by surprise so not only does it does this kind of handle you know when should we in the supersedence rules here um right you can go and set the so do not expire so this this does handle exactly what it says it handles the expiration of updates but what's even more interesting is it actually handles um syncing them in so i had twenty thousand plus updates in wsus but i had to set to three months and when i did my syncs between configman and w says you'd see the log lines that says hey i'm ignoring this it's superseded and it had been superseded at least three months prior to the super cd update so they didn't even get pulled into config man it made my six shorter and then i bumped it up to 99 and then all of a sudden all of my well actually still not oh no is there yeah there's actually updates that have probably been superseded for more than 99 months i would think um but it brought in the large majority of my updates right and so kind of an interesting tidbit here where you're bringing in lesson to config man but like just like jordan said though they still get scanned against they are in wsus they are not declined unless you do something about it um they're gonna get scanned against until they're gone and that's when you do this come to play yep unless you have this set up and checked yup cool that was a really good question it's a really that's a it's a really good question and a really good thing to understand too is it's just how does supersedents work and what are the icons and what do the pieces and that data really mean any other questions cody sorry no that's pretty much it i don't know i don't know there's we can ramble on almost forever but certainly a while yeah awesome we try to make wss interesting not hard it's not easy but someone's got to do it exactly okay frank do we want to do maybe the raffle now yeah okay cool yeah we can do that thanks guys first uh thank you guys for the session there really appreciate it yeah absolutely and then yeah okay sounds good to me perfect thanks ah weird i got kicked out do you hear me now i was just gonna say i was he was about that and he dropped cool okay all right so um first of all in the chat window i'm just going to paste kind of the term service just because the value of the the raffle um just our legal makes this to this just be aware if you enter the contest just just be familiar with those terms something that will be uh you'll need to accept it before getting the ticket if you went other thing just be aware the event it's just be aware if you're international on the call today there's there's probably a good chance that at least uh for most countries you may not potentially be able to fly into the us so just be aware of where you're at um second if you're already attending like maybe you're a speaker or something like that please don't enter because these aren't non-transferable so if you win it would just be you able to book it so i'm going to go ahead and paste in the link that we have for the randomization for the code so if you want to enter you should be able to click that website just let us know if you have any issues getting in sometimes it seems when we do raffles we have um basically sometimes we've taken down this swagit site so if that happens hopefully we can have a backup here but looks like things are working pretty well here so we'll give it maybe a minute before i click start draw and then we'll go from there nice so yeah guys check the chat uh he pasted in the link for the swaggett um go in there and join for the drawing here don't miss out on a chance to uh to grab that so yeah so just for context this will be for the moa conference in october just a really great event just you know really focused around system management i'll include the link for the session uh for the event in the chat window as well so whoever wins they'll get a full ticket to the event which includes hotel for four nights um [Music] and we'll have a code for the winner where you go and when you actually register you'll just be able to enter the code which would be the option which would include the the hotel as well pretty amazing cool we'll give it maybe maybe 10 to 15 more seconds looks like there hasn't been anybody entering lately so there's only 20 20 people so far so um five percent chance right here so this is uh this is pretty high compared to a lot of these yeah i see 60 people on the call so there's uh don't miss out if you guys are i know there's probably some that can't enter but all right so we'll do uh three two one and then we'll start it look at this suspenseful nice congratulations daniel saltzman all right so daniel uh you should be able to get a screenshot of like your end the winner so just email that to me i just put my email in the chat and we'll go through and get you the code awesome congrats thank you don't forget to stop at the patch my pc booth yes absolutely uh wow that's wonderful all right back to you frank i think yeah very cool kind of open session yeah so thanks again to our friends at patch mypc for coming in and sponsoring our session this uh this month very much appreciate you guys as always um [Music] so for the group um one thing i just wanted to touch on quick is 2107. so 2107 of configman current branch was just recently released and really a couple of brief call outs i wanted to make and it's mostly on some prereq changes so we changed the requirements for net framework to 462 and ultimately for eight so you'll start seeing that in your prereq checker if you haven't already started looking at that start looking at upgrading your version of.net the client is now requiring a newer version of the visual c plus plus redistributables so be on the lookout for that and there's a new prereq check to identify infrastructures that are still using sql server 2012. um there's about a year left i think just under a year maybe 10 months now left of support for sql server 2012 so if you're still running that start making your migration plans to move away from it and get on to something more current and then lastly i think this came with the last version but i'm going to mention it now the https or enhanced http requirement um i believe this was 2103 but like i said i'm going to mention it now because it's a big deal uh i'm not a big deal but something important for for your environment if sorry just accidentally mute my headset uh still it's a big deal if you're running http in your environment still the enhanced http option will be supported so that's sort of the simplified path away from http um but the recommended and best practice moving forward is going to be https pki certs and all that so definitely if you're not already thinking about those things the https requirement goes into effect with the 211 release so late next year so we still have a little bit over a year to get to that point but never too early to start planning for that so uh let's see yeah thank you yeah certs client comms it's uh 2111. not 2107. well that's what they said on the dock that it'll be 21-11 um it'll be well it actually says the release that comes after november of 2021 is how it was phrased um yep and yes.net 48 is all site servers i just had a customer actually um it's just a warning right now it's not a hard blocker for the net stuff but uh it is something to keep in mind so but yes it was all site servers they had a remote server that it kept coming up on um and then they when they reached out you know they let them know that it was all site servers and not just the primaries so any questions on any of those things or any ques any any questions on any of the 2107 stuff that's come out i know i didn't talk much about any of the actual feature releases in here but okay well what i'd like to do and is open it up to the group we've got a large group on today so we've got quite a few peers on the call and a bunch of experts on the call so if there's anything anybody wanted to ask about things you're dealing with things you're looking at that you're not quite sure um pop off mute throw it in the chat whatever you feel most comfortable doing but i would love to see some open forum here and seeing us helping each other in the community so there's anything that anybody has now it would be a great time for that can be anything small doesn't have to be something huge uh i think one thing he said actually is kind of interesting so um it's something that's called out in the docs but you always hear a little bit of mix back and forth so https via pki is is the official recommended solution right and not you know http is supposed to make it easier for people but at the end of the day right that doesn't encrypt everything there are actually still things that don't get encrypted uh it just encrypts the most important things um but but pki is still the recommendation then yes okay that's good to know docs kind of say it but you still hear people yeah just do http it's easy just check the box but at the end of the day right i mean it's better to have your own pki infrastructure and to be able to revoke those certificates easily exactly exactly yeah you get more control right like of everything like you said we can revoke certificates there's just more control in it um and so ultimately that's the recommended path but you're exactly right that the path of least resistance is just checking the box for ehtp yeah yeah for sure so i think we're going to see a lot of customers make a frantic transition when that date starts coming around to just check the box and move but long term hopefully we'll start seeing everybody moving to https oh yes good question this is timothy i've got a sql question uh at what point would someone recommend for config manager having a sql instance that is not on the primary site server um when you say at what point do you mean that like what size environment right yeah say like number i was a number of endpoints would be the probably the primary gauge but also if there was uh any uh other realized benefits from having it on its own uh own separate instance i don't know that we have um any specific guidance that will say you know once you hit let's say 50 000 devices that it will um uh that's the number that you're gonna wanna to move it over right so the big thing is is number one someone just put in the chat too is sizing it correctly they ask you know how much ram you got um so that's that's the big question right is you know as long as we're sizing things appropriately um but then someone also did just make a really good point about um the high availability so with uh to get true aha the database cannot coexist on the site server so um so that's always something to consider can can't it now though there was a bug but now you can host the database uh on an availability group right and do aha servers has equal across two site servers i think uh there was a bug at one point that didn't allow it because of a prereq check but i think it got fixed yeah the prerequisite i think got fixed at least i don't know if that made it fixed though so i think you can do an availability group with a primary secondary node on your two site servers and do aha local to your site servers um okay yeah that i didn't know it's interesting because so like it's it's an interesting thing right because justin's putting it in the chat here right there's there's technically documentation for it but i can assure you that if you go to certain conferences and talk to people that there are absolutely people running above those numbers and running below those numbers all over the place it all comes down to sequel how much ram can you throw out how much cpu can you throw out how much resources can you throw at it for sure that's a good question in the chat gotcha thanks guys appreciate it is this is this the link that you just paid oh so you just pasted it up to 100 000 clients sql server is remote from the site server compute yeah because this used to be a big deal behind the kaz's right this was actually this document originally was one of the big reasons why you would always be like no why are you building a cash you don't have a hundred thousand clients right um as well as you know the whole concept of sql server being on box or off box because i think this one actually has the kaz stuff in here too doesn't it justin you see this one has the cas yeah it's like posting it yeah i think that doc was only specific to if you're in a cast like a primary in a cash that's where i guess remote supports more i don't think that applies i think it's still 100k if you're in a primary site that's standalone so i don't think the code located versus not in standalone as far as i know i don't think the numbers change in that scenario only if it's in a hierarchy based on that doc um lost yeah so this one around client push so uh will that ensure that the client gets installed it's like yes but with huge air quotes right so there's a lot there's a dock on microsoft's site that talks about all the prereqs for client push so yes it totally will try now you can see there's at least one thing at the bottom don't install it on domain controllers don't install it on site system servers but you've got to be careful with client push especially if you allow fallback to ntlm that's technically a threat vector it's something to keep in mind uh and then it's only going to try to do site-wide client installation if it has discovered those objects right so if you enable site-wide client push like you have here and then you don't have network discovery 80 discovery or any discovery at all enabled you're not going to see a whole lot of things happen so you need to discover them for that to work you need credentials you need authentication you need to be able to actually get to those right so there's an accounts tab there that you have to configure and then on top of that there's a log file ccm.log it's in the installation directory the log directory for all things configman for the site server that'll show you all of those attempts to reach out to the servers you can have multiple accounts configured you can also have no accounts configured so if you have no accounts configured it's actually an interesting idea because if you don't have a client push install account set up it uses the site system server account which is the computer account which has a randomly changing password that nobody knows because it's the computer account password so it's actually kind of an interesting idea to use your site server as your client push account but if you're seeing that it's not hitting all your systems right there's a lot of factors you've got firewall rules and rpc being enabled and smb being enabled and open uh to be able to do all the things it needs to do what checkboxes do you have checked right what did that happen to be a site system server are you not setting it for servers um do you have the fallback to ntlm enabled or disabled are you blocking ntlm and kerberos is failing you know there's just little bits and pieces the ccm.log is going to help commonly i see recommended maybe i think actually justin just did a video that covers client installation pretty extensively i think we could probably drop it in the chat if he hasn't already while i'm rambling he did um yeah so there's other ways to deploy the client this is a way uh i personally recommend against it to be perfectly honest with you because it has it you can totally exploit this it is it has risks but you know i'm not a consultant i'm not going to come here and say don't do that but it's my recommendation use some of the alternatives yeah the client pushes a lot of interesting pros and cons too so you scale up and scale down one just from a security thing it's that it's a spooky thing because you typically you're gonna end up with at least one account in there that's gonna basically have access to or likely has access to way more than what it probably should right and then you end up with an account that probably doesn't have its password changing very often that somebody could grab and it's going to have local admin on everything um and the second that gets scrapped that's no bueno and i definitely would you already covered the ntlm the ntlm thing definitely i would recommend unchecking that yeah i think i think the other thing around client push is uh if you have automatic enabled just be aware like that won't automatically push unless they're in a boundary group that has site assignment too so if you're seeing some clients not push which i think is part of your initial question might want to check whether or not it's being assigned to the site via your you can go and look at like the clients or the devices and see if the site code is being populated if it's not you probably just don't have a boundary group set for site assignment which would cause the automatic push never to even attempt not even counting things like the prerequisites like firewalls which were mentioned one other thing too uh there unless this has changed i haven't checked it in a minute is you might the other thing is it doesn't necessarily download from distribution points so if you're in an environment that has a lot of distribution points and only a couple of management points and maybe you have an observed number if you use the right-click client push of the automatic the content is pulled from the management point not necessarily the closest distribution point um so if your management points are going over a big when as opposed to looking to download the content locally it's going to want to pull it from the mp because mp is always going to have newest content for it so just something to be aware of that's kind of in the the can cause high network traffic piece there i know that just now but that's why that can cause that high network traffic there was one other that slid by there golden rule for how many subs uh wait before wait before you get to that really quick i want to go back to that sql question about when to when to move to a remote sql database so the the article that that justin pasted in is old so that's guidance um from 2012 r2 and all that so our new documentation um talks about you can have a stand and i just want to clarify it you can have a standalone primary site on a co-located database with 150 000 clients um there's not a hard rule that says that a hundred thousand you have to move it but it is recommended that when your environment starts getting bigger um because of the high i o requirements to start moving things and then you want to start looking at that remote sql database so i'm going to paste in the new recommended hardware dock just so you guys have that one as well so i just wanted to go back to that quickly but yes we can now go on to the next question which i saw was is there a golden rule for how many software update points you should have i would say give the jason sandy's answer on that one of it really really depends on your environment right i mean if you have if you have 100 clients right okay one software update point's probably fine or maybe just use with b right maybe that's good enough for you right the other question too is do you really need software update points i don't know um that that's another half of that question to look at as well maybe the correct answer depending on what you're actually servicing is is zero maybe i want no software update points i only want to push updates via windows update for business or or something like that that's totally a viable option as well in general i think the docs i think say ten thousand i think devices is ten thousand or five thousand dollars oh it's uh crazy high this is crazy software update point can support up to 25 000 25 000 when running uh the software server on the site system yes yeah so 25k if it's uh if it's hosted coexisting with another site system role maybe it's on a dpmp or the site server and then it says a standalone sup wsus can handle 150 000 clients if wsus meets requirements right that's right staggering number it's huge yeah the other thing though that goes with that too though um is remember we were talking about the different types of databases is the the size limitations of the different types of sql right so as you kind of expand out into where you want to put a software update point maybe it's not on the primary maybe it's on some other server and maybe you decide to install sql express on that server just be aware that there are size limitations for how big that database can get um along with those lines too so yeah i think sql express is 10 gigabytes something we just kind of we're talking about the other day so um you might be tempted you're like oh sql express that's got to be better than wind right but once your database reaches 10 gigs it's going to just peel over and die i just dropped the link for the uh size and scale for uh the uh sup roll i feel like they just changed that recently too to check the github commits i don't know i feel like it used to be lower for some reason or maybe it was mp rolls that were lower i don't remember oh let's see so next question going i see jim here is a loaded question my current environment 2012 r2 in sql server 2014 same box planning to upgrade wondering what is going to be the easiest way to upgrade when you say you're planning to upgrade gym are you talking you're planning to upgrade the server os your 2012 r2 yeah i need th i wanted to upgrade the server os and the sql server since i know the sql server is probably going to be uh out of or not supported here before too long anyway so i wanted to do them both okay um i do have go ahead yeah in the past i think the last two times before i was at the company they literally stood up a brand new primary site new site code and migrated everything and that seems like both insanity and overkill to me so there's i knew there has to be a better way yeah um there's a couple of things you can do so we do support the in-place upgrade of the os now um that's fairly recent probably that's been a couple of years now they started supporting primary site in place upgrade for uh server os um the sql server upgrade you know i've done several in the past with customers and they're honestly usually pretty benign so that piece of it i wouldn't be too worried about um you know obviously just planning and timing for those kind of things but then the site server upgrade or the site server os upgrade um one thing that i've been telling a lot of customers unfortunately doesn't apply for you is to take advantage of the aha option do a failover and you have a pseudo server os upgrade done for you um just failing over to the other box but because your database is co-located you have to move everything then um yeah i was wondering if it would be worth it to even go down the road of setting up the ha and failing over but i wasn't because i had heard in the past that upgrading doing the in-place upgrade for the config manager site server wasn't exactly a great idea but if i don't know i certainly don't know yeah i don't either i don't mind yeah it is supportive this one's kind of right and yeah that's the thing it is fully supported but it's still not something that i love but uh jordan just posted in the other option which i think you just mentioned as well you can do a full site recovery so basically go through the essentially what would be a failover procedure um the um uh so basically it's that it's in place upgrade um h a failover those are really your options to upgrade your site server os unless someone can rattle off another one excuse me that i'm missing yeah the only good thing about my environment is i only have one server we're using uh adaptive one site for a lot of the content replication so i don't have distribution points it's literally just one server it's got all the roles on it but it's just one server right i i did just put a link uh it's it's a little bit of an older article but the process hasn't changed so it doesn't really matter uh from jason sandy's where he talks about that backup and restore scenario where you stand up another server with the exact same name the same way you don't have to do the same drive structure but i'd recommend it uh and then you know you can just stand up a new box use the same name and then do a site recovery that goes pretty well too i've had some good luck with that but it's got some some to knows that are listed in that article yeah definitely the one other thing i would definitely look at is make sure if it is a 20 tool or two server that it's not an upgraded 2012 r2 server that it didn't follow any of the pass on that um because it can get weird and sticky with that um also want to be really careful that your wmi health is good before you try to do an in-place upgrade in place upgrades usually work most of the time um yeah they're fine but when they don't they go super super bad usually there's no there's not much of an in between they usually work i would say probably i don't think i've i don't think i've ever had one that truly like failed failed or failed to a non-fixable necessarily state but i've heard some horror stories about the most part so snapshots are my friend in this scenario is what you're telling me if you shut the server off first that's the key thing okay so snapshots when they're cold is the big important thing because snapshots when they're hot and somebody correct me if i'm wrong here but hot snap shots still are not fully technically supported if the sql instance is on the same box because it's a difference in between the moments for it yeah i normally won't snapshot it ever when i'm doing any other type of upgrade because of issues like that and one yes one thing to keep in mind right is if if you are doing the backup that you you should be doing of you know your content and your database and such uh realistically right your your servers should be able to wipe off the face of the earth and you could restore to whatever os you want with the same name right that's that's something to keep in mind you can do that at any time you know the tooling is there the the recovery is fully capable of doing that that's the way i i personally prefer to do it on the ones that i've done just because i like starting with clean fresh servers um because you never know what things have lived inside of that server before unless you owned the server beforehand um but yeah yeah now i get it yeah this was stood up as a fresh 2012 r2 so that part doesn't it doesn't worry me so looks like i had a couple scenarios to test in my qa environment thank you gentlemen yeah someone did just uh chime in um easy thing to do let's say the upgrade uninstall w yeah that is something i should point out to you jim if you do the uh server os wsus has to get removed it's in the documentation here i'll find that really quick and paste that in the chat but because that is documented that wsus has to be removed first um before you do the server os upgrade so yeah one other interesting thing to point out since you were talking about wss just pasted an article if if you do an in-place upgrade from anything prior to 2019 windows server 2 2019 or newer any third party updates the categories will be duplicated because wsus changed the way that they create the hash for each category from sha1 to sha2 so basically if you're doing third party updates dell patch my pc really anything when new updates are published after the upgrade happens there will be a duplicate category that gets created in wsus for that new product and if you don't know that and you don't go enable that in your software update point there's a good chance new updates published after that in place upgrade won't automatically sync into config manager so we actually get a ton of support cases for people that do this and they aren't aware of that that scenario thank you it's a good discussion so far guys i uh i'm very happy with all the questions so far so thank you keep them coming though clearly have some experts on the line here that can help answer that's why we're here frank to make you happy thank you chris i know nothing well this is just exactly what i was hoping for was discussion and people asking questions about things they're doing and getting answers so who's ready to get into into me that mecca does work really well intune does as well and it's uh there's a lot of benefits to making the move so for our next uh user group meeting um that's what we'll try to focus on is a lot of the cloud management stuff yes if you have 2500 apps you can migrate your apps to intune it's totally a thing or do you come manager you can also co-manage them and still get all your apps from configman and deploy apps in tune there you go so that's that's an interesting point to make right because it's a little deceiving uh the in if you do co-management and there's the sliders right which are great love it but it's deceiving because you take the client app slider and you move it over to pilot intune or you move it to intune but the that client app slider in particular is different than the rest of them because that slider lets you do both right the rest of them don't but client apps lets you get updates from configman or apps sorry the client apps one lets you get applications from configman and from intune and that's different than the rest of them so it's a good thing to know um and you can actually also make them appear in company portal so you can still have a single point of service for your for your end users but come have apps come from both sides yep isn't there another slider in there as well that lets you do both mr cody oh yeah i guess that's true updates you can technically do updates yeah matter third-party updates specifically third-party updates specifically right so that's one of the weird the weird ones a lot of the times happens is dual scan yeah so if you enable the slider over to it you can actually enable dual scan you can get your microsoft updates all through your beautiful windows updates for business you know your whoopee joy happy happy days and then you can get your third party updates for anything else be a good old config manager if you want to yep and that was all customer feedback driven as i recall because initially when those sliders came out it was one or the other but a lot of customer feedback was we want to do both still so and i thought i heard something recently about cloud management gateways being deprecated in the near future or something like that um is that no longer needed or something right misread yeah i haven't heard that they're changing i mean they've they had they just added some new functionality in 2107 for the uh um scaling and stuff so i have not heard anything about cnbc with cloud dps or something cloud dp was just officially i think deprecated um and it's no longer a thing but like that the actual term cloud dp there are still cloud dp's but they're just part of the cmg now so but yes you are correct there that's probably what you're thinking of the cloud dp i think in this last release was officially killed off oh maybe that's what it is to matt the classic version of cmg yeah but no i haven't heard cmg specifically the service moving away so gaining momentum looks like there's still a cloud cloud dp node in 2107 though when i reinstall when i rebuilt my environment uh beginning of august um we used 2103 because 2107 was still like early opt-in i think um it actually offered me the option to build a new cast and i'm like what i thought they took that away i i don't have a cast we just got a single primary but it offered me the option to build a cast right off the bat um that was just really weird they're still supported and still it's still supporting they have use cases it's just that they introduce the idea of a kaz collapse right so they're they're letting people you know relitigate their sins of the past and and remove their cavs uh if they don't actually need it uh because there's some very niche use cases for it valid but uh there was a lot of people who ended up with a cavs that did not need it and until recently you couldn't get rid of it the story i always heard about the cast was one of the ways that it got created was like it was in the textbook at one point it's like drawn in the picture and so all the consultants were like yes this is the way you need to do it when realistically it was really designed for companies that spanned global coverage zones that needed maybe this primary to be in a different language and to have everybody being reporting this that the other thing and then to be able to funnel that data up to a top level for management owners to see it or or they tried to use it for network throttling right yeah which like that's it's a bad use case but that's don't use pull dps exactly doing cavs collapse with two of my customers right now so you can use both luffy and sccm so you can get updates for three-part updates uh no that's not something that's changed recently that's been around um for a while now it's just component of the the dual scan features of how that of how that works so basically it's scanning against microsoft for getting the updates and its policy for that and then it's scanning against w so that's for the third party updates and it does it yeah we actually have an article about that i actually i think i have a let me go see if i can find that real quick and i'll post in the chat the uh technet article on it they're not tech net what was the new blog that they created uh whatever the other one the other one it's interesting though right because to achieve that third-party updates from wsus and then updates from wolfp you you do the thing that everyone was angry about and deliberately trying not to do you enable dual scan that's really what's happening and and actually it's not just third party updates you you can get them everything from from both if you want like you can get microsoft updates from config man and from intune or from wolf b you're just gonna get really confused um but it boils down to to dual scan um you know you're gonna be able to talk to both and there's settings that you can figure to do it there's a lot of newer uh blog content out there jordan just linked it and you can totally do this it's fully supported it has been we used to do it actually internally at patch my pc uh we were doing all of our third party patching from our internal sccm infrastructure uh and then we would get wolfie for our updates and then we eventually dog-fooded one of our solutions and now we'll peer into uh what other questions guys here's one for um like compliance items or configuration baselines or something like that i i don't know a whole lot about those but i'm considering using them because our group policy environment is messed up to put it nicely to the point where i wrote this big hairy powershell script to put all of our settings into people's machines including current user stuff that was an interesting little event side adventure but um you know we're copying a bunch of files setting a bunch of registry settings um thank god for getadmx.com to translate all those group policy settings that we thankfully had documented so i could put them into the script but i'm thinking that maybe you know compliance items or baselines or something like that might be a better more well officially supported solution rather than my hacky powershell script uh yeah is that a thing or you know if and if we're considering possibly going to some cloud management in the future is that going to be a waste of my time i have to redo redo it all over again i believe that'll actually convert all of your registry settings to config baselines in config manager so they're good for preventing configuration drift it's totally valid what you're proposing right you would break things down probably a bit more granularly it sounds like you have a big script that does lots of things uh you might end up with lots of cis and or lots of bass lines but it's totally valid you know i've used cis to you know do the post configuration for line of business applications to to ensure that my distribution points all have uh deduplication configured uh to make sure the registry settings for my app are correct right you can do all those things and and or i've seen ci's used to maintain the xml files that control uh feature updates you know so there's a lot of ways you can use it uh my colleague andrew and i just posted the link in the chat we did a last year it's been a time flight it's been a year and a half almost we did a v smug virtual systems management user group around cis uh it's got some pretty solid content in there and yeah jordan just mentioned dsc right desired state configuration it's related right you you can do this in other ways too um but within configman there's nothing wrong with cis they're they're pretty awesome um you said if you're gonna go to the cloud though and go to intune and if you're gonna not do co-management then i don't know maybe you don't want to go through the trouble it depends on how many things you're doing but if you're pretty savvy jake said there's i don't know if there's a website there might be i think there's a powershell script or a module that does it you can feed it um registry info and it'll generate ci for you right so you know your settings oh yeah like you can totally script automatically generate ci's to to manage all these things for you yes if i could do it through script it like it because i've got all the registry settings in a csv and then i've got i had to do it so that because some of them are basically preferences rather than policy so you know set once and let the user change it if they want afterwards um i don't think i'd be able to do those through a configuration right because you just gotta get crafty so you create a script right you create a script and you have a set once and then you set a flag you need to maintain something somewhere right your your remediation script would set a flag maybe you have a maintained area in the registry that's like you know org dash remediation history and then you you assign some flag that says yeah this remediation has happened once right it's it's like a bit that says okay it's done so then your detection does two things it says a do i need to remediate and b have i remediated before right and then you check both those conditions and all of a sudden you've got a preference instead of a policy so you just gotta be creative i did link that website i found it it will create the check and the remediation script for you based on the registering entry via powershell ah cool thank you uh the only thing i'll mention with cis there are timeouts if you start getting crazy complex with scripts uh by default it's gonna be one minute so unless you're doing something pretty crazy it's difficult to hit that yeah i don't think i mean even the whole you know all the stuff that's going on in a local machine um can get done in less than a minute even with my script that's actually verifying each setting um well thank you is the ci script timeout configurable uh yeah so it's actually yes uh supported i'm not sure offhand so it's in the scf i'm not mistaken the system configuration file you can totally change it but it has a larger scope than you think so it's the ci script timeout now when i say ci script i don't just mean configuration items but i do right detection for an app is a ci requirement rules are a ci uh everything like that is a ci so you're gonna have a larger impact than you think when you adjust ci script timeout but yeah it's technically changeable it's a portable i'm not sure i can speak to that i want to jump back real quick here there was a question here from matthew about what was the value add over 80rs was that in regards to like about wolf b or or what was that question about i just was trying to understand a little bit about that before we get too far down the rabbit hole here like why would we use wolfie over an adr or something or why would or something else see if we get a question or under the come off mute or not all right never mind let's keep going in the chat for it sorry i'm all good i like your comment about adrs though being able to be available now though i think that's pretty neat i am so excited about that i used to have like a bunch of things about adrs that i could complain about like don't get me wrong i love adrs i think they're a fantastic method of getting updates out to create automation they've done a really good job with with for what they are right um they can be incredibly flexible but definitely available is one of the things that's missing that was missing and that was not the number of things that used to just get me angry or get me worked up about it have significantly decreased over the years i will say uh meaning you want the uh updates from and soon to be more like what you have in configman i'm assuming is what you're saying there matthew yeah i'm a little unclear right i don't know that the ui in general in intune is going to shift towards something like configman i mean it kind of has its the azure look to it right unless there's something he wants more granularity because you can't do that with the graph api but again you'd have to use the graph api for that right anyone had experience on wireless pixie booting go talk to two pint actually let's say there's a i think oliver killspash did a blog post about that you see me dangerous i agree dangerous yeah right yeah here we go yeah it was oliver all right i found it it can go higher i just don't think anyone would want it to [Music] there you go see it's fine um build wind pe with wireless support and i want to say that there wouldn't be that that was the beginning of a series that he did i want to say there's something else about that out there that then he was able to then convert that into pixie because once when pe supports it the rest of it off it goes it summons the dogs yeah so the other common one is ipixie right there's alternatives to the traditional pixie that you're used to that technically add other things uh so it's kind of like a yes it's possible but in a lot of people's opinions just because you can doesn't mean you should you mean you don't like re-imaging anybody who connects to your wireless network's computer it's a feature you guys are imaging go to intune man i mean it's just imaging by another name it's fine yeah i work for a law firm so we actually do have to seriously wipe machines and re-image them on a regular basis because of all these different iso compliances and probably hipaa and who knows what else so bare metal is we're we're never going to get completely off of on-prem stuff we have we're going to have to have you know the ability to image a machine forever at least for for the foreseeable future a lot of vendors are coming out with a bios deploy option for the os as well so eventually you might be able to go with that yeah we're we're pretty much a dell shop and i have seen some of that um and dell's the one that's leading the charge with that so you'll probably see it before others i've seen it in there and usually turn it off because to me it's it's potential you know a way in i guess but uh i hadn't dug too deep into it either yeah if it's anything like their uh bios connect uh with the vulnerability there just to stay away from the gen one i think yeah i turned that off too and i saw that and i'm like no way no thank you that and what is it um they had some sort of like lojack sort of thing on there too it's like no i don't want all this crap in here go away any other questions guys yeah got a little bit of time left couple minutes for a last minute question i have maybe a stupid question maybe i'm being stupid about uh ci's in a baseline i'm starting to use them more and more particularly script cis where you have a you know detection script and a remediation script and i find that if my detection script you know returns a result that the the thing needs to be remediated and runs the remediation script even if that remediation script regardless of how that remediation script ends whether it's successful at remediating the issue or not the baseline returns compliant because it ran the remediation script and i guess it just assumes that the thing is remediated i've worked around that at this point by having another ci in there that checks the same thing and doesn't have a remediation script um but is this is this something anyone's familiar with is that is that normal behavior that if you have a remediation script in a in a baseline like that in a ci um then it will always return compliant with the assumption that it remediated the thing in my experience yes now if your detection right the next time the ci runs it would eventually eval and tell you that it was um yeah so and i think you know the other consideration is um exit code so technically right you you could within your remediation additionally do the detection and throw a non-zero exit code and potentially populate the error stream and that may move it over to non-compliant so that's something to keep in mind i think i theorized about that as well and tried it and it and it didn't didn't change the result it still returned as compliant if it um if it ran that remediation script even if i exited the remediation script with a with an error code or you know false or whatever i'm not looking for but i'll try it again and um yeah if need be i'll just stick with the the idea of having an additional ci in there that that rechecks it yeah it should be that if the remediation fails it doesn't return as far as i recall from it i know that the timing on them can be interesting shall we say uh that's one thing i've always struggled with is looking at the reporting codes on it and the timing of it is that there's a there's some built-in buffering behind the scenes there where it won't let the pieces and parts run faster i think even if you set the remediation of the times down to a minute it won't uh it's gonna wait 15 minutes so one of the other things you might want to do and apply there is is revalidate it but apply some extra time to looking at it um to wait period on it because it might just be that it hasn't actually updated or changed the state yet because of how it's doing that buffering gotcha yeah paul that depends on what you're looking for out of it because you can change what the expected response is from it it's different whether it's true or not yeah so there's some check boxes and in like okay return uh you know mark non-compliant if item not found like there's some checkboxes in the ci's themselves uh and then depends on what you set right it can be a boolean an integer it doesn't matter uh but yeah it should if you don't return or if you exit non-zero it should say non-compliant but uh interesting if it doesn't i'll try it again maybe i uh maybe i went when i tested that retested that it uh i goofed something but yeah thanks oh yeah what you're looking for uh i guess in regards to ci's though we did link a video i could probably find it and link it again andrew and myself did one for a user group a while back uh could be worth viewing you know we do talk about some of these kind of weird caveats and such and provide some examples of them and all that fun stuff yeah it's definitely a uh when you're when you're first looking into that stuff it's there's a lot there it's there's a lot of gotchas in there for sure well we are just about out of time here guys um uh good question yeah great point there if uh if your remediation return error doesn't return on compliant then we'll probably can file a bug max um if we're certain you know what i mean so we had like you said you can do it do a double check and then we can take it from there yeah thanks i'll follow yeah definitely let me know i can run through it in my life as well um and see if i'm seeing the same things um all right well listen guys uh we're out of time here now i appreciate everybody coming in i really appreciate everybody um helping answer all the questions that came in from the group uh very very helpful and i again thankful for the group for bringing the question forward because that's this is exactly what i was hoping for we got a lot of really great questions and a lot of really great discussion and you know one thing that you know you hear all the time is that if you're thinking the question there's a really good chance someone else is or that they're dealing with the same exact thing so again thank you for everybody that's asking the questions and a huge thank you to all the guys that patch my pc justin and team for coming on and and providing content today very very much appreciated uh again don't forget head out to the meetup site meetup.com forward slash click to follow us we will have another event sometime this fall probably in the november december time frame i will be seeking out speakers if you're interested in that and hit me up on twitter or just shoot me an email or hit me up on teams i am we'll start building out that speaker list now so if you want to speak or you know someone that wants to speak let me know we'll get you added to the calendar um i think like i said earlier i want to focus on some cloud stuff next week maybe we'll do tenant attached and some other into management things so be on the lookout for that but again thanks everybody and have a great day and a great weekend

Channel: Patch My PC

Views: 2,772

Keywords:

Id: jqn_4P2Rew4

Length: 117min 4sec (7024 seconds)

Published: Wed Sep 15 2021