I Compare: PowerShell Edition - Mathias R. Jessen - PSCONFEU 2020

welcome to this talk here entitled i compare partial edition my name is matthias also known as is reset me the following hour so we're going to have a look at options for comparing things in powershell and we're going to have a look at how that influences sort order when we sort things and finally we're going to have a look at how we can influence some of this behavior using partial classes so on the agenda for this talk we're going to review the existing built-in comparison operators in powershell sort of what what primitives do we have available to be able to to make comparisons uh we're going to very briefly cover the difference between comparing for equality or equivalence versus comparing for ordinality or relational inequality uh obviously we're going to have a look at how we can order stuff or how we can solve stuff with powershell and we're specifically going to have a brief recap of what sort of it can do let's have a look at what group object can do and then towards the end we're going to dig into how we can create comparable classes with powershell for this purpose we're going to have a look at a couple of implementations of the icomparable interface and hopefully a brief look at i compare and i equality confirm so first things first comparison operators in powershell what the what is available to us out of the box first of all we have the dash eq and dash and e operators equals and not equals these test for simple equality we can figure out whether a number is of the same value of another number or the other way around we also have dash lt and dash gt for less than and greater than operators and then we have a set of non-strict variants these less than or equal and greater than or equal on top of that we have a couple of containment operators for when you have collection types arrays innumerable case types if you want to figure out whether a collection contains at least one instance of of mountain contains we're in is the way to go right these obviously also have negated counterparts so not in and not contains we also have a couple of string operators or screen testing operators the dash like wildcard patent meshing operator and the dash match regex pattern mesh operator and then finally we have a couple of type operators the s operator to test whether an object or an argument is of a certain type and it's a negative counterpart is not now why have i arranged these in this sort of weird weird matrix here well there's a reason for that because these operators all have a different characteristic here and if if if we saw a group by some of their their common characteristics uh we might be able to sort of draw a venn diagram that looks a little bit like this everything in the top two rows all the operators are what we might call overloaded that is they depend heavily on the type of the left-hand side argument and we're going to have a look at what that that looks like in real life the ones on the right and in the middle uh all have sort of a double role that is they can perform comparisons but they can also filter finally we can take this group in here and say that these operators here are less than greater than less than or equals and greater than or equals really deal with inequality deal with relational inequality and this is going to be to be important in a second finally we can group these into this thing kind of very strict and non-strictly in quality operators the first group is the overloaded operators and so uh let's break out the parcel and see what we can learn about them let's have a look at a couple of examples of how this overloaded operator behavior might influence the results you're going to get when you're planning so let's take some sort of the the obvious uh trivia example here uh i have two values i have two strings with the exact same content i apply the eq operator to that i would expect it to return true right i i expect these two to be to be considered the same and sure enough if we if we execute these uh the result is true but in order for powershell to sort of be helpful right powershell is supposed to be sort of a success tool for people who are not necessarily professional software developers and so we have a couple of behaviors that allow us to sort of compare apples and oranges and this is sort of where this overloaded behavior comes in so that means that when i say apples and oranges basically comparing numbers to numbers is not the same as comparing strings to strings and so in order to compare a number to string or string to a number we need to convert both of them into um to a type that can be compared right so the way to do that or the way that powershell does this is whenever it sees one of these overloaded operators less than uh eq or the containment operators it's going to have a look at the type of the left hand side operand and so if the type of the left hand side operand in this case the numerical literal one partial is going to interpret this as the integer one right what it's going to do is it's going to look at the type of the right hand side argument over here the string one it's going to say ah these are not of the same type right these are apples and oranges let me just make sure that this gets converted to something that's comparable to one so what partial does is it applies you know normal partial conversion rules for converting the string one to unit and sure enough we get the numerical value one and therefore the integer one equals the string one or the string one uh equals the integer one are always going to return true right they're sort of symmetrical by accident in that the string one when cast as an end becomes the integer value one and the other way around an entity value cast to a string um is is going to be the same as the string representation right this also works for ordinal comparisons right so again we sort of check for the inequality relationship between our two operands and so we we might do something like um you know one is is less than two yeah that that should be true sure enough possibly converts the string two to the integer two and this relationship holds right uh the other way around we take the string and the string value two says is this greater to one and once again partial says this is true [Music] but is this by accident or not what if we compared the string 2 to the numerical value 11. [Music] let's see what happens apparently in powershell 2 is greater than 11 except not really because what's really happening here is that we're we're comparing the string value to to the string value 11. powershell is silently converting the number 11 to a string that just says 1 1. and from an alphabetical point of view yeah 2 actually has a greater value than 1 1 right if we were to sort these alphabetically anything starting with the number one would become four anything start with with the with the character two and so this can sort of bite you right so this is this is really this is really important to keep in mind most of these operators these comparison origin partial are overloaded and it depends entirely on the type of the left hand side arguments what behavior we're going to get from the comparison algorithm the most previous example of this is the uh sort of true false dichotomy and so i have i have an example here where i'm going to take the the boolean value uh from the dollar true variable and i'm going to compare it with the string true and so if i do this if i do this sure enough it's going to retention right through 3 let's try this the other the other way around string true is it false no it is not false right so okay it seems like booleans and strings are symmetrical as well or are they if we do false equals the string false huh that's really weird especially because if i do it the other way around if i compare the string false the string representation of this boolean value to the actual boolean value false it says that it's true and so in this case we can't we can't rely on some on on any sort of um uh uh asymmetry uh or we can't rely on the on the symmetric behavior of of the the eq operator simply because they are overloaded and depend on the left hand side so again these might look equivalent but they are not and really important to sort of keep in mind when when comparing things in partial uh finally uh i think as you can see here there's even a ps script analyzer rule specifically for this case where people get a value passed from elsewhere they try to compare to null but if the value that we have over here has been cast to something that can have a null like or a a non-truthy value we're going to have a problem because they're going to try partial is going to try to convert the null value into the type of this object over here so again if a string an empty string for example this is going to this is going to be true simply because casting null to a string is also going to result in an empty string so again this only works by accident in this case the real way instead of doing this is to say null equals and then the input value because null is the only thing that's null and so either this value is known and you get and you get true because no type conversion will um will occur uh partial three and up uh i i'd recommend anyone to get into the habit of this if if you're sort of getting bitten by this by null checking like this use the use the type operators instead if you expect you know object to to be some custom data type then basically basically test whether the input value is of that type or not because null doesn't have a type and so if this has already been cast as an expected type and it turns out that the value is not actually of that expected type it means that the value is null and we can safely skip so again stop doing object equals null or null equals object simply just test whether the object is actually an object or not and then make your decisions basically next up let's have a look at the filter operators the filter operators what does that mean well um as i mentioned before all of these operators work in something called scala mode right that is we have a single thing we have a non-array and non-collection a non-innumerable object on the left-hand side of the operand and then we have some object to describe a criteria we're testing against the on the right hand side now we used um dash eq and and the inequality operators in the previous exam so i'm going to use the the spring operators matching like in the following but the same behavior will be true uh for um for eq and the inequality operators so let's again look at the simple expected case when applying a mesh operator i have a um i have a regular expression pattern here that is just the literary characters is in my handle is reset me right and so we're just going to ask the regex engine you know can this pattern be satisfied uh using this input i want to execute this and sure enough is found right in my is recently so far so good but what if i were to pass an array of strings it doesn't matter how many right but an array of strings as the left-hand side argument um to the mesh operator so i'm going to do this right the only change i've made here is that i've turned this single string into an array containing this single string right i'm going to execute that and i no longer get a boolean value what's going on here right like is it true or false i'm getting confused over here well it turns out that it is true and that is exactly why it returns value so what's happened here is that as soon as we supply an array since it doesn't it doesn't it doesn't make sense to apply you know a regex match against against an array of things what's going on here is that partial is going to unravel this entire array and then against each one of these it's going to apply this comparison and so again if we look at a a if we look at a collection with a few more examples here using the like operator again if i execute this i use the like operator to filter i get all the matching values back right i get all the values back for which light would otherwise have returned true when i executing this condition against it and so basically when we use these or when we use these uh these filter operators uh in non-scala mode or in list mode basically we're doing this right this is this is functionally equivalent of taking the collection piping into where objects and then testing against the value of the object itself and again down here the example from before we use the like operator the exact same right we take an array we pipe it to where object and then we have where object execute a predicate that just does like and then the pattern against it the the nice thing about this is that it is way faster than using where object right if if you just let powershell handle whatever in the background partial doesn't need to interpret and compile the script block it doesn't need to bind values to dollar underscore stuff like that right so that means that you can actually get a pretty significant performance boost out of relying on on these operators in filtering mode uh when it's simply the intrinsic value of an object and not a property or anything like that that you're that you're trying to compare against so this is this is really good keeping in mind right the first time uh the first time people usually get like this is they start doing things with powershell and we have this concept of commandlets that you know return one or more of anything or zero or more or anything right so if you just do get stuff cat service for example with the display name uh wildcard uh we might get multiple service instances back right and so if i try to discover uh services with with certain with a certain prefix on the machine right and i then take the result of that query and i try to apply um if condition to it for example right like these services have an even longer prefix you know i'm sort of trying to filter here this is still going to work down here i just need to be aware of the fact that service might actually be an array which is something that i might want to check for and so in particular 3.011 we have this synthetic count property and basically every single object to allow us to determine whether the result of um of the pipeline was one or more characters so even though this might not be an array we're actually going to get count one so again worth keeping in mind the nice thing about this is that this just works as expected even though i might be ignorant to the fact that you know concerns might have returned more so again partial trying to be a you know a bit of success runtime sometimes actually interferes with its usual usefulness a little bit uh so again it's worth keeping these these rules in mind another point of confusion is uh testing uh testing for what we might call falsely values right values that when coerced into booleans might actually be interpreted as false rather than true and so one of the points of confusion here is let's let's let's take this this array of names right two bias then an empty string and then rob and so if i expected that names might actually only contain a single name you know i i might do something like oh if if name equals uh you know an empty string then i'm just going to skip the entire operation right like i'm going to return out of my function the problem with with applying this equals empty string to my string array up here is that this expression is going to result in this empty string because it matches the criteria being an empty string right so then if we put just going to put some quotes around so we can actually sort of like see that it's an empty string we're talking about here and sure enough we can anti-string back right so the if condition [Music] now needs to evaluate this empty string and as i said before i think an empty value on an empty string value to uh to a boolean is going to result in faults right so that means that yeah the names array actually contained um an empty string right but we're never going to hit this branch because the fact that there was one out of multiple empty strings uh inside the names array now mean that the the uh the if conditional value is too false and so a safer way of doing this is flipping the entire logic around and doing uh less filter for names that are not empty strings that means we know that we at least have some values right that are non-empty right the other way of doing this obviously is to use the contains operator instead because contains uh contains this output operator so you might do something like let's get zero uh spring sure enough [Music] so in powershell seven um and previous versions of powerful core this is no longer true uh there's a there's a bit more going on than what i'm going to describe in the following sort of behind the scenes and sold object and over the last couple of versions we've also gotten a number of new features and and fonts optimizations to sort objects that gives us things like um sort of pagination right we can we can pick the top 10 out of out of the collection of uh of 100 items according to some sorting criteria and for that we sort of use a specialized um heap sort in in sword object but going back to uh pre-partial core windows powershell and sort of the the default fallback behavior of sort object basically what happens is that once we've collected the input and created our ordering matrix we're basically just going to pass it off to the array sort method the array sort method is optimized for sort of general purpose um ordering uh off arrays and so the way this is optimized uh is using a hybrid algorithm called introsort and it's called introsort because it's sort of dynamic it can sort of look at its own state and optimize based on that and so introsort basically starts by taking the input and performing a um a number of passes with the quick sort algorithm the quickstart algorithm uh is is fairly simple it's a classic divide and conquer algorithm and what that means is that uh the algorithm is going to pick based on an average uh opposing point uh that is sort of an expected median uh of the values in the array it's going to use that to then partition the input collection um and what we're going to see here with this animation restart is we have a bunch of unordered data right and so we find sort of a medium pivot point and then we just split the collection into two partitions we then divide those split those into smaller partitions and then sort those individually up until the point where you have sorted subsequences each partition is sort and stuff right so this is great for sort of small and medium sized sorting but in order to not get the worst per uh the worst case performance out of this quicksort implementation what eraser does is that all of this partitioning going on right here in the animation you know dividing and conquering we're only going to allow that to take place up until a certain depth at which point we're going to say okay enough if we continue sort of recursively devolving this partitioning of the input we're going to sort of escalate the the performance characteristics here so at some maximum depth we're going to say enough for the remaining partitions uh that we have here uh we're going to have a look at the partition size itself so if the position uh the partitions that we cut out if the first pass of quicksort has a size of less than 16 we're going to switch over to insertion sort instruction sort is extremely simple it's also pretty slow but as long as you only need to sort small arrays that you can basically fit within a memory cache line insertion sort is going to be pretty fast simply because of its straightforwardness and so here again an animation from wikimedia insertion sort pretty simple we just create sort of a and built a larger and larger sequence of ascendingly ordered values and at some point we will have rearranged the entire thing if however the the size of the petitions that we have left after performing the initial possible quick sort are more than 16 items large we can no longer rely on insertion sort [Music] being memory optimized by by the runtime and so what we're going to do instead is we're going to switch over to something called heapsort and heapsort basically builds an in-memory tree in which the um in which the uh chug node of of of each node in this tree is either smaller than or larger than its parent node and so this has the really nice property that after doing an initial pass as we're seeing here now we have sort of our tree structure our heap structure we can start ordering from either maximum or minimum and so in the case where we want to get the top 10 out of 100 for example with a custom heap sort implementation we can actually exit early right once we have the top 10 items in that list we don't need to worry about the order of everything else if the user doesn't care about it and so um in partial 6.1 some some changes were made so that instead of calling a rated sword we actually do our own heap sort internally so that we can um so that we can return a stable subset uh of of the items that that are being requested by the user okay now that we've talked about sorting algorithms and you all know that you know sort of does sort of like not the worst case uh performance and and that we've also made a few optimizations to actually make it really nice i think we can sort of put this behind us and we don't need to worry about sorting algorithms anymore so let's go back to sort of the the meat of figuring out how to order this so again going back we still have our input we have our order matrix to which we've applied this descending multiplication and so now we're simply going to unleash a raised sword or our algorithm on the on the other matrix down here swapping the input along with it and we end up with a collection that looks like this and as you can see sure enough the order matrix is now um is now correctly sorted and we get the expected we get the expected output okay so that was that was pretty simple right like we took the intrinsic value of the object we applied you know multiplied the minus one and then we sorted another way using array sort okay what happens when we when we start um when we start supplying multiple properties or multiple values uh on which to which to sort them because solar object does support multi-level sorting and so what what might that look like so here we have a slightly more interesting example right i'm going to ask sword object on the same already sorted input to reorder it first by applying remainder 3 that is the remainder after dividing the input value by three also known as congruence modulo three and then if we get into a situation where uh where two of the values happen to translate to the same result for this expression we're then going to go in and have a look at this second order property which is just going to be the intrinsic value of of the input so again in order to create our input our order matrix we now have to create a two-dimensional array so in our order metrics we now have sort of two properties to solve by right so we apply these property expressions throughout the the entire input collection and we push the result of mapping these property expressions onto the input into our matrix and so down here for example one modulo 3 is going to give you the result one two modulo two is three is going to give you the result two and so on and so on and so now we have some differing values in these two um in these two rows in our order matrix and so we start by sorting the the first level row the first level expression that we want to sort on and so the result is going to look like this right [Music] once again by the value of remainder 3 we've now sorted the entire the entire matrix if we look at the second order property down here we see that within the bounds of each of these groups we're already sorted right because the input value have the input order also happen to be based on the intrinsic sort value of this but what if we do something slightly more advanced so i'm going to expand the expression up here and say actually i want i want you to order this specific property descendingly without affecting the value of the others and so we can do this with with a property expression that has a descending key set to true so if we apply the the same rule as before as soon as we see something ordered descendingly we just apply a multiplication of -1 and then uh next in order to uh in order to then uh keep the order from uh from the first row in our ordering matrix we're going to sort of box these collections right beyond the bounce of any of these boxes we're not going to be able to swap anything because that would mess up the sort order of the first row and so we're going to sort of do it within each of these groups if we flip that around we now get the expected result we get these groups of three based on the remainder of of dividing by three but the internal order of each of these sort of subsets are now descending instead and finally we get our expected output we already went through a couple of examples of how stone object works some of the input options you have for deciding um how to how to rank around to start the input item so i'm i'm not going to go through um the same examples again just briefly listed as we saw before you can type this sort of dick without any parameter it'll try to order the items by by their intrinsic value their comparable value uh we can specify property names right but this is not just the property name this property parameter will also accept property expressions uh or calculate expressions right so we can pass a script lock to it it'll try to calculate the value of of that property expression against each of the input values and sort by that and then finally um we can actually we can actually pass a hash table consent containing both an expression and flags that might invert these soil order right so again in the case here specifically specifically saying that we want descending order for this particular expression rather than for the entire upper array before we move any further i i want to talk a little bit by group object as well the other way of ordering things right when we when we talk about sorting things we're ordering things in real life actually we're we're sometimes really just talking about grouping or categorizing things right when people you know talk about uh environmentally friendly sorting their their trash right they're actually we're actually talking about grouping their trash together um and so what does group update look like well group update has almost the exact same uh some user interface as as sort of object that is we we could take a collection right [Music] to group update without anything else and it's going to group them and competition this input collection based on their intrinsic value right so again executing this we're going to get a result where we can see that we have two instances of the value one two instances of the value two and two instances of value three and then each of these each of these upper elements these up records will also contain a grouping of the exact elements um that were grouped um we can also uh group um by the commonality of some of the sameness of some property expression just like with with sword optic right um so again like the example in the slides we're going to take the um the sequence of integers from one through nine we're going to group them by the remainder again after dividing by three and then to do that we're going to see that sure enough it groups them all together and then the name of each of these um these output rows or or output records is based on the actual value that was uh was calculated right group a group object uh at least since partial 4.0 as far as i remember has also has the s hashtag switch parameter and this is going to change the output format slightly as it indicates it's going to return a hash table rather than a list of of groups and this means that we can then sort of index into the into the output to get easy access to a subset of a collection that again shares some some common property and so again if we execute this sure name value this looks like a hashtag output i'm going to assign this to a variable so i'm going to try to make sensors there we go here again the index key is going to be the um the value of the expression by which they're they're grouped right and so in this case i input integers the key would be the integer value one group grouping also supports um multi-level grouping uh just like startup startup that supports uh having multiple properties instead of doing doing multi-level sorting and for this again you suggest you specify multiple conditions and you can specify as many property names or conditions as you want the only caveat here is that the default output as we saw before each directly in the group object the identifier is going to be this name property and name pro and name is always a string so that means that even if you if if you have properties like this that results in in an integer the name value in each of these records is going to be a string and so this this might be problematic if if you're trying to group uh if you're trying to group non-string or non-simple value types because at some point group optic is going to have to describe them using a string so you might want to you might want to keep after that grouping also has this dash no elements which which can be really useful when you're not really interested in the the values or the identities of the objects in the input collection themselves but you're only interested in sort of their distribution right we want some stats about you know if we split and partition the input based on again the property expressions criteria and how many how many uh how many objects are going to end up in each bin or each partition and so again i'm going to take the output from a get service and my machine is going to retrieve all the all the other registered services on this machine i'm going to group them by the startup type property and i'm going to tell group update but i don't actually care about the input items i'm not going to use them by anything i i just need sort of like statistical output on on on the distribution of startup type configurations so if i do this sure enough you can see here that the um the group property has been cut out i i don't have all the service instances in here anymore but i can clearly see that i have 83 automatically starting services uh 198 manual starting services and eight services that are disabled all right so okay so we've sort of looked at some some vague maybe not to realize examples we've started talking a little bit about the mechanics of uh of specifying which property expressions to sort solder group on but before we we jump to the classes part of this uh i just want to show like one real-life example uh that i encountered recently in which uh solar baked and group object was crucial to sort of pivoting some data around and so the example here is that um we have some input from an external system a software inventory this could be from sccm it could be from your patch management management suite it could be from a bunch of networks and it doesn't really matter right uh the data that we have here tries to inventorize uh installed applications on a number of computers and so for each entry we have a single application installation right we have a single product installed and registered on a specific machine so the first extent here we can see that on computer one uh visual version 1601.929 is installed and the operating system of the computer in question is windows 10. so for someone like me who works in in security and detection engineering uh this data is really interesting this data might also be interesting if you're working in uh in you know procurement or license management if you want to get you know a total sort of view of uh of the uh licensed software that's installed and how it's distributed throughout your network right so the idea here is that we have all these individual records again from something like a software inventory and we want to group them together by sort of distinct configuration right so in the example with the output down here we can sort of stitch together that we have at least two machines with the exact same configuration they're both windows 10 and they have this exact version of visual this exact version of power bi and this exact version of of webex right so this is sort of a distinct operating environment that we've seen at least twice throughout the network uh from a security detection engineering point of view this is interesting if you want to sort of lock down your fleet and make sure that nobody installs anything if all of a sudden you start seeing a small subset of machines and that have you know uh unknown or weird versions of software uh installed you know you might want to react on that uh again if the licenses the license requirements for a product change right and you need to either upgrade or remove a specific specific version this information might also be really useful right so okay so we have sort of this this flat array right this is basically an import csv right so we we have a set of rows with a couple properties and can we in some way take this data and manipulate it and sort of pivot it in a way so that we can end up with these groups of distinct computer configurations and so for that group object and sword object is going to be extremely valuable and so the first thing i'm going to do here is i'm going to take all of these records and i'm going to uh to use group object to group them by their device name right because if i need to figure out the total configuration of a single machine in order to be able to say whether it's the same configuration as another one then at least i need to know every single piece of software installed on that specific machine so if we start by taking this inventory up here [Music] all right so we have our csv converter to convert it to sparkler objects what i'm going to do here is i just say inventory and then group them by a device name now we can see that we had sort of have five distinct buckets here each of them describing every single thing attached to that computer right every single product installed on that specific computer so this is a pretty good start for each of these i'm going to take the the group i'm going to create a new object a new custom object that describes the configuration of this machine and so obviously it's going to have a name right this is just going to be the group name because the device name was what we saw or what we grouped on in the first place then i uh i'm going to assume that the operating system is going to be the same for every single entry because it's the same computer so for this reason i i have a look in into the group of elements that we grouped on i just take the first example and then i just copy the the operating system value from that it's going to be true for for um for all the others right finally the thing that sort of can tell us whether these computers are configured the same or not is whether they have the exact same set of uh software uh not just the product names themselves but the versions themselves as well and so to do this what i'm going to do is i'm going to create this software property which is going to be an array of strings describing this product and the product right so if i do this i get a couple of you know nice nice objects right here but in order to be able to again group them once more based on their software configuration i need to make absolutely sure that when i'm comparing these two rays that they're sorted because if these two arrays are sorted it happens to be that by accident they actually are right now but if these two were not sorted right if this group up here said webex power bi and then visio if i try to compare these two as strings in order to get the appropriate um identifier for the group group object i'm going to get to a point where if we if we look at these as sets of software right they're going to be the same but just because this list wasn't sorted correctly i'm not going to get the correct result so internally this array that we're going to use next we're going to sort that as well and so by accident we sort of get almost the exact the exact same result down here and then i'm going to take all of this these groups right and then i'm going to push them through a group object one more time and then i'm going to uh to group on these two expressions right here the name of the operating system right for two computers to be able to have the same configuration they at the very least must have the same os and then since our list of software has now been sorted calling dash join will always produce the same string granted that the two arrays the two software properties contain the same sequence finally we leverage the dash no element uh parameter switch to say you know what we don't actually care about this intermediary computer representation because again we only want stats for sort of this specific computer configuration again because we're grouping on these two properties these are going to be concatenated together and will will become the identifier for each group in the output and then finally we simply sort based on the the count of each of the upper property let's let's try to do this um without sorting it first so again i have my individual device configurations here with their software installation started internally like to make these intermediate objects type into group objects and as you can see here the two computers that at the top here that have the same operating system and the same list of software have now been grouped into a single one so that i can see that i i actually only have four distinct configurations across this this network of five computers finally as i said before in the decider output we'd like to group these again if you're doing something like thread hunting or [Music] or detection engineering being able to do this sort of stack ranking is it can be really useful so we pipe the the output from the final call to group object to sort object we ask it to sort on the count descendingly so we get the largest values at the top and then finally we uh we pipe this up with the format table and we should get something that at least looks somewhat like like what we're hoping for and sure enough we get this table down here that has the operating system name a sorted list of the versioned products that are installed on this distinct type of configuration and how many machines match this this fingerprint so again sort of nested uh nested use of group object and sort of really useful for taking flat or or sort of minimally structured data and pivoting it into into um into something that we can then reason about and do something useful with finally i want to touch on one thing that sort of trips a lot of people up what we've been looking at so far for our input here is uh arrays right we have we had an array of of custom objects from the csv here um we had an array of integers uh we had an array of strings in the previous excel with sort object so these are sort of like simple simple flat arrays right simple collections but what if we have something like a hash table or a dictionary and we would like to ensure that they get sorted you know according to some criteria that we can calculate over each of the key value pairs so as you might know hash tables are sort of infamous for being unordered right or or not having deterministic order so if we do something like this hash table and then [Music] ask partial to show the output you can see it's all uh it's all sort of jumbled in here right it's no longer in the order we inserted them it's no longer it's also not in alphabetical order so what's going on here the the problem with this is that if we take a question and we type it to [Music] this let's pass it like this we try and just type that through sort of like that's not going to do anything because a hash table is is is not an array and so sorry is go just going to see oh you have an array of one hash table i don't need to do anything right there's nothing to compare against so what some people will do is they will they will take the table and they will say ah let me turn this into something something that and sure enough the output we get here sort of looks like this is this is correctly sorted dictionary right a correctly sorted um hash table but powershell is lying to you what's really happening here is that partial has just formatted an array of individual key value attributes that are no longer part of any hash table right this is just a copy of the individual items that we got from the enumerator so the output of this is no longer a hash table and this can this could bite you right so if you try to do something like this and like overwrite the existing hash table you're going to have problems because hashtable is no longer a hash table it's an array of key value images so how do we get around that well the first thing that comes to mind is that okay we we make a copy right we we assume we can't do anything about the internal ordering of these keys so let's make a new a new hash table called this copy and then using the uh using data numerator we're going to enumerate the key value pairs in the half chip itself we're going to sort them by key as we saw before this works just fine and then for each key value pair up we've sorted them by their key value we're going to copy them into our new our new hash table say this and then we look at the value of copy [Music] damn it it's all unsorted again well the reason is that copying a hash table to a new hash table is still going to be a hash table and as i said before hash tables are sort of infamous for being completely unordered so what can we do instead well one thing we can do is we can create an ordered dictionary if we prefix the uh hashtable literal with this this magical type name ordered partial is going to create a specialized dictionary that maintains insertion order so what does that mean it means that the order of the keys in this in this new dictionary is going to be determined by when they were inserted right so if i start by inserting a equals one a is always going to be the first key of of this collection up here so if we try that instead they're a new copy of the table as ordered dictionary and then again use the exact same um technique enumerating the p value entries in the hash table sorting by the key and then finally copying it actually retains that in certain order and so again if you find yourself in a situation where you want to sort the dictionary you need to retain the dictionary functionality you need to still be able to index into it or whatever else you might want to do but you want it to be sorted well this is how you do it create another dictionary enumerate key value pairs of the existing unordered hash table sort them and then re-insert the order dictionary will retain an order by which you you insert them so now that we so know how to how to operate the solder object uh commandlet and the grouping commandments now that we have an idea of how to use them the big question becomes this is all fine for sort of intrinsic uh or building data types like strings or or integers or whatever you have um but what if we were to define some custom classes some custom data types and powershell and we wanted those to adhere to a specific default sort order when we passed them to something like solar object or array.sort so the answer to this question i think i already gave previously when we were talking about three-way comparisons uh in order to sort of implement a a common way of performing these three-way comparisons we have this interface called system.i comparable and the system.i comparable interface requires only the implementation of the single method compared to and so this compared to is this three-way comparison mechanism that we need to implement which when the current instance is less than or precedes uh the the argument that we're comparing against in the sort order we simply spit out a value that's less than zero usually minus one if the two objects can can be considered equal in the sort of varieties that is neither is larger or greater than the other we just emit zero to indicate that you know we consider them the same in terms of of ordering and then finally in any case where not none of those are true we simply output a number larger than zero to indicate that the current object is um is larger than or follows in the sold order relative to the argument that we're comparing this so how do we implement i comparables and well we have this we have this method secondary here uh fairly simple to to convert to powershell right we don't have access modifiers and partial um literals have square brackets around them and finally all of our parameters need to be preceded by um by sigil so this is basically the um definition uh in a partial class so if we try to implement this in a partial class i have a really exciting example here i have this new text type called some class i'm specifically saying i want to implement this i comparable interface where this clash really should implement this i comparable interface and so i've added a single single property to this class this is just going to be sort of a simple wrapper class and so what we're going to do first is i'm just going to hide all of this so first we're going to define this let's type this class without attempting to implement i comparable just let's see it doesn't it doesn't explicitly implement it so if i create a new instance sure enough looks right it has a value property okay what i'm going to do now is i'm going to create 10 objects with different values instead of public order so let's take the numbers from 1 to 100 and use that as the basis for our property value so for each object especially some class value and so here i'm going to use get random time to just get um 10 out of these first 100 integers in some random order so now if we look at these we can see we have 10 10 distinct instances of this sometimes x type the um the value of their property are all jumbled right right like this this is not this is not properly ordered right now so if it takes against this and i have that just sort of objects it does not seem to make a difference right powershell does not know how it should apply any sort of sort order to some class right power doesn't know about sometimes we literally just define it right we have to tell partial how it needs to be a so this is where system.i comparal comes in we declare that we want to implement it we implement our compare2 method and then this trick is going to be useful for a lot of sort of simple case types if we just want to sort based on the intrinsic value of some property that already exists on the data type it's as simple as simply devolving to comparing the two values against each other right we get another object we take the value property of the current instance and then we simply just call compared to on that and pass the value property of the other objects back in and so again assuming that you know we can assume that value is an integer on either side and assuming that you know compare two is correctly implemented for integers which we've seen before we can sort numbers right then this call right here should tell us either minus one zero or one based on whether the current one is proceeds follows or is equal to the other one in this order so i write this definition of sum class and we do the exact same thing again we created quite a few instances look sure enough random numbers all all jumbled together not really sorted properly now of a sudden magically these objects get sorted by the value of this property that we that we're comparing against when compared to nothing this is all you need in order to create partial classes that you can sort that are comparable right implement the i comparable interface by implementing this compareto method and then again in the simple case where you just want to wrap the the value of a property already existing on on this data type simply just offload it to that that instance all right now that we know how to implement a comparable with the vague examples from before let's try something that comes a little closer to real life and out in the real world i really enjoy pizza pizza is awesome and pizzas are different right uh one pizza is not necessarily as valuable or as worthy as the other and so i i thought it might be an interesting idea to try to rank some pizzas uh using i comparable and so that's what we're going to do now i have this class definition here for a pizza that again declares an implementation of the icomparable interface the pizza itself has only two properties the size the diameter of the pizza and a list of toppings a lot of strings and just like in the other examples we need to implement the comparative method returns an integer takes a an object and [Music] and we're then going to try to compare the two just like before type checking first off if the other value is null or no like then it should precede anything else in this whole one the next thing we're going to do is we're going to compare the pieces by size obviously more pizza better you know so the first thing we're going to do is we're going to do the comparison between the two size properties again we're just comparing an integer to an integer here but instead of returning the result of that and then sort of letting that be a proxy for the comparability of a pizza we're going to save the result of this because now we have some decisions to make namely if the size comparison resulted in a non zero value that is either minus one or plus one then it means that the sizes are different right either this pizza is bigger than the other one or it's smaller than the other one if that's the case then we'll just return that right and the very first step we're simply just going to sort by by size but now in the case where size comparison might have actually returned zero indicating that the size of the two pieces are the same what are we going to do then well what if what i've done down here is i don't know how you feel about pineapple on pizza but there should be less of it very simple so we're going to count basically how many occurrences of pineapple there is in the toppings list on each pizza using the dash like operator again in filtering mode and then we're simply going to just count the instances of pineapple that we found in the toppings list we're going to do that for the current instance the the current pizza and we're going to do it for the other one that we're comparing against and finally down here as i said i want i want less pineapple not more so again we need to multiply by -1 to make sure that the last thing in the sort order is the thing with diffused pineapples one might also be tempted to do something like just flipping the entire thing around so saying that the [Music] number of pineapples on the other pizza [Music] compared on this pizza you could do this and it would have the exact same effect i tend to not do that because it becomes sort of confusing it's way easier to see i can flip it back it's way easier to see that i deliberately intended to reverse the order of this entire thing otherwise i wouldn't have put a minus one in front of you all right so let's try this out i'm going to model this now there are type has been defined let's check creating a pizza let's do a 16 spur pizza with some cheese and some tomato very simple so this is going to be pizza number one and then we're going to create another 16 inch pizza but this time we're going to put a bunch of it now that we've implemented uh i comparable you'll all of a sudden find that we can actually qualify the relationship between these two pizzas using the less than and uh greater than operators so already now we sort of established that there's a there's a there's a sortable relationship between these pieces right sure enough if you do q1 p2 the lowest value is valued pizza the one with pineapple on it is going to be at the top here and again if we want to show that um are anything else it's the size that matters i'm going to recreate these these two pieces it's going to be a small one so now if we do again all of a sudden the one with cheese and tomato is on top because it is now a smaller pizza the size of it is less than the otherwise hoard pizza one more thing i just want to show here is that if you have a property of an integral type that is an integer or byte or a short or or long what you can do is you can actually supplant that with an enum value and your comparisons are going to work just fine what do we mean by that well in this case right here where our constructor again takes the size into pizza as an integer right integers could be negative so we could come into a situation where someone you know packs our our pizza ordering systems and and they order pizza which is minus five inches in diameter and i don't know top with you know shoelaces and you know we're not going to be able to make negative times repeats and i don't think we have any shoelaces that are fit for human consumption around so this is sort of this is pretty bad right what you can do in this situation instead is that if you have a sort of a preset set of of sizes um obviously you can apply the uh the range realization attribute here so we could say validate range between 12 and uh 20 uh inches for example but in a case where you have sort of a um sort of options uh you might want to use them so we could do something like um piece size here and so i could say that we only do a small which has which corresponds to a pizza with a diameter of 10 inches we have a medium pizza which is 14. a large pizza 18 inches and then finally we have the xxl or from this is called a family pizza 24 inches right there now that we have this enum type and again given that the underlying value of all these xenon values are integers we can sort of use them in the exact same way that is if integer is i comparable then you bet that an enum where the base time is an integer is also i comparable and so now we can sort of just plop in our new enum type here now people will be prevented from you know we don't need the metadata range attribute anymore because we only have these four valid values right and now nobody can come in and do you know [Music] pizza new you know negative diameter or a thousand inch piece or whatever because none of this can be converted to a meaningful uh piece of size uh value so this is going to fail uh way before we even we even reach the constructor rate so this is pretty nice but as i said the sort of the real nice thing here is that the email itself also implements compared to so this is still going to work just the same uh this size even though it's no longer an integer it's not an enum type where the base type integer still implements compare two and we can still um move on with the same thing so if you find yourself in a situation where you're using enums to sort of use preset preset data type values um this is not a problem you can you can you can use their i comparable interface as well so we've had a look at how to implement prince exploding we have you write default sort orders for our own date types with the pizza class that we just looked at before um but sometimes you might be in a situation where you need to sort something and you might need to sort it according to multiple sets of rules and it's not necessarily it does not necessarily make sense to sort of say that this is the default sort order right you might have a very specific use case in mind where you need to sort some objects of some type you might not even have access to the the source code the the the type definition right and so how do you handle that well it turns out that um there's also sort of an extrinsic facility for doing these three-way comparisons and so for this we have the generic i compare interface and so i compare looks a lot like i comparable the only difference being that instead of comparing the current instance to one document that's being passed in we simply just need this this external comparer to evaluate two arguments that are being passed in otherwise the rules are the exact same if x is uh is less than or or proceeds y in the in the sort order again we return we return less than zero and so on and so forth so let's try this so down here i i have again super simple data type a shoe a shoe has a size right maybe we could also give them a make in a color it doesn't really matter right what we're interested in in here is how can we sort on some on some properties of this shoe and let's say that i'm not interested in implementing i comparable for the shoe type or maybe i didn't even author this this shoe type right maybe this came from an external assembly and so if i want to do something like let's say create a generic list of shoes i create a list of shoes i'm going to have some shoes we're going to have the size point sushi really big size 52 and then we're going to have a cute little [Music] size 38 so now i have this list of shoes here and as we expected so it retains insertion order right so the order in which i insert these this result is still the uh the order they were seen in the list which is exactly what we what we would expect just like uh just like arrays can be sorted in place stocking lists and for lists it's pretty simple you take the instance of the list and then you simply call the sort method on that so if we do that now ah something happened method invocation exception exception calling start with similar arguments failed to compare two elements in the array so because we did not care to implement i comparable and there's no there's no other way for um for list but sought to infer the rules for comparing you know one shoe to another obviously sword can't succeed so how do we how do we solve that well if you have a look at the method signature here the the very first or the second overload down here takes one of these i compare overshoe into places so let's try to implement one of those so i'm going to say the uh we're going to do the reverse shoe compare that is we're going to rank the shoes um from always the largest size to the smallest size instead of the other one as i said the interface here is almost the exact same as i comparable but now the method name is comparison step y just like the example before since we're um since we're uh comparing on sort of an uh an inherent comparable property value in here the size which it's easy enough just to return x shoe size compared to size of y and then again as i said before we could have flipped this around i really like to just explicitly change the sign of uh of the thing and then finally just need to qualify the second one here there we go so now i'm going to i already have my shoot type defined and so now i'm going to define this compare that operates on shoes so and run this code so now we need to pass an instance of this to the list spot method so let's create one here to create a new instance of that and now when we attempt to sort our shoes with this external preparer provided it can indeed sort itself from the largest shoe size down to the smallest and so again you find yourself in a situation where either does not make sense to create a default sort order it usually does if you find yourself in a situation where that does not make any sense or the data type that you kind of extend the data type that you're operating on then this is a this is a nice way to go about this if you find yourself in a situation where you want to sort of cut lash copy the behavior of a partial default comparison so things like ignoring string ignoring string casing uh having um having sort of value conversions between different types you can also do that so again we saw before how eq is overloaded right we could actually expose that through a compare so we could say something like class you know ps compare and implement certifications generic i compare and so here let's say that we're doing string comparisons then what we can do in our comparison section is let's say string x says string y and then here we call up to system the management for automation the language primitives we do the compare which is the sort of the default partial implementation and partial partially then going to take care of comparing whatever this compare is going to encounter up in the real world obviously [Music] and there you have it basically a a partial complaint or a partial behaving compare that you can then pass off to either methods or disk types that take the take three-way comparison for the purposes of soldering or ordering so in summary you can achieve a lot with just sword object and group object alone right um play around with it uh and especially this this idea of you know supply script script block uh supply property expression uh play around with the with sort of how far you can take the input uh in terms of how you want to order that being said if you are sort of taking advantage of the ability to define classes at one time in partial i would strongly suggest that you look at implementing i comparable um for the reasons that we've seen before obviously if it doesn't make sense to say that you know you know are these sortable are these orderable obviously not if you find yourself in a situation where you think i i need a default sort order for this kind of thing i comparable is definitely what you want to buy them too if on the other hand you find yourself using generic collections uh lists dictionaries a couple of them are also specialized right we could we could create a sorted dictionary or or a sorted set in which um in which we would also need uh need to either implement uncomparable or um or pass in an i compare implementation for the target type that we're trying to sort or score in those collections so again using classes or extensively using generic case types you definitely want to dig into i compare one and i compare ice and demo code will be up online as i said at the beginning my name is matthias also known as is reset me and thank you very much for watching

Channel: PowerShell Conference EU

Views: 841

Rating: 5 out of 5

Keywords: PowerShell, Core, psconf.eu, psconfeu, keynote, Jeffrey, Snover

Id: zggWL-0gefo

Length: 80min 10sec (4810 seconds)

Published: Sun May 31 2020