PowerShell "Shell of an Idea"

hello friends this is don um glad to be speaking to you for a little bit i you know wish we could do it in person or at least live but between schedules and time zones and death viruses uh here we are but thanks for thanks for taking the time so i'm going to do kind of two parts here um the first i'm going to tell some of my favorite stories from sheldon and i idea the untold history of powershell um just about four or five kind of cool things i think people don't know about the show really kind of give you some insight into where it came from and why it works the way it does and we'll conclude with just a little bit of career stuff because i'm actually working on a new soft skills book for manning and it's a good fit it's top of mind for me um keep in mind you can always connect with me at concentrated dawn on twitter and would certainly love to hear from you so this is this is my agenda i'm not going to do slides because i hate slides so let's just jump in um i think the first thing that a lot of people don't know about powershell is ampsi amsi and this is something that lee holmes who's a member of the team back in the day from the beginning really a security kind of you know oriented fellow came up with and it stands for anti-malware scanning interface and i want to get into a little bit of history why this thing exists so you know back in the day we all know that bb script was a thing and it was a problem thing in some cases a lot of people used iterate viruses and the powershell team knew that that eventually powershell would be used that way too and so they looked at how the anti-malware vendors had jumped onto vbscript and what they wound up doing so if you're an anti-malware vendor you would you would hinge on the vbs file name extension or something like that and when you saw that file you'd open it up and they would try to parse the code and they would actually write their own vbscript engine to execute the code and see if it did something that looked bad or they would have signatures for bad scripts well really really quickly the the script kitties got wind of that and they started doing weird encoding tricks with vbscript so that you would open the script and it's just gobbledygook what puts the vendors in a bad situation like you can't you can't just run the script and see what it does right because it that would be bad but if they're writing their own vb script engine like now they have to maintain parity with this thing microsoft is producing and it was just ugly it was slow i mean i'm sure you've seen really really poor performance from malware back in our anti-malware scanners back in the day now lee wanted to do something really really robust but they had already been through this whole you know monopolistic anti-trust situation so he knew that whatever they did they needed to open it up to everybody so everybody who was who's doing anti-malware software could play along and that meant they couldn't just put something in windows defender they had to put something into windows that exposed itself as an api and defender could use that but so could anybody else right that way they weren't you know getting into monopolistic things again so what he came up with eventually was known as the anti-malware scanning interface and it's actually so when an anti-malware sees a powershell script right maybe it's hinging on the ps1 filename extension it actually uses this interface to call powershell and say hey do a first run parse of this for me don't run anything but if there's encoded stuff or any tricky stuff expand it out and give us the the raw expanded version of this and then the malware anti-malware software can look at that and say okay this matches the signature for a bad one or this is an okay one or something like that so we had to work really hard to get the windows team to buy off on that but it's in use today like everyone uses that today and the cool thing is that if the powershell team adds some new character or new command or a new way of encoding information in the script it's all taken care of the and the anti-malware software automatically gets to take advantage of that because it's actually power shell expanding the script and then handing it back to the the anti-malware software without actually running the script so i just think that's one of the coolest things in powershell that almost nobody knows is there and it does so much to improve performance of malware scans it does so much to ensure that your anti-malware software can notice can can identify bad scripts um there's a really cool little bit of architecture in there so the next one i want to talk about and what if i have a marker here well i'll just use my pen because i i need a white board but i don't have one and i don't know how to use the one in zoom here powershell was originally designed to have multiple pipelines now you probably already know that it kind of does have multiple pipelines right there's the success pipeline verbose warning error debug and information so the current versions of powershell you got six pipelines but that's not what i mean by multiple pipelines originally they had intended you to be able to spin up your own pipelines and even make them recursive so if you can think about it you might have a pipeline running some commands and passing data down the line right and at some point you might that i really do need a marker hang on okay this is the tricky bit about working in the home office let's see if any of these work you want to see it this way all right so you've written this pipe one yeah that's better and you've got a couple commands and they're passing data to one another and then imagine imagine a command like where object right today that simply decides if objects continue down the pipeline or not but what if it was able to say you know what some objects are going to go this way and other objects are going to do that way and we're going to have a bifurcated pipeline right and maybe this one's going to continue straight on to the end because we like those objects or whatever and this one can continue straight on to the end but we're going to put a loop in there meaning i'm going to put objects in there and if i don't like them i'm going to do some processing with them and i keep doing that until they meet some criteria and then they're going to get to the end and then we'll re-merge the pipeline so if you can really think about the the execution models like that's just a really really simple one you could have the idea was you could have as many of these happening in parallel as you wanted each one taking different steps and different processes massaging data doing whatever so this is one of snover's original architectural ideas and it's a great idea the problem is word did not get around to all of the developers and the developer in particular who was working on the the main pipeline execution model was thought he was just writing a command line utility and he wasn't thinking about all these different models and snovercano was off doing some other architectural stuff and didn't pay attention and so by the time everyone looked back and saw that what they had was pretty much hard bound to be a command line utility and this whole idea of multiple pipelines they were so far down the path it would have been too expensive to go back and start over they didn't have the time or the money and so here we are so it's a great idea that kind of never happened right so what's my next good story so a little bit about the core history i'm not sure everybody really knows why powershell came to exist um there's a little bit of sneakiness almost mutiny and definitely some hijacking so it's kind of an exciting story so we got to go back to oh around the late late 90s i think mid to late 90s intel at the time was designing all of their processors so their their processor architecture on sun microsystems spark stations unix machines running solaris which was sun's variant of unix and i mean this wasn't like a great look for intel right like spark stations used risk-based processors they were not intel processors so intel was using the competition to design intel's core business uh and they hated it and they said you know what we we want to fix this and so intel ceo got together with bill gates and they said look we need some help we think our hardware can do this but we don't have an operating system that can do it and so bill said yeah you know what we need to fix that windows can be that operating system and they put some money and some people onto the problem and this kinda was the beginning of powershell because what they realized is one of the big workloads that intel was using was a bunch of k shell so ksh uh corn shell scripts they relied heavily on these and so they said well you know how can we fix this and they grabbed a bunch of people from the services for unix team and they supplemented the team and they kind of put it all together and they came up with something called kermit not the muppet there's actually a children's book called kermit the hermit so it's about a kermit hermit crab named kermit so k kermit shell get it k shell corn shell anyway it turns out intel was sort of lying um they they had an idea they were going to use a linux distribution and they were already in the midst of customizing it but they kind of wanted microsoft as a backup plan and they kind of wanted to just push bill gates buttons a little bit and so it in the end nothing happened but what they were originally going to do what was going to be powershell what was going to give us all this amazing command line administration ability was just going to be a port of cornshell that's all we were we were going to get unix on windows i mean ironically you know all these years later we have that you can run bash on windows now um so you know all good things come around i guess but what happened next was jeffrey snover was was had been poking around and he had been hired to help microsoft get over this command line administration problem um another reason they knew they had a problem was when they bought hotmail and hotmail was a completely linux based you know web mail system uh and microsoft wanted to port it to windows and they had a white paper written up and that got leaked to the public it wasn't supposed to it's supposed to be internal only and it was i would say brutal but it was very frankly honest about windows and its lack of of scale because you had to push buttons to do everything so the company knew it had a problem and its other enterprise customers had been telling it that so they hired jeffrey and the first thing they they gave jeffrey was a bunch of money and they said you don't really get any head count but you know go start cracking away at this command line issue so jeffrey spent a few million dollars hiring contractors to write command line utilities and so and that worked and they got a bunch of great command line utilities out of it um you know some of the ad commands that we we use today still uh ultimately came from that project but he got to the end and said you know this this process of fixing this problem is not going to scale if i if i spend millions of dollars and get like 50 utilities so about that time he started really looking into windows management instrumentation wmi and wmi was interesting in that lots of people at microsoft were investing in wmi they were populating the repository with stuff but no one was using it it was difficult as an administrator to really get into it you could do some stuff with vbscript but it was a little sub-optimal and you had to know a lot about how the repository was architected if you've ever tried to do with my stuff in vbscript you know it's hard so jeffrey decided to write wmic.exe and this was going to be sort of a meta command line utility so for a few tens of thousands of dollars he was able to write this tool that made it easy to get to all this functionality that was already there and you know instead of spending millions to get 50 commands he spent tens of thousands maybe hundreds of thousands to get thousands of command equivalents because now he had unlocked everything that was in the operating system well he kind of caught wind of this this corn shell thing that was going on and realized that that team you know had kind of hit the wall um intel didn't want to fund this any further and it wasn't going to go anywhere and he said you know here's the hardest thing to get at microsoft head count head count and a mission that team's got head count i've got a mission i'm a hijack them and so he did he convinced the program managers who were running the the kermit team that the right thing to do was to go a totally different direction and that by creating sort of a meta shell monad and then using everything else that was already in com and windows and the.net framework and all these other things that had been written but were just hard to get to so powershell would just be a rapper around all this goodness just as wmic is really a rapper around a smaller chunk of goodness well everyone pretty much bought off on the idea and he managed to sell it to them and that's how we wound up getting powershell instead of a corn shell pork from solaris to windows um i think it's one of the best stories to know because it tells you something about powershell the point of powershell is to take a bunch of things that don't look the same and try to make them more the same and if you really start thinking about powershell that way it helps inform the kind of scripts that you're going to write right you don't necessarily need to do everything from scratch look if something is out there and it works it's just a little hard to use or it doesn't fit powershell is a great way to put a wrapper around it to make it look more consistent to make it fit to make it more discoverable to make it more you know easier for people to use and that's really what powershell is powershell did not reinvent anything everything you do with powershell now you could already have done you could have done it in net or you could have done with a command line utility or you could have done it with wmi or you could have done it with com or you could have done it with you know a billion other apis that are in windows powershell didn't give you any new capability it just exposed you to things that were hard for an admin to get to without getting into visual studio and writing some enormous application so powershell brought all that developer stuff all that stuff was already there it just brought it to our audience and it put a consistency layer across it it helped create discoverability so as you write your script the things to invest in consistency with the rest of the shell discoverability help files documentation right that's the value powershell brings the functionality is pretty much already there so focus on investing in those things um you know there's there's another great story about how powershell kind of got to where it is you gotta remember when when you're creating a product like this that's that's brand new and it's never really been done before you don't always know how it's going to land with your audience um and i want to point out windows mobile as an example a lot of people loved windows mobile as users but developers didn't there's a lot of stuff developers didn't like about it and because ios and android were already so huge for developers to invest in another platform they really had to like it i think it wasn't universally loved from a developer's perspective and look what that wound up doing to the product right it might have been a great operating system but with no apps no operating system like nobody wants the operating system they want their apps well the powershell team kind of into that they they knew that they not only had to make powershell appeal to administrators but it had to appeal to developers because if no one was writing commandlets those are our apps right no commandlets no then who cares no no powershell well so what they did is they actually sat down with developers and did what may have been the first usability testing on an application programming interface and they asked a bunch of developers look sit down and write a commandlet and they would watch what they did and they would watch the rabbit holes they went down to and they would watch you know the questions they would ask and they would make changes and they spent a ton i mean powershell version one was years in development and a ton of that time was spent doing usability testing on the commandlet api and so when it launched developers loved it they're like wait wait wait if i have already if i've got code like if i have code to do stuff and it's in.net or com or something else all i have to do is like this this this this and this and boom i've got commandlets and now administrators can use my stuff i'm in that's no work at all that was smart one of the best ideas powershell had was the ps providers right the idea of writing an adapter that could make almost anything look like a hard drive um admins know how to use hard drives it's a great way to explore data that's what the file system is it's a giant hierarchical database and it stores blogs and metadata and everything else and almost anything can be made to look like that if it's a data store of some kind but they didn't do any usability testing with the ps provider api like they did with commandlets and writing a provider is uh it's kind of non-trivial it's hard like there's a lot of things you have to account for as a developer and it's not just hey i've already got a great database i should just be able to put a thin layer and now everyone in powershell can use it and it wasn't like that it wasn't a thin layer providers they're still hard um ken hanson one of the program managers from back then long time program manager says you know i wish we'd had the time in the budget to do more usability testing because i think providers would have been more impactful and unlocked more things for people if we had made them more approachable to developers now you do have projects now like jim christopher's ships project that does make it easier like he does some of the heavy lifting for you but you know that didn't come out till well in and and by then everybody had kind of shifted over to the commandlet model versus the provider model so it's just another interesting thing that you know mistakes are made um sometimes amazing things are done sometimes you know you just don't you don't have the time and the budget to to do it every single time um but it's one of the reasons that even now providers are you know they're just so-so they're a great idea but they're hard and so they don't catch on so i think the last story i want to tell um is about oh you know what let's talk about wear object let me get my whiteboard back out here so i want you to think what hap what would happen if if where object looks something like this um look at that for a second and i don't know if i need to un-mirror let me let me hit my video because i don't know if you're seeing this right so we do my video settings um and if that didn't work then that will boom one of those two so looking at a pipe so we're piping something into this then where and then the parameter minus expression and then in quotation marks some property right this and then your operator and then whatever you're comparing it to so just as a general model i want you to consider that that's what where object was going to look like um i have to remember my video because it freaks me out to look at sorry so i don't know if that affected what you saw in the recording or not but you can let me know um anyway that's what object almost looked like it was quotation marks and as they dug into it and they started implementing that they realized it was really hard like they were running into just a lot of you know it seems easy to parse that but when i look at like oh man you know this could be one of 15 different things and we've got no way to tell it from hey man this is going to be hard to write and jeffree stover credits bruce payet with switching us over to where we are now and it's wear object minus filter script and then you get the curly brackets and then you put an expression in there and you can use dollar sign underscore to represent the object that's being piped in and and we're all used to that now we know that now we're all familiar with that syntax it's been around since you know 2006 but the idea of turning that quotation and that very sequel like expression language and going instead with curly brackets which represent a sub-expression and calling that a script block like that is an executable powershell runs that as a script in a pipeline that it spins up when it sees that command and it feeds in the object information to dollar sign underscore and then we can do all kinds of stuff with it like a lot of us just do it for simple comparisons but you could have an entire thousand line script within those curly brackets that created the concept of a script block and it is pervasive in powershell now you write an advanced function you've got to begin process and end script blocks that's where they came from you've got for each object and it has a minus script block and we use the curly brackets on that that's where that came from the idea of a script block being an atomic little thing that can even be assigned to a variable and that variable then represents that script and you can execute it that was that was a big deal like that's everywhere in powershell now and and it all came about because where object ran into parsing problems with the original syntax they had so again i think it's a fascinating story i think it's so worth understanding these things because it starts to unlock just little bits of of of of synergy like once you kind of know a few these stories you realize how how the shell fits together and how it came to be so uh you know one more plug boom shell of an idea the untold history of powershell it's available on leanpub.com and it's available on amazon hopefully in your region i'm pretty sure it's available in most regions um but if you're an e-book person then boom this is the hardcover by the way um i only made 100 of these these are gone but the paperback looks just like it it's you know pretty sizable hefty book oh and it's uh average i'm really proud of this every chapter just about starts out with a fantastic cartoon there's those fun cartoons as well i want to talk about one more thing before we go we're going to switch topics and i want to talk about your career um as i mentioned i'm writing a book for manning called soft skills and obviously a lot of you know i wrote a book called be the master soft skills is kind of the success part of that be the master has been discontinued at this point um everyone who ever bought a copy on leanpub will receive an email if you're if in fact if you did buy a copy go under your leanpub account and make sure that it is allowed to send you emails when i update the book because that's how i'm going to send you an offer code if you want to get a discount off of the the successor book from manning and you know one of my messages and something i had to really refine down for this book because be the master gets a little long-winded about it and heads a different direction but well i i guess there's a better way to do this when i came up through the industry um i was just like most people i didn't go to college i came out of an apprenticeship program for aircraft mechanic um so i worked on on navy jets but coming out of that is a lot like coming out of college and that i had no idea what to do where to go um i knew i liked computers but i had been told i had been told by my high school guidance counselor that i didn't do math well enough to get into computers so thank you sir for that fabulous life advice anyway i worked in retail for a while for a company called electronics boutique eb games primarily us-based i retailer video games and stuff and eventually i was a store manager and they had an opening at the home office and i uh wanted to take it because it was an opening into their i.t department and basically doing support of their appointed sales and the cash registers i got there and i had an opportunity to learn how to do as400 operations so i took that and we upgraded upgraded loose term to an os2 based point of sale system that sucked i mean it's hot hard and it was such a problem for us i i i bragged to my my cio one day i said you know if we ever snowed in for a few days i'm gonna rewrite this thing in visual basic and we'll have our own point of sale system and we won't have to deal with this crap anymore um and we got snowed in and i rewrote it vbscript and or vb eb4 i ran on windows 95 and that's what the company used variations of that and successors of that until they were purchased by gamestop which just acquired the company and i was kind of always looking for that next thing you know what's my next thing um want to make more money you know maybe better have a better schedule i work night shifts hated night shifts i left eb and almost entirely over the night shift thing they refused to move me on day shift um they still needed that nighttime point of sale operator i'm like i'm writing all your code like i'm i'm getting paid like 23 000 a year for consecutive i wrote this whole thing nope night shift so i left um next company kind of was always looking for the next step up they were a network consulting firm and so every major certification i attained i got more money cool so i did three or four of those got me mcse my cne my i don't know whatever else compact ase blah blah blah um found a position at bella atlantic got a couple steps up there but i was always looking for the next thing and it honestly wasn't until about four or five years ago that i started to feel the pressure of that so i'll be 49 in a few days so i was probably early 40s when it started to light weigh on me like i i feel i've i've done a lot with my career but like when is it going to be enough you know it gets hard the heart like the higher up you go the more successful you are the harder it is to find your next like the success got real small there's not a lot of step ups and it started to really weigh on me i'm like should i should i go to college i actually looked at an online university i was going to drop big money to get an mba um so i mean first i had to get the bachelor's degree right because i didn't go to college um because i felt you know that mbas that's my only way forward like if i'm going to step up and it just it started to really weigh on me and and i think that's where a lot of the genesis would be the master came from is is life in your career don't have to be a rat race it doesn't have to be a constant lookout for for what's next um let me let me tell you the steps here and then i'm going to tell you the most important skill you need to have in your entire career to keep you safe and fed the first is forget your job forget your career sit down and think about your life what do you want from life right now this is going to change over time if you're a young person you you may decide to start family get married do whatever else and and what you want from life is going to change that's fine this isn't a permanent answer but i want you to sit down and really think about it and almost right i don't want to be more of it but almost write your obituary um you know joe joe uh had a family they lived in a nice house they took good vacations write those things down and then start quantifying that what's that house cost you know are you gonna have two kids are you the dad or the mom that always makes it to every single uh football game i almost said soccer sorry rugby game and if you are what kind of time is that going to require away from work what kind of work life balance do you need start quantifying those things i need this much free time i need a job that lets me go at this time every single day i need this much money write those things down those things those quantifiable things those things that you can observe that is your success and you get to define it nobody else and it is not some intangible thing that you have to to endlessly chase after and go up and up and up and up and up you know what it is and once you've got it you can stop you don't have to run the race anymore once you've got what you need all you have to do then is maintain it so think about yourself define your life who are you and what does life look like and that's your goal quantify it that's your success now instead of just chasing the next title or the paycheck or whatever it is you think you might be chasing don't make any move unless it aligns you toward that success like okay yeah you want this is a step in the right direction i recently had a job offer very exciting job but i was looking at the scope of the job and the company that it was for versus what i'm doing now i felt you know i've had some frustrations at work we all do happens not every day is perfect but to do this job does not feel like a step toward my success it feels like a step away from frustration never step away from things never run away from something in your career always run toward something make deliberate thoughtful decisions the gps analogy your success definition those bullet points of quantifiable objective observable criteria that's the destination you punch into your gps and then every turn you make is to get there sometimes it can seem circus that's fine like life's not perfect you take what opportunities you can but but aim yourself toward your success and once you're there you just have to maintain and let me tell you the most important skill in your career because remember your employer owns your job right they have to give you the tools needed to do your job they don't have to give you anything past that they do not own your career your career is what gets you your next job and it's on you to pay for that career to feed it to nurture it to help it grow and to take care of it and the way to do that is to be a lifelong daily learner your most important skill is your ability to quickly learn new things and be comfortable with that be comfortable saying you know what we're we're switching everything over to linux or windows or something else i'm fine with that i don't know anything about it but i mean i learned all that i know now therefore i can learn more and i might look a little stupid while i'm doing it because i'm going to make some mistakes but that's fine i'm willing to learn you have to exercise your brain cognitive science tells us that if your brain is used to learning it will do it more easily it's just like going to the gym right if you're used to picking up heavy stuff picking up heavy stuff is easier if you're used to running you know 80 kilometers then it's easier to do that if you're used to learning anything then it's easier to learn i make a habit of reading the i go to wikipedia every single day and i hit the random article link and i'm not really care much about the factual accuracy of what i'm reading i just want to learn something new i want my brain to have to parse new information i make a point i work for pluralsight so i'm very fortunate that i have free access to the entire library i watch at least 30 minutes to an hour three or four times a week at least because that forces my brain to learn and i'm not worried about whether the topic i'm watching is is on point for my career path i watch software dove stuff i watch machine intelligence stuff i tend to watch like introductory stuff because it's high level and it explains to me what i'm looking at and it's how i keep up and it's how i make sure i can always keep up if someone came to me and said you know what we need you to take over the the data curriculum okay like i don't know it but i can because that's that's my most important skill is i can i can learn it i can learn it confidently so define yourself define your success and be a lifelong daily learner make your brain learn it will do it it will do it if you do it every single day it takes it i'm gonna say about a month of that before it starts to feel natural and then maybe another month before it starts to be eager like you want it like you get you're like a crack addict for for just learning a new fact or something you'll become fascinating at cocktail parties if you pick your topics right or if you don't you'll be dreary at cocktail parties but either way learn every day every single day including weekends a random article on wikipedia isn't gonna take up much of your time right you can do that in the loop so i hope that was a little bit of fun i hope there was a little bit of an interesting powershell story in there um you know i i don't work with powershell day by day anymore and honestly i i really only ever taught the entry level stuff so it's always a little intimidating to go to a user group because i'm like well i'm not going to teach them how to do functions because they probably know how to do that already but i am really into these backstories and um you know i i am really really into career management and owning your own career and really being the driver in your own life so again thanks for your time uh i hope you are going into a holiday season that is a little bit better than the dumpster fire that the rest of this year has been i hope you and your family are as well as you can be and please do keep in touch take it easy

Channel: Brisbane Infrastructure DevOps User Group

Views: 84

Rating: 5 out of 5

Keywords: powershell, don jones

Id: EBfLX6ZIDlI

Length: 34min 20sec (2060 seconds)

Published: Mon Nov 23 2020