Oral History of Robert Garner, part 2 of 2

hello I'm Roy Ogas and today is May 17 2019 I'm here again at the Computer History Museum in Mountain View California with Robert Garner and we're about to start the second session of Roberts oral history discussion thanks again Robert for your participation my pleasure in the discussion at the end of the last session you had come to the end of your stint at Sun Microsystems the year was 1998 and you were content contemplating a move from the company before we move on to the next phase of your story I thought I would ask you whether you have any other comments about Sun Microsystems that you'd want to bring up before we move on to your next chapter of your story no thank you right well again thank you very much for doing my role history it's just wonderful I liked the part one part so far - we'll hopefully be just as fun and I will try not to talk as fast and and and try not to interrupt you so one story I would a little tiny story I forgot to tell its son before I move on it was kind of emblematic of the way things happen there's when we were actually just before the Jeannie project started there was really a move to try to produce a really inexpensive computing device a diskless PC like device and there was a strong feeling we could get it down to $1,000 for a selling price or even less perhaps and so bill and I and several people were in a room and we were looking at the bill of materials for how expensive or inexpensive we could make a diskless PC so you know and get all its information over the network from other servers kind of anticipating a lot of what we do today and I I came up with this bill of materials that it was under much under $1,000 so it looked like maybe you could sell something for $1,000 and right in the middle of all this village or he gets a call on his cell phone and he starts going hey we just figured out we can get a bill of materials done for a portable system for much less than $1,000 and he'll put down his phone I said bill who was that and it was it was the head of Oracle Larry Ellison bill you just and then the very next morning I'm driving to work and I turn on KQED and there's Larry Ellison talking about how he thought we could do a network computer for under $1,000 oh that was the fastest transfer you ever heard from idea you know implementation idea to PR but the radio and that and he gave a talk you know that very Friday and he compared the network computer to the versatility of a pencil or something so what was the year that Oracle acquired some oh that was much later much later this is this is in 1997 or 1996 okay that was much later yeah and Larry Ellison had was had a real personal relationship to some sort with with Viscomi Neely and Larry had helped bring Herr Scott worked with Larry to bring some of his database software to son during the early days because it was kind of embarrassing the very earliest part of son I'm sure I told this story before but son was had had it had a manufacturing facility in Milpitas and our volumes were increasing and the rate at which we could ship products was being hindered by something which is rather surprising a friend of mine helped and the man who set up the manufacturing facility there and there was a backup in the system and there were actually boxes of Sons ready to be shipped in the parking lot they couldn't be shipped because the rate at which an entry the part number for that device could be removed from the computer was slower than the build rate so there were actually components accumulating in the parking lot and apparently the software were using was running on IBM mainframes somewhere in Southern California and was my friend looked at the algorithm it was a linear search algorithm through the parts list to delete you know order to add to say one of these components that shipped and that was slower than the time it took the manufacturing facility to build them so son was in dire need they may be running on an alcohol machine now they think about that IBM compatible one ball machine somewhere in Southern California so son was this is very early at son this is in that probably in the nineteen 1988 timeframe somewhere in there so yeah Larry Ellison and that whole thing came later but I but that's a good segue to why I thought it seemed like the time had come that son was being eaten from below by like I said before the intel pentium line had actually slightly surpassed same clock rate 300 megahertz ultrasparc to compared to a pentium to slightly surpassed us in performance and people were be getting more capable building shared multiprocessor memory systems and more capable systems so I I just felt the writing was on the wall that we were gonna get eaten from below by by Intel devices and the GE project it was a lot of fanfare but it was clear that it was a long way from being a practical product Sun was starting to purchase on a regular basis smaller startups and it was kind of disillusioning they would just purchase them and then eat them up and politics was starting to get more intense at sun it become kind of a big company so I was really longing for the opportunity again and to be in a small company where you have much more control of what your future is going to be arcing back to what Eric Schmidt had told me when I joined Sun you know whatever happens here or whatever you do it will become a product you know there's not you don't have time to cancel projects and start whole big projects I mean son had gotten to the size where we were starting either through acquisition or on our own large projects which would be canceled it would struck me is you know all largest companies this tech companies this happens to is it it's pretty inefficient so the startup model I think is much more efficient because you focus on a particular idea or product you fund it sufficiently if you bring it to market and you're tested immediately you're not necessarily just doing a project to keep people on the payroll you know or to keep there to keep their managers happy yeah there's sorters less infighting at a start-up in a company large company like son was at the time if you have project a and project B and they each think they each think the other one is not worth doing anyway you get all this infighting so I wanted to try to also move away from that and go back to a startup one of the popular journals magazines that time trade giant trade magazines at the time was up site magazine and there I read of a startup doing a network storage system using fiber channel and my first and it sounded pretty exciting they had funding they were about a year and a half old they had an initial product they were selling this is before the dot-com era has income things are heating up though in the valley so my first thought was well fiber channel okay that's all sounds good but is IBM doing a fiber channel switch a switch was the main product that brocade was doing companies had fiber channel well let me back up a sec if there's a broken sense fiber channel initially started as a connection between high performance discs you could string the discs together on one serial cable and you know somebody would get on the bus and they would take it talk and someone else could get on the bus but it was serial so if you had 127 discs all hundred or 127 discs would be on one Christmas tree serial loop so the innovation that brocade had done is let's give a port to each disc and let's make it cheap enough have enough ports so that we can you know perhaps some of those ports would be connected to serial loops maybe you would have a serial loop so you know give some switched technology to improve your latency and response time and which is great but if if if the big BM of IBM was going to do a switch the days would be numbered for brocade or the other company for that matter so this was this reminding me a little bit about the risk opportunity remember back in the early part of our risk work at Sun we be worried whether IBM would do the right thing and come out with a great 8:01 derivative in the workstation market and they didn't so here we are again I'm going well okay here's a startup is IBM going to Tromp over everybody and I so I poked around asked people questions and it appeared as if there was this huge battle inside IBM for on the one side there was fiber channel on the other side was their internal architecture so did you actually talk to people at IBM as well well I probably can't say that I certainly talked around quite inside information I tried to get inside information of what IBM was doing and it was seem to be not totally secret so because there were a lot of discussions were happening at public conrick standards meetings so I said okay let's go for it and in some ways that so the firm I decided the join was brocade communications in the mists the summer of 98 after 14 years at Sun and I came on board as the director of hardware engineering reporting to Paul bond derson who was the first employee at brocade and Paul had been the director of the i/o group at Sun Microsystems probably not the most well known group its son because you know son was focused on the cpu and it's sunny west Solaris software but but Paul had done an outstanding job leading the io group at Sun and in fact that's where the ideas for fiber channel came from you know it's son had parallel scuzzy ports on it's workstation where you have many lines in parallel just to connect to one disk drive so clearly you needed to make that transition Ethernet made it first to a serial line you needed to make that transition from all these parallel bits to a serial line to the disks and so the fibre channel work had started before brocade was founded Kumar was one of the architects and the other place where the serial bus transition happened was two memory gems that was so all these all the main interfaces and computers were Trent were high throughput was important we're transitioning to serial lines high speed sewer lines it was of course not as high we don't mean I see it as high today but it was one megabit per second and three megabit per second at that point so you knew Paul Banda Singh from before did you know anyone else had broken well what sort of thing I didn't know Paul that well because he was kind of over in different buildings across the bay but he knew about me and he certainly could get advice from right from something I mean the reason I said was I was wondering how much of it your decision to go to brocade was based on people you knew it was not it was not based on it oh no it was more based on I thought they had a great idea IBM wasn't doing it and it was really going to fly yeah I that turned out to be true it turned out that brocade entirely dominated the fibre channel switch market they became essentially the sole supplier to IC supplier to the entire high-end computing market so it just exploded no they did have to the Italian architects just like we get talented architects at Sun I can count on one hand the number of really talented architects who really understand it architecture other egos get too much in the way and and work with designer team so Dave banks on the hardware side and Craig Martin extremely talented architects and it really talented architect on the software side when I first arrived at brocade there was a design review for a tape out of there very high-end ASIC the next key switch chip and I sat through the design review and I asked a question I said well ok over here we have some asynchronous signals coming in and where is the synchronizer and it's like oops they have forgotten they needed a synchronizer and you know Dave there's more issues you have to deal with in a ship of that complexity with you know half a million gates and so Paul liked the joke that I already paid for my salary the first year because the cost of spinning an ASIC is extremely high that's a good thing for your yeah he wasn't your immediate manager he was he was call bond erson was my mean yeah so I think for him to feel yeah he was VP of engineering and I was director of hardware engineering I mean at that time the hardware team he was truly a scrappy start up the hardware team was only four people and over time I grew that to about oh I'm sorry seven engineers and over time I grew that to about 30 so maybe they were three four people on the asic and three people on the board it was a pretty tiny team incredible team how big was the company as a whole at that point oh when I joined it was pretty small I was like employee 130 something's pretty one a couple hundred people and again they were doing everything they're even doing their own manufacturing of the switch that but obviously they and everyone else moved away from that pretty quickly but so I had responsibility for all the hardware aspects the engineering design team verification you know the development compliance test and then field support for the one and two gigabit I'm sorry not not one in three and the products recalled the fiber channel silkworm 2000-3000 family by the time I was there you told me if customers had hardware problems they'd come to you and yes yes yes the support group as well yeah yeah I hadn't you had to do everything at a startup I guess the bloom ship was ten million transistors an eight port switch so that was what was fully defined and working by the time I joined and then and that silicon did work the first time after it came back so yeah I had it directly interface with customers and and manufacturing problems when we started manufacturing the silkworm 2000 they were getting random test fields and many failures in manufacturing and it took a while to figure out but it would depend on whether a certain other items were plugged it or not power supplies and there's this bus called the I squared C bus which is used a lot in and systems and it's a serial bus that links together chips for which you can run serial JTAG as it's also called through to read out the state of registers and stuff in ships and it's a poorly designed bus it was designed a long time ago it's just a single wire essentially and it's brant multi branch to not terminate it well so what I what we learned what I discovered is there were errant reflections on that bus so depending on what was plugged in and and what was worse is that the problem got worse when the carrot the pieces the characteristic impedance of the printed circuit board changed so the problem came upon us when also do we were getting printed circuit boards for another supplier so I had to and here supposed here's the here's a switch which is at the heart of your storage subsystem has to it has to work yeah yeah it can't not deliver bits from the disks to the processors right so we had to communicate with customers and tell them that you know we we found the problem we understand when it comes up you know the board's you have you know don't have this low characteristic impedance so you shouldn't have a problem and I think you know one thing that Paul taught me and one thing that I learned how important it was to really tell the truth to customers you know don't try to hide behind you know some excuse you know just be honest and tell the truth and and the customers respect you a lot more for that even though there's a problem they know that there's a future problem that'll be truthful and honest and that was a bit stressful but that was a great learning experience so so what were the type of customers that brocade had every major server company and the server company so all of them okay IBM HP Sun all the you know Dell compact all the major server companies were using brocade switches for storage storage area networks the application was for the storage subsystem yes service that yeah they were call is called the storage area network now instead of just having we're called jbods just a bunch of disks with the disks all linked up serially with fibre channel it was more of a network I mean you could have sub loops but people quickly learned that eisah the one of the problems with the j-bot scheme it really broke the whole idea is like Christmas lights string a Christmas lights if one is out and it screws up the whole net so if there's one ace akin one disc which is not playing by the rules you know all 127 oh there just go down the two so people like NetApp you know had discovered this problem early on and so we offered you know our price point was you know it was good this good price point it'd be nice as a little cheaper but it definitely addressed that reliability issue so highly highly used by its another role that you said you handled as Hardware director was the EMC testing kind of a Blackhawk area would often is done by specialists only yes truly that like that well that was a real eye-opener I mean we at Sun we usually had a separate group that dealt with and let them had a compliance but it I was early enough at Brocade where as a start-up I had to deal with it personally I mean I had a consultant and one engineer who was familiar with it it hit particularly hard because we had one product that the silkworm 2,400 which perhaps the folks third got a little carried away they there's really pressure especially from Greg gray as the CEO to reduce the cost of our components and so we actually he that live Tillman the head of the mechanical engineering group came up with a design that had no screws you know let's save the price few screws and maybe make assembly so it kind of clamped together and maybe that was a hard thing to do at that time but it leaks like magnetic radiation like crazy we would so we would go to Elliot labs in Mountain View and and put it in their test stand and then it was just leaking high frequency because you know two gigabits per second high edge rates little cracks around the frame it was leaking electromagnetic radiation all over the place we hired a consultant I can't read his name right now and he was a very folksy type guy he and this is a shocking introduction to me he comes in with an old tomato can and he you know he hooks a wire up to it into his frequency analyzer he's look at this tomato can around our silk room 2400 and he proclaims this is never going to pass very scientific instrument yeah it to me okay and I kind of looked at them and said but we have to get this the pass because we have to ship this you know they're to reduce our costing and they make Greg happy on them and so I spent literally weeks in the compliance lab trying to learn how to contain the emissions from the sport now one thing he gave us advice and doing it was to something that was counterintuitive to me at the time was to tie together chassis ground and signal ground in as many places across the printed circuit board as possible yeah you would think that that would would introduce external problems into the printed circuit board we did that and it did reduce emissions you know you do all kinds of tricks around the edge of the printed circuit boards they don't leak out the sides of the board I learned that it again the analogy of that I used before on the timing at ultrasparc you have this box in steel wool and you're trying to push the steel wool in and you push it here and pops out there electromagnetic radiation was like that if you if you kind of use copper tape and you paste it all up over here it comes out over there so it was like a delicate balancing act you you couldn't be too aggressive on spots that appeared clearly would be leaking spots you had to kind of average out you know your your your use of of tape and whatnot to cover holes you know there's always a trade-off as well as the size of the holes for ventilation versus will those holes in bit like kinetic radiation as the frequencies get higher the holes have to get smaller and that then impedes the air flow so there's this dance between those two as well and so high we were there at Elliot labs just nearing again what happened at Sun we called back that Sun we had the diskless boards in the oven and we were getting memory corruption errors on July 4th we spent all day trying to solve that problem because they weren't going to announce it on July 8th in New York City unless we found the problem I'm not sure I mentioned that last time but that's why we were there on 4th of July well this time is a third of July at night and we are on the flight path of Moffett Field and the the trainer fighter jets were doing a nighttime fourth of July show this time so they were coming in right in over our building with giant streamers with you know they admitted burning streamers out the back and they were doing a private air show which we saw but we were half jokingly they you know that perhaps are looking like radiations might bring down an f15 or whatever they were so what's the case design for the product dining house yes every absolutely done in-house and and so I could go poke it at the Clive Willman and say Clive you really candid me a lemon here in terms of like Rebecca had any compliance and but at that point it you know that was right mislaid it were those folks in the room with you no just my consultant yeah and then one thing was kind of fun is why we were at Elliot labs maybe not that night but I ran into Ron crane and Dave Boggs they're their firm land media there's dyrdek old man media was in the building near nearby so I would ask Ron has became kind of one of the analog experts of Silicon Valley and so I'm crying his shoulder and he would give me some advice so it's fun picking up with Ron again at that time but we did we did get the 2402 to eventually to pass what a mundane thing but an important thing so I can't tell you one more a brocade story not not really at DMC but again an interesting tie in with the people that sign and so a lot of press and analysts would come to brocade you know and ask why are you inventing a special high-speed serial link for disk drives when there already is one for networks called Ethernet you know and you know our standard answer would be well the reliability has to be much higher you know on Ethernet it's okay for the network to drop a packet or lose a packet because tcp/ip will later come around and resupply it but there is but you can't there's too much of a latency in the case of just getting something from storage you can't afford that so fiber channel is guarantees in order delivery of packets in order reliable delivery of packets you can never drop a packet in fibre channel and so you you need to have some type of flow control mechanism Ethernet never came up with a flow control mechanism I mean there's there are several standards but there they can flick with each other there's you know the where the receiver will push back on the source or the source will look at what's happening you know so there all kinds of but in the fibre channel and you can't draw packets and in order delivering but slow latency issues okay because you want it the whole packet to come back quickly you know this is de storage you can't wait Ethernet you can so exactly so we felt secure in that sense but the fibre channel guys were trying to broaden their market oh maybe we can use it for workstations and they're kind of pushing the boundaries so it's logical to ask okay so bill so there were proposals on the table for 10 gigabit per second Ethernet that time so the analysts were saying well with 10 Gigabit Ethernet coming why would anyone do 3 gigabit fiber channel and I asked them who's telling you this because analysts usually don't make up these questions themselves who's given you this advice Oh a well-known and respected angel investor in Silicon Valley Oh Andy bechtolsheim so I go to Andy Andy wait you're an investor in brocade he did invest some amount in brocade why are you telling all the analysts that our technology is is going to be defunct with tanking bit Ethernet you're an investor in the company why are you doing this okay he goes okay I'll hold off and the next week you know an analyst would come in and say 10 Gigabit Ethernet why are you doing 3 gave at fibre channel so I got tired of that so I said ok so what we're going to do is we are going to define a 10 gigabit fiber channel standard and so I with supportive and in many other people we started an informal standards body it was called hairy har I and we met probably every two weeks so this was the technical people who understood high-speed signaling at brocade and technical people who understood high-speed signaling at the Ethernet switch companies we met every week to define a 10 cubic per second interface to Ethernet that could be shared by both fibre channel and Ethernet yeah it was called hairy initially some really great experts in high-speed signaling but we decided that based on the nature of printed circuit board technology at the time we didn't quite think we were ready to do a single 10 gigabit link so we had for three and a 1/3 gigabit links running in parallel and that was called Zoe in the XA UI after meeting several months weary and I just was kind of that just took her head of the group I didn't really contribute from a technical perspective kind of kept the meetings under control and stuff but after several months we realized that we really had to be meeting in a public forum it was really against the rules of standards bodies and so we dissolved and the work continued on the ethernet on Ethernet side of things and it was announced and we basically said well we'll design Oh our switch chips to use this as a nurse which link and so therefore you since fibre channel is going to be 10 gig and even it's 10 gig you can't say why are you doing fibre channel anymore so it's a little bit of specs Manship those two numbers yeah no no and you know I'm not sure how important that was in overall for brocade later the 10 gig link 10 gig never went to a disk so the fibre channel community continued to do 1/3 and I think what was the next top 7 maybe but it at least got people understanding that there was no fundamental reason why the fibre channel couldn't go as fast as Ethernet and it had had this better characteristic of guaranteed in order delivery of packets that was the big debate at that time when people were very there was like two cultures there was the storage culture in the networking culture and the networking people didn't understand this storage culture the storage culture understood the networking people so during during this time we were the hardware architects a banks was beginning development on the next generation a third-generation ASIC to replace Blum and I think we called it Mothra which was kind of an odd name but had it was basically higher speed more ports working again with a long sigh logic we thought it'd be a pretty simple thing to do but it turned out that else.i logics new place and route algorithms were not meeting timing and our schedule was slipping and it was a little double of doubly frustrating for me because the team that had done the new place in routing algorithms was actually apparently a design team in Russia so I actually wanted to talk to them because actually you know know a little Russian because I took Russian in high school but I Ellis I wouldn't allow us that level of communication and these were employees of brocade no no this is also logic now I guess I'll say logics are doing our face and round for that's in Russia yeah and so it's a little frustrating and they were so they were they were slipping it seemed like there was a little we could do I mean the design was the design they weren't asking us to change in major ways their place and route software was just unable to meet the timing so I kept slipping and slipping I didn't say much about the CEO of brocade he joined a week after I did but he was a very authoritarian autocratic General Patton was his hero brilliant man he tend to want to do everybody's job he would kind of take over from the sales guys and belittle them if they hadn't done well take over from the marketing guys he was quite capable so he could kind of do their jobs and tell them they were not doing well a very very powerful person and he I could say a lot about him but not not on the record so he he got pretty upset and I think he wanted to have some heads roll because we were look like we were about two weeks late two to three weeks late and there was gonna be a conference coming up so he applied sufficient he was sufficiently angry its applied enough pressure that I decided it was not tolerable to stay there and I actually resigned in the fall of 2001 thing one thing happened interesting enough is that the is that the chip actually came back just a month later and worked first time so yeah I was off by a month and a half out of a half your schedule but it resulted in an enemy leaving brocade and a director or software left at the same time as well so what what is the story is brocade still in existence no no what happened with them eventually it's kind of a sad story and I won't say all information but one of the issues was you know Greg was such a brilliant person who could do anyone's job but he ironically he and this is not just me saying this it's in the record he didn't really trust engineers and yeah you know he had a he had a distaste for engineers for some reason and it's hard to have a startup company if the CEO doesn't like engineers your son was very successful because Scott not an engineer though trusted the main engineers but he didn't he had some mistrust and and that would bother paul baan derson and bother other people as well and then things happened during this time now we're into the dot-com era so the market is exploding right and even old son is coming back up selling this car to thousands and the market is super uber uber is just flying everywhere companies that don't even have products are doing it well well brocade had a product and it was selling well and Greg and we were a lot many of our customers were actually the financial institutions on Wall Street so when Greg you know did the road chose to go public he he was an amazing salesperson he can just sell your same shoes pair of shoes and repeat back to you for you know four times as much that people were frothing at the Brocade IPO when brocade went public the stock shot through the roof and continued to shoot through the roof I mean so imagine people had shares a pennies per share and it hit almost a thousand dollars per share you unwind all the splits so after brocade went public and its stock you know we were worth more than many countries in Europe in situation people would say well if I join your stock price is going up so much my option price is going to be here they'd already be way underwater breath even every month it was going up so why should I join I mean so Greg plusses heart to try to make it more appealing would say okay we'll give you a lower price you know but when the first time you contacted us which nothing was illegal apparently about that I know nothing about that aspect of it but apparently there's some 1930s law that says that if you give someone an option below the price you know the time they join you need to take it as an expense and apparently the company didn't do that and so this so even though backdating stock options is not technically illegal you have to take it as expense and Greg hadn't done that and so it became a big court case eventually great lost his court case inaction went to prison and I think that put a damper on enthusiasm is a morale hit for brocade and in addition the market for fiber channel fiber channel is a bit expensive and people were desperately looking for other ways to include storage and not have to purchase the you know storage or network so they eventually got the bought I think by Broadcom and you know long history and I think the original brocade is just on support now what about the front like the fiber channel product or did that survive it survived because they had stayed they probably had a few customers in the editing to support it and then InfiniBand came along and InfiniBand was kind of like fiber channel done better InfiniBand was really designed for high-end super computers to link together processors but since it also had an order and reliable delivery could be used as interface to storage subsystems as well but the storage community didn't jump on it right away so but anyways they gave another a set of things for a switch to do so you could maybe combine InfiniBand and fibre channel and maybe even a Thor net so you know those little gains of tactical things the future brocade company got itself involved in but they didn't pan out in a big way but so I was gone though I mean that that two years was very intense and I really actually did enjoy it even though there there was this issue of attention between the highest levels of the company engineering you know it had great great people I mean manufacturing people support people soft where people it was a fun place to be so I said I took some time off during that time it turned out that Andy actually asked me if I wanted to be an engineering VP for his next startup Kalea but my wife she probably did say exactly these words but in my words it was kind of like well if you might want to get to know the strangers strangers in your house you know you may want to consider not going to a startup company so I thought okay maybe it's time to get out of the startup fast lane and auspiciously above the hill just a few miles from where I live is the IBM Almaden Research Center so I thought oh that might be a fun place to work so I called up my my friend Winfred Wilkie who had been the VP of engineering at hal member I mentioned Hal and we hadn't met up and via Hot Chip conferences he was how remember it started to build spark chips and spark or station's spark servers and so we naturally got to know each other and he was then at IBM Almaden and he didn't want to do a start-up for personal reasons he's I need a Borg and his previous wife and had serious health issues which I won't go into here but so he was willing to just stay at a place like IBM ommited which is a place where you could really follow your nose and not have to necessarily follow somebody else's roadmap you could create projects that hopefully it would be an interest in the marketplace or or to IBM as a whole and I called him up and he had this idea that he wanted us to develop this really exotic server something you never would have thought of yourself when Fried's background was in particle physics so he was a physicist and as as a physicist you're used to being around novel thinking large scale experiments and big things and IBM certainly had those type of people in research and so Winfrey was right at home so he had this idea that he wanted to build a very different server servers traditionally are a little what are called 1u 1.75 inch high 19 inches wide you know 20 18 24 36 inches deep servers and you'd plug them into a rack and you plug your cables in there's a switch on top and there you go you know it's cookie cutter that's the waste server sir I mean you can get bigger servers bigger than one you you know if they're a multiprocessor there will be six you or 8ui IBM certainly specialized in those but Winfrey's vision for a server was you would build the server and more in a compact brick style or a cube style so maybe you know four inches on a side six inches so six by six by six and you would stack the bricks and they've communicated on their surfaces without cables without cables so they would you just push two together and that network connection would be formed between those two and since you have a lot of them maybe you have ten bricks by 10 bricks by 10 brick says so they were connectors on the sides and well there I'll show them there they were either capacitive or inductive okay not a physical good not a physical connector the idea was is that physical connectors or a bane of the industry you know I learned a lot about physical connectors during this time traditional physical connectors you know you want to put you know a recipe that works is gold nickel copper you put the gold on top just a little soft but it resists many contaminants nickel is hard and copper is a good conductor and hopefully that layer will last as long as your product does throw all the insertions and the insertions you want but how many times that you had a problem with the connector right and so the thought was is to instead of having a mechanical connector which involves force right you there's usually a terminology engineering people uses a male and a female plug and they have to be inserted into each other so if you have a box so if you want to have boxes that slide into place you know you push put one down put a note on top of it but one beside it you can't be plugging them into each other unless you've got an explosive charged it connected them so they have to you know and if you have two metal surfaces that are brushing there may not be enough brushing to wipe away surface contaminants so so the idea was to have I think what was first idea was inductive and I'd I said no induction and inductive things are slow let's go capacitors so I I designed the capacitive couplers as we call so anyways his overall vision so you know ten by ten by ten that's a thousand already in a small space you know twenty by twenty five twenty you're talking about ten thousand things in a pretty small space so it's extremely dense so now the question is how do you get the heat out of the middle so well since you might think of a nuclear reactor right look of reactor as rods and so you would cool it via cooling rods and the bricks would slide down over the rods and you would cool that way I have a picture will show in a minute so this was his vision and I was like that's a pretty crazy vision they're gonna let you work on that yeah and research you have to have a really outlandish idea you just can't do something more traditional because that's what development companies do so I know I was new to research so I said okay oh go along with that I mean the world has moved away from liquid cooling keep that in mind but he knew of other liquid cool projects in IBM was the last bastion of liquid cooling you know and it's high-end as 360 machines like they had gone to you know minus 10 degrees Celsius cooling and in the 360 successors so I I said okay I'll do that and I couldn't I couldn't beat the commute you know bike ride is 500 feet up hill but still it couldn't be so in May 2001 I joined the allman Research Center as a member of the technical staff which is actually a title that applies to wide range of salaries and people it's just a broad category they have not a manager in other words yeah I was I didn't start as a manager stuff there was really no management opportunity there really so it was it was getting out of the fast lane and getting back into doing technical stuff one thing that was true was that I never forget my first day I pulled into the parking lot there's a sign that says did you remember to turn off your car lights and I go how paternalistic is that they're asking me but I forgot to turn off leave my car lights on I thought that was really crass but anyways I soon learned that many of the researchers there really enjoy biting back at the hand that fed them it wasn't like everyone was trying to be a conformist but it certainly a research was trying to be the avant-garde corner of IBM corporate what does it address just out of interest the Dre you know IBM was always known as the white shirt and tie certainly not by not neither no no no they they threw all that out on the mid-nineties okay I think the marketing people for sure but yeah but that's but as the engineers that all that all typical Silicon Valley I mean it started Xerox PARC I think you know we remember we wore blue jeans and stuff and that got spread around Silicon Valley and I think that eventually made its way to the East Coast and yeah maybe not t-shirts on the East Coast yeah so certainly people at IBM Almaden were we're more casual the research was you know you think of research as possibly being frou-frou but the reality is is by via ball in research with almond and research was directly connected with the disk drive development so they were responsible for the next generation heads you know disk drives have become a massive business for IBM you know in the mid 80s I don't know if I mentioned but the 33 80s the giant destroys became the second largest selling merchandise in America behind airplanes I mean they're making billions of dollars selling destroys in the mid eighties so keeping that in and then of course a whole ecosystem of destroyed master started that's a whole history but keeping that roadmap going was important and so I've been research the research center on top of the hill Caudill road was the facility down on 585 you know 400 acre site with massive buildings manufacturing buildings and a totally separate part of the organization in that's separate but they were linked because they were expecting that the advances to come in the in Atlanta the hill so there was a huge science and technology at presence at IBM all in research and it was still going on when I joined so I could go down to Caudill road and see you know the advanced research being done I could see how they were for example how they would try to measure the bit error rate you know one bit air and ten to the minus 15 accesses you know how do you get numbers like that and you know the experiments they would do to try to show that that level reliability is this fascinating place to be and high-speed signalling that had to go on you know you're getting up into the multi gigahertz speed on the heads you know the head heights or measured in angstroms I think you know several atoms so there was just a really vibrant connection between IBM home and research and the disk drive development area so how did they end up IBM you think it was an East Coast company how did they end up having these two major divisions in Silicon Valley because of Silicon Valley well I I actually know some that history because I did a lot of with my involvement condition museum I did a lot of three on the RAM Mac development so in the early 50s IBM on the East Coast found they couldn't hire engineers fast enough and they couldn't recruit him from the West Coast cuz no one wanted to go live on the East Coast so they intentionally set up a research lab in San Jose and in 1951-52 and they intentionally wanted recruit from Stanford and Berkeley and plus whatever industry people still might be on the west coast so they set up the lab and San Jose there on on the building is still there on forgot the name of the street right now and they gave an open blank open option to just figure out how to improve data processing and companies and they did all kinds of experiments eventually they came around to describe someone at the National Bureau standard said thought of an interesting way to do a disk drive you take a book and you put it on a table with its spine up and open up all the pages you know imagine you have a head circling around and then so instead of pages you have a circular disk you would kick it and the head would be dead right that disk so that's what rabbitov reppin all have done it oh you and you know this is an area but now a new revenue this is the amazing part of this story is you ROI Ogas actually knew this guy yeah inventor or the national tour standards so he built a prototype but the guys at in San Jose thought it was impractical and they had the idea of just putting 50 disks on a single rod and it's kind of a diversion but so they were successful in inventing the important advice and in high tea technology the disk drive the very first one they developed a machine called the RAM AK for it but you know the Ivan 1401 which have been developed on the East Coast kind of took over the market as a stored-program computer and so they had to shut down their CPU development in San Jose and they just supplied it as a peripheral so IBM computers the 1401 and 360 so that's kind of why I BM doesn't have a big presence in Silicon Valley all I really want it from that whole effort in the end was the disk drive not the system so there was really no impetus to build a systems group in Silicon Valley for right vo so that was interesting history just starting now so so here we are back back in back in the year 2001 and the project was gonna be combination of software and hardware because if you have all these bricks those are the small cubes one can die obviously and so what kind of rely but you know what kind of redundancy are you gonna build into the storage code simple replication could work but now you doubled the cost of your storage an eraser code across many many bits perhaps eight perhaps more so that way if you know one brick goes you know you can rebuild the data from the surviving bricks so we had to develop the software as well so there was a project started called turtle farm and it endeavored to do this but not sufficiently staffed and so it wasn't really a successful there was an already an ongoing project that it started at IBM Research Calton GPFS journal parallel file system started by Roger Haskett and people in the late 80s actually it they actually started as this is hard to believe today given today we can pull out an iPhone and watch a movie on it or watch our movies on our laptop but in the late 80s a single CPU if you remember not have enough performance to deliver a video stream that you could watch it's a little hard to imagine because we we have gigahertz processors and we pull up you know we watch videos on our phone but a single high performance $10,000 CPU couldn't source single video so the son and SGI and many companies at that time we're trying to come up with a video server that use multiple servers to supply a single video stream to the user perhaps in a hotel you know maybe someday at home and IV a research you know Roger started I did it I started a project to design such a system now they did design such a system that worked but being researchers in generalists they designed a general peril file system that could both read and write so it could be used by supercomputers not for video streams but just for storing data so it automatically stripes the data across all the disks the keyboard is automatically alright so you have a large file system GPFS will take chunks of it and put it on each of the disks connected to servers your multiple servers disks multiple disks per server just stripe it across all the systems and then you any given client can now beginning bits for multiple sources so Roger and the early 90s went to the software groups in Poughkeepsie New York and there was a good match up the supercomputer groups there needed a pair of high-performance file system so Roger established this channel to market for GPFS where if IBM was selling a high-end supercomputer multiprocessor supercomputer you would you would likely get GPFS along with it for free so he wasn't selling it as an unbundled product but it was being sold with like Rs 6000 multi computers so and then by the time I joined IBM they began to sell it as an independent product so he could run on not just on AIX but it could run on Linux and it started to sell in the open market so gpfs was this huge high-performance parallel file system organization and culture that existed and that was one thing we could run on Ice Cube and we ended up doing that in fact we actually ran out of on Ice Cube because we were unable initially to get that distributed software folks were unable to get the erasure encoded software to work so we just demonstrated the GPFS was just to set the context when you join what was the state of Ice Cube was it a concept or an early concept so they nothing had been built yeah nothing it he was thinking of moving forward oh yeah so he asked me to help basically make it happen and so you know I I'm used to implementing things so I brought to the whole the whole endeavor you know the knowledge of how to build this system how to work with Silicon Valley companies to get the printed circuit boards built we we tightly worked up with Harold jewels in Bovington Germany to actually they actually manufactured the mechanical part so I'll show you some pictures in a second they took responsibility for the central processor car the CPU card I designed the network switch card they designed everything else the memory card the CPU card I linked up with Silicon Valley manufacturing firms to manufacture the boards one thing that was true at IBM almond researches that since it wasn't a traditional systems company they didn't have really technicians or us or staff to build things there was only one really technician Manny Hernandez who helped us a lot but we ourselves had to build these things I mean think our Harold Jules and his team in Germany you know did some of the design and I had firms and the valley manufacturer of the boards but we had to yeah and they stuffed him with components but we had to assemble them all so it was very hands-on experience so it probably took longer than one might imagine I have a picture here of ice cream so that will help so so well that's me and that's one feed bulky and that's dad's name so here's the ice cube we built as a prototype we put a three by three by three and on one side we connected traditional one gigabit ethernet so you could you could basically get everything on one face and we had a cute little LED system where you could see which way the the lights would blink to man which way the ports data is moving on the ports these towers you see my hand is on a tower there there's actually my idea how we ended up doing this the the towers they're aluminum extruded you know a rectangular cross-section they go through the bricks and there they filled with water or liquid coolant water plus glycol and there's a little overflow the top so they they naturally fill up a certain level they go out the other side so this is a liquid cooled system the cables on the floor over here too but liquid water from the building so it was very quiet the IceCube even though this is 27 servers no fans there were no fans and so you could be in a room with 27 servers and is quiet it's like a bubbling brook we really really like that yeah this is just a stand and there's a there was just some lab control equipment where you could turn power on and off individually to the brick so one of the ideas was is that if a brick failed you could disable its power thereby you wouldn't you know be boisterous and screw up the network or something so you could turn off a brick so how did the power connections work oh well yeah yeah the power connections were actually connected yeah they're not visible in the picture either there's actually a they called hidden hidden connector between the top and bottom so power went vertically so each brick has a power connector between the two and that's a real connect a real competitor because you're just placing the bricks down vertically okay and so the one in the real middle it gets its power from that Jason one no well no they're headed off so it gets its part the one in the middle gets its power yeah people would always focus on the one in the middle right what if that guy's eyes isolate so we would we would kill the one in the middle and Claudio I'm sorry Claudia liner wrote the software that does all the routing Ethernet traditionally wants to find one path through the network great this was an innovation actually during this time the idea of an Ethernet having multiple paths through a network all converging to one destination wasn't allowed by Ethernet so he changed the Ethernet routing protocols so that we could you could route through different paths through the cube one thing that Claudio did as well as since we were basically building a PC I was a little shocked to find that you know we needed access to the BIOS because we had our own devices and stuff you know the swish and that's not a simple thing to do it's a get access to a to a BIOS you have to have a license and it's all top secret so we were able to do that though we were able to get a copy of the BIOS and modify it which Claudio did so you said it was a liquid-cooled yes so how did so there was the a plumbing system well that's what I was describing there these towers I should have brought one these you see these aluminum she had Lumina thing right there yeah that's a that's a one inch thick by six inch wide by however tall you want to but it's rectangular cross-section filled with water liquid and then there's a but there's a return tube that goes up through it and so the water fills it up and then goes back down through the return tube and so that provides a cooling spot and then well here let's look at it in a neutral brick so here's a picture of the brick so it's a little hard to see but there's there's the opening for the tower right there and so the all the electronics is on a piece of aluminum with heat pipes that go over to the aluminum and so then when the brick goes on this was really clever I went free came up with this when the brick goes on we we have a little lever these two levers that we would rotate and that would apply pressure between the vertical column and the inside aluminium plate so we'd get a high pressure and we had to have a thermal material between the two because metal the metals not that great of a thermal contact so all the heat then would go in fact you can see a heat pipe they're a little hard to see but there's a heat pipe right there from the processor chip over to the plate I think we had two heat pipes our dims did not have a heat pipe so that was kind of cheating so we IBM had great so we use IBM software yeah IBM had use of companies like Aaron but a great thermal simulation software so we can simulate the thermals inside this closed box and we could see how warm at God and we actually had a thermostat inside so we could always measure the temperature inside Oh inside every cube had that tower with the cooling so the one in the middle again had its own cooling you know well well there's one tower that goes vertically through all of them all right and each each brick clamps up against it so one tower is shared by in this case three bricks so what you know so there are three by three there are nine towers two towers yes yeah and all the bricks on one tower share the cooling in that time so we could measure all the thermals and and saw that it work just fine it was that we could measure the CPU die temperature because the dies now have thermostat the wrong thermistor is on them so it all worked the only cheat okay so there was another cheat so the district we had 12 destroys tuna fish disgrace this is one they're all packed together side by side and the heat was conducted up to this aluminum plate over to the bar and we actually could get the distillate temperatures as well because they have thermistors in the disk drives so it actually worked but I'm not sure how scalable would be going forward it might have been I think the probably the biggest difficulty would have been in the dims I mean I mean IBM I'll talk about that in a second did this amazing liquid cool system I'll talk about I'm in a but this was very obviously non-traditional you've never seen a computer like this so I so I had fun bringing people into the lab and showing them our three-dimensional computer so the nice thing about it is if you add a brick a brick in it has storage as a CPU and it's a network switch right so it's a scalable unit so you're actually building a scalable switch each time you add a brick you've added more ports and you've met them or CPU got a more storage so it's a naturally scalable system so yeah is the other disk drives I understand this is a switch really well there's a switch in each brick and right and a disk talk to the process is the disk the storage you are managing or is this just a disk for the local software it's the storage Ramanujan yeah that's a good question there we used not a prom flash for the local boot device for the local processor we just use a flash flash - okay so the actual storage being managed is part of the cube in other words yeah so the software then would manage the distribution of all the data across all the disks and so we tried with turtle farm to actually create new software which would do that later we had an effort called Kubo's but I'll talk about that well I can't talk about it now that was a second-generation attempt called Kubo's to do the distributed erasure codeine I mean at the time places like Google are starting to do this naturally in their in their server farm so you were just had a small group here so right now what's the order of magnitude of storage in a 9x9 well there's there are 27 times just 12 disks so that's you know over 300 disks so whatever capacity you want to put on this drive today two-and-a-half inch disk rice or quite large yeah today what do you think avoids yeah this was these were just it would have been nice to use flash style just drives at the time but at this time in 2004 2005 they were kind of expensive so we still use mechanical rotating what we call rotating rust what about the software was this all custom soft well that's what I was saying turtle farm was the project at site on the first project he tried to do the distributed the erasure code and storage that that didn't pan out we ended up demoing the system with GPFS which was very reliable the whole team supported it at all but it was based on replication so for our demo what we did it actually as we hooked external Ethernet connectors we we showed a movie a banner movie and we were streaming it from multiple discs GPFS does that that was the original motivation for GPFS they would kill a disc and you'd watch the movie and you can watch that nothing happened nothing there's no blip done so it was it was real time and what were the processes in there where they just until these ready these our opinions our opinions what kind of operating system was the software running on Linux Linux all right and it's functional I mean it took a little it took a while to get it built because we had to hand assemble I have pictures of sand assembly in these bricks you know first you have six then you have ten then you have 20 then you have 27 we added a separate on manufacturing line basically in the lab does hand assembly cement and that and the picture of the individual module is that I entity on the right that's the capacitor oh yeah so let's talk about the capacitive couplers so so here's the so here is the casements boards that I designed it would all fit in one chip today but this is 2003-2004 so this is the switch chip this is a Broadcom switch to have eight port switch chip and this is just a convert to eight one gig these are 8 1 gig ports this converts to one gig serial over twisted pair for the external face ports and so there are six sides to a brick right just a cube so there are six capacitive couplers six so top bottom left right front back so each coupler it turns out this is the Zowie interface there are four channels you know these are yeah I forgot to mention this is I say one game this is 10k one kick outside ten gig inside so these are four channels each running at 3 and 1/8 gigabits per second so these are the this is the receive side and this is the transmit side so and it's friendship next to it yeah it is exactly over so that format capacitor between yeah so the transmit side talks to the receive side over here all they have to do is touch and you'd have your connection you have a capacitor there and there's a capacitor well that's one of the mysteries of physics as to how signals jump you know it's called the displacement current physics class no one really knows what it is some kind of near field effect where the signal just magically goes through the gap and so this is ceramic and it has to be polished to a certain smoothness to guarantee a minimal gap to get the capacitor enough capacitance so what kind of gap was there physical terms between the two yeah it was measured in microns they could get micron yeah like 10 20 microns yeah I'll have to okay look at my notes to get that number yeah you'd polish it and so they were they were pressed together by four well we had a very clever idea I think one of her somebody on the team came up with I didn't bring the holder but you can see in the picture there's a plastic holder around him and then there was a light spring just a piece of metal and so the two would come together there's a there'd be bumps and holes on the plastic part so the the bumps would naturally find the hole of its partner so they have perfectly aligned and made and the springs would hold together okay and we we we kind of wanted this is ceramic and so you can break ceramics and so to make it more robust we also explored using a we designed a capacitive coupler based on Mylar and we had that work but we didn't ever enter a building so my Lord would be and you'd probably have a pillow behind it it would kind of push against Neal the other one the other thing we did is I VM had great high-speed simulation software so we used that to design the connection on the back and this is where you learn a lot about the effects of inductance I mean just this is a differential pair that small opening is enough to affect the signal quality and so we we actually designed a 10 gigabit serial link so you could have for 10 gigs and in simulation it worked one thing that's nice about this is we didn't have to modify the drivers that drive the the four lanes of three gig the distortion that this provided was within the compensation capabilities of the existing chips I mean if you had full control you would you redesign I mean the fact that our serial lines work at all is amazing if you've ever looked on a scope at a 10 gig circle line or the lines that go to memory chips or that's all jumbo because because you can have many zeros in a row or many ones in a row you get different frequency components so you have high frequencies low frequencies and so they respond differently to the network and so you end up when you have a low frequency the you know through a capacitive thing it starts to decline in value right and short things are really fast and so you you have a pre compensator which gives more energy to the low frequency components and less energy at the high you have receivers that it's a it's quite an art the whole serial transmission and reception process but that this luckily at this time and with this design we didn't have to worry about that and the paper shows eye diagrams that show prospect was a little bit of concern I had at the at the higher 10 gig rate but this is this work quiet we had no no so even though it's wireless there wires inside I mean this this was a very high-speed connector here that we were able to use to connect them to the board so it works so let's say you you know you said when you started it was a concept then you eventually built some things yeah and you had prototype and then one so yeah that's the question is what happened to after that so it made for good PR but you know some people think that research is just a branch of the marketing department I think that's true in many ways but we really did want to turn it into a product so oh I guess but for terms of dates 2005 when we had this working so we did demo this to many customers no you know we have this working thing with water flowing through it right I mean it's pretty impressive you bring people in the room and they go hey this is cool I don't wear these in fact this man who was head of a large Japanese bank wanted one for his bank that's we were almost ready to build another one for him that was pretty funny did you make any that were bigger than 3 oh no that was so much work you would need a team a company a start-up to do that so so and it's expensive so but during the demos marketing more calm person at all it would bring companies in and they would go yeah this is really cool when can we have one when we get one what people liked about the idea was that they would run for a long time kind of like tandems non-stop idea right week said that if bricks die it keeps going and you buy tow provision at a certain amount you can predict the failure rate of disk so you over provision by 20% and it should last 5 years without having to touch it how many failures couldn't tell how many failed cubes could have taught well yeah we did a lot of research on that you in terms of what happens to the network how long can the network stay up you know what percentage of bricks need to fail so we had equations and graphs and papers and simulations and so you know if you over provision and then if you didn't know provision and not if you guys just add more bricks and and and bricks of higher capacity you know perhaps be at it but that was not a feature that software could handle anytime soon people would come in one person was very very the bruise really listening to one of the demos I was impressed I think he was a high school student he said now wait a minute you're saying they single one forever you have to over provision the number of bricks aren't you basically putting the burden on the customer to pay for your unreliability upfront yeah but that's reality so his point was if you've thought about it more was that you don't you wouldn't want to deliver a whole bunch of bricks that had low reliability because then you just have a pile of dead bricks so that would be Muse you two perhaps produce more reliable bricks to start with which might imply you have to do burn in an IBM had abandoned doing burn in years ago and many companies have companies used to do burn in a good way to test the resiliency of the components and then ship them to a customer but with improvements in manufacturing it was easier to ship it to a customer and then deal with the problems later so most of IBM had had already abandoned burnin so that was one of the then the negative sides to this idea is that you would have to have more reliable manufacturing not to overburden with the over with you with the redundancy people love redundancy they don't want to pay for it I think what our story there was though since storage increases in size so quickly you know you can add just add bricks so and whatever's failed so what does add more because do you want them anyways because they're higher capacity so it was reasonable story so now the question was what to do with it so one thing that happened is there was a person well we talked to the storage people of Tucson and they weren't they weren't interested other parts of IBM weren't interested because it might compete with the Blue Gene supercomputer work although we were a long ways from being a supercomputer it's kind of interesting that IBM itself didn't seem to be that interested in music yeah well IBM was trying to move to movies they were off-the-shelf stuff and here here's this new again which they had had funded yeah well IBM had done so much internal stuff for so many years it was really starting to be motions of hardware's becoming to monetize let's try to use off-the-shelf stuff why would he want another system which is defined strictly by us and isn't second sourced by anybody you know I understand that coming so-so so the question was what what to do next and in early 2006 happened to be that a VC from East Coast VC firm called new venture partners Frank romanovski was kind of bantering about inside am their model was trying to find a loan researcher who had a great idea for a product but he couldn't bring in the market because the volumes wouldn't be high enough for IBM to justify productizing take that spin it out you know make a successful company so he found our project now the only project with a problem or challenge with our project is it's not some simple thing as it's not a shelf it's a whole system's product with software hardware and everything it's a big enchilada and but he was willing to take it on so by October 2006 with funding from new venture partners we started to spin out startup company we called it a Savalas of all systems SED al I didn't like the name for reasons I won't talk about but and they gave us five million or something I don't know but since we weren't sure if it was going to be successful and IBM was resisting it as a spin-out we remained IBM v OS remained IBM employees and worked there and we hired about six people or something the exact number we hired five employees so if I was joined as IBM employees we hired five employees and the five that were hired were not IBM or not IBM Hearst no they was evolved yes yeah and we had you know board we had met with the BCS and and we changed the design this this design fundamentally is too expensive is too expensive if manufacturing costs would be just way too high so i changed it to more of a blade based design so you had processor blades you had storage blades so there's now the processor plates could fail independently the storage blades and a 2d backplane not a 3d thing and in 2d we had 20 port switches on the cpu cards it's on paper now we never built it and actually the reliability of a 20 no network is much higher than than the 3d because you can skip over nodes you know the the if you have 20 ports you don't have to go just your nearest neighbors you nearest neighbors and their neighbors and so you get this very rich network and we did simulations on that to show it was much better so and then the IBM lawyers an assistant we remain liquid-cooled which was a hindrance obviously because the world's moved away from Libman claim but I realized we could build a large RAK and do an air to liquid a liquid air heat exchanger so we could have air moving through the rack through the blades and this was standard technology yeah there is cooling companies out there and aired a liquid heat exchanger and then a you know a water connection on the side and in some sense this was mirroring what was happening at companies like Microsoft building the big trailers full servers you know water connection on the side but air inside and an air to liquid heat exchanger inside it's interesting you said that lawyers wanted you to retain why would lawyers have an opinion they were it was hideous iBM has really no track record of spitting out companies some companies do a great job like Cisco did I mean hated it they were really large we're really worried about solely an IBM name accidentally they the terms of the spin-out were just onerous I mean they were IBM we get the first right of refusal for anything that might happen to the company if we hired like Osama bin Laden you know they would roll back and pretend like that nothing to do with it it the fights that the lawyers were endless and and that was one of the problems so we we went to Sand Hill and we pitched it to all the VC firms I even pitched it to Mike point she was now at a VC firm up there my high school buddy who I'd run from I had learned for freshman class president with and lost high school he was he was kind of interested but the only person who really bid was bill joy at Kleiner Perkins and the other person there I can't remember right now so the database guy um yeah I was just commenting you said the lawyers wanted it to be water cold to remain water cold yeah because I think they asked you was so why give us more freedom that's probably from a risk point of view that they were no they wanted I think they didn't want us to be successful oh I see yeah that was there so ownerís are so hideous that they're okay I never asked them but you know while you're spending on Ice Cube all the ice cubes water cool this you know I was certainly willing to move away from where coal in the whole industry was so so we so we had this but we had to start off we started to work on this this air-cooled with the air to liquid and heat exchanger idea all the simulations aired the cooling simulations looked good and you call that ice rec called a nice rack yes I didn't bring any pictures we didn't build it but we had lots of drawings and then the the racks would have a type of switch rack which would connect to the other racks and so we had a very low hop count we could have array of 10 by 10 racks a hundred racks each with well variable numbers servers the number of servers and distres who are adjustable so you could have a disk drive rich server or CPU rich server so you could have anywhere from 20 to 50 CPUs and a very low hop count because each node has a switch so maybe a hop count of only five or six across 10,000 processors so we wouldn't talk to companies like Microsoft and other companies they were really fascinated from the network perspective I'm building a scalable network switch but you know Bill joy said okay let's give it a shot but he immediately checked with his buddies at Google and Google was just then starting to build out their large data centers and to be honest they didn't need the help of any startup in fact you know later Google build their own switch chips and what was interesting is already Google was this was very educational for for me and for us was already building data centers with power usage efficiencies pee-yew ease at one point - in other words you know for every kilowatt of power powering electronics they only needed 200 watts of cooling energy the cool most Pei numbers and data centers are closer to 2.0 you know twice whatever energy to run the servers you need that same amout again just a cool that's a pretty inefficient but that's kind of standard in the old Libre datacenter rooms they had already figured out how to do pee wees of 1.2 and they were publishing them it turns out that that figure was about equal to what our number was so the advantage of the cocoon was gone so the way Google was building their large data centers they hired large industrial designers and you basically built a data center with pretty you know reasonable racks high ceilings and you pull in outside air pull it through and push it back out to the outside and you can't get more efficient than that you know their intermediate schemes or you have air to air exchangers but you know if the climate is good not China let's say near an industrial plant pulling the outside air pass it through your servers wanted a pretty high temperature the temperatures might get pretty hot especially up high in the ceiling in fact I would go to interactive server cooling conferences and everyone notice I worked for IBM they'd come up and ask me how hot can I make the outlet temperature of IBM servers I would say don't ask me because everyone was pushing the envelope to realizing that you could run them hotter than the old standard set in the 60s and 70s with these air based systems with Noli birds you know in the room you could get away from those cold those and save a lot of energy so there was really no reason to do a liquid cooling based system and so we based on the fact that the terms acquisition terms were onerous and NVP owned too much of sub all I said like 70% it wasn't Athena pcs in July 27 and who a second round of funding we just elected to shut to start-up down fact one VC were talking to the forest basket was one of these we talked to one of the VCS while we were pitching to him he got a call about how they were shutting down rackable and he goes with his bent I mean isn't it you know so the whole transition was occurring where the major data center infrastructure was being done but the large company its Facebook Google Microsoft and no one needed to help us it was a complex set of reasons it wasn't just it was certainly not just pure technology yeah but that was a factor we were still because of liquid cooling expensive but even if you took that away I think no one wanted like a lunar Inquisition terms all that way no one really needed the help of a small company yeah okay so at that point we shut it down and it's a project you all went back to IBM with you and we never then technically never left so we finally say that the ones that were hired they have gone to other jobs well they didn't work at IBM remember they just went and found other jobs right they were working for civil and they just they just have a job there yeah they just wouldn't found other jobs so so I that point was thinking of leaving IBM but Roger Haskin who was the Godfather of GPFS so I've talked about before asked if I could manage a new project to deliver exactly the software we needed for icecap which was a way to distribute all the data across all the disks and if a disk fails in a racial code would you know distributed racial code would bring you back the data the cluster great as its called it first been pioneered in academic papers in the early 90s but there are really no implementations out there yet so this was pretty exciting I thought they are there already was an architect there really a brilliant man another case of one a brilliant architect of Silicon Valley Jim Wiley but Ben it might be a moment in his entire career he was a lifer I'd be ever brilliant man brilliant programmer when I say brilliant I mean truly put equal to three or four people normal programmers who were just astounded to what he could accomplish in one day either writing software or writing specs or understanding problems so I said sure I'll manage that project software project I named it Perseus for the Greek god that could slay sea monsters okay and that funding was initially coming from DARPA and a project called perks which was just the name of the next large supercomputer they wanted to build and IBM it had been awarded the contract you know thirty million dollars or something and some number a million could go towards development of the storage system software and this was the last huge raw liquid cooled were raw computer probably from IBM this perks machine the architect I'll fill in his name designer of the hardware this was the type of machine that you would find in the basement of the no such agency or a down dan brown digital Fortress moving it was liquid cooled the racks were one meter wide and two and a half meters deep each rack could accommodate a hundred eighty kilowatts of heat with one gallon per minute liquid cooling per rack the the anticipated cooling for the entire 90 racks was a water-pipe of foot in diameter each processor board a meter wide to you high had eight multi-chip modules for power processors and eight multitude modules for the interconnect which was a star network so every MCM could connect to every other MCM in the entire cluster and this Thor junus where were one meter wide four high with 384 to 1/2 inch disks in it and I suggest if they have an air liquid heat exchanger which they did initially they thought they were trying to direct cool a disk which was crazy if you had to ask how much the system cost you couldn't afford it I mean one of the the list price for one of the storage drawers was half a million dollars now they sold a property for less but that give you a sense the list price for the processor modules were close to a million dollars per processor you know the world have moved away from liquid cooling a heavily liquid cooled machine so maybe in the basement somewhere right where you couldn't bring in outside air and call your electronics so eventually the perks machine was announced as the IBM power 775 supercomputer so we have we had the Charter of writing the software so we we knew that this would be a good platform to write the distributed iteration 2 clustered raid software for so many disks lots of CPU power but in our heart of hearts we knew we wanted to get it moved to an off-the-shelf hardware so we didn't we were really doing this just for we were being paid by DARPA to do this for perks but in our heart of hearts we knew it would be processor independent and OS independent so we were doing for AIX and Linux on this computer but it had a power power 7 or power 8 processor core the interesting thing had happened right about at this time into this started in 2007 was that CPUs were now fast enough right these 1 2 gigahertz CPUs were now fast enough that you can run a read Psalm in a racial code in the CPU at full memory speed the whole storage industry like NetApp and all these companies did raid erasure codes in accelerator Hardware off to the side and so just like in the grand history of computing things get moved into the CPU as they go faster and faster and faster as performance doubles ever while used to double every 18 months not anymore so we that was the last thing that just got crossed in around 2000 and four or five a seat single CPU could do reed-solomon racial code at full memory speed without any special acceleration hardware and we knew this was true for the power chips and also knew was true for Intel so we had a one of the guys the team had every clever clever algorithm algorithmic way to implement a Galois field with four bits and you'd use four registers 64 registers in parallel and a very clever way of doing exclusive wars' and implementing the algorithm at full its full memory speed so you could do everything in software 100 operating system there were fears that you know maybe the operating system would introduce on certain latency and disk operations but that hasn't been borne out so so I I managed the team development of the software really didn't take a whole lot of management because Jim is such an outstanding architect was the team in place when it started or we brought a couple people in ok these were all people that IBM mostly yeah IBM Almaden wasn't exactly growing in size though at the time so it was kind of painful from that perspective you know they had spun out the disk people several years before that to Itachi IBM had decided well what happened is is they they kind of made a slip up then one of the enterprise disk state they did I can had the name later they out sourced it to a group in Japan and it quality suffered and it it was the impetus that caused IBM management to get out of the destroyed business so IBM sold all its destroyed business to Itachi yeah now most of kata Road is apartment buildings and a few a few Itachi buildings but and it's all concentrated now Western Digital but that was kind of the transition that got them out i destroys but so almond and i said to kind of IBM all but on researchers kind of reinvent ahead reinvent itself what other things we work on besides just destroyed a lot of people thought that IBM got out of the storage server business to just because they got rid of the discharge but that's not the case right yeah IBM still and to this day still delivers storage servers so so we built this highly complex what is it almost a million lines of code distributed raid software we announced it on 11/11/11 Africa that date November 11 2011 IBM we announced that she professed native raid or GNR for the p75 supercomputer which was mainly procured by since it was liquid cool perhaps they understood it meteorological forecasting agencies around the world the European Canadian companies government agencies obviously and universities but you know it numbers sold were limited given its cost umino the supercomputer yeah work do is reliable our software worked it was reliable looking at the comment about the clustered raid before I go on to the what we did next the reliability that theoretical reliability of the cluster rate is remarkable we Jim Wiley had done some simulations if you if you take some data and you randomly distribute it you have to remember where you put it but if you randomly distribute it and then a disk fails you know let's say during the rebuild of that disk another disk fails so let's say you have a to fault tolerant code because the data is distributed across the disks randomly when the disk fails the number of were called stripes or chunks of you know that need to be restored is a fraction of all the data it's it's in the case of a distribute across eight disks it's it's like eight over the number of disks that the fraction of data so now when the second disk failed the fraction is that fraction times seven over the number of remaining disks so the amount of data that has to be restored decreases geometrically very quickly and so the probability of failure during that restore is dramatically less and so the reliability on paper goes up to pretty quickly to hundreds of thousands to millions of years in fact three fault-tolerant de clustered raid on paper would have an average mean time to data loss greater than the age of the solar system and I I'll be I still need you know we did a lot of papers and simulations and studies I hope into paper but it's kind of like the key to long life whereas if you take data and just put it in a chunk put a chunk here you put a chunk Derek put a chunk here you put a chunk here which is very read normally does it and you have a failure here and a second figure here you have to restore all the data in that chunk which is just that fractional chunk of all the data and so reliably do that it's not very good compared to the cluster rate it's just incredibly dramatic and if you're not careful you can overhype it so some IBM sales people would say you'll never lose your data with RT cluster drain system now because the mean time to data loss numbers were greater than the age of the universe well that's not quite true because you have bugs in your software no software is bug free and not so much maybe because it's a bug in the software but the hardware does some weird thing right which causes your software not recover properly under some conditions so I you know even though we inherently the clustered raid has non intuitively remarkable meantime you know reliability and remarkably large meantime the data losses or just our randomly timed heat losses compared to conventional raid it was important that we that customers still realized that the data was that important they might want to back it up because you know there could be a bug in the software and the software has been shipping now since 2011 this is 2019 so for eight years now and install base is quite large but I'll get into what we did next but the reliabilities turned out to be really really good now did the clustered raid get traction anywhere else well we were the main we were the source of it and I at the end I'll talk a little bit about that it is done now industry-wide you know Google does do close to grade no I'm sure other companies do I I can't say for sure but because it sounds like has a lot of advantages of it regular it does because the reliability and the rebuild times are so short you know what you might send to different data centers is a smaller amount of data as well - did you replication it's kind of scary though if you if the metadata which describes where you randomly placed all the chunks if that ever gets messed up you've lost you can't reconstruct your data well he's dead predominantly backed up just ask - you just has to work yeah so you know sometimes I jokingly imagine that it someday if someone walked into the Google and changed a piece of code that lost access to media all the world's data would become scrambled it's it's people don't realize how sophisticated the the storage algorithms are on places like Google I mean I don't know all the details I'm always extrapolating but it's extremely complex based on the papers they publish okay so so now the question is what what to do next we've got this really expensive supercomputer thing which is impressive but it's very expensive so roger Haskin introduced me to a marketing guy at IBM in the hpc group matt Drossel and he had an idea of how we could get to market with cheaper hardware he knew about what I call a hidden channel in IBM through which since we were using off-the-shelf components we could deliver a complete this is in my words now a complete systems product to market bypassing all of IBM's procedures processes and VP's because of this channel we said ok let's go and the channel was in the x-series group IBM you know had x-series was just Intel based servers it had AI series which is which is this old older stuff and it had the X series which was the 360 stuff and in the I'm sorry the Z series excise II or whatever order you want and the Intel based stuff they had a group there called the intelligent clusters group so if a customer wanted to buy a cluster that group would get the components the CPUs that storage the networking assemble it and ship it to the customer so we said ok well here are the systems and there are the customers and so we went out and sold it and it are the same software obviously it gets better over time but the same software and there was no objections in other possible I would be on phone calls and people say who's the VP in charge of this project I would say Shh it was essentially run out of the GPFS group underground and it depended on support from the x-series people so in early 2013 we introduced it formally as we called it a solution as opposed to a product to make sure we didn't step on IBM internal toes we called it the the linux gpfs storage server or GSS and we sold in configurations up to a petabyte per unit and it sold well Nana gave us lots of field experience that increased the volume of gpfs sales but now you're selling a complete system hardware and software so guess what your revenue is higher so now you start seeing battles inside IBM who gets to take the revenue but again this idea of silina system gives you better better revenue they just saw people want software for free all right they for selling a system well I see there's hardware ok so I kind of get it so it in 2012 we introduced that the GSS but about a year later IBM unbeknownst to us well kind of decided to to divest themselves with the x-series group they sold it to Lenovo because their margins maybe perhaps weren't good enough was you know that Intel preservers were becoming too commoditized IBM just doesn't like doing commodity stuff you know they'd sold the disk drives now they sold they sold they sold the laptop guru to lap the a the think laptop still anova so now they sold the x-series service to Lenovo oh so that's what confused me because I know they saw the left yes that happened earlier that's nothing new slapped oh yeah that happened earlier this was now the server's and Intuit that IBM sold that whole group to Lenovo and to sweeten the deal they gave them our software as well our gnr GPFS native raid software to sweeten the deal so now we just given the software to Lenovo but that we needed a product so at that point that helped us introduce a Linux IBM power based product and we called that the elastic storage server and that was introduced in 2013 now we actually had to go through all IBM's process the procedures and VPS because it used IVM power line and the advantage to going with power is they have a worldwide distribution of spare components so field support was already in place I mean there are lots of rough edges to that because we were had many components it wasn't just one part number someone could call in on but we basically had the support the whole support of the Power Group and we helped revenues on their low end power servers we help augment their revenues and it's it's sold quite well and it's still selling to this day it's up to we're up to four petabyte systems when I left but maybe it's higher by now and we introduced both s solid-state disk versions and you know the disk versions so we had the industry's highest I ops rates you know per server 30 gigabytes per second throughput rates the fastest I ops and the lowest latency Oak Ridge National Laboratory we work closely with the labs on their next-generation supercomputers they announced the summit supercomputer with a 250 petabyte 2.5 terabyte per second ESS based storage system and Livermore they're still working on this announced Sierra with a 150 petabyte 1.5 terabyte per second ESS based system what timeframe was that that's just last year just last year so that's very recent so they're still working on bringing them up very high performance storage systems all running gpfs native raid on the market it's called IBM spectrum scale gpfs is kind of the geeky word for it so it's been selling well supported by my team in alman and actually iBM has awards they give up you hit a certain revenue number 100k per year I won't say the other numbers but up to up to a billion dollars per year and the intermediate one our our team got recognized for that revenue our small team of just five or six people because we you know obviously people in assemble the systems and stuff and manufacturing but we were supporting it in the field we were supporting customers there's like a start-up inside it so he weren't just a bunch of researchers on the hill we actually delivered a complete systems product and supported it so with the help of the group in Poughkeepsie you know the gpfs guys in Poughkeepsie with their help and their enthusiasm and a lot of respect because of people like Jim Wiley so was that the last time major thing you did you're now retired yes so yes so in May of last year with that that going well I retired and so 41 years in Silicon Valley that was 17 years at IBM the the group of us who worked on that are retired now many of the other guys retired as well at the same time and we get together every month that the group is called Romeo retired old man he do you know it all streets the incredible humor arise density over the gym Wylie yes so one thing that happened just to comment at IBM in the spring of 2013 arc decided just because of an investor's conference they needed more space and they divested and destroyed their library they had this large library of 5,000 pound scientific journals 20,000 books and they decided to get rid of it just to make room to have a conference and a physical library physical library books you know people just don't look at books much anymore but I we wanted to find homes for many of the so I contacted the Niels Bohr Institute and they had a worldwide distribution list of libraries and we were able to distribute about 5,000 bound journals to to other libraries Palo Alto mathematics Institute took all the bound issues of physic I mean the physics letters one thing that was interesting is that I was in the hopper as books were going being thrown into the recycle bin you know become intelligent toilet paper I guess how comes this bin of the Proceedings of the Royal Society of London from 1900 onward you know these books have been kept in Australia first that's like when the atom was discovered right and this is you know with the fruit and all this stuff is in it and I thought well someone might want that and so I looked at the list of you'd sent out and it wasn't analyst library didn't know they had it it wasn't in the inventory list so I ended up preserving its it wasn't thrown away so and a whole bunch of French journals were thrown away and I just watched that French movie about the French Revolution where they were decapitating the French elites we need to save those so that was kind of disheartening but but we were able to save him a lot of the world okay so that was was idea so I guess our next section will be on the Computer History Museum yourself yes so did you and you started long before you left IBM did you find retirement enable you to do a lot more now suddenly at the Computer History Museum or was it just that you just kept doing what you would do yeah more descriptive and doing I was really grateful was the right word for IBM for allowing me to actually start volunteering anything at the Christian Museum I started volunteering there first in 1998 but I'll get into the whole story but in their 2003 I basically started coming here every Wednesday and no one complained I mean it was a freedom that they I mean if I was at a start-up I never could have done yeah took time off he had more normal work hours at IBM maybe we still worked long hours but instead of being 24-hour days there maybe 10 hour days and and on Wednesdays I would work longer right if I was here I would just work into the evening I don't know if you remember it Xerox when I was working for you I used to stay up really late and I would wait till 11 o'clock to call my girlfriend reduce the phone bills so I was used to stay in a place that's funny no yeah so that that probably helped encourage your work here at the Museum yeah they didn't discourage it no one no one said you can't do that so and I was basically helping IBM's presence you know that they my p.m. doesn't have much of a West Coast presence and so so anyway so I when I started visiting a computers museum they had the Quonset huts at Moffett Field in about 1997 it was all in little Quonset huts and it was kind of a geeky dream you could wear like a fall Haleh computers where they go to die right and all this stuff was crammed in the Quonset huts and I started acquiring and donating small class old computers I found it idem 1130 I found at IBM 604 that was the two calculator I found like IBM trust memory units so I started donating stuff to the museum personally I enjoy I have some old classical computers I have a I have a collection of Dec computers I have an 18-bit PDP 15 in it and this is how I was gonna say you have your little museum home computing Museum at home yeah yeah 18 bit PDP 15 a 12 bit PDP ATM I had a PDB 12 for awhile and I had a HB 21 one 689 T 100 be calculator the original calculator used during the Apollo missions you know kind of had a xerox EDI Danny Lion itís unfortunate the Daniel and I gave the Danny Lions I gave back to Xerox PARC to Cathy Jarvis to her legacy lab but then the next year the marketing guys wanted the space shipped it all back east which has really didn't disappointed me because I hope to use that as a base to show it to people here so in 2003 I was part of the PDP one very early just for a few sessions of pd-1 restoration team which had been started by Mike Epona's and he asked if anyone he had noticed there I interrupted that sense he no he had noticed that that it was an IBM 1401 on German eBay and he said is anyone here willing to lead a project to restore it and so I raised my hand and said I just joined IBM alright a couple years before and I do nothing but IBM computer so I raised me and I said sure I'll do that because when I was younger I remember I didn't like business big business IBM was business you know do not fold spin mutilate you know I was into scientific computers you know Dec who's IBM I did nothing about their computers except maybe a 360 so I said sure I'll do that and then my next question is what's an IBM 1401 so then I look it up on the web I go holy it's a five-ton you know twelve kilowatt computer from the late 50s and so I was thinking what did I get myself into so I people told me they said you know there's an IBM retirement Club in the Bay Area why don't you put an ad in the IBM retirement newsletter so I did that I wrote a little ad that idea 1401 needs help you know please come help restore one and the next week this is in 2003 I guess or maybe early 2004 about a dozen retired IBM errs just show up here and they'd all worked on 1401 czar in manufacturing in San Jose and they were all interested to see if they could help and they all volunteered and many of them stayed with the project to this day so I add a team you know we divided it up into the I mean I'm a fortunately one isn't just a single thing it's like eight or nine things that need to be restored there's the I'll show a picture but you know there's the CPU there's the extended memory there's the card reader there's the printer than the for tape drives so you know it's like six automobiles and and a box full of 2000 brand circuit cards with 10,000 transistors so you know we had a lead on the CPU we got a lead on the restoration lead on the printer we had restoration lead on the tape Fry's restoration lead on the card reader and punch and and so we started but one thing we kind of noticed was that every week well the German machine came or IBM helped a IVM help pay for the shipping which is really nice you know I've never really had their act together we just said there's the computer and here we are and they did everything they went to the guy's house he'd started in his garage outdoor garage in Germany for 30 years his auto garage they painted up put it on a ship and it arrived in the loading dock here all paid for by IBM and after about you know we got it put together got the team working on it we continued to find failed cards in the cpu and like I said they were on over 2,000 cards SMS cards about this big I should have brought some and every week we would find Ron Williams would find one or two bad cards and one of the problems was solid state is supposed to last forever right transistors last forever yeah well it turns out that when they made germanium point contact diode they found that the glass adhered to the leads better if there was iron in the leads iron apparently adhered glass adheres to iron so guess what happens if you have iron leads in German moist summers for 30 years graph they rust so many of our diode leaves were rusted right at the junction point and many of the transistors the they put iron in the in the and to5 cans would either be rusted right at the leads are rusted on the cap Vicki you could push sometimes some of the transistors that they bent over the the leads had rusted through that was one of our common failure mechanisms and so I'm I was with two cards being discovered every week I was starting to think did I despite 10,000 rusted transistors yeah I think the cost for the system was about twenty thousand euros that's when I learned you should price in dollars because the Euro declined by the time we buy you know but I just pay you know twenty five thousand dollars for 10,000 rusted transistors there's a lot of corrosion in the mechanical parts as well we were able to borrow components from other pieces here at the Museum in storage swap is the right word I mean that's one area you could repair things you couldn't replace it with new parts well for mechanical yeah sometimes but transistors we had to have there was new old stock on the web so these are there was germanium transistors and we were and we had spare cards that we needed to we could always take them from the spirit cards but there was a new old stock we wanted to not the switch to silicon to keep it you know authentic germanium is a poor transistor technology it turns out you they never really did figure out how they work be honest so we had point-contact transistors I'm sorry the first transistor was a point-contact transistor you know done it Bill ABS this very second type of transistor was an alloy junction transistor you have a base and you melt on an emitter and rolled on a collector and have little wires that's the kind of transistor used in the IBM 1401 alloy Junction germanium transistor when they went when Bardeen and and yeah Ratan and Shockley was the manager you know they discovered by really experimental not by doing quantum physics that by putting little points on you know they were looking for a solid-state transistor but you can read all this in the history books by putting a little context on they're able to get one they didn't understand the surface states they really didn't understand why and it it turns out and it was always not reliable and the alloy Junction transistors are not allowed was a black art to make one and the problem was is germanium doesn't have a it's surface doesn't naturally oxidize at first they thought there was no oxidation maybe there's a tiny bit so contaminants get on the surface and they cause these charge pools which caused the transistors to do weird things either have low gain or leakage or loopy transistors which were a form of memristors where the IV curve is supposed to go up and over but our IV curves some on some transistors are loops so that when the voltage decreases the current doesn't go down the same line memory remembers where it's been so all these weird and they Bardeen they never figured out their whole conferences on how to get your medium transistors to work and they never figured them out I like to sit in so silicon came along and silicon naturally oxidizes and then you could even put traces on top and having a great circuits right that's what the fair shell did and all those guys noise and all that and so I'd like to say that if germanium were like silicon if it oxidized this place would be called Germany Valley not Silicon Valley because germanium is named after Germany [Laughter] you know Philip Philadelphia could have been Germany Valley if germanium transistors have stayed back east you know with the but the work being done in early transistors I probably would have thought of a different name though yeah Germany Valley does this but you know but silicon means Flint so this is really a Flint Valley Germany Billy Flynn family so anyway so now so so I'm thinking gosh we're finding a failed transistor couple field transistors or a weak you know we doomed so starting to get disillusioned by so by 2007 I was just driving along and got this call on my cell phone from Scott Bellflower who said his father had an IBM 1401 in the basement of his Connecticut home i Allah said what this man had turned his little tiny home in daring commitment into a data processing center he had key punches in the family room the kids old you know pictures in their pajamas with punch cars and he ran a business out of his home and he had a 1401 in the basement that he operated from operating 49 and operating 49 you used it from 1980 1980 s through 1995 actually late 1970s to 1995 95 yeah the most trailing edge store purgin computer we know of who were his clients yeah the internet was around in 1995 they were Westchester County golf courses you know bar tabs and golf scores and he would translate into the punch cards and supply so he had operated in that basement of his house you know the 512 kilowatt five-ton system in the basement of his house and and he had a single two three-phase power converter he turned it on the neighborhood lights would dim didn't tell his neighbors why you know it was kind of a secret non secret his wife divorced him at some point he said he didn't know why I think I knew why [Laughter] and you know maintenance became too hard after a while his friends were maintaining IBM stopped maintaining and so eventually shut it down at 1995 but during that entire time he had a dehumidifier in there but not since 1995 so I knew there was a chance that rust wouldn't be as bad so I flew out there I walk into his dungeon there's water on the floor I get oshit oh there's been a leak in the yard we said oh yeah okay but again you know it's below grade and so there was some rust in the back of the tape fries and stuff but but I said okay we'll go for it so in early 2008 of course it had to be on a rainy day we got a moving company he agreed to give it to you or well sell it and I unfortunately posted the price of the first one so he knew how much they asked for so we purchased it from how I raised some money purchase it from him and you know I had a new supply of cards well we had a whole system so we we had had to be a rainy day then I contracted with a local moving company where they called craters and what a job that was I have a whole movie on it but they've got you know how to get this five ton thing out of the basement and you know winches and they had to reopen an opening and it took all day and they load the load of the components on a flatbed the crane couldn't lift the two-ton CPU with the arm outs an example and the trees were in the way so it was quite the local TV station came out and filmed us we were on the evening news you know discovery in Connecticut this little sock salt box home and the shipping company a collister gave us a heavy discount on shipping it to noise ium arrived here we put it in the room I'll show a picture and I predicted that we knew the failure rate at SMS cards we had on the German system so I predicted there would be 26 failed cards and there were 25 filled cards so it was much more reliable and the amazing part about the story is that the very day in 2008 or a very weak you know every day that the Connecticut system arrived the German system started to work there are no more failed cards to rusted transistors so it was kind of like a sibling rivalry you know you're not going to answer me honestly the very week that the kinetic system arrived the German system started working but it probably was a random coincidence of the crossover point there yeah so yeah it did take me some work actually did convince the museum staff to get a second system they thought maybe we were getting well carried away but I I argued that if we were gonna have demos on a constant basis we need a two system so if one was down we could still give it down so the idea was to actually have two systems not well I was worried that this might become the main one but because the German system started to work I argued it both ways right but then with the German system also working and I said well hey we need to because - yes because that way if one's down we can rely on so we had we started we couldn't officially give demos at that time because the room was more of a lab not a public space didn't have two doors sometimes we didn't fight our friends in but it became a public space we modeled the room in 20 in 2013 2013 okay yeah camera my nose and I'll show pictures but from that point onward we keep it down with every Wednesday and Sunday except from the building power result once and except for just one exception a couple of months ago so be while one was down the other would always be working so we could have frustrating problems with the card reader let's say on the Connecticut machine or frustrating problems but the cartridge card reader usually gives us the most problems mechanical nature of it and then you go on for weeks or months but during that time we could do demos on the other system so that demo lab actually has two systems yes no you know two pieces yeah I wanted to show a picture so this is a picture of just won't fit on the camera it's it's what it's wide-angle because it needs to be wide angle so this is the lab before it was remodeled and on the left is the Connecticut 1401 on the right is the German one so that's the CPU that's at daelin acting like mo there over the top if they owns the Garrison Keillor computing history he has websites that contain so much information about computing history this George Aaron he was actually the 1401 design team in the late 1950s in in IBM and it's gonna find a funny story bill flora is not in this picture was his neighbor and before it was a member of my team for many years these guys would take walks together - finally they realized that that you know his walker partner was on the design team at the forty nine one and this is the the chamber under the famous chain printer that's Frank King better pictures of who was the lead on the chain printer Joe Preston on a tape drive and that's Ron Ron Williams who was the CPU lead and it was Ron's work we didn't have a car tester so you found bugs by just you know what's not working and and Ron he had the most volunteer hours of any volunteers museum ever because he came every day Don Luke and Leia and this amazing person here is a better picture on the next slide Bob Ericsson 92 years old when he passed away the smartest field engineer I ever met he had all the coolest jobs that I am he he worked at Los Alamos stir and they talked bomb testing when they had the IBM 701 there so he knew all the atomic bomb physicists so I had all these stories of what that was like and he worked in Washington DC during the decoding of the Japanese transmissions during World War two he was there when they decoded the transmission of the you know the Japanese generally shot down on the plane so he had amazing stories to tell and amazingly brilliant he restored from memory these unit record equipment this this these punch card unit record equipment 50,000 components each he restored them from memory there's an amazing individual he could restore watches even though he was over 90 years old he could his picture is featured in there so I one day I decided I needed to get everyone in white shirts and the old live in position because they used to do that right they used to sing company songs and all that kind of stuff in Endicott so I got everyone to put on a white shirt they had to narrow tie they had to rebuy their O ties yeah that's bill flora Jo Preston and some people aren't with us anymore Bob heritage is somebody I knew at Ivy its son and brilliant man he'd worked at IBM he did the he brought up the tape dry so he built a tape drive emulator out of analog circuitry on a PC so that we can actually emulate six tape tries we don't need to have working tape drives so that way we could debug the tape drive controller right so we knew we had a good tape drive which is broken right the tape driver the controller so his his tau emulator Hardware can emulate tape tries and we can use that to upload software into the system as well at thalen Ron began the Garrison Keillor of computing has hundreds of websites on the street community Ron Williams the 1401 Frank King a young kid from Berkeley had showed up for a while there's Bob Erickson 92 years old when he passed away there's me George Erin dawn Luke Alan Palmer tape drive Ron Crain Bayesian analog person for us on our team and amazing stories these guys did model rocketry at Black Rock Desert in Nevada on the side so they were building large Rockets all the time Stan panic so most unfortunately the old timers many of them or most of them have no past you know the only ones still with us or you got that wonderful opportunity of them helping ya or Alan Frank and Bill the others are now past quitting Ron yeah that's a wonderful picture yeah so that was now and then this is what the room looks like now this is the remodeling so they the German systems on the left and the Connecticut sisters on the right and one of this hand key punch is something that Frank King brought in it it actually is IBM model number one it's a hand key punch hand card key punch type 0:01 dates for the middle 1920s and people need to touch something right when they're in a space and so people come up and they you can put a card in and you push two keys at a time for a letter because remember the first punch card didn't have letters the first punch cards were just digits you know 0 to 9 and the letters were added as an afterthought you know the density of characters is so low on a punch card how many trees were killed because but you know the chain printers have been extremely reliable they leak hydraulic fluid a little bit 14a ones have been very reliable the biggest problems we've had is in the card reader punch units and the German one down here so that's that's it so you said your head demo is like twice a week four we still do business over here and you told me that that young kids were especially interested in this old technology yeah they're fascinated can you explain more well I think I think we just create an authentic experience you know it's like a time machine you walk in there and you lose track of the fact you feel like you're in some time warp well it's visceral even young kid it's visceral because it makes a lot of kids in particular love to sit on the key punches because you they make a big noise and they're commanding in fact any Vettel shines daughter and once had her in there for two or three hours at night she don't want to lead because she was like printing secret messages because they're holes right kids just love the key punches and they love the big sounds that Shane printer makes it looks like a t-rex dinosaur and that's a computer you know so parents you know enjoy experience I I know it sounds cliche but if you watch the demos people really enjoy them it's not just me projecting I mean I have you know it's just a man it sounds it's more visceral I mean sometimes the demo is interesting because you understand and like intellectually what they're doing especially people from that era certainly others it's just it's it's it's a fun experience yeah you know maybe the kids is more than latter than that yeah yeah so they way the demo works is that it's too many people have everyone keep on I should come in early so my recommendation was you choose one person from the audience and that person key punches they typically choose a woman because women typically did keep punching back then she goes up there and afraid you know their own projects into that person and then you take the names and people who've come earlier and then we print them on the printer so then we print out big block names of people when we run powers of two program two four eight sixteen which the chain printer gets louder we need more demo you can take what they punched out and show it on the screen on the printer no screens printer the printer we actually print it in a big block type if you need to come to one of our demos Wednesday three o'clock Saturday at 11:00 yeah yeah so then we print it out on that printer alright and that's what I mean it's the computer printer yes the 1403 a chain printer hundred lines per minute it's makes quite a racket it we we gave a system to what's called the tech works or the science technology innovation at Binghamton which is next to Endicott we gave them both we I mean I helped get them a fork IBM 1440 system and a 1403 printer and as a student project they built an interface between IBM PC in the 1403 and as an ANSI one slightly more advanced than ours and the students called characterized the 1403 is a cross between a machine gun and a typewriter because you know these hammers fire you know the page moves the ribbon is wide the Hammers fire and pushed the paper against the the rivet against the paper and the sound sound sounds like machine-gun yeah and we haven't done the computer music yet on chain penury computer music because we were afraid to break a chain we only have one spare chain but one of the engineers in Endicott the Binghamton facility worked on the 1403 and he figured out how to rebuild a chain so what we thought was impossible in fact the machine shop at IBM Albarn wouldn't take it on he actually built a fixed Rick was able to build a one which we're currently using it's it's this long string a little tiny 2 mil wires and all that the slugs are mounted to it so it's been fun so one of the things that the project did is it allowed me to really learn in depth the history of computing because IBM was very early in it right so I had a chance to over the years to talk with Fred Brooks exchanged emails with him you know mythical man-month in the 360 program Chuck Transcom who ran project manager 14 one program he ran basically all of IBM he's like the president of IBM for a long time learned a lot of history the predecessor to the IBM 1401 was called the world accounting machine it was actually started in the early 50s when IBM was losing business to machines Buhl in Europe and we need a more flexible counting machine they set up three projects but Kipps see Paris and Germany Paris one so the Parisians started to build it and it was they used a plug board instead to start program so it came back the United States Maurice Popov who was 92 and is still alive he ended up heading up that the the IBM research lab and de Gaulle and logoed next to nice later felt that was kind of their machine because friend underwear the designer used all the data paths from his machine in the world way accounting machine and just added an instruction set to it but no one they kind of half thanked him no one ever really thanked him for what became the world's most popular computer so when I called him in 1998 he said you're the first person in IBM has talked to me in 35 years and they'd built him a little scale model 14 and one so anyways a lot of fascinating history he's still with us so you say you told me that it had won a war and a war down yeah so in in 20 2014 tony salehoo who had restored a colossus decoding machine at the mint Bletchley Park if they're housed of Bletchley Park right now at the National Museum of computing they have an annual award for computer conservation so we won that award in 2014 recognize achievement in the area of computer conservation and restoration know I was in London so that gave me a great chance to to meet the British Conservation Society people and go out to Bletchley Park said you think I'm the museum we'll have a do another Prado like that just before I go on the next one and in May 2017 that same museum the National Museum of computing at Bletchley Park we had a conference on restoring computers and I gave a presentation on the fort wrote a paper and give a presentation on the 1401 in 27 yeah okay so one last story it was fun on the fourteen and one is in in 2008 we realized that 2009 was going to be the 50th anniversary of the 1401 so we you know we spent all this time getting the system work and I was thinking well where are where's the architect where's the designer you know this was the world's most popular computer in the mid 1960s and when I heard we two computers was a 14 I won so where's the dude and so we knew his name Trent Underwood and you'd google it and nothing would show up not beyond I think something you know maybe people wondering Boise and IIIi eventually found a hit on a music website where it said Fran Underwood is my father and I bought this CD for him who was the designer of the 1401 and the title the CD is IBM 14 I wanted a user's manual that's a piece of music I thought this is weird so I know he's still alive the review was written a year ago and what's this thing called IBM 1401 user's manual so it turns out that a young urban guard European composer Johann Johansen and around year 2000 finds this real real tape in his father's closet and it's of the funeral ceremony of their IBM 1401 in Iceland the 1401 like in many places was the first computer store broken computer in Iceland and they liked it so much they had a funeral ceremony they recorded on the tape and on the tape if you put an AM radio on top of a 1401 or other computers of that era and you run a program which tries to present musical sounds like a melody it gets broadcast through electromagnetic radiation there were no FCC requirements back then into the radio and it sounds ethereal and mysterious and so they had encoded a famous Icelandic classical music composer and they played that through the 1401 through the AM radio and a set of mysterious and weird so he was so inspired by that he wrote a symphony to the 1401 so it's the only symphony I know of written by a human written by a human not computer jaren to be performed by humans right not computer synthesized and dedicated to a computer and if you go look on YouTube there's even a modern our dance piece that goes with this woman writes on stage looks like she swallows a fortunate one it's meant to be birth life and death and it's been it was performed as part of Johan Johan syns road shows throughout Europe probably a hundred times and he did tours of the United States and I would I would go to one I went to one of his his shows in San Francisco and the young guys young kids are in the audience it's kind of grunge you know he have movies of old black-and-white industrial processes and retro and grunge is kind of cool these days and and I'd said we have one of those in the kids because we're so old in the audience the kids would go really so good have you two been fine yes it's all yeah and the music is on iTunes I mean the symphony is out there well that 1401 it's cold IBM 1401 the user's manual odd name and he got that name because a favorite very famous famous book called life a user's manual now by a French author but it is a conflicting name because and and and the movements are called 1401 1402 1403 and 729 in fact one day I was driving actually to Hollister which is where my father grew up and on National Public Radio Michael Berman on All Things Considered because now we're going to talk to the composer of Ivan 14:1 a user's manual I go what I'm driving down the road this is way too much nostalgia so there's a five-minute interview where he's talking about the importance of old computers and preserving them and the beauty of this symphony okay this is off the charts so you know I met him he came here later he wanted to extend his movie you want the extensions extend a symphony we wanted to do a movie based on the 1401 Morrison was going to do it we were trying to get funding for it but then okay they said did you see the movie to theory of everything about Stephen Hawking's wife I did then he wrote the music to them hey that kind of ice cream is gonna just see the movie arrival you know the thing he wrote the music to this very powerful music huh Johan Johansson Johan Johansson so we're I'm all excited like let's go and let's try to find funding but then he died of a drug overdose last year as bad so anyway so we have to back up so so I know that friend Underwood is alive because he his daughter bought him this CD it's not the father and the father has heard her father so I look on people finding websites and I and I there's this person named friend Underwood in Texas and about the right age I figured so okay so I tried to find information and there's nothing about I try to contact the house no answer is just nothing so I go okay I'll try the neighbors this is where year one has you know to maker lots in the middle of nowhere Lockwood Texas and I called a neighbor and I said they answered and I said is your do you have a neighbor by they have a friend Underwood and I hear some of the background say don't answer the question it's a bill collector I said no no I'm calling from the museum we're trying to find this man cuz he was the architect of the world's most popular computer in 1960s and so they said can you put a sticker on their door to have him call me and sure enough two weeks later I get this call from the daughter and he's on the line listening and so she called me back and so I kind of had to befriend him he wasn't sure why I was interested and she had worked for IBM moved to Lockhart tried to set up a Java cafe but Lockhart is strictly a cowboy country it didn't do well you know they had to rent out their house and so I I went and did an oral history over in his in his house and he has an amazing story friend Underwood he started as a mechanical engineer he wanted to be a design engineer if you were hired at IBM as a design engineer back in the early 1950s you know what they had to do they actually go to the assembly line and assemble three unit record equipment 50,000 components each by hand and then he learned logic design from Claude Shannon's first student and then he worked with this this guy named von Neumann in the advanced development lab ASD D and IB indica so you really brought himself up and they took over the worldwide accounting machine and gave it an instruction set maybe start a beauty my old it was he he was up there late eighties surveys and he had gotten a stroke some years before so his voice wasn't as well as you'd hoped I said your story is so amazing we're going to do a fiftieth anniversary of the 1401 you know and next year in 2009 you have to come if you come you'll get a standing ovation and he said I can't travel you're not blah blah blah so then he said well you need to contact Fran Fran Scone Chuck Brandt's Co and and he was the project manager so I call him and he goes I was waiting for your call he said I knew we were restoring a 14 oh well and then he said well you need to get the marketing guy he's in Saratoga California so I got him easy and now she'll Jacobs and so I had the three key people who just bracha ties this project was called space after at the time because it was during the era smoothing it correct so everything was about space it stood for something and and so we I worked on a presentation they gave a presentation the IBM John Iwata director of IBM communications and marketing sponsored and came he said these are like the three NASA astronauts of IBM because they gave an amazing it's on youtube if you want to see it and people have told me at the Museum was the best evening they've ever experienced in terms of the history of it and not a future looking event 500 people showed up and and Fran got a standing ovation and I people like the guy who was the architect of the of the Alpha dick one day I was in the 1401 lab and there's this guy cleaning her floor and it was he brought his son and tried to get a summer job it was dick the architect of the eye of the the deck alphachip who I didn't directly competed with his son I said he and he came to the 1401 event because he he loved the 1401 day it was this he thought it was a cool machine you know it's amazing there were people who had me to know the 1401 it's just remarkable but I would never think they would know it I you know the the the man who did small talk what's his name he guess what his famous favorite computer it's the 14 because it was his first computer at an Air Force Base around has a nostalgic reasons yeah Dave Liddell it was his first computer yeah so anyways it's a bit just amazing well I think it's an amazing project and you've given the world a wonderful legacy yes yeah yeah because yeah it helps realize they're gonna be forgotten well and it just it just represents that era when computers were kind of human-sized and they weren't even our room size room sized and they weren't they were intimidating because people overhyped them but they're not is today they seem to be tracking devices and personal invasion devices right there's nobody not even operating system on a 14 and one it wasn't hooked to a network right so it was kind of a tame animal and the company's you know the face of him were punched cards which we're kind of dehumanizing but little people realize what was coming yeah so yeah it's been it's been a lot of fun and and by the way one of the key aspects about the project is to get younger people involved because otherwise we can't keep it going so even though much of my team is now passed we now have six younger people on the team so they're keeping it in their 50s and 60s that's good including one guy in 40s no two people in their 40s or maybe 30s so and they're enthusiastic about it you have to kind of find people who who either retired or are willing to come in yeah work and talk about it yeah so it's still so I think we can keep it going for a long time all right so perhaps in the last segment if you could reflect on your very busy and long career you know what you say you have a number of lessons learned I mean if that's what you want to focus on or yeah be fun well you know I mean first of all I guess I'm most proud and grateful to have had a chance to work at some of these firms right to be in at least two places that were instrumental in changing the direction of the market you know the Xerox star and Sun SPARC projects actually represented even transitioning computing from kind of assist computing to risk computing right if that I mean to have the honor to be able to design your your own computer I mean a computer of your design I mean you work with people or it's not a single yeah a single person doing it it also a revolutionary design yeah you know together with the software and everything else very different from before yeah I mean I was just very grateful for having that opportunity to and to be on the ground floor the Ethernet you know that was they'll be able to work with someone like about Bob Metcalfe just you know and it's on and be able to work with people like Bill joy and Andy Bechtel shine just amazing and and of course it Xerox PARC even to get to know people like Chuck Thacker and Butler Lampson and Alan Kay by the way it was the name of that person who's trying to think of that I'm talking about talking yeah you know what so I I just feel really grateful for all that any we're talking about people I for some weird chance or luck I you know the CHM the Computer History Museum has fellows right and I actually worked directly with five with five of the nine fellows Bob Metcalfe Chuck back Arlen Conway bill joy and Dave Patterson directly with them not just know them and I interacted with seven others so I'm really grateful for that opportunity I would do it all over again it was just so so exciting and so wonderful so yeah some of my some of my cheap advice that I learned over the years was one of the key things I learned I think this started on the xeroxed our project was to truly listen to people I mean I thought engineers are not very good at that you know we especially in a competitive environment feel like you have to have the answer right away and and you have to almost in in meetings or in your talking to people and you know I told you to feel joy I would talk so fast he would butt down belittle people by a speed of talking but you have to at some point during the day yep sit down stop seek people out and listen to them because what I found was is that typically especially if you're confused you don't know what exactly what to do next the answer to the conundrum is actually in the organization somebody actually has a reasonable answer just they're not speaking it or they're not typing it an email message and you have to just go sit with the person talk to them and and give them time and and the answers or directions to go in are they're you know they don't need to be found outside necessarily or in a heated meeting where one's wasting their time that's people tend to clam up and be like yeah so I just got to sit down and listen to the people the answer is usually working there somewhere waiting to be uncovered the other key thing is it's better I have found it's always there's two kind of manager Styles one were you don't trust people and you treat them like you know mushrooms in a closet I don't think that works at all but some people have that's because it keeps you defensive and rigid and what I might've been wrong I better think about it you know there's that style and the era start where you trust people you know after all they wouldn't got degrees and you know they're probably pretty good people they may have other stuff going on but it's important to trust engineers first trust them first so it's better to trust people first and then ask questions later it's better to get to the bottom of concerns if someone has a concern don't just let it ride really find out would it and get to the bottom of it and and don't be frightened by what it seems to be a can of worms if a problem is so complex don't don't let that scare you because it's going to bite you if you've ignore it it's going to bite you and you know so you just can't ignore it you have to I mean some of the best managers I know of do them and they do it better than I do you know typically these startup companies would be we have problems surface on a Friday afternoon you know all you can do is we got to come in on the weekend guys you know but you have to do that you can't kick the can down the road so one of my other things I learned is successful organizations it seems to me it's best if they can have a balance of people that are kind of Loony free thinkers and conservative routine personalities you know Park was that way son was that way you know so I think it's you know other companies I know over that way so it's really important to do that it's also important not to hire too many uber egos because then all they do is they fight you just get catfight suit just one or two at most and then simplicity is always better and we always say that but creeping feature ISM it really infects companies because everyone wants to contribute in some way or they feel like they're not being worthwhile but you just have to resist that you have to resist you know perfection is always an enemy of the good and sometimes there's me a successful and unsuccessful endeavor is is you don't want to try to boil the ocean all the time if you try to boil the ocean it's likely it's not going to work so Oh one of the key things that I learned is that my definite my weighted our criteria for a really good architect the system architect is someone who understands their own intellectual limitation so that if they're really brilliant you know they can push the envelope and make a design complex even more complex you know they barely understand it but a really good architect steps back and doesn't design it he or she's limit because that way they can leave room to debug the problem right if you make it a little simple for yourself when it doesn't work because it's not going to work there's to be some case where something fails in a way that can't can't possibly fail you know if you leave some intellectual capacity there you can then debug the problem I mean I've known architects to design something so complex that when this thing's misbehaving and getting errors they just throw up their arms and go I can't debug it it's just too complex because they design it to limit of their intellectual capability so you have to step back from it and we leave room that was and so like I said I can count on one hand the number of architects was known to do that know a version of that was Dave Patterson used to have this maxim which was projects can only draw from a limited number of cleverness beings you only get a bowl of a certain number of cleverness beans and you can't you can't use more and then an old four version of that was the KISS principle keep it simple stupid but I noticed people tended to say that didn't tend to keep it simple Chuck Thacker was an example of that he would say that but then I thinks weren't that simple oh and also an aspect of good architects is they're willing to really do the hard work not just saying this is the design go implement it but attend all the design reviews and all that I mean problems and sit and listen to people you know really generous and gregarious and their ability to listen to all the problems it's also seeing the whole thought through to the conclusion rather than just head throwing it over the family yes yes well they're getting the feedback that way by listening to every man yeah exactly if you just throw it over the fence you're doomed yeah there's a there's a quote about this in the system's Bible played by James gall which is in dealing with large systems the striving for perfection is a serious imperfection you know one thing that's kind of funny I learned especially as son is that you kind of had this feeling that if there was a weekend between a problem and a Monday you could just get it solved over the weekend but typically what happened is on Monday the requirements would change whenever you did over the weekend became unnecessary so don't get suckered into thinking that because you have time to work on something you're doing the right thing it takes a while to learn that one Oh a funny one is it seems to be true though both it when you're bringing up a new computer it's so much joy and fun to design a computer build it and have it work I mean it is such a trip I mean because you don't know when you turn it on of course you've simulated and stuff you just made these days and you back then you didn't simulate it but you don't always gonna work but but then of course problems set in and what I noticed there what seems to be true is computers act up and falter after midnight it seems like they always get flaky after midnight and the other thing is you should never complain never compliment a computer while standing next to it because it will always screw up right after that one thing I've learned about software and a lot of people think that you can eliminate all the bugs and software but I'm absolutely convinced that's impossible software will always have bugs it's just that the rate of occurrence of the bugs the interval between bugs stretches out long enough such that it goes beyond the shelf life of the software package but they're all you don't see it you see you don't see it because of software packages end of life so they're just always be bugs in software and so therefore bugs are always good in the sense you want to find him as soon as possible oh so in my restoration work I kind of came up with a funny inverse to the normal law and the normal IT world in the normal computer world there's an idea you have to rush to get it out as soon as possible if you're if you're late someone else will beat you the market the technology changes so fast time to market is really critical well in restoring old computers or preserving old computers the older it gets the better it is it's so it's the exact inverse of the normal thing where the if you're late it's it's defunct but now the older so if you're in if you're restoring something it's important not to rush into it it's important to hold back there's no hard because actually your labor has even more value but it was now so it's really important to uh to move slowly and to realize this inverse law that the older it gets the better it becomes it's kind of excuse to postpone doing things so those are so those are the main so that the main thing is I'm just grateful to have a chance to be here in Silicon Valley when I was I think there's a valuable remarks for you young engineers who are starting their careers anything else you'd want to tell us these new engineers well just always you know always buck the system don't don't try to become a conformist right it's always better to I think to be a rabble rouser and and that's one thing that maybe it's a West Coast you know cowboy thing it's an American thing but it's always better to if you can buck this because things change so fast you have to buck the system because someone else will if you don't all right any other parting shots no thank you very much Rory for doing this with me well thank you I I really personally appreciate all your thoughts and the stories both in this session and the previous session and many many generations of people will enjoy them through that are recording maybe that's not thanks again I'm looking we're going to do this already

Channel: Computer History Museum

Views: 578

Keywords:

Id: abiaI4ED0D4

Length: 166min 25sec (9985 seconds)

Published: Sat May 04 2024