The Philosophy Behind MinIO with AB Periasamy

hi this is a bee a bee Anand Babu piercer me just did you my name so many no manao they're all fine they're all right I used to call me neo and then what happened was the community overwhelmingly started calling it men I will and then I asked them for a reason and they told that Mineo sounds small mini Oh sounds minimalist and they they understood our design philosophy from the very beginning they grew with us right they actually helped us grow and that was a very valid reason and when once we heard that we said okay we will make that official and be rebranded if you see the reflection on the logo right the birthmark last year we did that to make it official then you have a large community they have the power to overwhelm you right and there this is just community power now they you may even see me mistakenly like calling it mini oh and you will see that we our own our own team will call it mini I mean I they are all fine they are all right it really it set up what matters is the search engine sees that as a same spelling and they get to us anyway like so so what is mean I mean IO is an object storage it's a software it's a software only product and it's meant to do one thing one thing right and a to be an object storage yeah it's actually a complete alternative to Amazon s3 it's an Amazon s3 compatible object storage it's high performance and few things we will go into those details but what it is is it is a complete alternative to public cloud storage in specific if you compare Amazon s3 for public cloud mean I wish for the private cloud and we wrote it in a way that it has no dependency on anything else not even Amazon SDKs and tools and select the server component is literally a 44 megabyte static binary and all of the functionalities you want is just in that one static binary download and run it runs on a variety of environment like people running us on 5g towers on ARM chip the runs inside like Mellanox blue field type smart next to two smart store pretty much like a variety of environment power 8 power 9 x86 from a OS X laptop to a high-end Linux server right and a it's a highly portable high performance object storage hey MC the mini o client it looks deceivingly simple just as a tool but it's actually the reason why we wrote that tool is well applications use SDK to talk to the object storage server the UPS guys they actually write a whole bunch of automation code and they needed to automate everything right and the problem that they had was the traditional unix-like tools did not work and you and you need to be able to manage massive amounts of data and do all the automation around it and MC gives you a complete unique scoria tools like like LS CP are saying whole bunch of these tools but more modern more powerful it even works for a regular file system it a and then comes the SDK why did we even write our own st cave and Amazon SDKs are open source when we looked at the SDK early on the API is where auto-generated they were not idea Matic it was blotted with a whole bunch of other services that amazon has launched over the years and we needed something clean simple and idea Matic when a developer touches the SDK end of the day what we care is when you put a lot of effort on the server and if your application is talking to your server and they and the APS are poorly designed that's the experience they will remember right and we needed to actually put a lot of effort on the SDK it's a lot of work while it's not given that much credit it was an important thing to us even design wise right like apart from being idea matically in light-weight at that experience like Amazon SDKs would send objects in 5 MB chunks 5 MB is really small but they needed to do it because the applications will be sending this data over of 2g to like Wi-Fi they will flip flip over and they may lose data on an unreliable network so they needed to do small amount of data but we were actually seeing our adoption growing on high-end like deep learning environment mission-critical environment where they were running us on 100 Gigabit Ethernet you don't want to be chatty five and be chunks it would actually optimize itself automatically to 128 MB chunks so there are many reasons why we need the private cloud while it looks like public cloud is actually quite a different beast and we needed to actually put our focus on private cloud and give them that class of object storage with no dependency on anybody else and that's what it is and why did why did we even get into this object sorry you mentioned private quite well actually also edge applications I mean not just private cloud yeah it's the concept is getting there yes it'll be different literally I so the we are now like are when someone started doing the community centers like like a 8 node last bill actually pine 64 cluster right they were just excited about what we were doing and they just donated yes this was like in the first or second year so they will so that we will maintain arm support I was like okay that's hackers on hobbies like why will anyone run honest on a Raspberry Pi right but this is fine sixty-four Raspberry Pi is 32-bit arm it we we chose to be there everywhere it whether or not we'll make money it doesn't matter we need to be there everywhere that was the goal right and we supported it and but eventually we see that actually turning out to 5g Towers edge applications it actually happened at a later but I couldn't anticipate when we started then later on we even added more support will go into those details later that at the edge they don't want that to be a full-blown storage system because you could entirely lose that truck or whatever edge tower right and you should not lose any data they want the edge to be a edge caching CDN like storage but it's a full-blown object storage never has support for even that so yes so why did we do this now it looks so obvious particularly last year which transition happened in the industry that is private cloud finally started emerging right but when we started the first question like typical investors are other technologies would ask are you going to run this inside AWS I'm like sure it can run but it's of no use right so why would you run this inside AWS on EBS when you already have Amazon s3 it is designed to actually solve the problem when you're outside Amazon s3 they would laugh at this at that time because they thought that AWS will be the world's data center there will be one public cloud vendor and nobody else right if they industry believe that was true five years before but our bet was that it it is possible Amazon cannot be taken lightly they can do lot of things right and they are doing it right and but for us we can do one thing better than Amazon and our goal was let s3 win AWS but we will win the rest of the world bulk of the data that world will generate it's hard for me to imagine it will all be sitting on Amazon s3 so my point was bulk of the data world will generate will actually be outside Amazon s3 and that's the market we need to win and thanks for Amazon to opening up the market right that the market outside of us what salsa & nass the enterprise market they didn't get it so our focus was we knew that the world will produce an incredible amount of data he knew that ten years from now this problem will actually only compound and grow bigger and bigger if you built a partial brand it's here to stay and everything that you see in the market today like meaning back in five years they were all enterprise Sandnes type storage systems they had no understanding of what was about to come and that was our opportunity right that's why we got into that market and our thing is if we don't want to be just yet another object storage this is something we heard commonly when we in the first couple of years we thought it was a solved problem why at another object storage and if it was just at another object storage that I can do this better we had we had really no good reason to do just being marginally better it's actually a brutal market to be in a market where there are many players yeah and one is like slightly or slightly better over the others that was not the idea at all I actually saw that the opportunity was that everybody else out there essentially came up with the hardware appliance are they took a traditional file system design a really textbook theory and then added s3 api on top they did not understand this market fundamentally right what I saw was this is actually a complex problem it looks like complex but the fundamental thing was that it was not complex at all I saw that this was an easy problem when you find a easy solution to a complex problem that's when you can build a real business for me that we saw that this was an opportunity for us that everybody thinks this is hard but it's actually not hard if you if you have tried Mineo before you will see that mean I fundamentally is a web server it's really a web server with all the storage functions like a racial code bit rot encryption whatever they are simply web handlers it runs as the heart of manejo is a web server if that's the case it's real and and it's also stateless web server that's why a distributed mean I way you can crash you didn't lose any data and it's it's it's simple for me now I know that that because it is so simple I can grow fast I can take over the market and for me the important thing is that this is a market that will consolidate right this is a winner-take-all we knew that if we if you are marginally better you still go home with nothing and we saw that everyone thinks that it is a hard problem we saw it was a simple problem and we are open source right they can just copy our cord see everything but they can't why are they not copying our cord you will see by now many of these storage vendors are actually using our product inside their product but the problems they fundamentally have is the mindset issue that was for me an advantage every day if you talk to us a pile system guy they always think that everything is a file system problem they just somehow the storage engineers the developers behind these storage systems they like to do complex things they take great pride in building kernel systems they will for them they look down upon user space programs user space engineers they look down upon application developers they don't understand that the application developers list right the traditional enterprise IT a bookseller Amazon turned out to be a better infrastructure player because they understood the problems Google or Facebook they like from what I have to name it the application developers understood the problems and scale why because they had a different mindset that was the advantage we had right and we knew that when when we mature we will look nothing like any other object storage vendors and if we stick to our focus when we started we last three to five years right community will ask why not swiftie api why why not like NFS can you add enough as customers ask all the time I will tell them I can add enough s I can add file systems file system API we actually know how to build a file system because we built bluster but I let true this if I add a file system API to Minoru I will give you a media cur object storage and a terrible file system do you want that write the code the goal of my name is I will give you one thing really really well the strength for us is it's not like we are smarter engineers than others everybody has access to the same talent pool the strength we have is focus if you if you focus on just one thing even Swift API I asked the community pick one Zeus KPI or s3 API it's very easy to answer right they said s3 API then you have the answer right that focus that relentless focus of the minimalism that minimalism for us is not less or more it's actually a perfect quantity of something if you remove something if it becomes incomplete put it back if you add something if it affects us we remove it we always tell even the community when they submit improvements if you remove something your patches will get through faster then you added a new feature when you say that it's a web server is it like based on Apache like what kind of web server is it ok so that's a good question right it's the so the first why is it a web server look at the s3 API they have simply rest EAP is right and if you hire the average developer today to ask you write a web application ask this Prevage web developers and even an intern you know PHP or node.js and ask this ask this girl can you write me a simple drop box like interface right you know how to solve this problem it's simply they will have a file upload or package and they know how to do this what they will do take a standard web server and then write these web handlers and their application will go live next day right that's pretty much what happened at Amazon they understood that this is not nothing to do with Fibre Channel over Ethernet or any of these this is a web storage is a web service problem now what web server we are using web server itself has become a commodity every programming language has a package httpd package you just simply import it well so yeah in minerals case it's actually not Apache it's actually go language it's written in go lang is simply a go like a CD PD server so they that essentially gave us the advantage that market thinks that this is a very different problem and we actually think that it's a simple problem and if I average node node.js developer actually knows how to build application storage infrastructure and they find that this is easier they find that this is easier than Redis or a database why should object storage be part of kernel or some managed by some IT or somebody else most of our deployments are just managed by users themselves until it matures they start asking IT I want 10 petabytes data and I storage the an ideas why and that's when they realize that they are actually running Minoru right but we that simplicity and understanding of the problem with a very different view that that was the advantage we had are you gaining money from the service contracts or I mean how do you monetize this environment it is open source that's actually the one one of the most common questions right we we there are slides that Jonathan will address address on them but a what but Jonathan will go into details of how we actually make money but fundamentally right the reasons to address on the open source part if you see that a there are many open source startups they will say they are open source but then when you actually try it hey that feature is proprietary right to me if I am even proprietary it's better for me to like I'm better than open core because I never mislead right the brand is the most important thing that customers they even expect when they talk to a proprietary software vendor they exactly what they are getting if I told you I am open-source and I'm partially open source I'm kind of misleading right when I was case is hundred versus open-source and why that doesn't hurt the business we actually not about hurting the business we see that it helps our business but what we see is customers pay you because you solve a business problem and they believe in your brand that you are the best brand that they can put their trust on and it has nothing to do with the access to the source code actually that idea of holding customers hostage is the license key I think those days are over right today support they can they trust that you are the brand that they want to put their business trust upon because data is the heart of all modern enterprise and for them is hugely risky to to take some proprietary vendor even the large enterprise like storage vendors I don't want to name anyone but you will see even they have killed and hydrated their products multiple times startup are not customers see that open source is the best trust that they can have and also what is the best escrow github right the world has changed and open-source has turned out to be a huge advantage for us and if we believe in it like completely it should be completely open-source and we are

Channel: Tech Field Day

Views: 2,951

Rating: 4.8805971 out of 5

Keywords:

Id: k5i6Iw_BcH8

Length: 17min 3sec (1023 seconds)

Published: Sun Jan 26 2020