What's a Modern Data Platform, Anyway?

[Music] hi everyone i'm gaurav dillon founder and ceo of snaplogic and i'm pleased to be with you on our virtual keynote for big data london uh the topic of my keynote is around modern data platforms what is a modern data platform anyhow these days we have so many different definitions and so much has evolved and what i want to do is i want to share some experiences from the field from actually building lakes warehouses and marts and share with you our point of view on this topic well so so this is indeed the topic of the talk and now let me jump into some vantage points that i feel i have the privilege to share so you know snaplogic is a integration company we connect the dots for enterprises across the globe we have hundreds of clients in every different vertical industry from airlines through banking pharmaceuticals et cetera et cetera and as a result i have the privilege of having some unique information about the kind of challenges people are facing and the kind of things they're trying to do uh also by way of background i've been in the data business for decades you know prior to this i built informatica as a co-founder and chief executive for 12 plus years produced a public company that did a really good job of etl and so i've been in this business been around the business of connecting the dots for a long long time and i thought it'd be fun to share some perspectives um so what people want to do generally when you think about sort of many of the drivers around the the breathless hype that we have about uh digital transformation many of those drivers have to do with people wanting to do things faster they want to deliver products and services faster they want to provide better user experiences to their customers and they want to achieve amazing business results in other words what every chief executive wants is tomorrow's revenue with yesterday's expenses and as you can imagine in this day and age with the pandemic nobody is on the same operating plan that they were at the beginning of this year whatever they're doing it is somewhat different than what they started starting the year with and a lot of the challenges behind getting something that we think should be you know it should be a real real layup as we say in the united states or we should have it sorted as they say in england and the real challenges behind this are to some extent you know worrisome and to a large degree universal across the united states and the united kingdom so we did a survey with vansonborn to reach out to 500 id decision makers and these 500 decision makers came back with some information that to us was a startling for the state of the union in 2020 and b showed where some of the reasons why a digital transformation is so hard to do and pointed out to some of the opportunities to go fix those things and in fact perform those outcomes that everybody wants to see across every corner of the globe and every vertical industry so the 500 people who were bold 83 are not satisfied with the state of data warehousing and the performance and output of the data warehousing today even worse eighty-nine percent of it decision makers are worried that data silos are holding them back and in fact this seems to be the main stumbling block to moving data around to orchestrating things and getting those business outcomes that we so want to have and shouldn't we be able to have them in the year 2020 you know the cloud to some extent has been a wonderful thing but it also has created some under unintended consequences of creating a lot of business silos the cloud on the one hand has given us the ability to create capabilities you know we don't have to go out and buy servers and got you know heaven forbid air conditioning and all those kind of things to set up a new capability to manage a customer to bring on email to be able to serve a customer better through customer service et cetera we can set up these capabilities it's much the same way that we can consumer capabilities and and that has been a good thing but at the same time massive amounts of cloud and sas applications have come into the enterprise on top of what already existed in times prior and that combination is creating silos is creating just this new kind of mess if you would you know on the average companies have about 115 applications according to some of the research that we've commissioned but bigger ones have more 400 500 applications are not uncommon and these days sas apps websites as a business user might call them are not just limited to the id department or the finance department they're also in the marketing department they're also in customer service customer success and all across the company all the way to manufacturing with the internet of things and and this is this siloing of business in an unintended consequence is to our minds a very large problem we think there's needs to be a new way of looking at the problem from above uh the image that you have uh in front of you is of the dems flood control barrier uh a feat of engineering uh a combination of physics and you know metrology and other kinds of things that have been put together but to our mind a powerful graphic that we can if we harness our thinking and go about it the right way we can control these things you know i have so many fond memories of london one of which as i look at this is also of taking a cab ride and it was very slow in the cab and the cabbie turned around and said to me did you know that the traffic in london is the same speed today about 20 miles an hour as it was in victorian times and it stands to reason so so so it's true that all these cars are causing slow traffic in london and all those silos in the enterprise are causing jams and are creating an inability to orchestrate data to orchestrate services to achieve the kind of business outcomes that we'd like to see and we feel that there are a number of attributes that you should demand in today's modern data platform and you should be able to get them you know this will build out but let me talk about them each to each and share our thoughts about this with you we feel that enterprises need to embrace the cloud we don't live in an age where we worry about the actual physics of our cars you don't have to be a mechanic to drive a car you don't have to be an aircraft engineer to be a pilot today unfortunately the state of data is very much tied to the physical attributes of the underlying platform we need to get out of the hadoop zoo we need to get out of the worries about this format parque or that format we need to get out of edge nodes and those kind of processing we need to move towards the cloud and get away from the physical attributes i think that has been a big issue in how people have traditionally thought about big data in particular we need to go beyond batch you know it's not just etl anymore very truly it's a combination of elt but not just that there's many modalities you can move things in real time through an api call you can move things in batch there's certain data that batch is perfectly well suited for financial data in particular seems to be fine the reporting cycles are not instant but customer information the kind of things that people are doing on your website you want to have that happen in real time api calls and now event streaming with with new technologies like pulsar and kafka and others that connect these things together we need to go beyond batch for sure we need to think about data science as a first class customer of ours we need to think about an algorithm being as important a consumer of data as a analyst might have been in the last decade it's not just about reporting it's about algorithms giving them basis data sets with which to feed the magic of data working on data and new kinds of capabilities that can't be created any other way we need to think hard about deploying domain models we need to think about the same kind of domains whether we come into it from the production of data side or the consumption of data side we need to think about domains like what is the underlying customer definition what is the underlying data structure for a employee what is the underlying data structure for a partner so we need to think of these domains and try to unify as best as we can the domains and use one end or the other to try to push them together speaking of pushing the modern data platform cannot succeed on its own it needs to push its neighbors application developers around the domain idea apis and services to share models and to think about a modern data platform as a first-class citizen when they think about building a new application in much the same way as almost anything that you have that is manufactured you might be looking at this on a computer guess what the design of computers has changed radically because design now has to cater to assembly that is done with automation with robots so industrial design manufacturing has all had to change in that same way the modern data platform has to change and it has to push all the constituents around it to think about the modern data platform as a first class citizen as a partner at the table we need to employ a federated approach you know you can't if you just have a centralized approach then you get a backlash the business users are held back you don't get results if you try to get too much into people do their own thing you get sprawl you get again in these days of heightened awareness of data privacy uh data security um governance and compliance we simply can't have sort of an ad hoc world where people can go out and you know engage with data so a federated approach where there is a a way to have somebody authorize it where people can move very quickly but there are checks and balances in place in which some of the models in which some of the domains in which some of the um stewardship is jointly managed between business and id is very very important and last and perhaps most important we need to think of self-service a millennial is a computer-savvy person these are not people who are new to computers these are not luddites yeah the they were born in the age of the iphone uh and these are very capable people they're conversant they're able to engage with technology and we need to allow them to through automation through self-service we need to cater to multiple personas this has to go beyond id this has to even go beyond shadow id or line of business id this has to go towards self-service again with checks and balances and ways in which we can make that happen and and we feel and perhaps this is uh this is my wrap up here is so what is it modern data platform a modern data platform is something that brings back the simplicity of what we knew at the dawn of data warehousing where we had an organizing principle around which to manage data we need to bring that simplicity back and get away from the physical complexity you know nobody buys a data warehouse you assemble one well you can build one from scratch and much of the way you can build furniture or something but there's not a modern data platform for an organization that you can write to check and buy you assemble modern data platform using the best practices and technologies that are available to you at that time but in doing that we have to be mindful of my favorite steve jobs quote and that one that we have on the engineering wall in our company steve jobs famously said simple is hard complex is easy and you know apple's might in the technology universe the capitalization is a testament to how they have simplified the way in which we interact with devices all the way from macs to iphones and other kinds of things and you know i would be remiss if i didn't point out to this week's snowflake ipo and what a wonderful job they've done in simplifying cloud data warehousing and look at the reception and results they have achieved so it's not just the apples of the world but in our data business there is also this move towards the cloud and simplicity that is very clearly paying off and revenue growth and therefore capitalization so i believe a modern data platform is one that can adapt to changing requirements it is more dynamic because of the self-service nature of it and it can support changing models who could have thought that we've been a pandemic in january of this year whenever a fiscal year calendar year began we are certainly at a different place today and any data model any modern any platform that cannot adapt is not going to provide anywhere near the value that people are looking for and you know as i wrap up i would say to you we need to move past the three v's the volume velocity variety those kind of physical attributes and we need to move into what we call the four aces the four a's artificial intelligence it is a disrupter and a game changer we have to think of ai first we have to bring applications into our embrace we have to have access for anyone this is about democratization this is about martin luther for data you don't need a high priest to read a bible uh data bible you should be able to do that on your own and it's by using platforms that have a high degree of automation it's it's time that the platforms themselves use the data at metadata level of their customers engagement to do a better job to provide a virtuous cycle of better platforms by being able to observe some of the metadata around how people are engaging with them and i'm confident that as we do that we'll be able to achieve what we all look for from modern data platform and uh and that is very much something that is in our grasp if we change our point of view in our thinking as i wrap up i want to uh shout out to some people that you may have heard of some not you know in the very early days inman and kimball came up with some wonderful ideas james dixon did such a great job of data lakes people at snowflake and redshift have done a wonderful job of providing technology we work well with pwc and microsoft azure team and i think the consulting folks at thoughtworks have done a very good job of thinking about data mesh and other kinds of ways which we can put these things together so my thanks to them and also my pointer to you to be able to uh to be able to go out and do further research and work on this as needed so with that i feel our talk has come to a wrap and i want to thank you and wish you all the best in your journey to building a modern data platform we certainly can thank you very much you

Channel: SnapLogic

Views: 853

Rating: 4.7333331 out of 5

Keywords: cloud integration, big data integration, SaaS integration

Id: HfeaySYVN-I

Length: 15min 44sec (944 seconds)

Published: Thu Oct 15 2020