AWS re:Invent 2017: An Overview of Best Practices for Large-Scale Migrations (ENT212)

all right thank you well good morning everyone thanks for taking some time out and your busy schedules to hear from us and talk about best practices for migrations my name is Joe Chung I'm an enterprise strategist with AWS have been with the organization for just a little bit over the over a year I'm also joined on stage by Simon Clark who's coming to us from Dow Jones and we'll be talking a bit later about the story from that organization prior to joining AWS cells with Accenture had spent some time in consulting but the last kind of tenure of my time there was with our internal IT organization Accenture IT and before I left I was responsible for enterprise architecture a number of our core platform services our digital strategy and our approach analytics in a number of other areas and we had a goal to be 90% in the cloud as part of a digital transformation journey that we were undergoing in our organization when I left about a year ago we were 60% of the way there and last time I checked in with the team there were about 80% but as part of that journey we did a big lift and shift of our on-prem colo-colo data centers to AWS but also started to do some net new development over the last year just just tell you a little bit about the role of our team is to really share our experience or my experience as a former customer with senior executives CIOs and CTOs and just share all these the best practices that we've gathered not only from what I experienced as a customer but now that I've had an opportunity to engage with many more customers to share those best practices I've also been part of our migration acceleration program so it's a program that we'll talk about briefly at the end of the presentation where we're trying to package up all of this best thinking and provide methodology some investment and training to make that journey for those of you who are relatively new in this journey to the cloud and make that path much easier and stand on the shoulders of those who've gone before being an IT executive is really hard most organizations that I speak to they have they come from environments where there is quite a bit of tech debt you know is for those of you who've been in IT for a long time you know how it goes you develop new systems you're always focused on the next thing you never have an opportunity to clean up what you've done in the past because of the pressures to be able to deliver to the market and you end up saddled with a whole bunch of stuff that you wish you could have taken care of but then on top of that in this age where security is you know a concern for everyone and then you know kind of to add to that everyone is being challenged with this thing called digital transformation but more and more customers are coming to us and making the realization that cloud is a potential answer in that journey to retire tech debt to modernize IT to even improve security posture we've seen a pretty dramatic shift in thinking amongst organizations believing that you know cloud used to be this thing that you were sort of concerned with be afraid of - now more and more a belief that cloud can actually help your security posture but there are many different reasons most organizations I talk to these days are concerned about how do I get more agile how do I be more innovative we've worked with some customers who have gained up to 30 to even 70 percent productivity savings by being able to move to the cloud because you don't have to wait around for infrastructure to be provisioned and with these new wave of services and hopefully you'll hear a lot more later this week with some new services that will launch being able to take advantage of those without having to worry about instantiating infrastructure and laying down software more and more as you know datacenter leases are up or maybe there's some major Hardware refreshes I talked to also a lot of customers who are interested in closing down their data centers they say you know what this is really hard stuff we don't want to spend our energy racking and stacking servers we'd like to move up the stack take that capacity and move it to other things like the innovation side of the of the house and we mentioned digital transformation it's a funny term I remember when I first heard this I don't know six seven years ago remember chuckling very cynically hahaha we've been working with computers for how long now I've come to really appreciate and understand that digital transformation is simply about delivering customer value at a pace with innovation that we've never seen before in technology and being able to leverage capabilities like the cloud or analytics to be able to deliver those experiences and there are many more other reasons that customers are moving I've definitely seen a inflection point in the industry and just the interest that has come about in enterprises for migrating to the cloud but we get a lot of questions for example like how do I create the business case it used to be that some of the more key for leaning CIOs and CTOs just by sheer charisma and belief could motivate an organization to make a journey to the cloud but more and more everyone I talked to despite lots of statements of cloud first intents they everybody still has to go to the board or to the CFO and ask for money to be able to embark on this journey most people are not willing to bet their career on the accuracy of their CMDB so it takes some time to understand what's in their environment and not only understand what servers are there but the dependencies between applications who can help in this era where over the last couple of decades many companies outsource away a lot of their capability which is interesting because I came from Accenture and was part of that journey but I think many organizations are feeling like maybe they went a little too far and they need some help to help on that cloud journey or perhaps in source more capability back into IT organizations and a bunch of other things and oftentimes it's not the technology that we're talking about it's things like people process and call there's some pretty big companies in the enterprise space who have been moving News Corp has an amazing story and being able to move from 56 datacenters down to six saving over a hundred million dollars in that process to an out which is a utility company based in in Italy where they moved fifty five hundred servers in a span of nine months it's probably the fastest story of migration that I've heard where they've reaped benefits in 50 percent storage costs and the speed of provisioning new infrastructure to capital one who's really transformed themselves from a retail banking institution to really being kind of more like a high-tech institution with the number of software engineers that they've trained they've trained thousands of their engineers and certified them on the AWS platform because of just their belief and the agility and innovation that the cloud unlocks for them to companies like ancestry who've moved 10 petabytes of data their business they're continuing to evolve what they're doing in terms of their 2.6 million users and genealogy and moving into other spaces as well to even public sector institutions like FINRA so FINRA if you're not aware of them they monitor the US equities market for things like fraud insider trading by moving to AWS they've been able to process 37 billion events on a daily basis and be able to really accelerate the queries and the analysis of that data by over 400 times and there's lots of other customers BP ge Symantec Ticketmaster so we were seeing quite an uptick in very large scale enterprises and the challenges that come with those enterprises that have built up tech debt over many many decades of existence so what I'd like to share with you is what's the kind of the overall process for migration every company is obviously unique you have your own drivers for your business but for the most part I think you could say that the process looks roughly the same at the macro level it usually starts with senior executive sponsorship working with our account team creating a business case starting to invest in some foundational capabilities to exercise of portfolio discovery and planning which we'll talk a bit more about to the you know what we'll call the migration factory stage so once you understand what's your in your environment and you start to scope it out there are different mechanisms that you can use to actually migrate your workloads and we'll talk about some of the paths to the fact that every one of these applications or servers are going to have to land in some type of operating model the operating model will be different in the cloud and that's something that we work a lot with customers to work through and change their processes and the fact that this is a continual process you'll have to iterate on this no one gets this right on the first go címon we'll get into a little bit more detail about some of the iterations that dow jones went through and with each subsequent migration wave that you complete perhaps it's the first data center that you close you'll continue to learn adapt and grow so let's talk about the preparation stage executive sponsorships super important so what what do we mean by executive sponsorship because often times I'll come in to a customer and maybe things aren't not going as well as they thought it should be going and one of the first questions I ask is who's your single threaded leader over this effort and is that person someone with street cred in your organization and is that someone who reports to the CIO or the to the CTO how much buy-in do you have across your business because often times it's not just the CIO or CTO who can punch this thing across the line you have to engage with legal procurement security in the most forward leaning institutions or enterprises have engaged with HR partners because as you begin to become well-versed and your workforce begins to change their skill sets are going to change the roles are gonna change and therefore you have to think about their career path the JD's will change the job descriptions these are things to consider as you begin to embark on this cloud journey the next thing is to gain some foundational experience we've seen some customers even before committing to a large-scale migration exercise try to move some quick wins put some quick wins onto the board working with one customer where we have a 50 apps in 50 days challenge and it's interesting because those kinds of efforts I think are really important because they kind of act as a bit of a wrecking ball across because you're gonna have to deal with policy issues security network and having some type of goal that's timelined can create that sense of urgency and really expose those those inconsistencies in your organization on the people process and policy side and it can really help accelerate that journey so let's talk about some of these the other super important aspect in any cloud journey is having a team I often top sometimes run into organizations that say oh yeah we're serious about the cloud and I'll ask them about the leadership question but then I'll also ask about what's your team look like oh they'll say well it's 10% of Bob and 20% of Jill's time and that's our cloud team I don't know about you but you know when I have hobbies and it occupies some small percentage of my time but it's really difficult to make meaningful progress if it's kind of a hobby project and so having a dedicated focused team to begin your cloud journey is really important I can't emphasize that enough ideally its cross-functional in nature folks from your infrastructure organization folks from your application side perhaps application architects networking it's truly a multidisciplinary effort it doesn't have to start big we've seen teams that start with just a handful of people three four or five people but having that dedication and that focus and the air cover from senior leadership is super important then there's the process aspect as we mentioned they are as I mentioned the operating model is really important to consider things are going to change our operations integration practice likes this kind of floor box model but typically when you think about operating model it's these things around patching monitoring incident management these things will have an opportunity to change and we do work with organizations to tweak and evolve their these kinds of processes we also happen to have a service called AWS managed services that was launched last year that can help offload that now we're not trying to go in and replace the MSPs there's a lot of other things that managed service providers provide in terms of their full suite of services but when you think about the kind of the really the low level heavy lifting around these kinds of things AMS can be a potentially great option we actually use this at a particular customer because of the effort involved with tweaking your operating model around these processes and the fact that the AWS managed services team has already invested in the process has achieved many certifications HIPPA sakwun SAP - that can operate a very secure landing zone or a place where your virtual data center if you will and have all these processes wired up you can use that as a mechanism to bootstrap your migration and as opposed to investing a lot of time up front to do this and it's not a one-way door so we are with a customer that I mentioned before doing this 50 apps in 50 days initiative they're landing it in AMS and they're gonna use it as an opportunity to learn to teach their organization because their goal is to be eventually upskill the rest of the organization and it remains to be seen how much they'll continue to use of AMS but it's a great way to bootstrap your your migration and if you're interested in that please talk to your account manager and then finally the technology the landing zone as we like to call it your the virtual data center this is a super important part I will say that it's hard to get right in the get-go and Simon will talk more about that but really this is about your account structure your tagging taxonomy the networking design that you'll put in the amazing thing about AWS is we provide you a ton of capability a lot of LEGO building blocks but it does take some effort and configuration of that that's tailored and suited to your circular environment and then finally the business case and this is a super important component as well these days it seems like any migration at scale needs a business case and so what do we mean by that every organization as I mentioned before has unique KPIs and drivers for your individual industries and verticals but most people are not in an environment where they can ask for 50% more budget just because there's pressure to be in more innovative or more agile most organizations their budgets are capped so operational cost savings are really table stakes in any migration journey even though the end goal and the objective is innovation and agility workforce productivity is what most people are seeking the reason why AWS was even created was because about in the early 2000s there was a recognition that some of our engineers were spending up to 70% of their time dealing with infrastructure and even though we were pretty good at operating infrastructure at scale engineers were still spending an enormous amount of time dealing with infrastructure and that was really the reason why Andy Jesse who's our CEO created the business case and pitched it to Jeff Bezos for the creation of AWS cost avoidance the opportunity to not have to invest in that next hardware refresh and operational resilience this is something that when we went through our journey we didn't actually baked into our business case upfront we were sort of pleasantly surprised in our organization I think it was something like with a like-for-like lift and shift exact same architecture exact same servers we saw a 30% reduction in incidence particularly around network maybe that speaks more to our provider that we had when in our Colo facility but this is something that we see with many customers other aspects that you gain in terms of resiliency is being able to leverage AWS as region an availability zone architecture so taking some web servers and spreading into cross multiple availability zones automatically UPS the resiliency and there are many other capabilities like that I think Nike coined this term disaster indifference by just being able to leverage some of the h.a solutions that AWS provides to of course business agility the ability to innovate and operate it seems like every industry every company these days is feeling the pressure to to innovate I don't think there's any company that is not affected by digital transformation there are certainly being impacted more than others particularly media banking there's a number of other industries we use this slide a lot a GE oil and gas did a really good job of baselining really every dimension of what the factors that I just covered around agility and cost and resiliency in workforce productivity to where with their investment in the cloud they're able to reap 14 million dollars of year-over-year savings I don't know if you know much about GE oil and gas but with commodities prices kind of dropping out over the last five years they were under tremendous pressure to save costs and that was their primary driver but in addition to that they were able to take advantage of the AWS cloud and its capabilities to not only save costs but derive other aspects around innovation and agility so let's go a little bit deeper into the migration process around portfolio discovery and planning like I mentioned most people are not willing to bet their career on their CMDB so most people have to go through an exercise of understanding what's in their environment very few organizations keep their architecture diagrams up to date one of the things I would pass on as a best practice is one trap that it sometimes I see organization fall into is okay we don't really understand what we're doing let's just go super deep and try to uncover and understand everything every server the characteristics of those servers and what can happen is you get bogged down into this sort of analysis what I recommend is kind of a two-phase pass at discovery just take the high-level metadata that's base server counts what you best know about the number of operating systems and whatnot and take that information and use that for your planning and scoping purposes as opposed to kind of getting too deep into the planning and discovery and once you have identified the size of the prize and the business case opportunity then move into some deeper dive discovery we have a number of vendors that we work with and partner with including some of our own tooling customers will ask us well which one should we use and but really really depends I would say each one of these tools have their own superpower and it really depends on the needs that you have some are agent less some are agent based some focus more on network discovery where they can analyze the traffic between infrastructure and applications which can provide that high fidelity data model that's required for you to understand how to prioritize scope and attack your migration just to kind of call out a few of these risk networks help and cloudscape help automate content discovery for an application portfolio they work with second watch in Viacom to plan a migration of a thousand applications across three different data centers cloud Mize help nimble Commerce in terms of handling their their sprawl of actually the cloud so it's not just for your on-premise systems sometimes if not investment not enough investment has been made on the AWS side the sprawl can not only happen on Prem but it can actually happen in AWS TSO logic is another partner of ours and they're really good at more the business case aspects and some of the recommendation engines in analyzing infrastructure and recommending the infrastructure that you should be provisioning in AWS let's move to the the migration Factory if you will there are six patterns of migration that we typically share it's a riff off of the Gartner of five our framework that was released in 20:11 don't ask me which are we introduced I honestly don't know but usually it starts with rhe host lift and shift some organizations say you know what Joe we don't want to do our mess for less than the cloud we're only going to focus on Ryoga tech chure I'll tell you re hosting is a great strategy it can not only help you save costs but we mentioned the resiliency but I think it's also a great way to help modernize your architectures the mental model that I have is it's sort of like having a fish in this goldfish bowl and no offense to anybody's data centers that you've built up over the years but by moving to the cloud is sort of like dropping that fish into the ocean the resources that are available to nourish that fish open up dramatically when you move to the cloud we had an application just to share one story that had a really horrible user experience but it was mission critical to the business and the team and the business was very reluctant to touch it for fear of it breaking one of the things that we did though is we stood up in elasticsearch instance and we said hey we can use the cloud stand up in elasticsearch instance which is a solar based technology and pump the data into that it's got a great rest based interface and we just do it with a minor refactor of the UI you now have this practically google-like search experience for that application and the business was completely thrilled when that was shown to them particularly because it didn't take a long time and did they didn't have to touch a lot of application logic it's just one simple example of how even by moving your applications to the cloud it can help with even your modernization of those applications we have many tools that can help with the automation of the movement of those tools or those servers and those applications and certainly many organizations will do their first few manually as and begin to understand the process for how to engage in that the other path is a reap lat form we're seeing companies maybe perhaps upgrade their operating system because it's too old to being able to leverage services like our relational database service I don't know about you guys who've managed databases it's hard patching not only do you have to patch the underlying infrastructure but you all have to constantly patch the the database engine and we took a hard look at this as well because we thought we could potentially get 2/3 savings by simply being able to leverage some of these managed services there's certainly the option to repurchase why carry over systems that you don't need to invest in being able to use services like Salesforce or workday or success factors and then certainly we architected an option and being able to leverage some of the services like land and DynamoDB anytime you go through these kind of rationalization and analysis exercises you usually find what five maybe 10% of your servers are actually unused you can just simply turn them off and usually there's also some remnant that might be a little bit harder to deal with we still see plenty of mainframes and as4 hundreds and iSeries still kind of running out there what I would say in terms of best practice is a balanced approach not over indexing on one or the other and being able to be holistic in your approach and there are tooling and algorithms that we actually have available and in addition to our experience that can actually help recommend different paths as you're analyzing your portfolio there is a spectrum of I guess complexity in these migrations certainly coming from a VMware hyper-v type environment x86-based workloads is pretty easy I would say in the grand scheme of things and certainly on the other end you have things like mainframes it is something that we're gaining more and more experience on there are emulators that exist there are conversions of COBOL to Java companies like we pro and t-mac saw for investing heavily in this space - of course re architecture scenarios so just wanted to cover some tooling that's available we have a plethora of app migration and automation tooling to help with the journey our server migration service our database migration service by the way many of these services are two-way it's not just about taking your workloads to the cloud you can actually not take them back to some data transfer help that we can provide whether to s3 or using some of our storage gateways to move data from your on-premise centers to the cloud and things like snowmobile and snowball which are awesome mechanisms to move massive amounts of data from your data centers to the cloud and we talked about some of our partners the other thing that we're doing is we've invested in a migration hub so the other thing that our customers told us is wow it's great that you have all these tools but I'd love to have kind of one mechanism to be able to view my the progress of my my migrations and be able to see that orchestration so we've invested in a new service called migration hub and we've also last year released a application discovery service that you can leverage as well that can work either independently or alongside some of our partner tooling so just to kind of give you a sense of you know how some of these tools have been used you know over a half-million VMs have been migrated using our server migration service over 45,000 databases have been migrated using our database migration service our snowball appliances have traveled around the globe equal to more than 250 times it's a product and service that's really really been embraced and a number of interesting cases not only for data transfer added data centers but even for things like edge computing where some oceanographic research institutions are taking snowballs and sending them down into the ocean to collect data and then bringing that back up and then uplink it gets you a table us for for processing some of the organizational strategies I mentioned the cloud Center of Excellence one of the things you should expect is your cloud Center of Excellence or your cloud team will evolve some of the things that they may be working on depending on your migration approach if you're gonna for example do white-glove migrations you'll have your migration factory some customers do you advocate and go for a more DIY approach enabling their teams with the tooling so they can do self-service migration there's the platform engineering aspect of configuring AWS it's not a one in done activity it's something that you will have to iterate on to things like cost management and as your migration scales you know you have to expect that your team will scale I was talking with one customer and they were really challenged because they have a really small team but the expectations are very large so we're happy to help provide some data points and some organizational models for you if you're looking for some proof points that says hey my business won't let me hire you know the staff that I need to embark on this journey and certainly happy to help out with that I think another important best practice is visibility I came from a pretty heavy score carding culture at Accenture there was sort of no better mechanism to motivate people to move than some public shaming so we see a lot of organizations use scorecards like this whether it's the training and certification exercise or the migration process across different portfolios there's nothing like a bit of friendly competition to help spur the movement of servers and applications so as you scale this we mentioned you'll find yourself having to iterate so what I'd love to do is hand the mic over to my friend Simon and talk a bit about the Dow Jones story good morning thank you Joe my name is Simon Clark I'm head of infrastructure and operations for Dow Jones Dow Jones is a news corp company and employs about 6,000 people in 85 locations worldwide Dow Jones powers the professional world with data and news and analytic tools products and services you would notice for publications like The Wall Street Journal Barron's and Market Watch on a consume side and on the business side with Factiva news plus risks and compliance and newswise so what I'm going to do over the next couple of slides is actually take you through the calcloud journey that dow jones has embarked on and some of the iterations that we've gone through and the lessons we've learned through these iterations going forwards so I can give you some context to our cloud journey it started back in about 2013 the events zto of News Corp was Paul Schuster and he set out this vision that that all the news cup companies would have a 75% of their compute in the cloud within three years so that to Joe's point earlier is executive sponsorship so you know we have a big box we had executive sponsorship but you know what were the actual business drivers that we could actually build a business case around well I don't know about you but aging hardware is an issue and aging hardware especially in your service at 10 plus years old base they caused incidents right so who actually wants to go and invest in more physical Hardware when you could actually move out to the cloud so then again sir your operation of resiliency and you get cost avoidance so there's one business case there as Joe mentioned earlier the big drive for News Corp was actually to consolidate our data centers so why for those applications we couldn't move to a data center we move those super cloud and again saving more money and I think a key thing to chose Pontarlier digital transformation The Wall Street Journal back in 2013 we started noticing that people were actually moving more more to digitally experience and less and less people actually reading a newspaper and that trend continues and in fact last year we had that tipping point of where more people actually read digital of a need to print so we had to invest in actually and basically create a business agility for Wall Street Journal to be a digitized product but that also gave us then the scalability to actually respond to major news events and so we could which we could never have in the physical data center before and that's where I really drove our business transformation so in our first iteration and I really love his model of people process and tackett's worked for me throughout my professional career so we had the executive sponsorship from News Corp the Dow Jones leadership team all had performance objectives to hit for target we actually managed Viva cloud transformation and the initial phase of this via a partner called the chiusa who's an Amazon premium partner and they really helped us with our migration planning of the kind of 12 applications we have an eleven plus eleven thousand plus service we have the output of that discovery process really was a program roadmap and to Joseph on about scorecards that then defined the plan our milestones of how we're going to get to seventy-five percent and at what points we could benchmark ourselves the choose that also helped just actually create our migration patterns of how we're going to migrate to the cloud and all sort of resource models and the cost models associated with them one of the things the the engagement the application teams happened to actually understand what the migrations would actually mean to their many amount of work that had to put into that and what actually happened was the chooser really became our Cloud Center of Excellence which then with very limited engagement from the infrastructure and the operations teams and we're going to talk about that in the second iteration and what we did to address that the kind of migrations we did in this initial iteration really were lift and shift you know how do we move our VMs to the cloud we platforming how do we move kind of edged workloads and we had data centers out in Minneapolis and Chicopee you know how do we consolidate that and move that to the cloud and then finally as I mentioned before the Wall Street Journal how do we actually react attack that to become a digital product in the cloud we have continuous integration and deployment capabilities as well in the first iteration near etre processes we had were not standardized the deployment and provisioning varied by the the application and really the engineering team has carried on the kind of processes they had before in terms of cost monitoring well costs our cost was actually more about cost monitoring rather than cost optimization using cloud health one thing we did do to actually drive down and keep some control over our costs was actually by reserved instances malazan to actually manage that manager claustro file going forwards the other thing I'll say about cost and as you embark on your cloud journey is you're going from a server server environment which is capital intensive to an OP X environment which is a cloud so that's something to consider in the conversations with your CFO's in terms of how you need to flex your budget and I think the other thing you need to consider and going forwards with cost is so I just lost my train of thought so so anyway moving on to tech are the initial tech structure we actually started out with four accounts and two of those accounts who actually shared accounts so with great power comes great responsibility so as more and more applications went to these shared accounts they had the admin rights to use those accounts and actually make changes as more more applications came in and more people have these admin rights the actual potential blast radius of a change became very large so we had to do something about that we also started hitting some internal limits with an AWS in terms of actually pairing our VPC structures and back then when we started our migrations we actually use VPN rather than Direct Connect so it's really important to get that current direct connect back to your back to your data centers and then finally we moved a lot of our active directory to the cloud to actually help us fare so at the end of our first iteration you know we were successful we've moved a lot to the cloud but what we actually created was what we now refer to as our legacy cloud yes such a term can exist so moving on to iteration 2 I would call this the landing zone the the second coming of cloud in Dow Jones and I think really the key thing we did here was actually work with Medusa and actually create a cloud operating model you know what what roles and responsibilities do you need in the organization what skills do you need in the organization who is who's going to perform those baserunners kind of racing model we've then actually created a cloud engineering team and a cloud operations team to actually manage cloud migrations going forwards and this was really to enable essentially because cloud is a core competency that your organization should have and you need to invest in your people to choose to did a fantastic job as a partner leading our migrations but we needed to invest in their own capabilities to manage cloud going forwards so what we've actually done is we've actually embedded the engineering the actual infrastructure and operations teams into the cloud migrations in landing zones and that really gives those teams and makes them kind of cloud relevant and also cloud capable also we've also invested with AWS our accounts him we provision that everybody in technology has access to AWS essential so anybody can access that course free of charge and we've also worked out by looking at kind of racing model what skills and knowledge education people have in our teams and our retained organization need and we actually then provide training to those teams as well so moving onto process you know I'm a big grossest guy I came from an Operations background before doing this job and the big question here is how big should vio be in DevOps developers don't like being called up at 3 o'clock in the morning you know so we need to think about how big is the operations team going to be and what we've actually embarked on in Dow Jones is what we call the cloud process reengineering program and we've actually identified 15 processes that we need to have in place to actually manage cloud successfully versus on-premise and we've really kind of focused initially in that first wave on six and I think it goes back to the landing zones you know I've really you should really focus on your tagging standards and your structure and the policies around and tagging in your in your landing zones this will help you drive enforcement you can use policy engines to actually vents if you don't meet certain standards will actually spin your instance down but it also helps drive cost optimization as well and just to give you an example we've recently in our def share environment enforced tagging and we actually reduced our daily cost by over twelve hundred dollars a day now it doesn't sound a lot but you'd multiply that by 365 days that's quite a lot of saving and that's just in a pre-production environment also tagging is so important as Joe mentioned before who'd bet their career on a CMDB well our seemed to be was pretty void of any cloud assets and we've actually worked closely with with actually engaged with Amazon throw serve and we've actually now just piloted how we've integrated Amazon into our remedy CMDB using cloud rail and an config so that's that's an opportunity they're also monitoring and logging you know we snuck into conversations about when we use cloud rock watch when we use New Relic infrastructure and how do you then integrate those back into your monitor monitors for an Operations perspective change management you know we've continuous integration and deployment how do you reflect change into your sim DB and then really from a Service Catalog perspective how do you actually make that Service Catalog available to technology to engage for new requests or engaged for support and ultimately from a security perspective you know it's very important to have a defined patching strategy we're now looking to move to a kind of monthly cadence for patching and redeploying our ami is back into the landing zones to keep the latest policies up-to-date in various environments and I think another key thing is the use of automation we use terraform to actually deploy our landing zones and when we first started it took four weeks to provision an account into Amazon through a new application we're now down to the point is below a week with the use of automation and scripting and I think the other thing here to consider is the migration process when we started on migration journey a lot of the AWS capabilities with migration hub did not exist if I was actually doing it today I would actually definitely look at those services and in fact for our database freedom project we're actually looking at using that but that's another story and then finally for a technology perspective really think about your account strategy how many counts do you want we've actually gone from four accounts now up to a hundred and twenty accounts within AWS and you know the key thing about this is it you know you need to think about this and you also need to think about your landing zones because landing zones are so important you know landing zones are infrastructure as code that is going to give your end application engineering teams the diff the boundaries and the policies to work within which can be enforced from a security perspective as well one of the things about security is really important which is you know visibility of um data and we've recently had a an issue where we had some overexposure of some data with an s3 bucket and we actually work with our InfoSec team and we actually developed a own homegrown solution called hammer which actually assesses our a de Bresse environment looks for overexposure of s3 buckets it looks for em issues with our ec2 configurations and actually gives the engineering teams a chance to remediate that and if it doesn't within a certain period of time we automatically resolve those exposures I'm using this tool and hammer is something we're actually looking at open sourcing which should be open to anybody here in this audience so in conclusion we've gone through two iterations of cloud our initial migrations and then the second coming a cloud with landing zone we're not finished yet there will be more iterations you know we've now created a foundation for continuous growth we're currently sixty-two percent of our journey towards the seventh towards our migration of total of 75 percent to the cloud and the final thing of mention is without 75% we got to remember that 25% of our estate is going to be left in data centers so that's another conversation to have with your CFO as well to say look we're gonna have to continuously invest in our service state as well in addition to cloud so with that I'm gonna hand back to Joe all right great thank you so much Simon it was awesome - I think here from Simon it was also interesting that in our journey from my time before AWS a lot of the parallels and some of the same challenges that we face particularly as you embark on that journey and have some migrations underneath your belt and I think it's awesome to have some companies to engage with and speak with who've faced some similar challenges so thank you so much I'll try to speed through these slides because I'd love to leave some time for questions and some Q&A from the audience every migration does start with a hybrid scenario using something like a PC and a VPN as Simon mentioned when you start to get serious usually you have to invest in something like Direct Connect which are provided up done by providers like equinox and other networking providers and then that picture will evolve we also have more and more tools that can support not only the AWS cloud services but also some of your on-prem data services things like our DevOps tooling around code commit and code deploy or code pipeline and certainly a lot of our vendor tooling can operate well across these environments now so whether you're talking about an app dynamics or a new relic or service now to be able to manage your catalog across your data centers you can also use platforms you already know our VMware offering which is provided by our friends at VMware where you can essentially vMotion your servers right into AWS it's an offering that they manage is a great way to get started to being able to use platforms like open shift or pivotal Cloud Foundry and these are all great and friendly environments to run on AWS there's a lot out there that you have to know don't expect everyone to come away and memorize all these aspects it's the reason why we've invested in a migration acceleration program we've packaged up all of these steps the best practices and methodology to make available to you as customers we've also invested in our partner ecosystem around those tools like risk and TSO logic added data and cloud ah mais to GS is and born in the cloud s eyes and our own pro serve organization we also have training that's available for your organization training is an extremely important component to enable your people to get the skills that they need to operate in the cloud and some investment we understand that the migration bubble as you do that migration to AWS can be sometimes a challenge and this mechanism is a great way to help make the business case tilt more in your favor to help you make the case in your organization what does this look like we usually start with that business case exercise either done by our cloud economics team or a pro serve organization or partners we usually start with a migration readiness assessment and many times the focus of the assessment is not on the technology but actually on the people in the process how accurate is your CMDB let's take a look at of your security processes how well-documented are your architectures things of that nature and then we move into a foundation building and migration readiness and planning it can include something like those quick ones that I mentioned but it also moves into preparing your organization helping you set up your cloud center of excellence being able to start to tailor and design your landing zone to tweaking your both your operations and security processes and models and then finally helping you with your migrations either through your own resources and your teams or perhaps bringing in a partner we do have a partner migration competency it's a pretty high standard that we're working with the ecosystem to make sure that we can confidently refer people to your organization that will help you in the journey and understands AWS well just to give you a few examples Accenture is the one who helped them out with that 5500 server migration in nine months they're also helping coca-cola out with their 600 workloads to organizations like cloud technology partners who help the taxes with their migration and moving 70% of what they have to AWS to other organizations like slalom who helped Travis Perkins in their migration story and there are many others and many more who are coming on board and basically it's a co-investment by both organizations and resources and training to help make sure that we're being consistent with all our customers on how to approach migrations I mentioned training we do have a two-day instructor-led training course for migrating to AWS that might be something that's of interest to everyone here and with that I'd like to close thanks again for your time if you have any questions please come up to the mic this presentation is being simulcast I'd love to hear any questions that's on your mind that either Simon or myself can answer so if you would mind stepping up to the mic appreciate that thank you there's a question here yeah quickly I'm not sure if there are any CF force or finance team members here but explaining to CFO the need to refresh hardware is sometimes challenging yeah they always want to juice the hardware out right yep they want to go five years ten years so do you have any white papers any data behind as a hardware age is out you know your uptime starts going down but I mean is there any real data that we can share with yeah so there's we do have data we have some assets that we can share we also have we've dealt develop some board level materials to share with our customers because oftentimes we understand the need to educate not just you know kind of all business partners including the CFO so that is something that you can work with your account manager to get at but we do have both of those assets and we also have a team that is engaged specifically on CFO conversations and it is not uncommon for CFOs or legal or procurement to actually come to Seattle for an executive briefing center session that's been a great way to get the whole business on board so encourage you to pursue those options but just to make you aware of some choices that you have there okay great question what one more question if you don't mind so the other aspect of moving dollar budgets from capex to OPEX right it's it's it's usually shelling because once topics starts going up have you considered an option where RI dollars can be put into the capex bucket or is that a no no from the accounting teams yeah so we're given that we're not an auditor we are not allowed to give guidance and those kinds of things I would encourage you to talk to your auditor for your organization we have seen some movement and looking at that from the regulatory agencies who define those rules it is something that seems to happen a little bit more on a case-by-case basis but I would encourage you to talk to your auditors about what are the sort of the parameters in which you can capitalize our eyes because that you know that I would say maybe one in five organizations I talked to say that capex to OPEX movement is actually not a good thing particularly for those EBIT of sensitive companies so that's something that I would just ask that you work with your accounting firm to validate allowed to talk to anyone who has done it to here in this group but what did your team do I mean did they go country with our picks for our eyesore so what we actually did is we we purchased the our eyes but actually then spread it across a a period of time so we didn't just take the objects hit in one go we actually spread it across yes okay so that's what we did it from a financial perspective all right all right thank you thank you question over here yeah hi I'm Shane and I actually worked for Solon and your presentation you both mentioned percents that you're migrating towards for each Accenture and Dow Jones what do you find in your experience with the different migrations that you work through that organizations consistently don't migrate into the cloud like the processes and technology and operations that they still maintain themselves is there consistency on that sorry I'm maybe I'm not quite following so like when you said Accenture migrated 90 percent and Dow Jones is targeting 75 percent yeah it's like the ten percent and the 25 percent those things that are getting migrated is there consistency that you find in organizations of the things that they don't migrate to oh I see I see okay so I would say particularly in lift and shift scenarios there's a couple things that I've seen there's one option where you lift and shift very quickly but you're also lifting and shifting your operating model organizations that have decent operational hygiene meaning everything's sort of the same across the environments it will feel kind of all the same even with the stuff that you leave behind other organizations have a bit more diversity in processes and basically they're looking the cloud to help sort of clean that up and so what they'll often do is use that as an opportunity to essentially do kind of ops efficiency and ops optimization and automation as they move to the cloud and of course when you do that whatever you behind is gonna be very different than what you have in the cloud so there's no right or wrong answer it really depends on what your organizational objectives are part of the reason why we are so focused on doing a readiness assessment is so we like to understand those things so that we can make the best recommendation for your organization so you find that what people leave behind can kind of vary from company to company absolutely absolutely cool yeah and you know people have various reasons for what they leave behind you know we had our own reasons for some stuff that we left behind but some of it oftentimes is the technology like mainframes and as/400 it's obviously gonna be very different than the x86 based workloads cool thank you yeah you're very welcome hi there I'm Jessica with privated solutions and I too have seen the phenomena of the Hobby projects that goes on so aside from that what has been the the biggest challenge you guys have faced when trying to transform your team into from what they're doing today to this new technology and skillset and what was the best way to overcome that yeah so I mean sure I genuinely think looking at the operating model understanding what roles you need in your organization in terms of the cloud and you know getting down so that kind of racy because you you do need a partner so important but you need to invest in your people you know we really did get down to the level of and we've had to invest in people as well sometimes those skills don't exist in your environment you actually have to go out and actually buy them into the organization we recruit them into the organization which we've done but again there's a lot of people you've got to make sure your infrastructure teams kind of cloud capable and that's what I meant about the kind of the skill gaps assessments and actually then saying what training do they need to actually bring them up but there's nothing like at your hands-on experience and the fact that my opera infrastructure team is now are actually helping build and then deploy the landing zones that gives them the expertise of how what they're going to support in a production environment so I think just get them involved right from the get-go they understand how you can be utilize a partner and how you retain the orgonite information and that knowledge in the organization and not just with Department maybe just to add to that I would just say that this is where change management frameworks really help greatly and there's this journey because it is as much of people change as it is a technology change so maybe the mental model that I would give to you is there's kind of like I don't know a few concentric circles of impact that you need to consider the first is your cloud Center of Excellence and what you need to do to enable them then it's your infrastructure organization we had I've seen this mantra of you know let's move our folks from being admins to engineers and so for example Capital One completely redefined their infrastructure roles to be more like software engineering like from the most junior roles all the way to their tech fellows then you move into the application area usually coupled with cloud we see a lot of move to DevOps and agile in that journey that's another kind of wave of change and then the most kind of like you know the most kind of bleeding edge organizations are even engaging with the business where how they finance projects are changing moving to from a project mindset to a product mindset so the waves of change can ripple throughout the entire business I think there are some awesome change management frameworks that can help with that and if you're coming from in organizations that used to that it's an easier jump it's not just a hosting arbitrage kind of endeavor the people and the process change can be quite dramatic and there are some different ways that we can deal with that actually did a chalk talk yesterday on people and cultural change and it's a really hot topic my question is regarding the migration migrating tools I might have missed it my question is when we design the solution for the migrations is there any criteria on what basis we can use SMS versus I go for our double take or I go for a TSO or something you know just to give a good yeah so it really depends on the partner and who you engage with our pro serve organization partners usually have a set of tools that they're comfortable with and enough to be able to handle I would say most migration scenarios from a lift and shift perspective for like x86 I think there's a pretty good list of tools that people have come to depend on a lot of them are kind of really kind of dr technology based the way we did it is you stand up ami and AWS you replicate the data from the server over to that over a period of time you basically you know shut down the old system let the data kind of you know make its way over do a quick checkout DNS flip boom that's how most of these kind of tools do work but if you're looking for some more detailed prescription that's something that we can connect you with some of our partners are pro serve to help you through that selection process like I said they each have sort of their superpower so it's hard to say you know oh you should use this one or that one without knowing a little bit more about your environment it just a high-level not really the deep but deciding on certain percentage of the servers we can go for SMS versus I can go for any others just to give a good business case so are there any use cases which we do I don't know if we've had situations where it's like that you know use this for this tooling and this for that for the most part the environments that I've seen with customers in my own is generally they pick one tool for that least lift and shift type migrations because then also it's about making sure you get efficiency with your team's who are actually doing the migration right because it takes some time to kind of get good and going we basically set a bus schedule every week are some companies do it every month some people do it twice a month and they have migration parties and basically you know you come you do that checkout you cut over and then you kick anything out that didn't make it we'd also see companies mobilize like a small like kind of on-call team you'll inevitably miss a firewall rule or you miss a port opening so being able to have those resources right there on the spot who can make those changes and then anything when it gets you know I used to have this mantra no scrambling on bridge calls the minute people start thinking on their feet bad stuff happens right so you basically say up will call failure you could get kicked to the end of the line and you move on thank you yep yeah Craig Burnie had a question regarding your security option infrastructure cloud engineering team and how you integrate those into your development teams do you take a more prescript approach where you sort of APRI can like terraform templates and say this is what you should be using or do you integrate your cloud engineering teams into the development teams in like a DevOps fashion we actually a bit more prescriptive so that's the kind of purpose of landing zones of Chris's policies and the security is actually baked into the am ice so it gives him the kind is what we want to do a cloud is give the engineering teams of flexibility but with guardrails and that's what we do okay thank you yeah there's our pro stripped team also socializes this concept of deaths Deb sack ops so how do you and using DevOps processes and technology wire and security from the get-go so that developers and operators don't have to worry about that sort of after the fact and that's something that's picking up quite a bit of steam hey Joe my name is Joe as well on your spectrum of complexity slide you showed mainframes being taken along as to get there and most employees has anybody that you're aware of successfully migrated or modernized their mainframes that started out as a legacy mainframe shot yep so we do have a few proof points I'm happy to share with you some of those later but because it's on the harder scale there's obviously less of those proof points it's not in the you know many many hundreds of organizations that have done data center scale migrations so we are working with a number of companies across those three migration paths for mainframes emulation so we do have some customers already running mainframe emulators on AWS we have some companies who've done the COBOL to you know your language of choice conversions and then of course there are some who are undergoing REO architectures the I was just talking to my brother who is in a company that has a mainframe I think the challenge a lot of times with the mainframe we architectures is that once people start to realize hey we're gonna do this it just sort of it's more kind of like typical blocking tackling issues it's just like hey we're going to get off the mainframe and then the requirements just pile on and just it just kills the project so I think that's the challenge of using more like a j'l and DevOps and that so that you know and trying to work with the business that we're like hey we're not going to be able to do this BIGBANG conversion of the mainframe and use more of the strangling the monolith pattern and doing MVP and iterating up to where you can eventually shut down the mainframe so that's something that I'm paying more close attention to because banking retail insurance lots of mainframe still running around thank you well thanks for sticking around for those of you who are here for the Q&A and have fun at reinvent [Applause]

Channel: Amazon Web Services

Views: 5,115

Keywords: AWS re:Invent 2017, Amazon, Enterprise, ENT212, Migration

Id: 4NuXuQOGLKY

Length: 62min 42sec (3762 seconds)

Published: Wed Nov 29 2017