Building a Scalable Architecture for Web Apps

oh then I'm conducting is on building a scalable architecture for web applications and a Mac secure cover across the 60 Minutes a pretty interesting a transition of how you know how you'd start off with a very simplistic deployment in terms of an architecture and take it all the way to an architecture can support millions of users is infinitely scalable can support as many transactions as you want and is you know the common lingo that we use higher high you know high availability and resilient and so on and so forth this over up agenda you know starting off with why scalability is important then covering some of the variables and factors and then you know really the cuts the presentation starts where we actually build our own scalable architecture in incremental steps and you hear these terms across the presentation what some of you might have heard them already in applications you're building today vertical scaling vertical partitioning partitioning horizontal scaling in horizonte partitioning and what they mean what are the ramifications of each of them I'm going to cover load balancing and various techniques for the horizontal scaling and then what is that the partition it's partitioning cetera and then sort of end with some tips in platform considerations etc so to start with you know why is scaleable important but 2.0 word but some of you know is one built one way to look at it is that what's a you know scalable architecture presentation doing a rich web 2.0 conference now that we look at it is if you look at web 2.0 applications successful ones like facebook flickr any of the successful applications of some UX might be building scale abilities actually the one common attribute an aspect across all these applications you some of them might be built in flex some of the model building at Ajax for all the need to have a scalable web architecture a scalable back in architecture a scalable deployment to be able to actually service in the kind of transactions kind of users that you're looking at you know while marketing can result in instant successes and if you look at many of the web 2.0 you know experiences and technologies and paradigms if you get RSS Ajax etc they channel use XML is a base protocol which as we all know XML typically has more metadata than data automatically results in greater bandwidth consumption and therefore you know greater amount of transfer great amount of load on web servers or application servers etc many of the paradigms you know use protocols like RSS or well comment is sort of a push page version of HTTP but use HTTP as a as an underlying protocol HTTP the pull based protocol it's a polling based protocol typically you end up with you know a numerous number of requests hitting your server for data that might you know that or events that might take place once in awhile so you're looking at any slow web 2.0 application that uses these paradigms uses these technologies is going to require a scalable architecture you will be look at the dynamic language landscape today which is becoming fairly popular you know people are coding in Ruby on Rails Grails and many other JavaScript of course and PHP and many dynamic certain features a static language is don't again require set of performant architectures and scalable deployment and so on and so forth so in the end if you want to build a successful web 2.0 application that services millions of users you know technology why some were rich were user interface perspective you know there's several presentations today they'll cover myriad technologies from user interface standpoint my presentation is going to focus on the backend deployment in terms of how are you going to take that application and create an arc create an architecture that will basically offer zero downtime and scale to infinite transactions so what are the variables that you looking at considering in any typical application there's scalability which essentially is that you know how much can my application serve or how much can my deployment serve how many users how many transactions etcetera there's performance which is you know sometimes confuse with scalability is actually different from scalability in terms of performance talks about optimal utilization of resources so how performant is my application how well is it be utilizing the CPU how well is it be utilizing the memory how you know how well is it performing an application can be scalable but may not actually be making optimal utilization of resources responsiveness is very important in a web 2.0 landscape today user experience is something that's a crucial in critical and so how responsive is your application availability is all about you know at any given point in time what's the probably the application will be up you know uptime is probably one variable to measure availability downtime impact again I'm trying to sort of cover these variables because when we start architecting the solution we get to a practical posture resin illogic you know look at each of these and make sure that our architecture of the deployment we have satisfies all these requirements the extent that we want to down time impact is something again some people at times when building an application don't consider down time and practice about how the down time would impact your users how many users you could impact for instance one of the typical elements will be come out is don't put too many eggs in one basket is you put two tea suppose for instance a one monolithic server which serves a million users if that server goes down well that architecture goes down then essentially I'm inconveniencing a million users compared to putting eggs dividing my X across multiple different baskets so we can when you talk about downtime impact we're talking about what's the impact on the user where is the type of impact on the user is the downtime who's just gonna you know just going to not get the service or is he gonna lose data or easy cuz there will be some other implication of the downtime so that's something that you want to consider when you're building an application obviously cost is a crucial factor when it comes to building any architecture at your moment lastly maintain and Shepherd so you can build a really complex design that offers zero downtime can scale infinitely but if it's very difficult to maintain there obviously doesn't you know it's not it doesn't give us any benefits so what we want you know if you look at the variables you want high scalability availability performance and responsiveness and we want low downtime impact cost and maintenance effort that's what we're going to cover today so the factors so we spoke about the variables in terms of what we want now what are factors that influence these variable since this no means by no means a comprehensive list but you know you talk about platform selection Hardware actual application design the deployment architecture what kind of monitoring you have in place all of these factors can influence the variables that we're looking at and we'll be covering some of these today we know this presentation out of power all of them but this one will give you guys a brief preview so let's stop all right what we gonna do is we're gonna start building an example architecture for example application you know I haven't specified any specific application is going to start with a simplistic application and what we're going to do is we're going to build it up in terms of scaling in iterative incremental steps so we start with a very simplistic configuration and we'll keep adding one components to it and and and you know changing the architecture the way we'll be doing this and this is a interesting sort of step-by-step process that you know subconsciously everybody uses when it when they're creating a stable architectures we inspect the current architecture we identify scalability bottlenecks single points of failure s4 basically if there's any particular for high availability we need to make sure there's no single point of failure in the architecture so we identify as force we identify availability issues identified downtime impact risk zones and after you've identified you know one or more of the above in the existing architecture we apply vertical scaling vertical partitioning horizontal scaling horizontal partitioning and we repeat the process once again go back and inspect the architecture and we repeat the whole process once again so let's start with what most applications look like they look like a single box when you start off when you deploy that is the first time all applications today typically consist of an application server or you know an application is deployed on map server or web server just Tomcat rests in for Java for you know PHP you've got Apache with with the PHP engine or whatever but but an app server and then a back-end database server and typically when you when you deploy your app for the first time you don't have any user and you have you still not venture funded you most likely put both of them on the same box and start with app server a web server app several DB server on a single single machine now the first step that we take to scale this up words if you take that single box and you know now you certainly have thousands of users or a hundred users or five users the relaxants in your application you're going to add more CPU you going to add more RAM and that's what we refer to as vertical scaling so what again scaling is increasingly hardware resources we're changing a number of nodes I've got I've still got one node and inside that know what I'm essentially adding more and more CPU and more and more ram so it can it can service more users but until what extent can I keep doing this you know well you know some of you might say we can buy a 64 wave you know son processor based machine it's going to cost more than a Ferrari Enzo and you know your manager is not going to be pretty happy about this so how long do we continue doing this so that's where we look at some of the advantages disadvantage is a very simple to implement all of those upgrade the box but the disadvantage of one is a finite limit there's a certain limit to which I can go the hardware does not scale linearly so you know if I have one CPU I had a second see if you were the zombie I'm gonna double the performance there's actually a performance penalty each time you increment we in PV is an upgrade that machine is going to be a performance penalty which keeps increasing and for some point in time you would have diminishing returns riding additional grading that box by adding additional CPU RAM or capacity is not gonna actually serve your purpose it's gonna actually diminish the returns of course requires downtime for upgrading the box I need to bring down that box change the hardware they put it back up something that we don't really want to do because it's compromising on our high availability requirement increases downtime impact obviously because the fact is the downtime you know the number of users that are gonna be accessed in this particular box is going to be able to get access that bit less service an incremental cost increase exponentially to processor box is not double the cost of our single processor box it's more than that if I look at a 64 weighing machine it's not it's it's actually more costly than 64 independent machines so the incremental cost is higher so that's where we get to the next technique which is vertical partitioning by services so now what I did is I actually separated out the app server and the DB server into two separate boxes so that's what that's what vertical partitioning essentially means it's deploying each service or a separate node so I take each service that there right now my service consists of an app server and database server it could consist of a mail server could consist of various different modules or services I can take each of them and deploy them onto separate boxes the moment I did that I essentially increase my scalability I've essentially increase the number of users that I can serve so I've increased poor application of Araby what this means is in here in this particular case it might not be that apparent because if my DB server goes down then my application probably is down if my app server goes down the my application is probably down let's assume that my application also consisted of let's say Pam if I got Flickr if I found building Flickr from building puto and one of the things that I do is mail notification so each time somebody puts a post I send out an email to notify all the users right so that requires a mail so that'll be the mail in a separate box I let mail server goes down that does ami the rest of my application goes down right just Tomatoes our component that's going down burn down is actually increasing my for application availability a the interesting thing about this vertical partitioning is it allows task based specialization and tuning everybody here you know most of you know that tuning or web server is very different from tuning a database server a database or what it be requires more RAM or web so typically requires more processing capacity a web server is always going to service large number of socket connections therefore tcp/ip optimization and you know your kernel settings etc will be significantly different from what a database server would be a signal to fill my heart it'll be server would be tuned so when you automatic when you when you had all of these in one box all these servers on one box you couldn't actually tune that box to specifically cater to the requirements of that particular function but now when I separate my very vertical II partition by service I'm able to optimize in tune each box for the tasks that it is performing reduces context switching simple to implement for out-of-band processes what I mean by Auto band process is a moment I have multiple servers now here I'm not talking with a physical server but an application server database server etc it's actually a very simple process to just drop to pick up the radio a server and drop it into another box so the moment I'm looking at two things that have a process boundary two components or two modules that have a separate process boundary they're running in separate process is very simple to separate them or vertically part X in them they requires no changes to the application and the flexibility increases negative is suboptimal resource utilization which means that it is possible in this architecture thereby database server is very underutilized you know CPUs never Peking etc my web server CP is always speaking et cetera so I because I've separated out vertically partition my services it might so happen that in the different boxes that I have actually at suboptimal resource utilization in some circumstances so now the term vertical partitioning important thing out here denotes taking services so actually think about it is you know I but so is one service to service three I'm just doing a vertical line and taking each of them and moving into a separate unit to separate load or a separate server and it can be performed at various layers so you can do it at the clear the server layer that data layer at the hardware Europe I talked about some of this going forward each node each separate slice that you create is performing a separate task so yeah I mean our vertical when I was when I looked at the vertical scaling I had a single node and I was upgrading its hardware when I look at vertical partitioning I'm dividing now my application across nodes but they're heterogeneous nodes which means each node is actually performing an independent task of functionality now we move ahead because so as I said this is a finite scalability right because now I've finished moving my app server MIT v7 do separate boxes after thousand I reach two thousand users my application cannot take it anymore so what's the next step so I move to horizontal scaling well typically most people you love this when you start of an application will typically go through this route of scaling you move your app server a TV server into two separate boxes and the next step that you typically take is to start scaling your app server because that's the one that's going to take that's the one that's taking all the user requests all the user connections are not performing all the computations etc so the typical process through this is to have to take a load balancer you know deploy the same copy of your app server across you know multiple nodes and and load balance it to a load balancer this is typically referred to as scaling out an application in case of vertical scaling you talk about you you refer to the scaling up a server here you refer to the scaling out an app server now let's first before we get to details on load balancing etcetera let's understand the term horizontal scaling you know horizontal scaling I'm increasing the number of nodes but now there are homogeneous nodes so in the previous scenario and vertical partitioning I basically partition my application across multiple servers and they were they were hydrogenous nodes I'll kill the homogeneous nodes meaning every node it's got the same copy of the app server each node performs the same task cell is identical again can be performed for any particular type of nodes it's an app server and DB server we'll talk about how we do this for a database going forward so when we when we look at load balancing so I want to spend some time on horizontal scaling because now to get into something something slightly more complex the deployment where I'm horizontally scaling my app server I need a load balancer load balancer essentially performs the task off dividing the load across three different app servers that you saw on the diagram so certain considerations in terms of how I can load balanced my app server and I'm going to go through several slides that explain each of these considerations firstly you know whether you go for a hardware or balance or software load balancer other load balancers is invariably faster a software wood balancer is typically more configurable though these days Hardware balances actually come with significant okay after make a quick announcement so there are t-shirts will be distributed outside after the session getting back to the session so software but these days hardly node balances are also quite customizable in fact almost equal in custom equally customizable as a software or balancer they actually give you more bang for the buck hardware load balancers these days actually fairly cheap and can service millions and millions of requests and therefore actually give you more bang for the buck as compared to software load balancers and typically load balancers also combine in something called HTTP acceleration which I'll talk about a reverse proxying or HTTP acceleration of our slide on that at the ending so I'll talk about that no cook at that point in time the next thing that you want to talk the next thing we will look at when you load balancing your app server is how you will perform a session management there are various options in terms of how you perform a session management one option is called sticky sessions sticky sessions refer so how many people have you ever heard of sticky sessions so sticky sessions so fair amount who knows refers to a fact that when I use a typical load balancer the way it balances load is each time it gets a request it be so determined which machine to send it to under various algorithms will can run it can do this and round-robin it can do the same leaf utilization so it actually figured out which machine is is the lowest position right now and send the request to that machine and so on and so forth sticky sessions - is sort of a element to that algorithm the sense that when I user starts a session with your app server the first time he starts the first request the user sends he'll be sent to one particular node and then all subsequent requests of that user will be sent to the same node that was selected for the first request the advantage of this is that that particular app so was already loaded that user session and so on and so forth and then it doesn't require me require me shared session store I'll come to that but people understand HTTP is a stateless protocol right so you need basically you use you know cookies or HTTP URL rewriting etc for session management where essentially you maintain a session store for each user now if our user the quest goes to app server one and the next request of that user goes to app server to how do the app server to fetch the session data that was created an app server one so in K in various ways to do this one of the ways is one of the ways the session management is I said it sticky sessions where if each subsequent requests then will absolutely no issue because that really node will have that session because the unfortunate thing out here is that progressed server goes there all the users they were kicked into that particular server now unfortunately lose their entire session if their next request goes to any of the other app servers then this session data is not available there to real log in and there's any important data in the session like if they filled up a shopping cart with 20 items they sense you'll lose their entire shopping cart the other issued sticky sessions is invariably it ends up creating an asymmetric load distribution they call sickie sessions you cannot guarantee they're all your servers are equally equally utilized at all points in time because might be saddled certain set of users who happen to coincidentally land on one machine who are the heavy users and the light users happen land on the other machine or in case of a downtime this is where the issue set of compounds if one of the servers goes down then all these subsequent user sessions that will go on the other two machines then when this machine comes up you know you'll find that this particular machine which is a which is just a mop is going to have relevant residing that's a load or let's say utilization the other two machines because the fact that now all the user requests that happy that happen to come to the load balancer with met one are down time let's say are all going to the first two or the two machines ever up at that point in time so rather alternative to sticky sessions is what's called central session storage where instead of each app server storing a user session in memory you actually have a central storage could be an NFS mount well could be an AR DBMS most be most circumstances that we use in ideas like a my sequel database or ms SQL database or a post test database or whatever and each application stores its session in that central session store and now this is completely stateless which means the load balancer does really care by the request code could send requests to any app server each time the app server gets requested loads the session data from central session store and has this actually accessible to it at all points in time of course this automatically means they in the previous situation there was no central session store here now we have to situate two problems one is that we automatically created a single point of failure if my central session store dies it goes down all three app servers won't have assets in stores so the application is down which means after now to do something about that session store and the second thing is that there's a digital this can network i/o of some form of the other each time the session is being accessed so it Sam the session is being read it's being read from the database each time the session is being written to it's being written to a database I involved if assuming the session stores deployed in a separate box that's accessible to all three machines it involves a network call it involves a SQL transaction involves a database write in was a disk i/o etc so obviously increases latency but again all of this honestly is very marginal zero really revolved about it you know variable to the average an extent will just set an element that you need to consider the third technique is called clustered session management and clustered session management in sub having a central session store what ends up happening is all the app servers use a common protocol to communicate with each other to replicate the session data so each time so instead of now managing the data this is more of a people are familiar the earlier topology was a centralized topology this is a p2p topology appear to be a topology where every app servers appear whatever it is created it immediately broadcast some message to the other observer saying as a new user session created whenever any of them changes a session it again broadcast a message to all of them saying look this size should has changed this is a new date so on and so forth or data gets deleted from a session or session expires etc or if it is done through mutual arbitration through mutual message passing between the servers now since some advantages and disadvantages again the advantage is that there is no central session store and I still achieve statelessness which means I get send any request to any particular so the disadvantage is that the message passing between each of the servers the number of messages required exponentially in creases with every new node that I had every done add one more node now my CI have to be broadcast to all the nodes each time the session changes so quickly very quickly you can start having a lot of data transfer into that traffic happening between every one of the servers in rare scenarios which doesn't really happen too often or does really happen most of them several scenarios you can also find a situation where hypothetically a user requests reached an app server before a message reached an app server and therefore the app server may actually have steal session data so users first request went to app server 1 and user immediately send out a second request because he's got a multi-threaded client that's actually fairly fast on the client side and before the app server one could send out that session information to app server 10 they use a second request auto reach apps hour 10 and the session was not available and therefore the user basically got a login form so again that's depending on me again if it uses a regular browsing user it's a very low chance that are set up inside a control land environment will actually take longer time to pass a message than a user's request coming from far away but if a user actually has an application that's sending requests by rapidly O's an automated application on the client side it is actually possible so something you need to consider of course you can combine them and again I make sure that everybody's acquainted all the different approaches so I can actually get sticky sessions with a central session store or I can have sticky sessions with clustered session management so I can actually combine sticky sessions with any one of the other two approaches and so in sticky sessions with central session store I get the advantage is a central session store which means that you know I I have the ability to well if I'm using sticky sessions are not getting the statelessness but I have a central session store that's accessible to all the machines so machine one goes down then machine to machine three can still serve that users request but I also the advantage of sticky sessions which means that each time the users request the still going we see number one so now reads of the sessions do not need to go to the database each time so because my sticky session and and settle session store reads don't reads opposition do not generate Network IO or disk IO only writes to a session generate Network IO and disguise so again around with these combinations I figure out what's most appropriate for your application typically you know based on our experience the kind of applications that we build what I would recommend is if you have a small number of app servers clustered session management is fairly easy to set up but again remember this is only for a standard browser user browsing an application and normal speech kind of a scenario or any other scenario you might have those situations where a user doesn't get effects it doesn't gets very easy to set up because you don't have to set up any central session storage and if survived EVMS you have to set up configure and optimize another machine but ideal circumstances I would recommend using the central session store and use the key sessions only if you have to it doesn't only give a dramatic advantage and in fact one advantage is the bespoke one so sticky sessions right now is that my app server once in after the first request doesn't have to keep going back to the database to read back the session but you can actually you can actually achieve that same advantage by using caching as by creating an element a chanel to cache along with the central session solidago that a side going forward so ideal circumstance of what I would recommend is the central session store don't really bother with sticky sessions unless you really have to and it's a fairly simplistic approach all the flues deploy a central database that all your app servers can talk to so we spoke about section manage will be spoke about a load balancing you know selecting a load balancer now let's talk about the element that I spoke about let me started with load balancing which is the load balancer now now now the moment I had three app server the man load balancer I have a single point of failure which is a load balancer itself so the load balancer goes down it doesn't matter what 15 up against I was behind it you're my applications never be available so typically most load balancers will support either an active process or an active active server fall that scenario right so you can configure two Hardware load balancers or daisy-chain you know three are there multiple Hardware road balances or even in software you can actually have multiple software load balancers solve as a single load balancer appear as a single load balancer to the external world so that if one load balancer goes down the other one takes over the name says self explained really active passive means and one low balances just standing out there passively it's not really doing anything more in the active load balancer if it goes down the passive load balancer is got a heartbeat with the active load balancer it detects that or automatically takes out the assignment of the IP address and so on and so forth and such form the task the active one was performing in active active scenario both the load balances are active now typically the way this is done active active setup is slightly more complex so if I if you actually start out with this out suggest you start out with an active passive deployment and then move to an active active one but an active active typically the way this will work is one load balancer will be serving X number of app servers another load balance already serving X number of app servers and both will be configured to take over the other other load balancers load in case it goes down now if people remember about attractive load balancer is that solo work if both of them are actually at peak utilization right if both the load balancer actually hundred percent utilization the one of them goes down the other ones now gonna have to take a Porno percent load the option will work so the only scenario it can work with their 50 percent utilization which we generally speaking you're very similar situation to an active/passive of course you only scenario where active active can actually give you copy only reason Roo actor activist cost benefits otherwise you go active passive the only scenario where actor active can help is when your load balancers there are being daisy chains are not two or three but you know more than that a number so if you have four load balancers of fire load balancers the only active active route you still get a constant bandage because the fact that you don't have to double up your resources you have lower to n type of an architecture so that's basically as far as eliminating the load balance a single point of failure is concern so at the end of horizontal scaling which is what we were talking about before we rattle off into road balancers and session management this is what now previously we had one app server one DB server now this is what our architecture looks like this this particular box represents my web cluster which consists of two load balancers and three app server nodes and all of them are talking to a DB server so the end of step four after a horizontally scaled I've increased my availability because I've got multiple web servers any app servers any of them can go down and my application still survives my load balance I can go down my application still survives I haven't had made any changes we applications all at my deployment level ok fairly easy to set up but again this has finite scalability for a couple reasons one is I can't infinitely scale this I can't have hundreds and thousands of their app servers behind a bunch of load balancers and - I still have to tackle this guy out here my DB server now typically at this point this is exactly how the application will move you start off by scaling out of your web servers your app servers you know multiple app servers you know one database server and now you're going to start facing what will like the scalability issues area DB server level the my device is a single point of failure and also a scalability bottleneck in this parallel scenario so now we'll apply once again vertical partitioning earlier we did vertical partitioning at the services layer so we divided our two services app server and database server into two separate boxes now we will not actually do it in the storage layer so we actually separate out some functionality from my DV server and moving to San and honestly before you start considering you know most people start off with thinking about clustering my database and replication and so on and so forth this one simple step first automatically provides you significant boost in terms of performance and for almost any given application most circumstances you will find you know people you know take some time to actually size up their application area so you'll actually find most people will find oh my DB server speaking the load average is very high etcetera and you start adding CPU start adding Ram but it doesn't change anything it doesn't actually really increase your performance you know rather it still needs to be high the reason for that is not CPU or Ram that achieving consume now that that's actually causing the problem it's this guy they'll actually notice if you check out your CPU States but CPU utilization might be your lower which might be high whereas CPU that show you an i/o eight at all points in time it's actually because the fact that your disk can't keep up with the CPU so first step then you typically end up picking is move your data into a SAN so a storage area network with one or more stand devices each of which have you know eight sixteen twenty-four forty-eight hard drives and they can churn out data at extremely rapid speeds so now you once again are able to consume your CPU you utilize your sugar to the fullest extent and you get a performance based so boost performance the DB seller cost increases costs but Sansa gain up a pretty cheap these days you can actually build one your own but even buying a standard regular sand device is not fairly expensive these days the sand side this purple little box which are not gonna expand on is the science by itself literally we look at storage management there are hundreds and hundreds of different possible configurations and so many different things you can do it storage in terms of optimization what kind of hard drive to select what kind of sand protocol to select I scuzzy FC SAS acceptor and things of that and that by itself is a you know fairly lengthy dialogue I've actually got a presentation up on vaquita director comm it's a presentation had conducted and one of the other seminars which talks about storage specifically and only storage in terms of how to build scalable storage solutions you can actually refer that if you go to that URL but this percentage is going to cover more of the the application architecture as in the storage architecture so once I've finished vertically partitioning my next bottleneck is again going to be DB server because my web app servers I can keep scaling I can keep adding app servers okay at the sand side I can actually keep scaling my storage area network also and again like I said it's a science by itself you know details can you know you can refer some of the stuff that's available online for that but my fortunately the game will become this little guy out here it's my database server so we won't step six which is horizontal scaling for the database this is where we start talking about replication clustering etc and things of that essentially the same horizontal scaling then I did for my web servers what I talked about when I said I want to scaling for web servers homogeneous nodes each node performs the same task in same function right I essentially sitting behind a load balancer so that all web servers appear together as the terminology used for this actually the word cluster is the most misused term in in deployment architectures but the terminology used for this is the cluster of servers a class or a cluster which essentially appears as one unit to the external world so theoretically it appears as one single machine but internally it's actually multiple different machines acting as one so it's a homogeneous cluster and we measure that for our app servers now let's look at how we're going to do that for our database on the database side we have two options for creating a homogeneous cluster or for scaling out our database or for horizontally scaling our database all three mean the same I say one is only scaling the database server I mean that our multiply one database our node into multiple database our nodes each of which perform the same task so when my app server talks to the database server words it is talking to one unit as which one database our actually underlying it are multiple nodes the portafilter I have is a shared-nothing cluster or a real application cluster or which is also called a shared storage cluster because a storage out there shared let's first look at shared nothing clusters share nothing cluster essentially is nothing but taking my database so which is one database server and she loved that deploying three or five or how many ever I need for my application or two to start with and it's actually they share nothing they're actually independent independent nodes in fact if I go back to our web server horizontal scaling by the way when I go back to my web server or azan ttle scaling the sticky sessions diagram or the clustered session management diagram that I showed to you was I shared nothing architecture out there because I had multiple web server multiple app servers and they weren't sharing anything they were all independent they were all independent handling requests when I look at my central session store architecture in my app server horizontal scaling that was actually a shared architecture it wasn't a shared nothing architecture it actually ship multiple app servers was sharing a single central session store so just to clarify the term shared nothing out here but I look at share nothing so now coming back to a database context what I'm going to share nothing architecture for databases it essentially means multiple database server nodes which are sharing nothing each of them as a complete copy of the database here identical in terms of the fact that each of them can handle well they're almost identical I'll get to that shortly but they can all handle requests from the app servers and they don't share hardware architecture chief typically this is a cheap food database an application again the database the application part could be done at any level it could be done at the idea miss level if the item is natively supports it or there are drivers and proxies available through which your app server can talk to again doesn't require too much modification of your application code but these proxies or drivers will handle all of the the replication bit so they'll actually talk to multiple databases the apps are we'll assume since they're talking to a database driver or proxy that they talk into one single database only okay note here however did I mean when I talk about these three the actual database file may be stored on a sauna it could be a central San that does not make this a shared architecture the sand is actually a separate physical device it could be same set of sixteen hard drives there are three partitions one partition is being used by dv7 one partition might be server two and one by origin by DB server three but all three of them have their independent database files all three of them are replicas now if you look at an application in shared nothing architecture again just like we went through horizontal scaling you went through various considerations if you choose the shared nothing database cluster there are various considerations when it comes to replication let's look at those considerations first if I have three database servers I can choose a master/slave arrangement or a multi master arrangement must end terminology itself makes it clear in a master/slave arrangement is one master or you know a certain number of masters and these masters essentially it typically one master it's this master essentially is where all the write operation to perform this masters intern trip responsible for replicating out all the changes to independent the slave nodes slave nodes and the video operations can be performed across the master or the slaves write operations of the form only to one master multi master scenarios where you know you can write to any of the databases and it will replicate out the changes to all the other database nodes that exists multi master scenarios typically always more complicated because there's a there's a conflict possibility in a single master slave scenario since all right they're going to one master there's no conflict possibility but if you have multi master and the same laws modified at the same time in two different masters then when they're close replicating you are now have a need for conflict resolution which can be specified either as a automated set of rules or they'll have to be a manual intervention at that point in time to actually resolve that conflict in most circumstances I would say pretty much all circumstances you'll never need multi-master you'll typically always look at replication from a master/slave perspective you'll always have one master with a set of cascaded slaves and again out till you can cascade slaves so you know you'll have one master replicating out to five slaves which internal replicas repeat replicas now - and other two slaves each and so on so you can set up your your replication architecture in the most convenient fashion possible the next element to consider when you're looking at replicating so one element was either multi master or master slave I would recommend we go master slave next I'm going to consider do you want to do a sync a synchronous replication of synchronous replication a signal application means that a master receives a right the moment it finishes writing to its own copy it returns the response to the client saying that or everything done then it is responsible for replicating that change out to all the slaves it guarantees that replication will make sure it happens but the response the client is immediate and actual replication is performed asynchronously in case of cygnus replication on the other hand the master will write to its own database and will also make sure that all the slaves write that change to their respective databases and only then return a response back to the client so change will actually made synchronously and again there are advantages and disadvantage and pros and cons in terms of scenarios and situations as to which architecture you choose in the asynchronous replication response to client is rapid clan gets an immediate response while in the synchronous replication situation if you have 10 slaves and one master then you actually n modifications have to be made across all slaves actually if you guys a world of two pcs the two-phase commit process basically before I commit can be can be confirmed by that master all the slaves need to have committed that that particular data change and only then will the master respond back to the client typically so I'll tell you where the sinner you know what are the implications on your application if you're using a synchronous replication or synchronous replication your lights are always go into one single master in the master slave scenario now let's talk about the reads it's the region load-balanced Ross all this suppose have five database servers one of them is a master if I want to let's say it's a salary table if I wanna select you know all employees having ten thousand all right essentially I can send that media query to any of the replicas or any of the five database servers and essentially get B response that I require but there the concept called critical reads there are scenarios in your application where you can't afford because the master is asynchronously replicating to a slave there's a small chance that before the replication to the slave takes place your application file the second query to a slave is actually getting stale information in most circumstances like searches web search is basically basic use pro user profile views etc is not make much more difference but the moment you get to importing elements important portions of your application such as for instance if you're calculating if your company for actually making a payroll entry are you computing a person's bank balance your applications like those you know afford to have stale data because based on the data they get they're going to make certain decisions which are going to impact the integrity of the data itself so you can actually perform for instance if I $400 draw my bank and immediately fire a query to withdraw another hundred dollars and if that query goes to a slave which actually I then I quarter the first transaction I'll be allowed to withdraw a second hundred dollars but so obviously not something that you can have in an application so we're looking at critical reads they will need to be critical read can only be sent to those slaves which are as as updated as the live master is and again you have to bother with all of this because your driver or your proxy or your then your Dao layer is going to take care of this set of keep track of you know where that commits are being made what data's being written where especially we choose the right software but the idea now is that each time I send in a read request that read request now cannot go to all of my nodes it can only go to some of my notes mind you is the only for critical reads so if your application doesn't have too many critical reads you know easily go with asynchronous replication is absolutely there's firstly the chance of you getting stale data itself is rare once again going back to our previous assertion that a bunch of database servers that are actually in the same land which is laying same line segments which is what pequeno configure each is going to be will end up passing data far more fast so they're receiving a subsequent request from a client so most circumstances you want to run into that situation and most applications getting data that's a few seconds or in this case a few microseconds or milliseconds stale is now to make mature difference to your application so in most circumstances async replication is going to suffice for you but if you are looking at mission-critical type of transactions where data integrity is very very important a single application can still work for you but secure application may be a better choice the next consideration to make when you're looking at replication it should you do should you look at an application at the ID BMS level or the oh well actually this is not really a consideration this is part of a platform selection if you're if you select an odbms that's suppose for application then that's the best solution because the database natively supports a replication is going to be quickest and most likely the most reliable solution in the most leveraged solution for application any which ways if your ID Bemis doesn't support your application then there are various third-party tools which work at the DAO layer or the driver layer which can actually provide replication for databases that don't support our application like C JDBC Sequoia etc they'll actually work on the drywall layer and actually provide an application to our DBMS is the dual supported natively interesting thing about you know these kind of tools which work at the driver or da earlier is unlike a native replication tool they can actually support heterogeneous databases which means I can actually have replications and where one of my notice and ms SQL server another node of the Postgres database the third node is my sequel database if i use native ID and I can't do that because MSS Phil's no order replicate to my sequel database but if I use replication that drive earlier then basically I can have heterogeneous databases now of course this is just an interesting you know piece of trivia in most circumstances you're not only gonna actually have your heterogeneous database notes the only known benefit that I have heard for having heterogeneous database notes in your single horizontally scale database cluster is that if there's a vulnerability discovered for mssql and your ms SQL server is compromised then all your ms SQL servers won't get compromised because you have a heterogeneous database cluster most circumstances your database cluster is going to be behind a DMZ or behind it's going to be in a separate subnet it's not going to be directly accessible to anything except for your application servers behind a firewall so the possibility of that vulnerability type of a situation actually impacting your database cluster is significantly lower and so I would really count as an advantage in most circumstances I would say ideally choose an RDBMS that actually supports an application natively that's the best way to perform a replication as compared to trying to do it at the driver or do layer so that finishes that finishes replication and finishes a shared mapping cluster when I start off with the horizontal scaling our goal was to take one database node and convert it to multiple database nodes such that all all of them act as as one the two ways of doing that was one was a shared nothing cluster and the second way is called a real application cluster this is actually two clustering right now I think the only database that supports this is Oracle and I believe mssql will start supporting it soon but as over right now Oracle is the only database that I know that supports a real application class they're essentially a real application cluster you have your database as an as a as a software installer multiple nodes all talking to a common single storage file or a single storage partition so in a shared nothing cluster each database node has its own copy of that database in a real application cluster each database mode it's talking to one single day there's only one copy of the database now it's honest and so it's all redundant in and you know hi available all that stuff but it's only a single copy of the database and all these database servers are talking to that one single copy all right my system needs to be a clustered file system to support this so you have read at GFS or Oracle as its own file system called ofs both of these file systems actually support the article we are advocating cluster I'm putting this here primarily for academic purposes last I checked this is a hopelessly expensive solution and shared-nothing cluster is going to actually work equally well or in fact as is argued in there some white papers available for this in many cases even better and again I'm not going to sit and argue about the merits and demerits there are scenarios where a real application cluster may actually perform better then I said nothing cluster and vice versa but for all practical purposes both them will work for you I share nothing cluster created or PostgreSQL my sequel costs you nothing Oracle real application cluster is actually going to be fairly expensive as an exercise so my recommendation is yeah trying truth I'm getting used to this actually I just bought this a couple of days ago but my recommendation is that trying to the database which supports master-slave replication typically use asynchronous replication because synchronous is not required with slight in slow down your application further because latency will increase right your dao layer to ensure that writes are sent to a single DB so this actually shows you the architecture rights are sent to this one master okay and even critical reads if your application has any are typically sent to the master okay and all the reads are load balance between any of them so reads are sent to all we actually should have been a 1/1 more red arrow out here but reads can be sent to any of them and writes are sent to one master which replicates out to the other nodes so this is a this is our final architecture at step 6 opera horizontally scale the database have essentially got multiple web servers behind a load multiple app servers grande load balancer talking to a cluster of databases which is talking to a SAN and I've actually shown three separate partitions out here just to signify my recommendation of going for a shared nothing cluster if this was a real application cluster then the sand would actually only have one single partition to which say DB cluster talks to so now my database servers no longer a single point of failure right any one of these database server nodes can go down no problem ass and by itself is redundant and in fact I can actually have multiple sign devices inside the sand so there's no there's ample redundancy here and pull redundancy and I am put it on let's see here which means now my application is highly available there's no single point of failure and it's fairly scalable I can keep adding well knows that I can keep adding DB nodes but it still has a finite limit I can't keep adding DB nodes on to infinity and I can't keep hiding web node on to infinity and let's look at the database side because that's the one that's gonna give out first in fact you actually will to keep hiding app servers fairly rapidly as you as your user base increases but where you start again facing the bottleneck is eventually you reach a level where your shared-nothing DB cluster is going to give away and why is that let's actually quickly explore a shared-nothing cluster as a database cluster as a finite scaling limit so I have to obtain an example out here let's say your application the ratio of reads to write says choose to 1 which is I mean in fact in most applications are actually much more than this the reason why I shared nothing database cluster works is because you always have more reads then writes to most databases and most OLTP applications ok so your rights are all going to one database replicated out to all of them and your reader being load-balanced it's what's being served by so right sir actually when I say going to one database eventually go into all of them all nodes are performing the writes and only one of them is performing a read so every read operation can be handled by any one of them now if you have a scenario where your each to write ratios close to one if I have two databases then each database is performing rights 50% of the time and reads 50% of the time because if I have so let us give an example we have eight reads that essentially results in in case of two databases it reads since it's close to 180 s means have four rights so all four rights are going to go to both the databases but the a tweets that will be divided into four reads for this guy and for each for this guy so that means 50% of my database is doing rights as people sometime it's performing reads the moment I need to add more database nodes that ratio starts leaning towards the database note only performing writes more and more and performing actually participating in reads lesser and lesser so moment I have four databases I essentially have 20% of my time being devoted to reads and 80% of my time or something with that devoted to rights as far as each database always concerned you realize at some point in time we'll reach a scenario where adding one more database node is going to give me very marginal increment in the total capacity that I can handle just because the fact that every node has to handle all the rights and then it's only the reads that can get load balance right so at some point in time the number of Rights in my application will increase to such an extent then I think additional node is going to actually become it's going to become futile oh it's going to give me very marginal very marginal advantage which is why we now move on to the next and probably relatively the most complex mechanism for scaling your application to the next level in this off this is what a couple of more steps office but this this is pretty much the Nirvana of scaling in terms of any web a web app or any online application architecture which is vertical horizontal partitioning of your database essentially vertical or horizontal partitioning vertical partitioning talks about taking inside your database if I've got you know 20 tables I just slice them take ten tables and move them to one database and take ten tables and move them to another database now each database intern can become a cluster so now I automatically increase my scaling finite limit because the fact that I have divided my my table vertically you can it this can be done not just on tables not sure that the table then it can also be known as the column level so for instance if I have a user's table which has got you know 50 columns I can divide it into two tables for 25 columns each and one table or other database and therefore utilization you know sometimes I might need only the first twenty five columns sometimes the second twenty five and sometimes both and again depending on that I'll have to essentially modify my application vertical by the way when I talk about the step seven the reason why I say that it's complex is because now you will either have to model if you come this far without accounting for this you'll actually have to modify your application or ideal circumstances take into account vertical or horizontal partitioning into the original design when you built the application it still limits my scalability to a finite limit all right which brings me to horizontal partitioning of a database what is what with partitioning I mean it's like it's very easy to remember what equal partition is I've got my table so I just draw a vertical line and divide and take a set of tables I move it to another node in horizontal partitioning I'm taking a single table or all the tables actually in the application and slicing it horizontally by taking a set of rows and moving it into another database node or another database server so you typically choose an entity and this is very easy to understand psychologically a very simple statement for instance is that if you have an application what you're essentially saying when you all won't be part is your database what you're saying is here is my cluster number one okay which has let's say five database nodes and that's the match limit that I will put in that progres Kuster this cluster is going to serve 1 million users as soon as it's you know as soon as 1 million users of registered and are working excited on things that I'm gonna take the same able sucker identically create one more database cluster and basically put the next 1 million users out there so that's basically that's what that's the sense anywhere horizontal partitioning means that identical tables so if you look at the diagram you know two clusters that have formed all of them have the same four tables serving a million users each application coder do driver or a proxy know so now again nor since I've divided my rows the app servers will need to know where to send the query right all your power only one database cluster my apps are always create a pool of database connection so that one serve and send all the queries there but now I need to write a layer at my data access layer I'll need to write some code that will figure out based on my query where - based on you know which user I'm querying for or what I what data I'm querying for where to send that particular SQL query one of the negatives of horizontal party sitting is that SQL unions must be must be performed within code well you actually can do another database layer also some proxy some proxy scripting databases actually support that but basically what I'm trying to say is that if I want to take 50 users from database that's the one in 50 users from database go and generate a report I can now do a simple select star from this and general union of both of them because they're across databases that actually have to select those rows and do a union in my in my code what are the techniques that I can use for us on the partitioning the various techniques you know there's FCFS which is first million users go on custom one second million users go in cluster to there's a round robin which is you know user one goes here user to go 3 goes here there's lease utilization which is typically useful when you're actually scaling up so you start out with one database cluster the moment you reach let's say 50% utilization of that database cluster you put in a second one now you want this one to fill up because it's less utilized in the first one so you'll actually let this one fill up till they both come to 50 maybe we'll put a third one maybe let these scale up to 60 you know percent utilization put a third one so the least utilized least utilization algorithm will make sure that it divides your users again I'm using users here is an example ok the entity that you choose to R is not be partitions up entirely up you it could be used if I look at for instance of school management application you can horizontally partition by schools by teachers by classrooms by users however you want to choose in your application you typically choose one entity in most circumstances and essentially make sure that your so it's essentially the replicas of your database structure they're all identical database structures across those servers or cross those clusters but each of them is serving a finite number of users you could actually have a hash base the horizontal partitioning where a hashing function decides you know you compute a hash and then you you know decide which particular cluster to send that particular row to you could use of - based method so you say user IDs 1/2 million will be stored in cluster one or all users whose names start with a to M will be stored on cluster 1 again there are many different techniques available for horizontal partitioning you can decide which one you want to choose ideal circumstances round robin and least use are very easy to implement and actually work for most circumstances I have to understand that if you use around except for ash base and maybe even value based in all the other circumstances while you have multiple clusters each serving a set of users you also need to have one global user lookup map which contains your enta whichever entity or partitioning by it contains the entire list and which database cluster there's that particular entity map to because your application server needs to use that map to be able to actually actually figure out which your database custom to send that particular query to so this is what my now my my setup would look like I what my app servers which talk to a lookup map to figure out where to send the database query then I've got two database clusters and they're talking to make one you know a storage area network which can consists of n number of sand devices so one of the things to note by the way when I talk about horizontal partitioning and vertical partitioning here for scaling it's not the same as table partitioning that is you know if you refer a video a method still manual or many of these database manuals they talk about a concept of table partitioning this is not table partitioning table partitioning is nothing but partitioning data across multiple tables within the same database on the same node the only advantage that table partitioning gives you is that you can actually spread tables across multiple disks and by spreading data across multiple tables you're actually creating a match natural sort of a clustered index so that you can director for instance you know all data for the month of January in the table January all the data for month of February in the month of February table February and so on and so forth so basically this is not the same as table partitioning it's quite different from David partition we're actually talking about separate nodes out here now the next thing that so now at this stage I've really finished creating a scalable architecture that can scale pretty much infinitely because I can keep horizontally partitioning my database clusters and I can keep adding web clusters that talk to order so each cluster is going to deal with assert each database clusters will deal with a certain number of users let's say 1 million users in total in fact a logical way to organize this and what you know what typically is called sets basically what I do now so what I typically do is now I take this and I'll call it one set okay and you know what I mean instead of now talking about a scenario where I infinitely scale just one set I for easier manageability I divides into logical you let's say that one set for me one set that I deploy for my web application will cater to ten million users and typically it will consist of three database clusters two web clusters you know four or five load balancers firewalls or whatever and that becomes one single set and a moment I reach 10 million users instead of adding more database clusters or more web more app servers here I just create another completely fresh set again this cause were both goals you know in line with the philosophy I mean looks like which means I have each set consisting of app server clusters database clusters and storage area networks there's a global redirector which is required because typically your user will access an application using a single URL you know there are practicum or whatever web service that you're providing so google.com so there has to be one global redirector which will determine which set to redirect that user to then inside that site everything is essentially servicing that particular users request some of the negatives of horizontal partitioning and these are negatives that you unfortunately live with because when you finally reach that millions of users and billions of pageviews your architecture will look like this it can't be any different from this and so these are negatives that you gonna have to live with is that aggregation of data across sets is complex so if I want to generate a port consisting of data from here and data from here I provide special code for that users may need to be moved across sets of sizing is improper and that requires migration and Suribachi complexities etc and global application settings and preferences need to be replicated across sets since the same application deployed here and the same application deployed here you want the page to appear green across all your app across your entire application if that's an application preference you obviously need to you know manage it across all your sets and all your servers etc again these are all Emily need to consider or keep in mind by step eight you pretty much finished building an architecture that will scale couple of other things I want to add are sort of more like a wish list or interesting things that you would actually now want to do to try and optimize the performance or the utilization of each of the sets that you've created so that you reduce your costs so you'd start with caching in the next step that you take once you build this into architecture you'll try and put in caching wherever possible and every time you're looking at this guy or taking place you're increasing latency reducing performance and increasing CPU weight the moment' divide caching you're sorry decreasing before so more you are chasing you're increasing performance you're decreasing latency increasing responsiveness per user and actually optimally utilizing your architecture infrastructure so you can have object cases you can have session cases if you remember the session central session datastore I spoke about where each time I use a request comes in it has to read the central session store and fetch that user session data if you actually catch that session data you know that call to the database does not need to be made and you actually have a higher performance application API case page cases many applications can actually create page cases Facebook does that for instance it cases all the widgets and pages that that it creates so if there's no change they can just solve out a static page instead of actually having to recreate their entire page so the software then use for this is memcache D is written by danga soft it's a open source freely available platform independent distributed memory cache fairly easy to integrate with with any application it's got client libraries in literally everything Python Java etc etc terracotta is a Java only open source cache it's if you're writing a Java based application terracotta me actually give you higher performance then memcache T because you don't have to see realize these realized objects memcache they actually stores them in its own serialized format but terracotta actually stores them as photos in you know in JVM so the civilization D civilization reduces plus it supports l1 and l2 cache caching as compared to memcache sleeping only one level cache again caching by itself is a science away you could take an entire presentation on casing itself so I do I'm not gonna get into details of caching out here you guys can research research that on your own independently step 10 is adding a HTTP accelerator if you're writing a web application in most circumstances you will gain significant performance advantages by adding or either hardware or software base HTTP accelerator in front of your app servers typically one h3p acts in fact it should be accelerators and load balancers most in most cases come as one combined application either hardware or software so you're actually getting both the advantages of both the functionalities in one single module and typically HCP accelerator they're actually known as HTTP accelerators the reverse proxy load balancers layer 7 devices various different names but they're typically intelligent ones will typically do the following they'll redirect static content requests to a lighter server in most cases you'll be using an application server like Tomcat or or Apache where each process or the process itself is fairly heavier each thread is fairly heavy so you use it to some images and static content etc so you might want to put us on the same box or on separate boxes one more a lighter web server like like PP PD or or any of those lighter web servers and your reverse proxy or HTTP accelerator will redirect static content requests for lighter HTTP server it will also cache content based on rules so images that pass through the proxy reverse proxy or through the HTTP accelerator will get cached so next request does not need to hit your app servers it can be served from the accelerator box itself uses the asynchronous non-blocking i/o most web servers today Apache Tomcat now I started using async and IO but Apache and many other web server actually you don't use non-blocking either use blocking i/o which actually slows them down requires a greater number of threads to process incoming client connections requiring larger amount of memory reverse proxy generally uses async and IO async non-blocking out so this actually reduces memory requirements and actually reduces the number of connections in fact and so in that's the next point what typically end up happening is that if there are 50,000 incoming user connections the accelerator the HTTP ID server will probably only make 50 or on later 200 connections to your app server those will be HTTP 1.1 keepalive connections so on the front end side it's serving 50,000 user connections but your app server seeing only 100 socket connections or 200 Chalker connections thus improving the performance or enhancing the performance of making the most of that app server box is compared to wasting resources on that particular box some of these solutions that are available in jinxes are and pole bar are both open source software base reverse proxy load balancers and HTTP accelerators that you can use of course there are Hardware ones available step 11 which is the last step that I'm covering in this particular series is again ARP you've done caching and HTTP acceleration some of the other cool stuff that you typically do in in application you use content delivery networks if you've got a global audience you know like Akamai or any other CDN that can distribute your content latency people don't realize across submarine cable or satellite is significant when you cross that line big Pacific etc that the significant amount of time consumed for data to flow through to you seriously reducing that latency and increasing responsiveness IP any casting is you know allows intelligent load-balancing by sending a user's request to the closest available node automatically through IP routing tables again I don't need to be I'm not gonna go into the details of that but it's something that you can use for your application to increase responsiveness and increase you tendency because IP any casting automatically takes care of routing the packet to another destination if that particular server is not available a sync non-blocking i/o if possible for a network for sure but for even for disk there are libraries available that can give you a synch non-blocking i/o incorporate a multi-layer casing strategy so you know l1 cache is in process with your app server l2 cases across a network boundary and l3 cases across a disk boundary or on disk you can use grid computing this is again not relevant for most applications actually but but there are some application so you can actually gain advantage is true through grid computing and parallel processing across nodes quickly just to cover some platform selection considerations you know you're going to be taking spend some time in selecting your programming language in your DBMS to make sure that it allows you to perform all the architectural elements that I spoke about and also is best and most relevant for your you know best suited to your particular application so I mean I'm just putting three three variables out here but you're actually going to select programming languages scripting languages are dbmss caches servers application servers operating systems you go to you will select a large number of different platforms and each choice is going to have an impact on your flexibility and your performance and your scalability options and so on and so forth to choose them with care all the techniques that we learned today can be applied in any order so you know I covered them one by one I covered them in the path for the process in which you typically end up applying them but they can actually be applied in any order some of the other tips around speak about is horizontal partitioning and vertical partition of your database is something you should try and incorporate or design early on because that is one of the things and require dramatic changes to your application if you actually do that as the last thing in your scaling process loosely coupled all modules you know implement a restful architecture or whatever and perform application sizing on going figure out which way to scale identify bottlenecks identify single point of failures and so on and so forth and that sort of culminates my presentation dream so thank you and if you have any questions and I'm done most of the the presentations goal was to actually cover application layer scaling and not your layer two layer 3 devices like routers and switches etc most routers and switches come with native support for mirroring bridging and having having sort of n plus 1 redundancy built in so I can actually have multiple switches multiple routers at each step in my architecture etc and then I can handle during my network planning it doesn't have any impact whatsoever on my application or application design or or the way I put my application etc but it's obviously it's obviously an important aspect all of us have faced situations where we're trying to troubleshoot an application downtime we need to figure out somebody pulled out a network table from somewhere in the application went down so clearly it's a critical piece and it's a it is important the other question is on vertical partitioning for databases right and it might pose it and chain for example if you're using two three tables you can this to actually select when you vertical policy across database here to select table specifically such that they in general you know most people when they design applications or applications will actually end up being a bunch of tightly integrated modules in many circumstances most modules won't even have interdependencies we just end up tightly together into a single gem instance will be this easy to code that way single package you deploy it as a war file and just go ahead and upload into your app server and so in there people ends up happening is you know database server becomes or database becomes like a 600 table massive database where in most circumstances you can divide them into units single loop these 20 tables never talk to the other 20 tables for my application never need to talk to both of them together etc so you can use that for vertical partitioning so that's let's how you make that similar challengin horizontal partitioning is that if you have a lot of static data then that has to go across all the yes figuration data has to go across all servers oh you can store it in one central server and that can become another cluster so again depends on how you want to architected and I would suggest the latter approach because the fact that configuration data is loaded one pseudo really need very performant architecture for that but instead of I will replicate that data throws all database clusters you might as well have one separate database cluster just for current global application data yeah technical nature you mentioned Sequoia and C JDBC and you also said that it's all of them now they've been taken over by continual yes the I think Sequoia is the open source version can continue into the commercial version but you did also say that it's preferable in most cases to use the databases native clustering mechanism versus using a driver or proxy solution what were your thoughts on continuing peckers it's primarily intended for my sequel started out as a my sequel clustering solution and it seems like now that my sequel is continuing its clustering implementation is maturing what what are the benefits of using I am continuing talks about the benefits of having a hit heterogeneous database environment so you can actually have multiple different data we can have a my sequel pass suppose B SQL database [Music] [Music] you know all this so there's a greater degree of comfort level with detailed application native native replication is also faster and it supports out of the I mean it supports principle I look at synchronous replication it starts becoming fairly complex to implement at a DA or driver level as compared to native replication but you know two-phase come out to be natively supported the debase layer itself so ideal circumstances if there's a Metro clustering platform it's available that's native to the reader base was doing it as a base layer I would lean towards that as a choice rather than rather than third party tools but having said that you slow knee for PostgreSQL the application very successfully and haven't had any issues with it it's a fairly mature product post peer school itself doesn't have any native application and slowly works fairly well using simplistic triggers and additional metadata columns very attached to the database so you know so again it depends again on the platform of the database that you database are using if you have the liberty of the luxury of choosing it from the very beginning then I would say it you know start by choosing something that has some native support for application but again you know I don't contradict myself there partly but the thing is that PostgreSQL is a very good database we actually use it very very actively you know it if you start looking at database performance etcetera Postgres uses something called MVCC which is multi-version concurrency control while my sequel and ms SQL use something called locking in arbitration so typically when you want transactional support in a database you can either do it using locking or you can do it using MVCC oracle and Postgres upon the MVCC route and my sequel and MSS could have gone the locking law though the latest version of ms SQL actually does have a flag that you can then you can use to switch on MVCC but again depending on your application I have found in most OLTP applications MVCC is actually faster if you have a lot of scenarios of deadlocks possibly taking place or writes the same database table etc mvcc's actually faster and better than locking an arbitration and therefore post this might be a appropriate database on our DMS choice or oracle might be an appropriate aramis choice for that progressed scenario we choose Postgres in that case you don't have native application but then you have stoney so like I said all of these choices need to be made at the time when you're actually you know making a platform selections but you might be constrained by having to choose one particular already BMS then you might have to look at third-party tools for replication right so when you're retrofitting or when you're trying to scale and you haven't really thought about clustering when you began the process you would probably have to use something like continuum - that is correct and it is being used in production environment in many many successful applications so I mean they have a good product it's not a definitely something that works a partitioning whatever Rohullah wakil you are doing let's say take an example where I want to deploy my application for say 50 million users and but this logic side saturates in terms of performance for say 200 users now when I want to consider number of nodes if I want to partition number of nodes itself I will become say 50 thousand to see one graph or something so how do you think of architecting be fine well first problem that I would solve before even looking at trying to figure out how to manage there is no application where I would recommend managing fifty thousand nodes for the kind of usage that you're talking about so out first try and solve that particular situation so you know I'm staring to your second question sizing is not a simple mathematics of dividing by the number of users or dividing by the number of transactions in fact in most circumstances sizing becomes the biggest challenge and typically invariably has to be done you can do some piecing P products in but typically will have to be done on a production live environments you'll actually see stuff happening larger bolts and then based on that you know come to a sizing conclusion based on that decide harm would expand that cluster so it's it's not a it's not a very simplistic aspect of of course once your application is been in production for long if it does not have a drum hour or a fair amount of time you get a reasonable sizing exercise yourself you'll typically know in your mind they're okay for me a set consists of you know twenty-five servers you know seven of which are database servers 15 of which are you know web servers two of them offer a session store you will actually have that set in your mind and then all your blues each time you want to sort of you know expand from that point onwards you just deploy one more set and you start you know registering users out there but it takes a certain amount of life duction environment time before you can actually get to that mental you know sizing expanding well if you're doing face recognition this is not going to be a it this you know a application that's available to 50 million users simultaneously on the internet it's typically going to be you know either using the either even if even if that is the case then the complex part which is the face recognition algorithm will likely be run on the user's desktop rather than being around on the server side you I really would not look at doing the computation of that part of it if there's an requirement where I'm in some situation where I'm so there's a certain part which saturates for say 200 or 500 losses and also have to provide this particular functionality life possible in general there's a security well in that if that's the case then you have no choice you have to deploy whatever 50,000 or 1 million 1 lakh or you know half a million modes or whatever it is there's no real other choice if that's the business requirements of the screen like I said I would first try and tackle the problem of not having my application saturate at 200 users rather than you know trying factor sort of deployment across a large and as per your experience I mean suppose if these 15 50,000 node I have to handle so managing it free or I mean what could be a correct choice I mean what that by itself honestly is a lengthy I mean there's so many things that you can do like you can use puppet which is a tool for configuration of simultaneous you know X number of servers there are various tools you can use that allow you to manage server simultaneously upgrades hour simultaneously patch server simultaneously etcetera and things of that but that serves the system administration realm more than the application design realm but we can take that offline independently yeah Jackson and all those technologies you were mentioning about the application servers and all I just want to get some of you know citations from you like you are mentioning about Apache and Tomcat and not what if we know if we go with JBoss and all I mean can we can be cached using jabish kashi and all caching mechanisms but j2ee by itself you know has caching built in in fact it we the reason for j2ee to exist though these days truth be told it would spring hibernate and quorum and MVC frameworks jQuery is actually going out of fashion but your ejbs and and objects are by default cached in a container like JBoss etc and things with that so you have teaching mechanisms built in out there you don't even have to bother looking at alternatives again without looking without knowing your application I can't comment on it but these days I would strongly recommend steering clear from j2ee and moving more towards sort of goes with frameworks yeah just because it sort of becomes easier to manage it's more modular you know easier to deploy is more performant except for and things of that but we use this hibernate ORM and it doesn't have a default we need to implement that well you need to implement that in the sense you need to configure it more than implement you don't have to write any code for caching and hibernate I again personal opinion I and this is something like incessantly filed my developers about now and then I typically try and tell them not to use and less wealth in certain applications hibernate is very good in certain applications that prefer not to use hibernate because dot pickers of caching limitations so anything with that I think it does a wonderful job of caching etcetera but because I personally don't prefer a situation where we're an automated piece of code is writing my SQL queries because then I can't hand weak them well I but it does allow that but because it gives you a simple ORM interface developers don't tend to look at the queries and therefore many circumstances when you look at optimization you'll end up set with a scenario where it's actually the hibernate query that's not fully optimized or giving you sort of the best result etc so they're doing a fairly good job of that in terms of improving it and enhancing it just the fact that it abstracts that layer out you know I prefer native SQL wearing directly etc but then you can use a new arm like I batis for instance which actually gives you the flexibility of writing your own queries and gives you an ORM so gives you an object view and then you know for a cache you can actually implement terracotta memcache D etc thank you

Channel: Developer Summit

Views: 60,313

Rating: 4.1540232 out of 5

Keywords: Bhavin Turakhia, GIDS

Id: yeyugGZO2qc

Length: 83min 59sec (5039 seconds)

Published: Mon Feb 25 2013