The future of Drupal MultiSites

hi welcome everybody to drupalgov2020 my name's nick shu i am the track host for ops and security so you'll see me quite a bit when it comes to these these sessions and introducing folks um yeah i just want to point out that this is you know this is a very early bit of software or you know this is kind of the first time doing this especially for me so if you have any feedback or questions or don't hesitate to ask i'm going to be fielding the questions um but i'll get straight to michael so um so i saw michael present in it was drupal south 2017 and he was the keynote speaker and i just saw so much passion for technology um and that resonated with me a lot so so everybody's in in for quite a treat today for his his talk so um i will cut over to him now and and we'll get started all right thank you and yeah hello everybody um super happy to join and [Music] this year's unfortunately that we cannot sit together but it actually makes it easy for me because i don't have to travel for 30 hours so um but uh yeah today we want to talk about drupal multi-sites like nick said i'm very passionate about technology and i'm very passionate about not using multi-sites so um yeah let's jump right in um like nick said i'm michael schmidt or people call me schnitzel in the internet i am the cto or head of technology at amazio and we are a hosting company and we do draw a lot of triple hosting and especially really big drupal installations and so that's where my knowledge comes with multi-site so before we jump in i shortly want to overview give an overview of what multi-sites are so multi-sites really have been started in the drupal community quite some time ago and the idea was how could i run multiple drupal installations with only one single code base um the reason originally came from like super cheap posters where you could only upload one folder but people wanted to run multiple drupal sites and and so what they did is they figured out a way how you can do this we're still having different databases and also different file storage so that's important to know that and a multi-site has this same code base but different databases and different file storage and also each site of course has its own domain and so the idea behind was really like hey let's do easier upgrade and security patches because if we have a shared core base i apply one single change and it applies to all these sites and that would be really great and so at the end it's all about centralization having centrally managed systems should make things easier important to know is that domain access is not a multi-site so domain access is actually the same database with just different domains while a multi-site is the same code base but different databases so how do we how does drupal internal solution actually look like um when you install a multi-site so we see here we have some shared core modules and themes and then each site has its own modules themes files module themes files modding these files and they also have each their own database and you have a file called site sites.php where you define which site actually reacts to which domain and that's it so and that's a super simple and what exactly multisite can do now what i see is that this system now has been used for a lot lot more than it ever was intended to be and so people today they run multi-site installations with 500 a thousand the biggest one i've seen are like 3 000 sites that are all running and these are like not just like simple sites these are like enterprise grade or government sites that um they really need to work and there's a lot of different people that are working with them and handling them and so the problem with this system that we see here is that you cannot do individual updates so you always will update all sites at the same time and if one of them breaks and you need to roll back code this will apply to all sites so there's no way for you to do this there's also no real way to test core upgrades because if you update one side it upgrades all of them now what you can do you can run a whole separate installation of a drupal multi-site but that's going to be very complex again and then also because the individual sites they don't have code the core code in them it's very hard to do a development environment because you will give everybody access to all the sites automatically and so some people they would like to have individual git repositories for each of the site and then permissions but because it's one huge drupal site it's never going to be possible to only allow access to only one site for one developer and cannot access the others so it's going to be a mess and quite fast so we set out and we try to find solutions for our customers together what can be done and we came up with containers and and container images and the big reason there is specifically a the cool thing about container images they can inherit from each other so you can load them and i will see this later and also because each site creates their individual container you can have them individually upgraded you have individual security capabilities meaning you can define who can access what so let's look at a regular container workflow how do you actually deploy a drupal site with containers the first thing that you need is some kind of container that has php and php config in there this does not actually have drupal code yet it's just the php and everything that is needed to run a drupal this usually exists in a git repository and um it's like it said it has everything that you need but it does not actually contain any drupal code and in our case we use official docker images so the php for example will automatically update it when the php community releases in your image so you always stay up to date with security and then so these images here there are multiple people out there that maintain them and one of them is for example um is our company we um have an open source hosting solution called lagoon and we have drupal container images that we maintain but at the end there are other people out there that do the same and so you maybe don't even need to maintain them yourself if you want to use this and so let's continue there um now what we also have we need to actually bring our drupal code and so this is then core module themes and any custom modules and this is then in another git repository and the important part is this from here so this container image can now inherit from that previous or the upstream container image and you see this here so in the docker file for the site i say from php7 to cli drupal so i can use another image and add additional stuff like my composer json composer lock and then i can run composer install and so in that composer json i then install all my drupal the trash whatever modules i want to install in my um in my site so this image uses container images as upstream it brings super core modules themes whatever and the build process of the hosting solution actually will create of all these files it will create running containers and we'll run them so at the end together we have one fully built container image that is capable to run containers now this is the standard and workflow if you use any container-based hosting solution this is what's going to happen now what we can do is we can bring in a base image and that's where the metric is going to be now we start with the exact same so we still have our drupal container images and we still have an image that has a from there but we're going to call this the base image now this base image looks exactly the same as our site before and that's really cool so we still use the from the lagoon we have all the composer stuff in here it's just a regular drupal repository and it also uses these upstreams you can add drupal core and any modules that you want to share and and because this is this is a regular drupal you can actually start this drupal and test it and verify that it works so this means you can do automated testing you can make sure before you publish a new image that people can use this and what we do with this instead of creating actual containers we publish this image to a container registry that then other people are going to use so what we're going to do is together we're going to publish this base image now publishing meaning you can upload this to a container registry like docker hub but there are many others um that you can share and containers around or container images around and now the really cool thing is the individual site now will just bring their site code and they from or they use the upstream this space image so how does this look like so in our case the docker file says organization base image and this is the the image that we just published before it will then bring its own composer and composer install but in the composite json you can see that we don't need to install the drupal core anymore and also we don't need to install any of these other modules they're already there so all we need to install is additional modules that we maybe want for this specific site so this image uses the base image of that substream you can add any customizations via the composite json at the end it's just a package manager but it's really cool because this is docker and you can run this locally you can completely run this locally without a problem and if the upstream changes something let's say there's a new php version you can just update this and rebuild and the changes will go through all the images and will update but beside of this it's just a regular container workflow you have containers you push them and done so in the end the actual containers that will run will use all these three images and together they will form your actual fully built container image and that is going to be used to run later now what can happen we cannot only do design images once we can do this many many times so we have for example size one that is our production site that brings modules and themes we have site one development that also brings modern themes we maybe have a production environment for the site too that only changes a theme we maybe have even a site three that doesn't change anything it uses all the core and images that are there and you can have as many sites that you want in this system so it really gives you the flexibility to add them to update them to add additional sites if one of the site needs deaf environments no problem you can even um update individual ones so you don't have to update all of them at the same time because you can define okay maybe the site three is um is a test site and so you update that one first and only when that one works you can then update all the other prod environments or you say i update the dev first and then all the prods so you have full flexibility and how to do all of this so let's look at the advantages of all of this each of the site has its own git repository its own containers at its own environment that means you have no problems in terms of upgrading them individually and you can give individual accesses for uh for users so if you have a developer you can define okay that developer only has access to one specific site and not to the others and also these sites cannot interfere with each other if one of them has really bad code the code does not run on the other one and so you have no problem there so if one fails there's no impact on the others and then core modules are still centrally managed which in the end is the core idea of drupal multi-sites we want the central place where we can manage all of this and we can also have multiple environments per site so if you want to have staff staging production qa acceptance and whatever each of the sites can have individual amount of environments what that specific site needs local development is super easy and so we worked with that um with the australian government so with golf cms and this is all based around this and so in with govcms which is the centralized triple hosting solution for all of australian government we have around today 300 production sites that all are based on the space image system each individual git repository and or each individual site has their own git repository and in this case they can only upload themes and so we make sure that for specific sites they can only upload themes others they can maybe do modules so you can also have different permissions but each site has its own development and staging environment so if a team updates their sites their theme and this then just automatically updates the development environment and and that goes in and what we can do we can update all production sites within a couple of hours and this can be fully automated and we basically just need to monitor this so let's say um drupal brings out the new core module or a core version we update the base image and this will update all fights fully automatically from one single place so in conclusion multi-site installation with base images are super cool they're easy to use you don't have any stressful deployments anymore because you update hundreds of sites at the same time you still have individual permissions and access per site and of course everything i told you now actually works for much more than just drupal you can use this for any other system and we have customers that use this like for laravel and other other systems because we don't depend on the application itself anymore to support any of this and yeah if you want to see this in place or in action we have them environments we show all of this so if you want to give it a spin play around with it talk to us and we're happy to help and that's it sweet cool so we've got a couple of minutes left and i've just been going through the um the forum so um so the first uh the first thing was asked a few times by uh uh i'll read michael's so actually no sorry uh sample sample setup so is there a sample set up that people can go and look at this type of approach yes there is one that we're currently upgrading it doesn't exist right now so i cannot give you access to it but it should be there in a couple of days if you want to see it shoot us an email and or contact me and send me an email or contact us at amazing io and we're i'm happy to give you access to the environment um as soon as we have it up to date um because there are a couple of changes that we're implementing right now but yes they will be there cool um the next one oh actually uh yeah this was michael so the downstream composer.json isn't aware of oh sorry the auto refresh is killing me uh so the downstream composer.json isn't aware of upstream composer.json went when you have dependency conflicts um yes and no so what i showed you is a bit of simplified version uh for a 50 minute talk it is aware of upstream composer jason how this actually works is we're also publishing from the base image composer jason we actually publishing a composer package for this as well and this is how the downstream composer json is aware of the upstream and you will see this in a specific um [Music] in the demo how this exactly works cool i see there are some questions in the q a as well yeah i was trying to go in but it but the screen just says uh closed for me i can see that okay i'll let you feel from that um so the easy one the close-up of the t so this was drupal uh no the devops days austin 2017 they did a monster of devops shirt that i really like um have you played with multiple frontends for a single end or multiple backends asked stewart yes um we have done this and we actually have it in place so i would but i honestly don't really understand what this has to do with with that specific of drupal but yes we can also run multiple front ends with a single um back end that's possible then another question we have 50 seconds left um how do you handle databases like each site will have its own database and you pull correct every in site has its own database and you can use like site aliases to synchronize them and so that's like any other environment and then sean asked are there any performance reasons why having distinct project is a good idea yes and you can individually scale them so each container can scale automatically and so each side can automatically scale so if one side sort of needs a lot of traffic like in covid situations you can only scale that side without actually needing to scale all the other ones or all the servers and i think that's it if you have more questions join our booth i'm happy to answer them there no problem at all thank you

Channel: DrupalSouth

Views: 608

Rating: 5 out of 5

Keywords:

Id: baRBWDn-J0w

Length: 19min 0sec (1140 seconds)

Published: Wed Nov 25 2020