First Class Development Workflow in Drupal 8

alright hey everyone and get started now I'm kind of going to be walking around a little bit but let me know if you have any trouble hearing me so we're here today to talk about Drupal 8 the new workflow tools that we have available to us and kind of hike and leverage those in all of your new projects what I'm going to present today is just a opinionated framework I guess of tools that we put together at last call media these aren't like internal tools these are all tools that are available through you know open source means so I'm just going to talk about the pieces that we use in all of our new bills and how we've kind of stitch those together the sort of like end result of of this work is the scaffolding project which is linked to in that github link right there if anyone wants to you can check it out it's open source and feel free to flip it check it out issue any create any issues or whatever you need yeah so promising we had an issue at last column where basically we were storing all these new projects and the the process of getting a new project bootstrapped was getting way too complicated I think this this issue really compounds itself in Drupal 8 where you've got a lot of different pieces that need to get put together in order to do things kind of the right way so you know that the issues that we were running into were like how to pull in Drupal core and all of the contribute and furthermore hatch when we need to so you can use something like brush make that's one solution but then you have things like external libraries and in Drupal 8 we're already starting to see this shift of modules depending on external pieces and that's something that drush make doesn't necessarily do very well there is a great tool out there which is composer which we wanted to leverage but which was really hard to kind of bootstrap onto something that already exists so we were looking at not only how to pull in our Drupal dependencies but also our external dependencies whether those be PHP SAS libraries whatever you have need also an asset pipeline so we wanted to have like first-class asset compilation meaning we wanted to be able to compile SAS we wanted to compile JavaScript we wanted to have it minified and that stuff doesn't really make sense to do in Drupal Drupal using aesthetic we're now capable of doing some of that pipeline within Drupal but it's a separation of concerns thing you don't necessarily want to do all that stuff in PHP also sort of the testing process so for all of our new sites we wanted to be able to things like b-hat test and unit tests and that was becoming really cumbersome to set up on every single project coding standards so it's pretty important when you have a lot of developers working on a project that you're following a single sort of style guide and we wanted a way of enforcing that across the board and last but not least environments so this is a tricky one right you have your production environment you have your development environments and then every developer may have a totally different stack that they use locally so we wanted to kind of eliminate all of those issues and make it just turnkey to get started on a Drupal 8 project and really have a lot of these best practice problems solved so yeah and then basically where we were coming from is that we had projects that were on Acquia we had projects that were on Pantheon we had internally sort of built projects and everyone was doing everything a little bit different sometimes you'd use gulp sometimes grunt sometimes compass and so we just wanted to to make it standard across the board so I talked about the coding standards these are things that we don't want to change between projects so in other words we work on I don't know 50 different projects in a year probably and so we want to have a consistent set of coding standards we want to have a consistent set of development tools that we use that would be you know things like brush and gulp whatever and we wanted the interface to those tools to remain the same so when you onboard onto a new project you're not figuring out do I need to install grunt do I need to install gulp how do I compile things what are the commands and the other thing we didn't want it to change was the deployment we want that to be nice and standard whether we're deploying to Acquia Pantheon wherever platform the things that we want it to be flexible in between projects were the packages that are actually used so we didn't want our scaffolding tool to specify the Drupal modules that we're going to use we wanted that to be completely open-ended front-end frameworks so maybe we're using foundation maybe we're using bootstrap maybe we're not using either of those custom code is obviously going to be different on every project and the actual content of the pipeline's what happens in there so like do we minify javascript do we not that stuff we want it to be flexible as well so we wanted the interface to the asset pipeline to be the same but we wanted the pipeline itself to be able to be different and the tests that actually go into it are going to be different as well so that was our problem the solution that we came up with was to build a scaffold to use for starting projects and I want to stress that this is our scaffold but anyone could build something like this it's very easy and it's a very good exercise in terms of figuring out what matters to your organization and kind of capturing all the pieces that are important to you so our goals are quick to set up capture those best practices we want to standardize on tools we want the standard developer experience across the whole thing we want the asset pipeline yep this is all kind of stuff that we covered the two that we didn't really talk about here are the extensible build process so we want to be able to execute builds and to kind of have that be flexible again when we were deploying to different environments it needs a to be able to accommodate things like aqueous settings or Pantheon settings or whatever and then we want it to exclude dependency code entirely and this is sort of a radical idea for for Drupal sites that you're going to have a repository that doesn't have any of your Drupal modules your contribute or code anything like that committed to it but it was kind of important to us that we grab that stuff fresh every single time so I'm going to talk about the tools that we use and again these are just examples of what you could use you can pretty much swap most of these out for something else and achieve a similar result so we're using docker we've really bought into the docker idea and that's not to say that we use it always in deployment like in our production environments but when we want to spin up a development stack the easiest way to do that is using docker for us so I'm going to pull up a code sample in a second here docker is basically just a way to make these portable environments that are written in code and anyone can onboard onto so you run docker compose up and you get a full stack and in our case our full stack you is a lamp stat plus varnish reddish Redis and solar so this is a docker compose file and this again is on that github repository you're welcome to peruse this on your own but so we have all these different services and when we actually want to start our application we don't need our developers to install anything other than docker so we just run docker compose up - Dee it starts the application and then they have that full stack locally so that's a pretty cool thing so the stack that we're using the actual containers are Debian based but you can run it on Mac Windows Linux whatever yeah the other thing so I mentioned not including the dependencies at all one important piece of that is using composer and using it fully so excluding anything that is vendor code so we use the drupal.org packages to specify our dependencies and that's really been awesome for us you just run composer install when you pull the project down and it pulls in all the PHP libraries or modules Drupal core anything else you need you can also apply patches this is huge as Drupal 8 sort of matures this will become less and less of a thing but for right now there are a lot of patches to contribute and so you can specify those in your composer Chaisson this is the one for the scaffold and we just specify here that we're going to use the drupal.org packages and then we require a couple of cages and like I said the actual application here the scaffold project we don't require anything beyond Drupal core as far as modules we use brush and we use console and then beyond that it's pretty much all development dependencies feel free to stop if anyone has questions MPM so yeah one of the big things was one of our asset pipeline to be completely detached from PHP we didn't want interdependencies and we wanted to be able to essentially not can have that asset pipeline be a factor in production deployments we wanted to build our our artifact and then deploy it and not have to worry about nodejs or anything like that so we use NPM in our build process to kind of get our dependencies gulp whatever and then ship them off you know I'm going to skip the package file it's nothing to really look at Bower is another package manager it's more used for like front-end stuff so in our scaffold project we pulled in foundation ZURB foundation and we do that using Bower one of the really nice things about doing this is that we can then update foundation to whatever in the newest point release is just by running Bower update and so we're not actually required to have like a sub theme for the ZURB foundation theme and update the Drupal theme in order to update our CSS framework so it really separates the concerns very nicely all right static code checking so this is something that I said was important to us we wanted to enforce those standards Drupal core has really led the way on this and so we basically just hooked on to what Korn is doing yes lint is a JavaScript validator or linter so core defines a set of rules that you should follow for Drupal coding and we literally just include those rules it's very simple our es lint configuration is just extends the core yes link coder so PHP CS is a static code checker for PHP the Drupal coder module or library I'm not really sure which it is these days is a set of rules for PHP CS that essentially follow the Drupal coding standards so it can do things like detect certain security issues if you're using database parameters in an insecure way but also things like spacing and and the things that the little paper cuts that drive us off crazy so we do this during our CI process and and it makes things standard yep there was a question up front yeah yeah we we do encourage people to run these these checks locally before they push anything up but in practice a lot of stuff ends up getting pushed up to CI and and rejected they're better there than you know on production uh so yeah be had is a thing that we use sparingly it's slow it is a behavioral driven test tool for Drupal or for any web site really really you know you can use b-hat in a ton of different ways some people use it as a full test suite for their site we tend to use it more just to test really complex like JavaScript functionality it can actually execute JavaScript using something like selenium or phantom Jas and click through your website to see what happens the other thing we use it for is to test command line functionality it can go out and execute drush or whatever you need so in our scaffold tool we have two tests to find we just hit the home page using b hat and then we execute drush and if both of those run fine then we know at least the site loads both on CLI and in so we recently had a client that was pretty concerned with their front-end performance and so we wanted to add a tool that would do some sort of basic sanity checking on front-end performance so we came across this tool Fanta mosques and Fanta moss is a wrapper around phantom j/s which is a sort of JavaScript headless browser tool and it lets you specify some metrics that that you want to test for so you can say assert that there are only I don't know 20 HTTP requests made when you visit the home page oh yes so this is that the tool and it's a command-line tool we use it we wrap it in gulp and then these are all the things that you can monitor and it's really in-depth as far as like what you can monitor you can monitor how many non gzipped requests are sent or responses are sent how many domains you hit how much how many different dom elements you have all sorts of stuff but we run that as part of our test process and you know it's just about finding those smart metrics to to set in there for like the initial settings and then you can assert that they're there kept throughout the life of the project yep so is this something you can use to learn a performance budget like yes exactly yep yeah it's just if you kind of like codify your performance budget and then put it into the Phantom master configuration you can just check constantly that your that your performance budget is being that okay on to the asset pipeline in here think most of this stuff is probably stuff you've run into before autoprefixer may or may not be it is a tool to do all of that crazy browser prefixing that we all love to hate so the WebKit border-radius whatever all that stuff you don't need to worry about that anymore Auto prefix are pulled from the database of can I use comm which if you use that site before is really comprehensive in terms of what browsers support what prefixes and attributes so it'll prefix your code if you type in you know border is is probably not a great example because I think most browsers just support that on prefix these days but if you were to type in order radius into your CSS and set a border radius autoprefixer is capable of saying okay well IE i don't know eight doesn't support that and you've specified that you want ie eight support so we're going to prefix this for you so it's really dynamic and it lets you just write your CSS without worrying a ton about the backwards compatibility of the prefixes anyway so as as browsers advanced and you know you let's say over the life of a project you drop ie9 support it you can drop that from your autoprefixer config and it'll drop any of the prefixes that have been added for ie9 I've just had also specified that you just want browsers greater than two percent market share or whatever no model prefix throat to those routers yeah that is awesome and I assume that that 2 percent market share would update as the can I use database updates yeah so that's a really cool feature and again it can kind of let your your CSS just become leaner over time so you don't need to maintain it sass any less people in the audience all right so yeah we use that less people yeah I imagine there probably are but ya know we use Sasson and we're pretty opinionated about it it's just the tool that we chose and it's very powerful not a lot of really want to say about that we use it with foundation which I'll get to in a second and then as far as other tools that we kind of have wrapped up in our build process we use minification of both javascript and CSS in auto generation of source maps so your s CSS files or your dot J's files when they get minified and you're debugging them in the browser you can actually trace it back to the the line of s CSS or JavaScript that it came from to wrap all this stuff up the tool that we chose to run all this is gulp you know you could do the same thing with grunt you could probably do the same thing with a lot of tools but gulp is a really nice streaming build system to JavaScript task runner you can customize all of your tasks and there are just thousands and thousands of different plug-ins for it so we use this to tie it all together and this is the the common interface that we have for our developers this is our gulp file in the scaffold you can see we're including a whole bunch of plugins and then we have our tasks the steps that we chose here are install build test and note that's it install there's a top-level task here called install that has two subtasks and when you run the install task it goes through and and runs the subtasks so in this case our subtasks of install are install the composer dependencies and install the Bower dependencies down here oh I forgot to check yeah so check is static code analysis and again we have a couple of sub steps we have PHP lint which is just bare-bones does this PHP code execute then we have PHP CSS using the Drupal coder module and then we have our es length check so this disk task actually doesn't need the site installed at all to run and so this is the first thing we run on CI just check make sure the code looks good if it is we can proceed to you know whatever testing steps we have down here under test we have just the top level test a command and that runs those subset sub steps which are B hat and performance and so we've talked about B hat performance is Fanta Moss it's a little bit more code here than I'd like for the Phantom ass stuff but essentially it just runs a command down here we have a build test or build task is where we run the asset pipeline so the sub steps are build SCSS build javascript build copy and then we have a build watch to sort of just sit there and watch for changes I'm not going to dive too deep on these steps unless anyone has specific questions but that's about it for our gulp file and then what we do is we include a configuration for so that stays more or less the same between our projects we could edit it there's no reason not to but we just prefer to edit this config file and this specifies all the different packages of s CSS that we're going to compile gives them you know a source the maps directory relative to the place that gets output the autoprefixer prefix we use so we just chose last two versions here and the destination so this compiled CSS gets output into the theme directory the theme directory is dist subdirectory and we include our bower components in there as well so that we're able to reference those from our sass files javascript pretty similar copy is let's say you had font files or images that were included from a bower package you could copy those into your themes disk directory specify the patterns you want to check for the PHP analysis JavaScript patterns and this is where the performance testing goes and so you're asking about the performance budget before and so like this would be a search there are only five requests on the home page if there are more this test will fail so when we run a gulp test and it goes through runs p-hat and faith Amma's it'll check against those those metrics that is up there so the theme base themes are dead I think that in Drupal 8 with all of this package manager' magic available to us there's very little call for using a base theme if you're going to be essentially implementing the whole thing anyway if you are going to go with a packaged theme obviously that's not that's not the case but what we prefer to do is just create a bear theme it does have a base theme of the stable core theme by default so it gets all the templates and stuff from there but we just pull in the foundation library on top of that and then we have a very minimal implementation where we just sort of like wrap the messages tabs bread crumbs in the pager in foundations markup so it doesn't look great out of the box but it's all you need to get started on a project and so you see yeah we dive into that and so this is the theme directory very minimal and we just have our scaffold SCSS where we pull in foundation and again that would be included from Bower so that would be the latest version we run foundation everything which essentially outputs all of the foundation classes and then we can import anything we need on top of that in here another important thing is that foundation allows you to recompile with your own variables so we set you know things like link color and padding for the header and that kind of stuff in this global data CSS file and foundation gets compiled after that and so foundation is every time our own custom build so it keeps things nice and lean and we don't have a base theme to really worry about inside our templates ugly obviously we're using twig here this being Drupal 8 and we just have a couple you know status messages that kind of stuff pattering libraries so I know Chaz is here and people knows the pattern labs we stole that idea and kind of gave it a Drupal flavor this is a really part taken but I backed away as far away okay all right borrowing that it is out there and so yeah the idea is that you have this living style guide and it's it's very minimal our implementation here is passing through the Drupal theme system for better or worse we specify different patterns that we want and those can be atoms molecules or atoms elements or molecules I think is what we support your your theme can implement style what does it pattern stop PHP so your theme can actually have classes which is kind of cool and in here you specify your patterns so we have a status message pattern we have a typography pattern we have a list pattern and then in some of these let's take a look at like status messages so the difference here between something like pattern Labs is this is passing through the Drupal theme system so we're calling theme status messages on this theme we give it the message list and what we get out is the exact markup that you would get from from running when you see a status message on the site so here is what that looks like right there so we've used this as kind of our way of separating the markup out a little bit but we also sort of acknowledge that rupal has a lot of weird quirks and we don't want to be writing Twigg templates that won't work with with Drupal so at least for the time being this is the direction we're headed in all right so the last but not least all of this stuff kind of depends this whole idea of not committing things to the repository is great except that then you need to put it on Pantheon or Acquia platform or wherever and you're going to need those dependencies and so this depends on having a CI service in the mix to kind of pull all this stuff together and the CI a implementation is actually really really simple we have those kind of common interface tasks that we run and those correspond with how we build this site so if we look at the Travis yeah mo for this thing you can see that we've got a nvm install to set the node version correctly we've got an NPM install and then we run gulp install which remember installs both composer and packages packages here we have a composer validate command but really it's just gulp check if check passes we go on to the build step and then we would go on to the test step assuming all that passes we could then release this thing which means pushing this to an upstream repository so in this kind of setup you're going to end up with a an art scaffold repository and you're going to end up with an artifact repository and your artifact repository does contain all your dependencies we built a script for this let's see is really just a shell script and this you could replace this with your own implementation very easily so it's just push the down stream and so we specify where it needs to go we specify the branch and a commit message and it clones it clones the the downstream repository builds the site and pushes it up pretty easy yep how do you guys believe in you had a platform of sex using this because they know what clever mistakes they kind of have their own build system in place if you don't even have to worry about it Jenkins or Travis we haven't we haven't deployed a platform yet but yeah I mean if there is a built system on the platform I know like platform has something I know aqueous building pipelines you could totally use those yep I just yeah historically Drupal hosting has not done any sort of CI and so you've needed something like that yeah as far as where this thing is going in the future for us we're thinking about you know separating out the web accessible files so we don't want to have people be able to hit vendor libraries in the browser we want a lean kind of production mode right now you're getting all of the composer like dev dependencies and everything shipped up to your artifact so we're thinking about how to how to set up a build process that doesn't have all of those extra dependencies we want to get PHP unit in there and running and and set up so that that's easy on each new project keep going with the pattern library I think that's a valid idea sort of this hybrid of like a pattern labs approach mixed with the Drupal theme system and then asset pipeline so the tools just keep developing for this there's stuff like browserify now where you can include different stuff from s CSS and JavaScript and have that get pulled in automatically and then things like sprite generation and stuff like that and then finally I think there's just a lot of work to be done in the community around figuring out what to do with these artifact builds whether they're a good idea at all whether they're going to be replaced on each hosting platform by a CI service or what that's about it we have a few minutes for questions if anybody has any okay we're still deciding Game two those guys yep so the slides are actually linked to from the repository yeah they're right here

Channel: Bay Area Drupal Camp

Views: 3,238

Rating: 4.875 out of 5

Keywords: BADCamp, Drupal, Drupal Camp, BADCamp 2016, Open Source CMS

Id: 7veL-Gll9jA

Length: 36min 15sec (2175 seconds)

Published: Tue Oct 25 2016