Rocking the Gradle with Hans Dockter

the first thing I would like to talk about is why cradle why another build system right can you hear me well even in the last rows so requirements for build systems have changed tremendously in the last 10-15 years I remember when n came out I was so happy right I could I could finally compile and jar my stuff up and copy some stuff around that was all I wanted to need to do basically and I finally didn't need my ID anymore to do that right so that was before test-driven development that was before agile development so I don't tell you how old I am that was a definitely way before DevOps that was definitely way before the whole idea of continuous delivery right and all this advancements in software development put a complete new load on requirements or onto the build systems right building the jar and copying some stuff around and compiling is the tiny fraction of what you need to do when you automate your enterprise you have a heterogeneous software stack Scala JavaScript mobile applications C++ and when you want to do full continuous delivery you want to you need to unit you how many guys you need to you need to standardize on this heterogeneity you want to have a standard binary dependency management for all these different platforms you want to have a standard workflow and it's staging that the software often it's to collaborate java needs to talk with C++ or vice-versa etcetera etcetera you have you have a software stack that has grown in size the average software stack is much grow much bigger much larger than it has been ten years ago so you have you have performance issues feedback feedback cycles are often way too long so when you want to do agile development it's even more important to get that in shape right that means you want to integrate very often teams that work across continents want to integrate very often not as in the old days where everyone was doing a job and then after one year you try to integrate it right so the requirements have changed tremendously and out of out of those requirements Gradle basically was was created the heart of Gradle is doing the heavy lifting right for for the requirements of nowadays project automation and one important important part of that story is it's the it's the difference between declarative and imperative build systems right what what does it mean how does this affect the design of cradle I guess you all known imperative build system in a java world which is m and you all know in the clarity of build system which is maven so who of you is using and mostly for building java stuff who views using maven who view is using Gradle thanks Peter for them so yeah that's that's kind of the market share we we used to sometimes end is more than may even sometimes the other way around okay so let's start with some demo of grail and action to illustrate some of the concepts I will be talking deeper about in a couple of minutes so this is the imperative layer of Gradle right you we don't call it targets we call it tasks and you can start with an empty Gradle build script right and create a task and you can execute it and it says hello world Wow there's more to say about this imperative layer but you have this this is this is the low-level layer of Kreitler right so now let's say you want to build a Java project right well you could now do it the end way compile task create a compile task a process resources task test tasks etc but you have a more collaborative way of doing this right so here we have a Java project production code lives in sauce main Java test code lives in sauce test Java some might notice pattern and okay what can we do well we can't a cradle hey cradle this is a Java project and if we now ask Cradle what we can do with this oops we have now a full lifecycle of working with the Java project right can you read well in the last row otherwise they are still there are more seats available okay but it's not impossible to read okay so you have a full set lifecycle for dealing with the other project you can build you can test you can compile you can jar you can Java doc et cetera okay so let's try this out let's try to build this Java project we have some compile error well we need or Apache comments collections right so let's add that so we tell Gradle hey get your repositories or get your dependencies from maven central and we declare for compiling we need comments collection here we go run it one more time yeah another error another compile error what is missing well J unit right so we have to tell hey cradle for compiling and running the test you need a unit good we add that boom we now build and everything should be fine okay so so pretty similar to may even write you just say the Java project declare some dependencies and you stick to certain conventions and everything works out of the box okay so if we look now in a built there's a build directory the convention is that the jaws end up in the lips directory well this jar doesn't have a version number so let's declare a version for this project 1 0 and let's reax acute the build and here we go there is now a Java standard 1 0 dot jar ok now let's add some requirements the requirement we want to add is we have integration tests we want to run as part of this project right and those integration tests have specific external dependencies that are only that only applied to the integration tests like XML unity okay so how would you solve that requirement with mainly I want to run a integration test with specific external dependencies within the same project I compile my production code and I run my unit tests maven build masters provide your solutions right so you would you would write your own integration test plugin so the recommendation as the recommendation would be create another sub project right basically the maven way is that you don't have your integration tests in this project the maven way is create another sub project for the integration test that works right it's just not the way you want it to do it okay the tool wins the framework wins another time right but can be done that way how would you do this with ant pretty easy just copy your sorry yeah please with with an an IV oh yeah and an IV you could easily solve this yes yeah absolutely that's that's my point with an you would just copy another job our target another copy target for copying the resources and and then a test target that executed with IB you could deal with the external dependency management and would easily solve this problem not very declarative right but you could easily solve it which is good the absolute quality of end right it hardly locks you in until things become unmaintainable right yeah but that depends a lot also on the skills of course of and built masters so car definitely wouldn't accept the attribute unattainable for the Netflix for good reasons I'm sure so okay how can you do this with Gradle well you could do the end way right you could say I create a compiled task and et cetera et cetera but can you can you solve it declaratively so here we have the integration tests right source integration test Java so let's let's tell it to Gradle hey cradle I have sources I give them the name in tech integration tests they live in source integration tests Java and the resources live in source integration tests resources okay and if you now look at the tasks you see there is an intake test classes tasks you did right I could have declared it directly all those tasks but I have here in declarative element sauce at intake test and my declaring I have sources by telling cradle to what I automatically get the how what can I do with those automatically get an intact test classes tasks that compile stores make sense without explicitly declaring these tasks so that that's the so it's a declarative way of saying I have additional sources right let's let's execute this in tech test classes class oops again compile error after compile arrow okay couple of stuff is missing a unit is missing ah or cradle person is missing which is part of the production code well in my integration test I usually need my production classes to do the integration test right another thing that is missing is Orca XML unit which I which I need so let's take ray LaHaye for compiling those guys I need the output of the main source set the Java plug-in is adding two source sets with the name main and test one represents the production code the other represents the unit test and I can access them I can I can ask them hey main source set give me your output and this output should be part of the compiled class path right for the integration tests okay now what do I do with the external dependencies well I declare an additional scope right a dependency configuration how we call it I call it in tech tests and I tell it this intake test dependency container I should contain all the dependencies of the test runtime container which has all the dependencies of test compile et cetera et cetera those ones here compile test compile test runtime they are added by the Java plug-in and I can further at dependency containers right so and now I add a dependency to this intake test dependency container and then I can check Gradle hey the compiled class class for the integration tests should have all the dependencies assigned to intake test right okay so now let's compile the whole thing and let's see if that works yes successfully compiled ok let's continue let's execute the tests right so we create we create an integration test task specify the test classes door saw sets in tech output classes to end the class pass for running the tests where we say the runtime class part of the intake test we haven't defined that yet we define it runtime class path is the output which is the output of this intact test object the compiled integration test plus it's compiled class path right that's what you need running the integration tests okay so we run them and everything should be fine well not strictly so where we are okay let's first see that they are that they are really have been executed our nice test reporting and personal integration test has been executed right plus person test which was a unit test so one thing we haven't done yet the Java plug-in defined a particular life cycle for a Java project this integration test task is not part of the life cycle yet you have to execute it explicitly but if you tell cradle built a whole project integration test won't be executed so the Java plug-in it's a check task as a life cycle task to a Java project and all the code quality related tasks hook into this check task so the check task depends on J unit tasks they depend if you add them on a check status on fine park tasks you get the idea and now we tell the tractors hey you also depend on the integration test task moon we can now tell Gradle build a whole thing and yeah everything is built including that the integration tests are executed so so you took basically the standard life cycle for a greater Java project and extended it with your own requirements right okay so now this first this Java project is not a Java library it is a java application it has a method it has a class person that has a main method right so you want to you want to be able to do something with this application you want to build a distribution with start scripts and you want to you want to be able to run it from the build script to see if everything is okay so how could you do that well I already told you how to do that again two ways you could do it the end way create run tasks that that that that calls the main method create this zip toss that builds the distribution etc but you can tell cradle hey this is not a jar normal Java library this is an application the application name is super app the main class name is orc cradle person okay so and you see now by adding this you now have an additional in zip task and install app task and to run task let's call the run task the run task compiles everything and executes the main method the application basically write the install app task builds the application and you can see there's known install directory with the super app directory which has a bin directory with the start scripts and the Lib directory with the needed with the necessary dependencies right so and it is extremely easy to extend cradle to extract the specifics of your company into into a plug-in and to share it with the rest of the world so the application login is an example how we extend the Java plug-in in in the cradle cord institution but you could easily extract stuff with cradle so I created a create a directory here I create a file called integration test cradle and I now go to my build script and cut out all the stuff I've added for dealing with the integration tests and now I go into my project and say hey apply the integration test plug-in so and that way you have defined now your own extended standard this is this could now be your enterprise plug-in apply to any to all the projects in your company and there is now the notion of integration tests being part of your normal Java story right and going back to yeah how could you do this with maven write it norm the answer would be create another sub project which works but that's not the way I want to how I want to model that in grading we have integration tests that are that are testing the whole distribution they should live in a separate sub project but we also have integration tests we have 30 or 40 components right which are component specific with which only test the IDE aspects and those integration tests they belong to this project I don't want to have a tool that tells me no they must live somewhere else to score doesn't know I want to live it there that's where they that's where the code belongs now if we would follow this pattern where that that we would have to create for every component a component - integration test project we wouldn't have in Eclipse nor for the project we would have any project at one point that's the scalability thing the Refresh in Eclipse is not your best friend right so so we think it's extremely important that you're able to model the stuff exactly how you want way how you want to have it right so declarative yes rigid no only when you want to be Richard with your developers you can tell the story with Gradle but the build system shouldn't be Richard with the build masters of the world they know they don't name on the main better than the build masters we provide standards if they suit you use them but you must be able to extend them and to tell your own story and now become basic it's that this lies at the heart of cradle there is a fantastic interview a couple of years ago between arashiyama the gang of four arashiyama and Bill Venice on our team are calm and Arish talks about the dangers of doing frameworks big frameworks right and and he's saying the danger with big frameworks is at the beginning it's fun you get all the functionality for free but once you haven't requirement is's that is unanticipated by the framework you start to fight with the framework well and you start to lose 100 percent right this was the case with EJB to container manage persistent who has survived this no geez and this is the case with some real systems which I also have survived so so what is our answer to that so at the heart of cradle is a built language right a toolkit because that's Irish Gamez recommendation build toolkits and optional smaller frameworks on top of this toolkit right a toolkit that has no opinion with some frameworks that have an opinion because the creator Java plug-in is a little framework it has an opinion it has two opinion two production code lives in the same project it's the unit test code so the guys from LinkedIn have a different opinion they have separate projects for production code and unit test code so and for good reasons that one reason is they want to have clear class pass in Eclipse Eclipse doesn't have any class pass scope right so so how would they how would they deal with that well if granted we're just the framework we could say well just the blood of Java block in then your production code project has a test task well but it doesn't really hurt and your test project has some production code compilation step it doesn't really hurt well just go ahead somehow somehow it will do the job right and then and then you hire a new person right and this person works on a production code runs the test hey tests are successful well there are no tests right in a production code so he commits the stuff stuff is broken so then you decide hey every new developer must get his build system briefing telling me all the pathologies all the stuff that is weird it shouldn't be as it is but he has to be aware of no no no that's not how you should model your production domain and that's not how you should model you are a bill to make the same principles of you know of good design you apply to a production code you should apply to modeling your bill domain and in fact there are a couple of smells if you read the refactoring book for you can you can transfer to the bill domain forcing someone to create a sub project just for a couple of classes right that that's for me a lazy project smell corresponding to the lazy class smell of the you have in your production code right unnecessary abstractions etcetera etcetera there's give a whole talk on that topic so okay but what is the greater solution right it's the solution is hey don't use our opinionated little Java framework right but don't go back to Adam and Eve to the lowest layer and now use the Gradle task to do everything on the low level use the build language you have you have very a very rich language to describe a Java project right sauce sets etc this is a domain-specific language for telling your your story for Java projects right so and with this it would be very easy to tell the story what LinkedIn Java project is so integration test is also an opinionated plugin one integration test for a project it might be exactly what you want for your company but maybe you have a couple of teams that are saying hold on sometimes we have multiple sets of integration tests per project right with different external libraries now you could tell tell them or then create another sub project right no that wouldn't be the solution the solution would be extensible you can create your own language elements you could easily I don't have the time to show that like like there is a source sets element to declare sources you could add an integration test element where people could just use it into creation test tell the source directory and everything would be set up a task to compile them a task to run them etc etc and that would be the solution right you might have an opinionated framework but you also provide a language element for people that need more flexibility but for but for them it will be very easy I don't need to do much copy and pasting they say exactly what they need integration tests and that's the whole purpose of of writing the sls we come across a lot nowadays of companies who are now using Scala preferably for creating DSL for domain expert domain experts in the financial industries domain experts in the biotech industry etc etc right they don't they don't want to know anything about Scala they want there are experts in their domain the same is true for build systems software developers shouldn't be experts in build systems they are experts in what they want to do with their software projects and you should provide them a language where they can easily describe the requirements and not have wiki pages of the wiki pages or they have to deal with that so this is this is or when you can read this in the last in the last row you'll get this book but what build system is that and for some reason where we can you give you you can realize what it is so this is a build system from a company caught emitted logic they do a new building automation pretty complex stuff big building automation right so not home at home not private homes and they have a hundred plus developers and also a hundred plus sub modules and they have a pretty complex matrix depending on the building type and the customer type they must assemble distributions right which are specifically configured so this distributions take a subset of their of their components configure them in a specific way everything on a CD or whatever now you speak whatever and ship it to the building right very high QA requirements there's no way to easily have a hotfix right and enter domain which is pretty sensitive you don't want to screw up the climate how do you say and you know what I mean so climate control so what you see here was necessary for a normal developer to add another sub module this was a lot of copy and paste and those were all different property files they had to touch an extent to add their their new module to the android framework and those were smart guys but they pushed end to the limits right right they wanted to have some something that models their domain but they couldn't hide the complexity so this was extremely error-prone and developers were completely they hated it a dead one points boycotted it is that we don't add any other sub module it's fixed us we don't deal with this anymore so their pain was so high this this is a critical as this is their software stack right the pain was so high that they switched to cradle when greater was zero point five point two our first big enterprise built right so and then one of the things they laughed was that they could create a DSL for their guys so now a software developer doesn't need to do anything with n they they're not even aware necessary that they're using Gradle let's say hey for the for this a product we use we need two sub components we have some dependencies for the dock we have this documentation text requires license and specify a launcher this is a specification this is a story that makes sense to you with a little background iíve given you right for that they're building some kind of of application and distribution this big progress fight okay so one thing that may even really added to the domain was the idea of standardization absolutely and sanitization based on opinionated frameworks with the built by convention approach and they did that long before it was hyped with something like Ruby on Rails so I really give them credit to come up with such a mechanism very very early and and many people love maven because it took them out of there and nightmarish completely non standardized built environments right so and and we for a standardization is a very important aspect specifically for for larger shops and but the main point for us is not that Greater provides some out-of-the-box standards for you like this is a Java project this is nice to have right but the real value add for companies for us is that Gradle is a standardization toolkit that allows you to define your own standards for all the stuff that are done that doesn't come out of the box with the build system and I guarantee you there is a lot I mean if I look at the Gradle build what we have to standardize and and do for our automation which is on a lot not so large scaled and then any significant Enterprise build we think this is this is this is very important that it's easy for you to standardize the stuff and and deploy those rules across the whole companies how to do this we will learn later so another thing so even if you don't care about standardization if you say hey I'm from a pretty small shop we don't have many teams right we just want to have a good build system cradle is not a Java build system where you might be able to squeeze building other platforms somehow into it right it's extremely open and flexible for any platform right Java Android C++ Java Script etc and we are very excited that we are working together now with the Google Android team to make cradle the next default build system for building Android apps and libraries so that that's great news for everyone I think you know also for us this way for the Android world and because there are some issues with building Android apps right and and the Android the Android SDK people what they did before they were thinking about tools they were writing a build spec what should a build system for Android to be able to do and they created the terminology like product flavors and build types and they could take this back or we together took this back and extend it the greater the language to capture that you can say hey Android I have this product flavors a free version and a paid version you can specify that and and under the hood you get all the complex logic out of that and it's pretty complex you know with all the emulators and the specific testing requirements this yeah you can specify the what and to how it's done for you and it integrates deeply with the with the existing Gradle build language so when you specify a free product flavor as a three source set gets automatically created which then can further configure etcetera etcetera so even if you don't care about standardization the fact that cradle is extremely open will will make it a first how to say a tool of choice for you know modeling any domain right and so and give you a very good experience whatever whatever software stack you're using yeah one interesting thing about a declarative elements they are credible I mean it's very hard to query an end build to ask ask an input for anything hey what are your source directory is what are you test directories right you might have some conventions but you couldn't ask the build you could have some heuristics or whatever and having the clarity of elements you care you know I haven't have an object graph can say hey give me all the product flavors that are free and whatever overall my software stack oops so there was the declarative layout but at one but you also well at one point the what is one thing but as the build master you also have to defined a house somewhere right what how should things be done so cradle also has a very rich imperative layer so in that respect we think graders also the better end so on the low level and I will show you some examples why we think this is the case so task dependency inference okay let's take this example from the integration test and now let's add the requirement hey I want a jar task that just up my compiled unit tests for whatever reason I want to have that okay I tell crater hey tests all set give me your output and you specify a naming pattern and and you now can execute this test jar task and what is kind of interesting is the classes are compiled the test classes are compiled and then the joyous build did we tell this to cradle right no we didn't the only thing we specified was that it should use come on then it should use the test output and this output object is a rich object that knows a lot not just that it's not just a string that represents the part right so see here's the Java done a test one hour in cradle the output objects know what needs to be done to create a Content so the test output object represents the directory of the compiled unit test classes but it also knows that it needs to compile tasks to create a content of this directory so you encapsulate that in an object that's reasonable object-oriented design compare this with the end you have a property with the name class sister right and then you have a target compiled tests which pushes the stuff into class sister then you have a test target which depends on compiled test and it's the class sister you have a test jar target which depends on compiled tests and just the classes to her now let's assume the rules how you create a binary of the unit tests are changing let's say you don't compile them anymore right let's say you must get them from some zip zip file in some binary repository you write could be with end you have to go to all your build scripts and and change to depends relation right you have to say everywhere don't depend on compiled test now depend on extract zip file right you have to repeat yourself you say you need to class sister and by the way class is to depend on compiled test used to clap right and so you have to specify this rule over and over again when you change it you have to change it over and over again in greater you can specify classes to a property as a rich object and tell this property hey you need this task to create your output and then you just tell cradle hey you need this output property and the wiring of the tasks happens automatically right so just one maintainability advantage right another thing smart exclusion I mean this is the build master tool let's say I want to build a it depends on B B takes usually 50 minutes some talking generation and now you're debugging something in a you want to you want your working with it you want to execute it you know you have to execute it five times a minute and now B always takes so long so what do you do in end Tommy well yeah but you you might not want to touch the B but would be away another way you might also don't want to touch it you could introduce skip properties unless right okay I mean better than nothing right so but you have to say it explicitly I want I want to define this property to skip this task but then let's say B depends on C and E you also have to say or them also have to define the same skip property for C and for D so you're told and already B depends on C and D and now with the skip properties you tell the same story another time right with greater you don't need to do anything I mean skipping is just a built-in feature and when you when you execute a and skip be the task C and D will also be skipped because B depends on them if none of the tasks that are executed depend on them it's simple but hey it's it's it's cool and I mean other built system don't even have the notion of skipping right you depend on the mercy of their plugins whether they are capable or not and that's what built master painting yes I mean when I have to do execute something over and over again always have to wait I mean it's ridiculous I mean so hey one of the highlights are doing fine time incremental bit who is always doing a clean when the end build is executed when the maven build is executed oh a lot of brave people out here so who is not doing who is usually not doing a clean in MMA even if you don't do a clean it doesn't give you that much performance advantage so java dog is still generated the tests are still executed the the jars are still created the only thing that javac only recompiles the Java files that have been touched but only those not the ones this fire depends on so we have a pretty unreliable compile so you you take quite a bit of risk for not getting much out of it so therefore most therefore I got used to always do a clean right I mean are you using it you built the and checksum tasks to make things faster yeah and has some some stuff where you can where you can where you can check whether output input has changed or so okay let's do it so okay let's see how grater behaves here so we have another normal Java project now let's do a clean build only once just to start from scratch now let's do a normal build and a non clean build boom everything is up-to-date right very fast so now let's change the production code let's remove this field name - and let's rebuild again AHA this was picked up by Gradle right so everything was recompiled the buy code has changed so the job was rebuilt so the class pass had changed for the tests I need to be recompiled and me we executed okay so this up-to-date here means there are no resources to process so did that therefore this is always up to date but the other stuff needed to be react secured okay by the way compiled Java and process resources are created by the classes tab so this is an aggregator task when you say hey I want all my binaries this task depends on compiled Java and process resources to do the two steps that are necessary to create a binaries ok now let's do something else let's remove right let's react Secutor built and ah that's that's interesting it was recompiled right but the bytecode hasn't changed so the J's up-to-date test only to be recompiled etc okay let's do something else let's change the source compatibility layer which is not a file it's a property of the build right will this be detected yes and of course the bytecode changes so everything needs to be redone right ja needs to be rebuilt etc let's change the version of the j-unit dependency for compiling the test will that be detected I hope so yes tests are recompile the rest is up to date but has a recompile and we executed now let's now so far we only changed the input now let's change the output of the previous run right let's remove one of the compiled production classes let's reacts acute the build and Boon recompile but the bytecode was up-to-date so the rest is up to date okay so okay so the interesting question is now this is kind of - this is the behavior of the default greater task how can you use incremental build of cradle for your own custom build logic right can you use it so we have here a mountains XML file and the requirement is for each note of this XML file for each mountain node we want to create a text file where the name of the text file is the name of the mountain and the head of the the head weather and the head of the mountain is the content of the text file so let's just do some groovy stuff we don't need to care about here I create a task generate mountain files with some groovy XML parsing that generates those text files right so let's execute this task isn't that cool gmf you can use camel case task execution on a command line and greater will find the correct task so the big question is now only now first I want to show you that the task did what it was supposed to do yeah let's refresh the Eclipse workspace you have a mountains directory okay to text and Mount Everest text okay requirement solved now let's go back and we execute the task will it be up to date what is your trust in Gradle come all the way from Germany to show you some crappy up-to-date shaking no of course of course it's up to date it's not up to date let's try it again okay today check for custom build code is part of the commercial cradle offerings no no it's not let's revisit this guy here is not my friend for various reasons first of all I mean analyzing what the what what the input and output whoops I don't want it analyzing what the input and output of this of descripting implicit code is would be very hard right the other thing is I talked about declarative build scripts right and in Gradle we always try to separate the declarative from the imperative right the build script the normal build script should be spec like so now we have this the scripting code in there you can easily imagine if this is getting out of hand you have you've completely messy non-declarative how to read build scripts so this is something I don't like so what can we do to improve that we can create a custom task we call it convert Mountain XML and we say hey the input file the mountains XML file is a property of this task the separator feed : number and the output directory where the text files are generated then we have a task action the action that is actually executed by this task right and and now let's get rid of this guy here and make it more declarative say this task of type convert modern XML the output der is built here mountains the input file is source mountains XML and the separator is a column hey this is much nicer right now let's we execute okay we generated now let's react secure with other clean hey everything is up-to-date free and open source for you so now let's let's see but if it's really working here we go kay two texts and one everest text okay so let's change the mountains xml the numbers anyhow crap but let's change them to another curb number and you see no longer up to date we generated if you execute again up to date one more one more thing to try out if we change the separator which is a real property of the task not a file to semicolon we execute it's picked up this change and you see yes please reload we now have a semicolon and what is really cool here i think you didn't need to tell Crale anything about incremental build we just created just provides you a way to describe the input and output of your tasks and this very powerful functionality of an incremental build you get for free and this is so simple you can go home now you understood all you need to do and you know have incremental build but there is more you get out of that let's let's specify an XML file that doesn't exist well we know it's an input and you now get a nice error you don't need your own checking if fire doesn't exist make a nice error message you get that for free another thing that is pretty cool is you can now say Cradle clean generate Mountain Files so for every task you have you get a clean task that only deletes the output of that task another sharp tool for the belt master let's say something is corrupted but you have a build build output that is pretty expensive to create you can very sharply say only create the output of the sky and how can we do that well you told us what are the output doors of that task right so it's pretty easy and you see it's really it's deleted and another thing is you don't need to do a make do well we-we-well automatically creates all the out all the files that are declared as output directory before the task is executed little stuff a lot of goodness we can derive because you you are you can describe the input sample of tasks so we started okay so next topic expect the unexpected expect clean is we're talking to empty chairs that's what I so the bill domain is a complex domain there are many requirements I'm always surprised to hear them right and so many requirements are out there we I would never anticipate as a build system so somehow you have to write a build system you can in any software for very complex and unknown domain in a way that it embraces the unknown not in a way or unknown keep out of here right this is the way to do things right and that's another thing that it's at the heart of Gradle so just we just look at an example we have another another Java project and this Java project has quite a bit of spring related code right so and we want to create separate jars for our spring code that relates to the springy B layer for a spring code that relates to the core layer etcetera the default choice added by the Java plug-in by default it contains all the binaries we tell it hey don't include our spring related code right and then we have a spring Leiby jar as set a spring core jar and the spring message in jar so one thing you can now easily do with Gradle you can't a cradle hey build use the assembly task which is a dynamic task run it and it it builds all the archives you have declared right it's it checks what what what archives are there and it builds them all okay so and you see built by convention as lips directory and here we go you see the naming pattern in place the appendix is spring dB so the name is the name of the project rules spring DB - 1 0 - JA ok that was the plenum preliminary now I don't know if you've heard that spring source has changed their license agreement the license of the new license equipment says that every job that has the name spring in its name must contain the picture of Rod Johnson in the meta and directory of the job can you believe that so here's the picture of rod it's a goodbye present to rot I think so here we go now and build masters of the world how do you solve that with end right you could say okay we don't use the default jars event anymore in this enterprise we have our own macro death and every jar must use this macro death which then does something like that okay maven extent is simply plug in okay how can you do it with Gradle oops damn sorry wrong button okay here we go okay incredibly can say hey Gradle for all ties tasks of type jar which are matching a certain pattern JA archive name contains spring for all those jars at the picture of rod to the meta F directory so it's this blue code or is it real to go to be true right so let's execute Gradle assemble okay one cool thing guys up to date haven't changed but the other ideas have picked up all the meta F probably has changed hopefully let's see let's see what's in the zip in the jar I mean you know fast our video you see rot JPEG and to be sure we look at the standard jar no rot JPEG okay hey problem solved right now QA ex extremely concerned the fees are very high for violating this new agreement and they want to do you know checks now in ten whether those jar really have the picture on the other hand there are some really big bills out there where the jar takes 30 minutes to be built with all the documentation generation so they're saying hey guys give me a task that only built the spring jars I'm not I I fed out things to do than to wait 30 minutes for a jar to be built that I don't need right okay how can you do that well just do a little refectory extract variable or spring jars you have an object model right tasks is a soft type tasks container with type jar returns another filter jar container matching returns another filter jar container hey you know this is a rich object model so this guy here represents a filter task container now let's apply the rule or all tasks in this filter task container should have this map should should contain the picture and then use this filter task container to create a task spring jars that depends on the filter task container a one-liner no you just say hey cradle spring jars camelcase execution boon only the spring jars are built okay now let's add a little bit because it's fun the next requirement you need to create you want to create some checks on files for the jars to better track them right so you can appear area to those spring jars that you say hey for every jar create a checksum task that's great we'll ever check some tasks yes because ant has a checksum task and created ships with end and as part of the greater distribution so everything that is an end is your friend so we execute in the spring jars and you see built lips here we go checksum is there okay now the next thing is in a bit a little bit the next requirement I mean this is now only part of this project but you want to apply this stuff to all your projects in the enterprise right so oops sorry so the first thing we need to do the first thing we need to do is at the moment the picture is it's part of the of the source tree of this project so we want to use a picture of rod that is in a repository and can be retrieved from every project so how do we do that well we declare a repository a maven repository somewhere repo at Gradle or create a demo then we create a custom dependency configuration and add the image that isn't this repository come Micom spring rot one Oh JPEG as a dependency like you can energize the dependency you can also add a JPEG as a dependency that would work with maven or with Gradle or with ivy that you can do that right so and then in the front section of the meta-inf we just say from configuration spring boom we now retrieve the image from from rot from a remote repository and that's the first step to be able to share this logic across projects you see rot one okay peg now the next thing we want to do is to share this how can we do that well let's cut this guy out let's go to get up my github repository I have a couple of plugins there and there is a spring door cradle file which are already put there it's exactly the same code I just cut out of my build script right exactly the same code so I now take the raw URL see this is this is just ul to this file copy it and say apply from boom this is a remote cradle script I create a plug-in and say clean spring jars and unzip what's picture is there now every project in your enterprise can use that it gets even more interesting later when we show you how you can auto apply such plugins that you developers don't even need to apply them explicitly okay so before you tweet SpringSource has changed the license agreement they haven't as you can imagine but the set of requirements that's the typical thing that happens to you when you when you need to do automation this is not a bizarre requirement I've seen more bizarre requirements I've seen requirements like this unit test this one unit test only fails always when you run at the first time when you run at the second time and when it then fails it fails when a dense soup when it succeeds it succeeds and there's nothing those guys could do about it because it was a complex test the infrastructure team was involved they didn't want to do the changes to fix it so there are always situations you have to deal with and it doesn't help you if there is a tool that says well it shouldn't be like this well bloody hell I know that I guess but it is like this and it is not in my power to change that so should i disable the test is that better of course not so you need and this is a when I came across this requirement I mean I haven't thought about this or we when we were creating greater but in Gradle you can hook into the into the test field into the test runner you could you could deal with this requirement and then the other thing is if you have such a pathological requirement even and you would ask us could you please in the next release of cradle at a method fail only after so on so many runs we would say no no no our API stays lean this this crap no we don't if you are big framework the only thing you can do to accommodate the pathologies of the world is to bloat up your API because people can't can't extend it right so you have to wait for the next release and then yeah either way they they screw up the API to have you or not so but I would say hey use the create a toolkit to express that you could even just as a side note extend a greater test task easily to say hey test tasks I add another property fail after so many runs just for your enterprise build infrastructure you can extend even the API of the greater task but please that will never be part of the public API of the greater distribution but you can model that so multi module builds multi-project build I should decide on a terminology so here we have root directory it's called mighty project and we have three components API shared web service so three terms we have now component module project yeah I always I don't know at one day I have to decide which one I should use but anyhow so how can you and they have dependencies between each other how can you model this with Gradle well the first thing you do is to say hey I create a setting start Gradle file and at a cradle those are the components the sub components they take part of this single built API shared and web service okay so you can now ask Gradle hey show me the projects and sorry and you get such a report you got the project structure can even be nested it can have it can have multiple hierarchies okay that was the first step we need to do the next thing we created build script in the root project in mighty project and tell-tale cradle what is common between all the components so for all the components we say they are Java project all the components should get a repositories from maven central and all the components should use a unit for a to for compiling and running the tests and all the components are version 1 0 ok good now the shared component doesn't have a build script on its own it just you it's just happy with with the stuff that is defined in the in the root project so we go to shared and we can now say hey Gradle build shared and shed is built ok boom so API is our next candidate ok shared one oh ja API is different we needed a build script for API because API has dependencies that need to be defined right so the first thing API depends on shared it needs the output of shared for compiling so in cradle we have a specific notations for saying this is a project dependency this is a dependency to another component versus an external dependency right and it also needs comments Lang for compiling ok let's go to API and let's run the build and what is interesting now is that trade will automatically build shared before it build API right so which is yeah pretty cool when you have a larger codebase and your D bagging your application you you often don't know in which components you're you're you're changing stuff is cetera and then when you then run the bill to figure out if everything is working you don't want to manually rebuild the components you have changed but you also don't want to do a full build because it takes too long right so okay shared is not fully built it's not tested right it's only only the joys produced API is fully built okay now let's deal with web service web service also needs to have a built screw on its own because it's a it's a war project it's extense the Java project and it has also dependencies Commons collection Commons IO and a dependency on API so let's build web servers that's what we call a partial build and you see first shared is built an API is built and then web service is fully built okay so let's go to API so and the question is if you did it change to API as a developer and you you build API the tests are running etc are you good enough for committing the code why not exactly so I would like to know so I tell Gradle build API until TPP build all the dependent components as well and you see API is fully built and tested and now web service is fully built and tested web service consuming API right now you're good to commit this partial built that only builds a subgraph of the components plus incremental build is a traumatic time saver to get the average bill turned on you could even say hey build the components of API that depend on API and also built fully the components that is needed by API so shared is now also built and tested ok the last thing I would like to show with mighty project builds is you can of course go to the root project and build everything and that's what we're doing together with the dash dash profile option cradle creates a profile report that shows you the performance characteristics of that built write total build time startup time how long it took to configure the build how long dependency resolution took for the for the different scopes and along the tax execution took yes so we take performance very seriously so you could even do stuff like here we said this is common for all sub projects now we can do something what we call filtered injection we can specify the stuff that is only common for libraries for example in this case for API and for shared and then we defend these define hey libraries are all the sub projects that don't have the name web in its site you can use whatever filter you want that don't start with WEP and that way you can be fine in your route build all the stuff that is common for all the components the stuff that is common for the libraries that is common for the web project that is common for the documentation projects etcetera so very fine-grain way of injecting things not single inheritance based model right you can do some other cool stuff you see here all those built or cradles then what build or cradle is what built our cradle you have now multiple components so you could easily take cradle head cradle I want to have a different naming pattern for all my sub components the build file name should be that it should be the project name dot cradle even this is something you can configure and then you just rename the builder cradle of web service to web service store cradle we should have made this new default behavior of cradle but now backwards compatibility wise we can't change it but it's easy for you to actually do this to do this right okay oh and you can import easily such a multi module built into Eclipse we go we go to our multi see the mighty project build we take Eclipse built a model we tell now Eclipse import everything we could even specify tasks that should be it should be run before and after the import and we say finish and everything is now imported into Eclipse select working set multi-project boom here we go and there is now a creative dependency container in Eclipse and the source folders are correctly configured etc okay now we coming to one up to the hat to the highlight I have still a little bit of time left built masters delight what do we mean by that the cradle wrapper is one of those pieces we we really like let's demo this to you so we have here a normal Java project and we created the build master would create a task called wrapper nothing else just toss wrapper type wrapper and if the build master then execute this task the following is happening if you look here this task creates a crater W it created W be 80 and in a cradle directory it creates a rapid directory with greater rapid jar and create a wrapper properties and this is the greater wrapper properties file so those four files you would commit to version control the next time one of you developers updates from version control to develop a can build with Gradle even if grated is not installed on a machine the developer would just say create a W instead of cradle and then does all what he wants to do with Gradle and then the wrapper would go to this distribution URL and download this specific version of cradle and installs it to the machine of the developer if you upgrade to a new version of Gradle two weeks later you just change to wrap a properties file with a new with the new URL committed to version control people next time people do an update from version control they're used in the new version of Gradle extremely convenient for developers one aspect the other thing is historical builds compliance you now have in version control the build system that is needed to build this software right it's nailed down so and no longer support issues like it doesn't build with my version of Gradle say well interesting used to wrap up right so um okay next power feature init scripts so a lot last section so let's say in all your projects you have the requirement that the repository should be retrieved from maven central so we have here normal Java project we cut this out that the dependencies are retrieved from maven central and we just take some init script and we say all projects repositories maven central this is some script somewhere on your machine right so you now can go to this build testing with Gradle right where we just removed to the repository declaration and you can say hey Gradle build the sky but inject this init script into the build and it works right the repositories are retrieved from maven central everything is fine although it was not specified in the build of the testing with Gradle project now let's do some cooler stuff Grail has a very deep API you can hook into every aspect of the build lifecycle so for example what about a warning when the tests are taking long when a test method takes longer than 20 milliseconds to run would be cool so the cradle' object here is available in an init script it represents the Creator runtime and we now tell to create a built right run time hey crater after all your build build projects components are evaluated right to the following give me a root project and now access all projects in this in this build and for all tasks of type test right hook in a test listener calculate the time it took to run the test check whether it was too long and then issue a warning with the name of the test okay now let's we execute testing with Gradle with this init script and you see you now have those warnings now you don't always want to specify an init script right so what can you do to make it common behavior you take this code you take you go to the Creator distribution and every creator distribution has an init or D directory you create now a script in this init or T directory my inner door cradle whatever however you call it paste this code go back to the command line now you don't need to specify the inner script anymore run the tests and you see the warnings are printed so and now the full story is the following you take a greater distribution is to build master at the stuff you want to auto apply to all of your projects in an in a dirty directory and then you point to wrapper properties to this distribution right in a wrapper properties file you can point to your enterprise credit distribution with the energy directory with your enterprise customizations does it make sense and then all you developers will have this behavior including the picture of ROM Ross Johnson in a meta director II right tell me exactly it's completely transparent they don't have to read a wiki how to set up my local build system so and this is a really deep API I mean it's one thing to define a repository it's another thing to hook into the test in the little cradle Java and then running the test and get events back and do something with them of course you can also use init scripts and say hey this init script is all only for CI so stuff you only wanted to do in CI you can tell the CI hey use this init script that it's not happening on local builds for example init script could check and now we're back to the build language hey create a build give me all your repositories did you dare to declare your own repositories in the project build and not use the company repository build fail right you want to allow that on a local build because people want to try out stuff but maybe not on the CI build right so yeah you can go yeah it's it's Gradle is deep you can go you can go into the dark guts of cradle to to model your requirements so the other thing you can do is in in a project directory in in one of your projects you can UK or in any of your projects you can put it created or properties file right and you can specify certain properties let's not talk about a demon right now you can specify the Java home that should be used to run the build and you can specify the JVM arcs that should be used by the JVM that is executing the build again no build wiki that is saying please set your end ops or cradle ops to that value which is anyhow then global for all the projects you can lock this down it's version control it says its historical it might change over time right so so that means when creator runs it checks is the JVM arcs property set if it is set it fires up another JVM that then executes the book right and if you use the demon then then things are much more efficient because greater than has a long-running JVM that is warmed up and it's much more responsive so so so then crater just checks do we have already a demon JVM with those JVM are setting well then we use it right ok so I'm running out of time so yeah great let's very good and integration you can reuse any end tasks right with Gradle you can even import end builds into cradle and creator reps the end targets at runtime you can inject properties into androids from cradle so with end we have a very deep integration with maven we integrate very well on a repository level so Gradle is able to generate palms right to publish artifacts with the palm to to maven repositories or maven anis a cradle and a stent spawn understands maven snapshots on a dependency level maven is two things maven is a build system and maven is also a protocol for dependency management on a protocol for dependency management level we are fully compatible with me right so if different components different teams interact in a company via a maven repository they the other team won't notice whether you're using Gradle or main right we are working on also on a way to import maven build convert maven bills to cradle so in 1/3 and it which it should be Oz in in four weeks or so you should see a first first conversion plug-in that's my creation is a very interesting topic with my creation it's uh it's mission-critical right I mean for many builds so you want to you want to do this in a very safe way and what we usually recommend is use a build system that doesn't force you to tear apart the existing input structure if you have a build system that says well what this end but is using I can't reproduce you have to completely change your source structure well this is an N built hundred people are using every day and this is this is this is code that is constantly coated is constantly developed so what do they say well and you have to branch I suppose well I suppose you have to branch then correct so then every day have fun with the sinking of the different repositories have fun with comparing the results never do build migration like that and if you have to choice choose a tool that doesn't force you to do it like that but we usually recommend this don't touch the existing input structure until the end to it for example as long as its as its life at a cradle built that consumes the same infrastructure and Gradle is extremely flexible so we had companies they were using dirt class pass files from Eclipse as their master dependency metadata well I wouldn't give them a book for that but I told them don't change right now you know we want to have come we want to we want to be able to compare results right so we wrote an adapter that translated the dot class pass file into the Creator dependency notation and then we we we started to build up the functionality of the end build with Gradle and for every relevant output of the build the archives the Java Doc's the tests that are run we wrote Gradle tasks that compared and the archives exactly the same that are produced by the Gradle build compared to the end build and the same tests around is Java doc to same I couldn't imagine to do any significant build my Croatian without such tests it's a nightmare and I've seen people suffering heavily from doing build my creations without such a safety net I would never do that because what is the same tomorrow might be not today might be not the same tomorrow right this this is this is code and the development this is a moving target so you want to have always the possibility to say it's my greater bill doing the same and it was not long any longer doing the same you have to figure out why at one point the Gradle build is a it's capable of reproducing all the functionality the tests are running and then you can say ok I switch right now the Gradle build is my production build and then you can start to say ok maybe finally it's time for the doc last pass to be derived metadata and no longer masta metadata etcetera etcetera but test test-driven build my creation is the way to go a little bit background information so Gradle is a is a very active project right we have a vibrant community and very proud of our community yeah I think we have very very very nice team of developers so the the founder of the mojito testing framework is part of the Gradle team the author of the Spock framework one of the founders of selenium so yeah very they create a core committers and and and part of trail aware we release frequently with the milestones people didn't realize that we release frequently but now since one all every six weeks there has been a new created version 1 2 1 1 4 2 1 3 will be out there in a couple of or in four weeks so we practice what we preach with continuous delivery and and we are able to do this because we are highly automated we have a one-click release and with a pretty complex release process right and we have 8000 unit tests to guarantee the quality so that's yeah so are many significant projects have switched to cradle you know the first one was hibernate couple of years ago and SpringSource has has completely switched and almost completely and many enterprises are using cradle in production or for quite a while even before one all was out so so and it's getting it's getting more and more there's a there's a there's a strong momentum going on and which is important when you when you choose such a such an infrastructure technology right you want to be sure that that it that it will be around also in five years right so there's a company behind cradle called cradle where which provides support subscription now all the professional service model training on-site consulting remote consulting prioritize open source development custom development etc delivered by created core developers so yeah that's it for tonight thank you for your attention

Channel: InfoQ

Views: 44,978

Keywords: java, sfjava, hans dockter, gradle, gradleware, maven, ant, techtv, breaking open, marakana, tutorial, stackmob, opensource, Java (programming Language), open source

Id: 5a91dBLX8Qc

Length: 84min 39sec (5079 seconds)

Published: Wed Oct 10 2012