Using Azure DevOps Artifacts with Java/Maven Projects

hello everyone welcome to this video um today is going to be a short one i've been dealing a lot lately with a lot of customers in devops and implementations and so on and i've been like noticing most of the time that most of my customers don't actually know what azure artifacts is or they never bother to use it or whatever and so i thought you know maybe this is a broader problem than i thought and i thought you know to tackle this using a very quick video just showcasing the features of azure artifacts let you know exactly what it's used for um so first let's understand when do we use azure artifacts so azure artifacts best case scenario to be used if you have ci cd pipelines that have dependencies among them so for example you need to build project one first in order to you know generate a binary and that binary is going to be a dependency for project two and um you you need a way to store these binaries safely and securely and you know away from prying eyes and but you also need them to be available all the time you also need them to be there regardless of where you are and so on and um what azure artifact is basically does that it allows you to share these binaries and share these packages you know regardless of what programming language you're using and basically have this available for your programs that you're writing and so on the other the other use is that it you can you can actually cache the binaries or the packages that you're actually using in in your in your applications just in case they disappear from the internet so let's assume you have this application you know enterprise application that you're working on and that enterprise application you know depends on some json parsing or xml parsing library that's out there from the internet and so every time you compile your application it's going to basically grab a hold of this at least compile to a new machine like like a devops agent or something it's going to grab a hold of this binary or this package and download it and you know use it as a dependency to be able to build your own software but then what if this package disappears off the internet you know it doesn't exist anymore or this version you know or the the repository where you're working off is down or you know something like that how are you going to handle it then so then you need like a backup repository actually in in such case the azure artifacts is going to be the main repository where you cache um all your packages that you're using so that you are able to find them regardless of whether the original repository where this package came from still exists or not so i think this is a very neat feature so whatever we're going to do most of it is language agnostic so it will work regardless of whatever language and and we're going to to discuss it but i just wanted to let you know that i'll be using java to do the demos for today and the the reason i'm i'm using java is just so popular the maven tool sometimes it's a little bit complicated to to manage and so i think it's like the best case i can demonstrate what this what this tool can actually accomplish um so what i have here i'm just going to go and delete this project you don't need to to see this one and i have an existing project right here i'm going to delete it i'm just going to start fresh and i'm just going to open a terminal window and let's see so i'm in a directory right now i have a folder on my machine called source and this is where i keep all the sources i'm going to go make a directory and let's call this you know devops and so this is my you know project that i'm working on let's assume this is a large enterprise application uh and i'm gonna go see the devops and i'm just gonna open visual studio code um and what visual studio code is going to do basically allow me to create those you know java projects that we want to work with so i'm i'm just working with fresh products fresh projects there is no reason for me to do that except just to show you what uh everything from scratch so i'm just gonna go here create new project java new project i'm gonna use maven you can also use gradle or um and or anything like that to do this but i think maven is going to be the the standard that at least the most popular um thing that people use with java and i'm just using um a project template out there like the first one i found i'm no by no means a java developer even i don't even know java i just know how to build those apps because i i've been exposed to it a lot in the wild before and so we're just waiting here until this archetype is generated so now it's asking me for the group id i think this is probably something like the organization you work for artifact id that's like the project you're building so project one and one oh snapshot it's going to be fine and then the package as well um actually no let's do that again i think the package here should be the same name as the project so let's try that again artifact id project one wonderful snapshot and this is going to be project one as well and yes so now we have a project one folder right here and it just contains a source file i'm not gonna touch that i don't care about it and then the pond.xml which is basically the descriptor where all the dependencies are that that you're using so that's very good we can do the same in project two so let's do the exact same thing one more time and create project maven this archetype is fine this version of it is fine and we're gonna save it you know right next to it in the same destination folder okay good so now again same thing and this is going to be project two and this is the version oops i messed up something here okay and this is gonna be project two so that's very good and we have two projects right now so let's assume i want to make project one depend on project two so this should be built first and then whatever jar file comes out of the project one i need to use this jar file in project two so the way people used to do that before goes of something like this you go to the dependency here and you you know just can add a new dependency and then the group id was this the artifact id was project one the version was 1.0 dash snapshot and the scope is a system because we want to reference a system file and then a system path so and here this would be where the jar file were so you would do something like you know base dire and you know lib and then project one you know and then dash 1.0 dash snapshot you know the name of the of the thing dot jar and this is where you'd expect this to be so you kind of have to go to um let's first save this one and then i'm just gonna come here and go to project one and i'm just gonna do maven compile just to compile this good and then maven package and that actually you know runs the code and and gives me the binary so that's good enough so now i should have a target folder and the target folder should contain the jar file so now what i should do is i should you know come here and create a new folder inside this one i call it lib and then what people used to do is just you know copy this file from here to here and then here in the pom.xml you know in this in this system path location you would be able to build so let me first you know um like just cut this out and then do this let's test again and let me now go to project two and show you that if you don't have the file in place this will not compile so as you can see here this one is missing you know no dependency information available so it did compile but like the dependency is missing so if you actually had code referencing you know this jar file it wouldn't work and then it wouldn't compile of course um and again if i wanted to package so do something like maybe package again now you would have a problem because you actually can't resolve this you know so let's bring this back to system and put this like this again and now do the maven package all right now it works okay that's good so now i have a um a target file as well this this works in a in a typical you know one developer environment but now let's assume there are two different teams responsible for this you'd have to come and update the version of this every time and you know reference the new version right here and it's just it's a big mess and so what you want to do probably is have a central location where you save those jar files and then every time project two builds if you want to compile project two every time it's going to go ahead and seek the latest file and then you don't need this one anymore this is kind of the setup that we want to do so the first thing i am going to do is just create a new project on azure devops and new projects i'll call this devops artifacts just to show you let's do a private project that's fine okay good so now that we have that ready now we can go to repost and we upload the code so let's go ahead and um we wanna go to terminal and so this is the folder where i have everything so the first thing i will do is just i'll create a um i'll do get in it just initiating this as a git repository and then we'll just create a git ignore file so touch dot get ignore and the reason i want to do a git ignore is i want to ignore the um you know the lib and the target folders so you see after we've done a compile we have target folders right here containing all the jar files and the compiled classes don't need any of that in my code i also don't need the um you know the lib if whatever libraries are there that you depend on um it should come from a from a repository not from a local folder on the on the project itself that so to disencourage um you know bad behavior uh we're just gonna add a git ignore to that so um get ignore and um all right now let's do just a target and lip we don't want any of those um and that seems to be good so just checking this was done properly and it it's done properly so now we can get add all and then get commit there's commit that's good so now we have all our files in the repository now we need to add the origin and i'm just going to go grab the url of the origin from here and that seems to be the one so get remote add version and then this is my origin and now i get to get push and then the branch i want to push to okay now that's uploaded now i should refresh this page and find my files on there all right that's good and you can see here it didn't bring any target or lip file so we have a very clean source code only the source exists here no jar files nothing in it all right and now the next thing i need to do is just to actually show you how to now we're going to use this later to actually build our our software but let's let's work in a local build first um how do we use azure artifact now how to integrate azure artifact into our product so we come to artifacts the first thing we need to do is create a feed a feed is was is basically a package repository um so um the feed is going to be you know devops artifacts i think it just devops is fine i'll just call it you know or you can call it with the same name as your organization it's it's supposed to be the same you know the name of this app usually that's like the best practice so if you're building you know um pro just call it project if you like project app or something like that um that's fine and then who has access to it and again you can just choose um usually it's members of this project that you're working on members of this devops organization or you can specify the people who has access to it and include packages from common public sources this is what i'm telling you about is the nuget and mpmjs and you know maven repositories and so on so if you reference a public package that's coming from one of these public repositories it's going to be cached on your azure artifacts feed and then the scope so you can do an organization-wide scope that means this feed right here is going to be available for all your applications and all your projects or you can do it like project based so only for this application so i want it only for this application and don't no need to do more than that so what i want to show you as well is that here we have actually other binaries that are referenced so you can see here like there are you know other um other dependencies maybe like the j-unit dependency right here um and i don't have that this is going to come from the maven repository um so again this is my feed now i need to connect my pom.xml to it so again this is where we're technology agnostic completely so if you're using python like pip you can do it from here gradle maven npm visual studio and there is even ways to do this um you know without without like like just by command line if you want like to upload a zip file into there or something you can just do so so this is all i need to do is just to add this as a repository into my palm.xml and so i'm just going to copy this and i should have a repository section in my pom.xml so i'll do this for the project one first is there a repository section here there isn't so we should create one so we come here and do repositories if i can spell it right let's see and then we can paste this i have a formatting tool so i can just you know format this document to make it look pretty and um that's basically the url where it's going to try to you know synchronize all these packages that's good for this one we can do the same here and i'll just copy what i've done in the first one like so okay that seems good now but how am i go okay so this is a private artifact repository of course so how how am i going to reach it so there has to be a way for me to authenticate against this url and of course i don't want to put any secrets on the prompt file itself because this is a code file could leak out via code or anything like that so the way you do it is is in the um in the in the maven tool itself you're going to be able to do this so we come here there's an m2 folder for maven and that contains a settings with xmln so i can do cat settings.xml just to show you and well here we have the feeds and the name of the feed the id the username i want to use and the password i want to use so now let's go ahead and create this and the way we do it here in azure is via public personal access tokens so you just come and create a personal access token here i'm just going to create one and so the name actually you should call it something like project dot app dot you know um artifacts or something and so it should be something very unique so here i wanna read write and manage packages so i'll create this and this is the username and this is the password so i already have the username copied so what i can do is just you know just edit this and this is also going to be the the name of the feed itself and here we put the password for the feed don't worry by the time you watch this video these passwords will not exist and i want to copy that and i want to paste it here that's good so now my local maven repository can actually reach this now if we look into this project artifacts now the username is wrong let's do that again that seems i picked the wrong username i think here in the distribution management section as well so i just want to make sure i connect to this properly and again so this is just not as simple as this i need to paste this one okay so now as you see the url is different i think the first one had a problem with it and then there is the distribution management section it does not exist so we gotta create it and again do something like this i'm just gonna copy copy that paste the same thing again okay and again i will put the repositories in it okay good so this is this was project two i'll again i'll copy this part so that i don't keep repeating myself okay maven deploy so now it's uploading the jar file that it's compiled and you can see it takes on a like a date placeholder on it and so on and now if we refresh this now you're going to see now this you know project one and the version of it and again um you can just if you if you want to use this then you can just connect to this feed like this so now when i use my dependency i don't need all that other stuff in this other pom.xml in project two instead i'll just do this okay and as long as i have the repositories here and the distribution management set properly and then project two should compile even though now i'm just going to delete this just to prove to you i don't have access to the lib folder at all now so again now let's go to project 2. and let's do a maven compile good so now as you saw it downloaded the project 1 from our url right here and then maven deploy and now the the output of project 2 is in itself another jar file and so i i'm also going to need this jar file to be there now you can exclude the jar file from being uploaded um if if you don't want to but why would you i honestly see no problem having your repositories up there and there you go now you have these two dependencies completely resolved um so now i just need to upload this to my i'm sorry about this one let's do it here and get at any changes get commit edit on the xmn good so now my code should reflect um those changes now yeah so that's very good so now we can create a pipeline and um you know our pipeline is going to run this on a build agent the only problem we have is our build agent will not know how to access this artifact um so that's just what i'm gonna show you and now we can build because we did this we can build those two projects independently so if if we hadn't done that if we hadn't um you know used an artifact repository you know as a central repository that you can just put your packages on in a typical scenario you would have to build project one in order to build project two and that would prolong the build time and you know generate redundant versions and and stuff like that but now we don't need to do that you can just simply build project 2 by itself and in fact that's what i'm going to do i'm going to use a classic editor here because i just don't want to double with with the ml it's it's almost midnight my time and i'm pretty tired um so yeah um we're just gonna use this that's fine and this is a maven build that we wanna do so i'm just gonna use the maven template and what it does it just does a maven build i also don't need to publish you know test unit results or anything like that so this is the location of the pom file so i i just don't want to build everything here i want to build project two only i'm just gonna call it you know project two maven ci continuous integration is short and here i don't want to build everything or any pom file you find i just want to build pawn files into project two so this one basically um that's good so now we're building this the only problem with project two is it depends on project one well here i'm not building project one i just want to build project two so project one jar file is going to come from the artifact so let's save and queue and i want to show you what's going to happen so we're just gonna go see the logs for this and track what it's doing okay so as we can see the build has failed and the reason this build failed is it cannot resolve the dependencies and there is a very simple reason why it cancers all the dependencies it doesn't have permission to do so so um in on my machine what we've had to do is we've had to go to the m2 directory and modify this file in order to like grant it permission or grant my maven like this this dot m2 settings with xml is basically contains the the key where i where i can find this repository so every time i run maven maven knows where my repositories are and how you know has have the key to access them um but here we're using just a random agent from azure devops and that random agent doesn't know how to reach my repository so it's gonna need to do so the way we do that is just we modify our pipeline very quickly so we can authenticate built-in maven feeds these are just you know um it's a checkbox that's just gonna make sure that we authenticate the maven feed before before actually you know running the the build and so let's save and queue okay awesome so as you can see this maven build job has been completed successfully and so it could authenticate this time because of the checkbox we did um if you're authenticating to a repository where you don't have access to just have them you know generate for you personal access token and you can literally just i think there is a job specifically done for it and we'll insert this for you so i'm not sure yeah may even authenticate and that basically provides credentials for azure artifact feeds and external maven repositories so that's just in case that you're contacting a feed that is outside the boundaries of your project which can be the case in some sometimes so you put it before this one and here you just you know show me which feed you want to do so i want to connect to this feed and then okay where am i going to get the credentials from so you can create new credentials and it will save it somewhere and then retrieve it um on the machine on the on the build agent itself now what we've done here you can do with any other language you don't need to use um java for it it's if you're using anything else uh here is the um the the feeds replicating from maven central so again these are feeds being used by our project so again instead of our project every time going to maven central to grab the repository i've cached the version i we use here now if it disappears from maven central i don't care i still have my insurance copy of it just to be sure that whatever happens i'm going to be able to build my own software and i'm not going to be stuck with you know disappearing dependencies from the internet or something like that and you can you can of course reverse this decision if in case you ever want so again this feed will host everything you know pip twine dot net nuget whatever and again universal packages as well you can just simply use command line to push whatever file you want as long as you know your applications are going to need it you can just push it up there manually if you have a random jar file you can use gradle to push it like if like some some of my customers have something like um um a telephony library or or a payment gateway library that they use and it comes in the form of a like sealed jar file that they don't have the source code too so if you ever want to upload something like that you know you can just use command line to make sure that this jar file is uploaded to your feed or you can use graded for example and that's it for today i hope you learned something

Channel: Zoom Speaks Tech

Views: 4,875

Rating: 4.9540229 out of 5

Keywords: azure, devops, artifacts, maven, java, pom.xml, repository, package management, nuget, pip, gradle

Id: 8iyockDkIY4

Length: 30min 53sec (1853 seconds)

Published: Sat Feb 20 2021