Anchore Open Source Virtual Meetup - September 2021

great so welcome everybody to the encore open source virtual meetup i think we have a a set of interesting talks today uh we have a little bit of bookkeeping we'll get through uh but looking forward to this we're gonna take kind of a loose approach with the talks so we'll do q a um after each talk so if you have any questions uh you can get them in the in the uh chat so the speakers can can address them afterwards or we can kind of have a q a session directly after each talk i'm sure there'll be some interesting questions got some good stuff coming today from uh dan luring from encore and dan lawrence from google joining us we'll get into that in just a moment um so please uh you know housekeeping stuff you know uh try to try to meet your line uh if there's noise in the background just makes it easier for everybody else to follow what's going on uh we like i said we are recording this and there will be an email that will follow up afterwards uh for where to get the recording uh usually those land on youtube um but but the email will have the details on where to where to find it um so you can watch it again or pass it around to other folks that couldn't make it um and yeah we definitely encourage kind of interaction and chat during the meetup so feel free to use the zoom chat panel to converse and we can you know the the speakers will keep an eye on this i'll keep an eye on it and see if there's any questions we need to address towards the end um and and we'll make sure we get to those um so we have a pretty straightforward agenda for today uh we'll do uh welcome introduction uh and and we're gonna kick off on an uh a poll here towards the beginning and the poll is around um an in-person and in real life meet up uh at kubecon so we just wanna get a sense for how many folks uh would be joining us and who is it able to attend kubecon this year in los angeles like in person so we're going to open that poll uh now so brian will kick that poll off and it will stay open um through the talks and we'll close it at the end and share the results so that way if people join a little late or whatever they can contribute i'll do a quick reminder in between the two talks uh to respond to the poll if possible um and so folks can find out but that will help us with our with our planning and hopefully we'll see folks at kucon angkor will definitely have representation there i may see dan lawrence there as well i expect um hopefully so looking forward to that so the the main event today is is a couple of talks that i'm excited to hear about uh starting us will be dan luring talking about uh smarter ways to get data out of gripe so kind of using the the rich data output that group gives correctly and then how to format that how to you know use that to integrate into your pipelines and and the right uh format and data that you want and then dan lawrence from google is joining us he's been instrumental in driving the sig store project and he's going to talk about some supply chain security with six store projects and using sift so looking forward to that as well um then we'll we'll wrap up uh we have some additional kind of general discussion time and some some uh potential topics if we have time if people are interested uh like i said we also have some some additional time uh in our schedules towards the end of this after you know another 30 minutes if we want to go over if there's really good discussion and people want to hang around we're open to that and can help facilitate that but we'll plan to end the agenda on the hour so we'll go ahead and get started uh so dan luring will be um taking us uh through the first talk uh remindered brian to fire off the poll here so folks can start have time to respond to that and uh and to be clear that's the open source in real life meetup at kubecon in los angeles and then i will hand off and off to dan luring if you want to go ahead and get your slides ready we're going to talk about gripe output formatting sounds good yeah i'm ready i will stop sharing oh nice momentary grid of a bunch of people okay so uh hopefully you can see this um i'm moving the zoom controls out of the way so anyway uh yeah this talk is smarter ways to get data out of gripe and uh we'll talk about what we mean by gripe and what we mean by data we don't mean this data although we can if you want to um but really quick my name is dan lehring i manage the open source engineering team at anchor the icons are kind of dim here but definitely if you want to follow up with me uh please do so we have uh i have a github luring uh on twitter i'm dan lering and you can also feel free to email me um we'd love to follow up about anything we're talking about today or anything uh sifting gripe so my my team is the one that maintains safety great so moving right along uh gripe what is it for anyone who doesn't know gripe is a vulnerability scanner and in particular it's a vulnerability scanner for s-bombs it's interesting because a lot of people are using gripe to scan container images and grape can also be used in workflows to just scan file systems or a local repo or something but when you think about it the real correct abstraction for what how vulnerability scanning works regardless of which tool it is is the s bomb it's it's this list of packages with information about each package that can be used to figure out if this package matches up with a known vulnerability that's been reported somewhere um and so a lot of uh vulnerability scanners that are that are doing this sort of thing have an implicit s-bomb so they're they're being pointed to a container image let's say and under the hood they're generating an s-bom like data structure because they need to understand what software is installed in that container image um but then what they're showing you is just vulnerabilities and uh the gripe takes kind of a different approach uh it can do just in time s bombs internally if it needs to but you're also uh s bombs are a first class citizen with gripe so there's a companion tool sift that somebody's heard about um that we also maintain and so sift can produce s-bombs that are useful for a number of reasons not just for vulnerability matching but you can actually feed that directly into grip and we'll take a look at that in a second but um but in general gripe does get used for container images you can run right from within a running container to see the containers file system and scan that for vulnerabilities you can scan just regular files on your machine or on a server or whatever um ends up working out really nicely um and uh here's a link for uh more about gripe i'll have another link slide at the end though um really what gripe is if you think about it is this a data tool so uh gripe most of the logic of the gripe code base is just intensive data processing um and so there's an input and output to this equation on the input side there's actually two things that are primary one is the s bom which we just talked about and the other thing is vulnerability data now with gripe it ends up managing a lot of the vulnerability data side of things for you so um so my team's responsible for building and publishing uh vulnerability databases that greg consumes every single day so there's there's new data every single day being made available and behind the scenes gripe is accessing that data uh pulling it locally so it can do a super fast local analysis on on your machine um there is configuration exposed so you can you can uh see what it's doing and kind of change how it's behaving if you'd like to but but in general you don't worry about that but nonetheless it's still a first class input to what gripe's doing so what's the output the output is really rich analysis data and that's what we're here to talk about today i just wanted to set this up so gripe has really rich s-bom information that it takes in really rich vulnerability data and as you might expect the output is really rich analysis data that shows you uh the pairings between identified software packages and published vulnerabilities so to show what i mean let's go to a demo can everyone see the terminal sweet um big enough so anyway let's let's look at how this might work so um the first thing i'm going to do to kind of show off that s-bomb concept that i mentioned is actually run sift to generate an s-bom on a on an image that i've made it's just a dummy image that i know has a lot of vulnerabilities and that'll be really useful for when we want to look at the data that's coming out of gripe so we'll do this yep you can tell i've run this command already so um we're running sift against the image uh just local to me test latest and we're using that output switch so that zip is gonna generate a really rich json output and we're gonna save that to disk i'll just call it test.json so we'll run this really quick and that'll give us an s-bom we won't see the package information here of course it'll be written directly to disk and we'll use that as an input to gripe and that'll be a really great springboard for the grip features i want to show off something interesting to note is that uh you can feed uh an s bomb to graph you can feed uh you can an image reference from from your local docker daemon or from a registry um and it ends up being really powerful to use s-bombs as an input because what that means is the analysis is much faster because it doesn't have to spend time pulling down the image from the registry or loading it from the local daemon and it doesn't have to spend time unpacking that image to see all the layers and and do the merged file system and and then start analyzing from there um so s bombs are kind of a lossless representation of the input that gripe needs based on that container image so anyway it's just a really nice pivot point because then we can we can show off gripe um so cool so now we have this uh this s bomb on disc here um so we'll start feeding that to gripe so we'll do gripe um you can tell grep explicitly use an s bond that's already uh on this somewhere um so we'll feed it uh this s-bomb and we're just going to run this without any special output options for a second and see what it does okay so you can tell that it found some things this list is very large i'm scrolling up let's see if i can see the top no i can't even see the top so the top has scrolled off you know out of my buffer um and there's a there's a ton of vulnerability matches being shown here you can't see the headers because they've scrolled off but what this is showing if you don't know is this is the package name then we have the package version uh fixed information that that we know about from our upstream data sources either uh we don't know anything or it has been fixed in a later version or the vendor has explicitly said that they're not going to fix this vulnerability we have the vulnerability id and then the vulnerabilities severity so this is useful to get kind of a summary view of how bad you are in terms of uh known vulnerabilities with your image but it turns out that greg knows a whole lot more than this that it's not letting on in this output view and that's what we're going to talk about here uh there's there's a there's a few output options for gripe so if i go to gripe and then just under the help you can see a few of the flags we have here the one we're going to talk about here is this output option so you can do dash o or dash output and there's a few options for values you can provide you can by default it's table but you can be explicit about that you can output cyclone dx but we're going to talk about today is specifically json and also we'll get to the the template output as well so let's take a look at the json output okay so that's a lot of data um okay it's still going all right it just finished um so so you'll notice that's a lot of data and just scrolling up a little bit you can tell that um there's all sorts of stuff being shown some of it may or may not be relevant based on what you're trying to do um but you can tell at the bottom here we actually kind of introspect the image itself we talk about its layers its size uh references to uh repository urls um and then you'll see some vulnerability data up here but i kind of want to make sense of this because this is a lot here um and the first thing i want to talk about is this tool called jq so if you haven't heard about jq i really want to pitch it to you and see what you think because um jq is for querying and manipulating json data i first found out about the jq tool probably like seven years ago or so and at the time it was this awkward like necessary thing i had to use for what i was trying to do in a workflow but at present day i'm in love with jq and so i want to show you a little bit about what it does that i really think is cool um the first thing i'll do just to save time oops just to save time i'm going to uh uh write this to a file um we'll call this sprite.uh test.json oops i want the json version actually you know what i just thought one sec in demos okay cool so um type this get the json format this way we don't have to actually run a graph again we can just use the the result of running right to show what we want to show so um here i'll i'll look at this output and i'll just show you uh first i'll just show you jq itself so if i do that then i can just type jq and then uh i like jq even without any options because it it uh you can see that it's doing some syntax highlighting so it's identifying these objects their keys and their values and this is still a lot of data so what i'm showing you now is not a whole lot better than what i just showed you but it ends up being really nice for exploring the data in a little bit prettier way than comes directly from the tools and so one thing you can do is you can pipe this to less and maintain the colorization and so you can you can use less if you're if you're familiar with less you kind of control how you're looking at this data you can use less searching functionality so you can look for certain keywords and stuff and so i find this really useful so at the very minimum if there's a takeaway here i want to talk about uh the value of using jq paired with grapes output and the same things end up applying to sif's json output as well but so what can you do with jq i mentioned it can be used for querying and data manipulation um you can end up passing in these filter strings uh that describe what you want to do with the json data so one thing you might want to do to get started is just find out what's here what are the high level object keys so you can just pass keys and it'll tell you okay for the object that it's receiving it's going to show just the high level keys on that object so matches is where the the the main event is and we'll talk about the matches property that ends up being a giant array of vulnerability matches but it also shows other things that we were talking about with um more about the image which is the source more about the tool itself that did the analysis that's in the descriptor section um and the distribution identified for the image so now what you know let's let's look at uh actual matches so we can we can access uh properties on the object we can do something like matches we can pipe that to a length function and i'll just point out that um if this if this is hard to follow uh there's really great documentation i wanted to show so this is the the home page for jq where you can download it you can also just use homebrew to install it um but they have really great docs and i'm referencing this page all the time so it can show you all the different filters you might want to use um for accessing properties or certain array indexes how to chain multiple filters together so i just wanted to point this out and this link is in that links slide i have at the end that we'll send out so you don't have to memorize this uh url but so what you end up doing yeah we can we can see the length of this array so oh my gosh we have uh 1751 matches total um we can start to to look at this so let's let's look at just um we can map each vulnerability uh match to just the artifact name so uh we could just see the name we'll pipe that to less oops and so now we can see that all the list of the software that's that's vulnerable within what i scanned um i can then uh if there's duplicates in that list i can type that to unique to kind of get the distinct list um and it'll actually automatically sort so i have a list of vulnerable software here that can be useful um but then what you end up being able to do is is really do some creative manipulation so you can map these vulnerability objects to new objects where the data shape is what you define so you can say um inside this map i'm going to create a new object uh so instead of showing me all the the the noise that comes with with gripe's raw output i say noise it can be useful to you depending on what your needs are but usually your needs are such that some of it's interesting to you and the rest is not so in that situation you can create sort of a a compact view so i could say i want to create new objects and i'm going to have a package field on my object that's data is sourced from gripes output at uh artifact.name maybe we want to have the version so that's going to be present i happen to know where it is in uh in the gripe json and let's see maybe the cveid so that that is found for this package so i could do cbe and that's going to be accessing vulnerability.id so um now i can uh run this and just to clarify we're running uh we're passing in that matches array that gripe has and mapping it to my own array that i want to define um that has these fields instead and so this is what that looks like so you can start to see how i'm creating new data structures sourcing data from what gripe gave me but in ways that might be more meaningful for me either as a human because i'm just interested in a a more focused triaging of the vulnerabilities that are in my software or uh it can be used for uh getting exactly the format you need for uh downstream and a machine workflow so if some other processing needs to input json but not grab json maybe it has its own needs or or you need to do some transformations it's great for that and the last thing i want to show in the talk is you might imagine that with all the filters that you can apply and all the various kinds of data you might want to construct jq can be really cool for quickly finding out some information about your json data but this can start to get unwieldy over time especially if you're you're interested in a very particular format and you understand the format you want but expressing it in jq might be cumbersome and also you wish there is a way to persist that that new format so a few months ago we added something that's really powerful into gripe and it takes advantages of go's templating engine so we saw that there's different dash o options for gripe the one i want to focus on on just really quickly is the template option so if you're not familiar several tools are doing this now but they're taking advantage of the the templating syntax that's built into the go standard library and so um this is documented here and again there's a link for this in the slides but you can you can you can start to put together really powerful template files and these are text files that use this syntax to define a certain format and it accesses objects and properties that come from some data structures and those end up being the same data shapes that you saw on the json output um so it's really nifty because you can now create a persisted file that you can check into a repo that defines an output shape and it might be that you want a particular format and gripe doesn't know about that format that's okay now because you can create this template and even if greg doesn't know about the format that you want you can define it yourself and use that and then it's as if gripe knew about that format i'll just show that really quick a really quick example um i had uh you know i wanted to use uh gripes output in csv format and so graph doesn't have a dash o csv option for example so what i was able to do is create this csv template file and this is just using that syntax from those docs and again i'll link to that in the slides um so i could i can just kind of represent that data structure i want here i want like a header row where i say package version cve and here i have a syntax for uh ranging across that matches property and you can tell that these are pretty much exactly the same names as we saw with the jsonishers you'll notice these start with capital letters because they're actual go um structs and properties so for each match we're gonna we're gonna grab this artifact name the artifact version and the vulnerability id and so and then we're gonna you know close that loop so what does this look like uh we'll take gripe here we go so just to clarify we're uh having gripe uh look at that s bomb we're choosing the templating engine for his output and we're passing in a explicit template of uh csv.tmpl we'll provide pass that to less just for readability boom here we go so we have uh nice csv data we could have written this to disk or passed it downstream to another data consumer but this is a really nice way to create a format that gripe doesn't know about offhand we found this to be hugely powerful on our side um so recently we did an integration with gitlab's container scanning feature and gillab has its own ecosystem where there's a particular container scanning format um for reporting vulnerabilities and gripe doesn't support it natively but we were able to really quickly define the gitlab template um uh in this in a template file and then pass that into gripen then greg can just immediately pass data that's in the correct shape from the get go as opposed to having write as output to disk somewhere translating that etc we've had some a lot of interest in the templates from the community we had someone last week show us a a really cool web page that had been rendered it's using the go templating syntax but accessing ripe's data to render a full html webpage um and and one thing we've talked about recently is having the templates actually uh saved in a kind of a reference directory within the gripe repo so that folks who are interested even if we haven't baked it into one of those options we still have a list of templates that might be of interest to you so with that that's all i have get back to the slides here i uh we'll send the slides out but this is the the links that i showed today in case you're interested and uh that's all i had thank you so much thanks dan appreciate it uh we had a few questions so i've been watching the chat and we've answered a few uh while you were talking but i wanted to bring some up that you can respond to directly um for sure um one from uh mikey is is is the json the sift json output format and anchor specific s-bom format and can gripe use cyclone deactive spot s-bom as input as well that is a fantastic question the short answer is that this is like what we're focusing on next so right now the sif json output is proprietary to sift it's basically a json representation of the sift business objects um that it uses um but uh so it's proprietary it has a lot of information i mean it basically has complete information about what stiff knew about during the analysis but it is it ends up being a proprietary format which we realize is not ideal um so with on the cis side we already support uh various standards like spdx and we're working on expanding that on the sif side but to complement that to your point uh mikey yes we're working on having gripe uh take various standard formats like cyclone dx and spdx as input uh in addition to the s-bomb format um and we think that's gonna be really powerful because we see like the ecosystem having more tools where what's what's ending up in these uh repositories or registries are actual standards-based s-bombs and we'd love for uh gripe to be a really great consumer of that information and really valuable to people who are um using that data downstream in their security workflow so great question short answer is we don't yet but we want to get to it really soon great uh we had one other question from uh kera i'm sure i've mangled that name apologies uh which is can you show an example where gripe detects vulnerabilities that are not bundled with the os distro um sure if i understand the the question there's there's multiple uh kinds of vulnerabilities that we can detect uh there's certainly uh uh uh os distro vulnerabilities like debbie and maybe maybe that was showing in the um json output in my example but we also detect uh languages for sorry vulnerabilities for language ecosystems so there's there's vulnerabilities specifically about npm packages for example um for you know python packages or rubygems or uh you know maven packages or whatever so so there's vulnerability data not just about uh like distro package manager packages but also things from other ecosystems like for various programming languages i can i can show that or if that's satisfactory you can take my word for it yeah maybe we could follow up towards the end so we could um if you want to do do some more screen sharing at the end we certainly uh we'll have some flex time there we could circle back on that if we want to great okay um unless there's any other questions i think we those were the main ones that we didn't provide some chat response to dan can can look through the back scroll and see see if there's anything he wants to chime in on oh uh from chris can angkor also do licensing scanning to make a complete s-bomb dan i'm asking dan learning about i'm asking that question for chris yeah okay yeah um so that would be more on the sif side yeah so so sift uh right right now um part of what it detects uh as it's generating the s bomb based on packages it's finding is if possible shows uh what license applies to that package and i believe it's actually an array in case it detects multiple and it reports that in its output um i think there's more features we have in mind but this would be great to ask the community about like what to do with that information do we want to vet uh the license claims um based on uh licensed text or whatever um and do we want to enable more reporting based on that licensed data those are things we're thinking about in the future does that answer the question i believe so i think we have some more interesting questions coming up we're at uh york's answer asking about some looks like false positives let's uh so we can get to the next talk and make sure we have enough time for for dan lawrence to present uh we can interact with those on on slack or sorry i'm not not on fly uh in the chat window and then circle back if we have time at the end to talk about those topics more interactively i think they're great questions we want to get to those but i want to make sure that folks that have to leave at the top of the hour that we can get through the rest of the talks too so excellent keep the questions coming much appreciated um well so i'll just say uh sorry zach but um definitely in case you know worst case we don't get to it in the uh zoom chat definitely follow us uh follow up with us offline there's anchor community slack anchor dot com slack that we're in all the time if you want to ping us there and also uh the information that was in the slides about us on email or twitter or something yeah great thanks dan okay uh one more reminder that we have the poll still open so if you haven't answered the the poll question uh please do so it's just about if you'll be around for kubecon la in october here in los in los angeles california and uh if you'd be able to attend the in real life meetup so i'll just plug that one more time and then without further ado we will go straight to dan lawrence from an engineer from google who's uh very much involved in and driving uh some of the the cosign and the sig store projects from the linux foundation and he's going to walk us through kind of those products and some supply chain security stuff with sift as well so dan lawrence you have the floor awesome i should be sharing my screen i've got the whole screen shared um i've got a couple slides i'll go over but i'm also going to jump back and forth between terminal and do some demos and stuff too um i also my talk also features jq quite heavily um i think i use it all the time without even thinking about it so thanks for giving everybody an overview on jq uh right before this uh so i'm going to be talking about the object and doing some demos and showing how to use cig store uh and some of the tools there uh that are for signing containers with sift and gripe to get some useful supply chain security protections on your s-bombs if you have them so let me just i've got the whole screen but i'll do full screen mode just for the slide so they look a little bit nicer um cool yeah so a quick overview of six store i've got a couple slides um there's a lot to sig store so i'm gonna go over it quickly and i'm sure there will be some questions but if you have any um you can hop into our slack or read some more at cigstore.dev has a bunch more on the architecture and how all the pieces fit together then i'm going to show how they can be used with sift and gripe here and then jump to some demos combining it all into something useful hopefully if they all work so the overview and kind of the mission of stake store i like to start with this story when it comes to supply chain security most people know uh have been yelled at by their it teams and security orgs uh by now that if you find a usb thumb drive on the sidewalk or on the street outside of your house or your office you're not supposed to pick that thing up take it inside and plug it into your computer that is really really scary it's pretty obvious at this point because so many actual attacks have happened this way that even in sensitive buildings they even block off usb ports so people can't do things like this so this is a no brainer don't do this hopefully everybody knows this unless this one surprises anybody but when it comes to using open source software there's not really much of a difference between plugging a usb drive if i'm on the sidewalk in and doing something like npm install a package name this isn't even specific to npm you know any package any package manager in any language you install is typically coming from someone you've never met you have no idea what's in that code it could be malicious but attacks are happening more and more often like this where bitcoin mining ransomware uh credential stealing uh code is getting inserted into open source packages and then people are getting tricked into installing it so this is pretty scary and the root of the problem is you know we can't really verify where it's coming from or who produced the open source code so it's a pretty easy target for uh bad people to slip malware in so that's kind of where we started with sigstor um let's pretend we could you know if you imagine a world where that is safe and you could find some random source code artifact on the ground uh what if we could actually from that artifact look up and find all the metadata supply chain about where that came from and verify it so plugging in the usb thumb drive would be safe um installing an open source package so when you verified all this and trusted where it came from would also be safe um so that's kind of what we've been building in the same store project there's a whole bunch of different components that fit together to start making this possible but i think we're getting close and i'm excited about it here is the oh sorry yeah most of this demo is going to be going over the cosine tool which is a pretty easy way to get started with the rest of the sig store components and it's what we'll focus on today with our demos because it's dedicated to signing containers container sign cosign that's how we get the name it can like the name says it can sign your docker container images um it works with containers on every oci compliant registry we've tested it with you can sign it with a whole bunch of different types of keys you can use yubikeys you can use kms systems on pretty much any cloud it is built into a whole bunch of different transparency logs which i'll show a little bit you can do fancier stuff instead of just plain signing containers using things like the update framework or tough or in total at attestations which i'm going to be showing a bit more with these sift and gripe s bomb demos and then there's even a cool version of signing that doesn't use any keys at all when it comes to signing packages and code one of the biggest problems is keeping your keys and not losing them and not having them stolen uh so there's some modes we built in uh to six door involving all these complicated uh systems which i think are on the next slide um to make that all work and i'll do a couple quick demos of that too when i switch over to my terminal this is kind of the overall architecture and all the different moving pieces it takes to do that keyless signing stuff i talked about we don't have to go through all of this because this is a pretty short talk but if you go to sixdoor.dev there are much fancier diagrams explaining how it all fits together um under the covers though it's not really keyless there are keys involved um they're just managed for you and they're generated quickly and deleted so you don't have to worry about it but if the serverless people get to call serverless serverless even other servers then i think it's fair for us to kind of steal that buzzword and call this keyless so let's jump to some demos i guess we'll start with the cosine demos and then i'll come back and talk about how it works with sift and gripe and s-bombs the cosine tool is here on github this is where i started actually so it's github.com cosign if you want to follow along or try this out yourself all you really need is this tool and then some containers in a registry um and there's instructions on how to get started and install cosine down here i will open up my terminal make it a little bit bigger um hopefully this is being for people to see um awesome so i've got a container in a registry i'll show what this looks like there's another tool which is really cool for working with containers and seeing them called crane which i use with jq all the time so i'll show my image here uh container images are really just json and tarballs to stitch together so i'll show what that manifest looks like here first uh type that to jq so it looks pretty boom so here's a single layer container image that i have in my registry you can have a bunch of layers you can have a bunch of different things and i've got an alpine one we'll use later for some s bombs but this is the one we'll start by signing first when you want to sing with cosine the easiest flow is to generate a key here locally so we can do that i'll do cosine generate key pair there are no flags you don't have to worry about algorithms or anything like that you type in a password here what did i type i've got a public and a private key which is basically how signatures work if we want to look at these i can show you what they look like here is the public key it's pretty short this uses basic ec dsa or elliptic curve cryptography which is compliant with all the different government regulations and it's supported by most of the different hardware things and cloud providers so that's how we settled on this algorithm pretty short this is the public key if you want to verify stuff this is what you hand to your end users to do checking of your signatures the private key is right here this is normally the thing you want to keep private um and not show people but they're actually stored and encrypted so you don't have to worry if it leaks or anything like that you need the password to use this because it's encrypted um it's not a huge problem if you accidentally check one of these into a git repo um and there are even some nice flows you can do just by checking all this into a git repo and keeping the password somewhere else you can do signing in ci and rotation and fancy stuff like that um a lot of other systems store them unencrypted which means eventually somebody's going to forget check it in and you're going to get some and your emails and have to rotate your keys and stuff so let's use these to sign my image one flag here which is the key and then the image name i'm using gcr for this but you could use docker hub clay and the other registries my password so now it is signed and the signature has been pushed up to the registry anybody with that public key can now verify the signature pass in the public key here no typos and it got verified awesome so printed out all the stuff we checked here's the signature itself and this is just more json so you could pipe this into jq and look around at it but the signature payload that we signed just as the url for the image and the digest of the image in here too by default so it doesn't really say much it doesn't say why you signed it it doesn't say commit it was built from or anything like that by default you can add a cosign um you support annotations so you can do the dash a flag and add some key value pairs here i'll say we hold far if you're doing this for real you might want to pass in a git commit a tag a version something representing exactly what software went into your container image if you're building inside of ci so you have some more metadata you can check later um what we can do now is if we verify we'll see that there are multiple signatures here twice and then get appended to each other um yeah so there's two signatures let me pipe that to make it a little easier to read um cool so here's the first one it says what it is and the second one we can see that foo equals bar pair um and if you want you can check that here during verification and it will only print doesn't match that um i'm doing this verification here locally but there are a bunch of different mission controllers if you're trying to deploy to kubernetes or for runtime environments to do this automatically so you can have a list of public keys that are allowed to deploy and it won't let you deploy things unless the images have been signed so here are the basics this is a simple flow with keys if you wanted to use a kms system you could set that up yourself too and set up off to the kms system instead of generating the key and having those files locally i'll do one more quick one which is the magic keyless one this is still experimental uh so i turn that on and now we can do all this again um except instead of passing a key we just won't pass on here and now we'll see this cool flow we're actually going to sign it with um an automatic certificate we're going to retrieve from a save store service that is authenticated against uh my email address so we'll go through a little browser flow here if this all works cool so i'll go back here before i click okay we generated keys here in memory so they never touch disk they're random get a certificate and based on those keys from the six store service and now we have to authenticate with uh an email address i'll do that click this it does the open id connect flow proves that i have access to that email address stick some stuff into a transparency log which is cool and now it's signed again verify here we should see more keys again here it's going to go through gq terminal mode that just always does jq when it finds json yeah so same payload up here before uh we've got the certificate in here uh base64 encoded and then my email address which is the one i did the open id connect flow through so it all got signed again with no keys this time so you can verify it and just tell people your email address and they can check and prove that you signed things and you don't have to worry about losing keys so this is uh some basic primitives you can use to start to build up supply chain security signing by itself doesn't really get you security it kind of lets you build up your own flows and permission models um let's talk about how there's a whole bunch of different ways you can use this you know for gaining deployments to production stuff there are also some cool flows involving s-bombs and that's where sift and gripe come in uh so talked about s-ones before they contain all of the information that goes into a binary or container software bill of materials there are a couple different formats a whole bunch of tools for producing them consumption is just getting started those people start producing them so the biggest use case so far is knowing what vulnerabilities to watch out for and that's where i really like the sift and the gripe flow a lot of image scanners kind of combine those two just shown before where you scan and you get vulnerabilities but it doesn't really make sense because the s-bomb components you know things in your container don't ever change they get stuck in there once and the container image is static and immutable so why bother downloading and extracting and scanning that container every time you can produce that s bomb once at build time and then do a whole bunch of different things with that s-bomb which is way easier to do than downloading the container each time so a sift you can produce an x-bomb and then with gripe you can check what's inside of it if you're signing your images and the s-bomb is produced at the same time you send the image it probably also makes sense to sign your s-bombs otherwise the s-bomb could get tampered with and then if you're looking at the wrong contents then that's a problem too you might not look at the right packages to see if there are vulnerabilities inside them at the same time vulnerability data isn't static if you scan an image a year ago and it had no vulnerabilities in it it's not that useful because vulnerabilities get found in software all the time so you have to constantly scan and look for this stuff later so there are a couple different tools in cosine for working with s-bombs if you have say the sp something in the spdx or the cyclone dx format cosine can store those s-bombs in your container registry too and then you can sign them just like they are images let me see if i can get this command to work and remember the syntax uh so i have some um this one's already sitting here i think yeah cool so i use gripe to generate um or sift to generate and that's on the spdx and the json format here so let's look at the it's going to be huge so these are all the packages inside my alpine container that i generated earlier we can use cosine to upload this so it's not just sitting here in my local directory but it's uploaded next to the container image that i generated this from see if i can get this to work that and then the name of the image this one was for alpine i got that right awesome yeah so it's uploaded the spdxs bomb to container registry so it's sitting right here next to the image so if anybody that wants to pull this image can also download the s-bahn itself you don't have to worry about storing them somewhere else if you already have a container registry download s-bom and then that image name and boom it spits that back out so this is a nice way to distribute your supply chain metadata next to your container images and you can sign it with the same keys or different keys if you want to this was generated using sift earlier yeah that was the command i did except i had dash o spdx instead of json here for json i'm going to show a different flow that involves the scan results here and this is a feature in cosine involving attestations i don't have a great slide on these so i'll try to talk through it and explain it a bit as we look at them the the first signature i showed earlier um here and show what these look like again um since before but the signatures don't really have any metadata and i'm explaining why you signed something it's just kind of a boolean that you sign something uh what did i do so it's not that useful you have to make a whole bunch of implicit promises and implicit guarantees about why and when you'll sign things you can build up some basic stuff in one of these signatures with the dash a flag there for equals bar but it turns out there's a whole rich data model for this already called in toto attestations and they live here in this repo and the way i think of an attestation is a signed statement about a subject so it slightly formalizes those implicit things we just talked about a statement can be anything it can be you know this is cool i like this container it's red something like that and then the subject is what it refers to so instead of signing the container itself now we're signing a document about that container and referencing the container in that document so that's what an attestation is you can put anything you want in here um there are some well-known ones though like provenance um this attestation format uh contains information about how a container was built beyond just the commit it was built at if we contain the build tool the environment the commit of all the build tools that were used to build it that kind of thing and you could do this for any type of artifact not just a container what we can also do though is use attestations to capture scan results because it turns out for compliance and a lot of regulations you want to know how frequently your images were scanned and you might want to write a policy that says something like only allow me to run containers in my production environment if they've been scanned and they don't have any critical cvs within the last two weeks because again these results get old so we can do is we can put timestamps into these to show when the attestation was produced and then sign those so they can't be tampered with after they were first created so now what that becomes is a signed statement that says as of today wednesday september 1st you know 1 52 i scanned the container there was nothing bad in it so then later we have a record showing that we did this and if something bad happened vulnerability was found the next day and we know it was good today when i deployed it and we can tune our policy to rescan and produce these attestations again later so let's uh break apart that flow now i've got i've got the json version here generated from sift i'll redo it um this is going to get me a json sbom using the sift format here and that i've attached the spdx version to my container but instead of attaching this one or signing this one we're going to produce a scan result using gripe so gripe can do just in time scans from the images but that's not efficient and we already have the s-bomb locally so let's produce a scan here we'll take a look at it i picked an old alpine image here just so there would be a scan or there would be something in it so it wouldn't be empty um yeah there is one medium severity vulnerability this has been patched in alpine if you grab a new version uh but in the one i grabbed i think about a month old um there's this vulnerability here so in the json format save that to disk and let's pretend our policy allowed this so it's fine we said we just don't want high severity vulnerabilities um so now we're going to use cosine to produce an attestation here that sign with our key again and store that with the container so we can do i have the command here there we go instead of sign now we're going to a test um we're going to use this key to sign it um the predicate that we're producing is that that we're using is that scan.json and we're going to attach that to the alpine image i have here so this will happen i'll enter my password again and this all works with the same key management systems and keyless email based flows i talked about before too the ad station is now stored with the image so your policy system could check this and prevent you from running these if the attestation doesn't exist or if the data inside of it is wrong let's take a look what that looks like and then there's a bunch of jq to actually make it look good because it's json nest inside of other json there's a long jq payload but we'll build up to that one we're going to verify it this is going to show us all the different attestations that are on that image it's long it's json embedded inside of json with base64 encoding but this one was actually signed correctly by that key so we know the data in here is at least valid if we want to actually show what it looks like i'll put those jq things back we're going to pull the payload out of there we're going to base64 decode that and then we're going to take the data out and actually if i up here i think we'll see the time stamp yeah there's a timestamp in here somewhere sorry my terminal formatting is bad um here's the full one that shows the skin.json again that we got exactly out of here and cosine attaches its own time standpoint we actually do the signature so here's the scan result in json format showing that one vulnerability inside of it and this gets signed and uploaded with the image so it can't be tampered with for public artifacts you can use some of the rest of the sig store components to publish these onto a transparency log so people have a central place to discover these and they know they're not getting a special version of this and the version that has been distributed it's the same one for everyone so you're not hiding anything but for private images you can skip that and just use your own registry with your own permissions model cool so yeah that shows uh this wraps up the demo but it shows how to sign images it shows how to assign s boms and then how to use these tools to test vulnerability reports which might start to make these s-bombs actually useful in your policy systems and your production environment that's all i've got we've got some time for questions here for people that had any while i was going just talking for a while non-stop thanks dan that was great awesome to see the story end to end some interesting some things there i hadn't seen before the the keyless stuff is particularly compelling so that is something we've all we've all done that before lost the key i think there's some some chat questions there um is there just specifications do i prefer um they're they're all fine i don't really have a preference here on the format um the more important thing is how they're generated when they're generated um trying to do it as close to the build process as possible to get the most accurate data i think it's the most important piece stepping through slower yeah sorry i had to go through that pretty quick through a lot of commands there a lot of this is in the if you go to sig store cosigning there's a lot of these examples checked in there in the read me and docs too i think there's one about six store authentication from mikey yeah is there any issue using system authentication which based on a password to gain a pki key pair okay yeah so what actually happens there is we generate a public private key pair um for asymmetric stuff but then we encrypt the private key so it's like a wrapped key and so that's uh set up using a knackle secret box um if you want to know exactly how that's encrypted so there's no issue there you could keep it unencrypted and some tools like openssl we'll do that by default all these signatures are compatible with those tools as well if you have other systems this is just an extra layer of protection in case you forget to get ignore it and check it and push it to a repo which everybody's done i've done before so it's one more uh something to help you out a little bit and for my mistakes cool um so there are a couple more great questions what i'd like to do before because we're at the top of the hour i want to do a couple announcements and then we can jump back in you know we reserve some time if we want to follow up but i want you know if people have to leave i want to make sure we get a couple of more announcements through um real quick and then we can jump back if if you have some time dan and folks have some follow-ups so here let me um share this really quick to get through it and we'll hop back ready to go there we go okay um so i wanted to again do a quick announcement we have the open source in real life meetup at kubecon it's tuesday october 12th uh no presentations just to be networking and getting to meet each other uh can everybody see dan can you laureen can you give me a thumbs up if you're seeing the slide correctly okay just making sure um yeah and so it'll be at tom's watch bar uh this will be in the recording so if you want more more follow-up you can review this part uh tom's watchbar at la live so it'd be really close to the to the conference center uh we'll have beverages food stuff like that uh also some new uh sift and grape swag which will be cool uh and we'll have a drawing for a drone quadcopter um so you can take observability to the extreme but space is limited so we have the registration link here uh if you want to follow up and register there so this will be in the in the recording again anchor dot com slash 2021-kubecon meetup go ahead and register there so we know you're coming and we can make sure we have the right right amount of space for everybody so that's uh that's all of our formal stuff uh the poll should be closed for the um or closing uh for the the uh who who's able to attend or knows they're able to attend uh that should be getting shared out here momentarily uh but i want to be sensitive to everybody's time uh we'll continue on um and be able to continue the discussion in a few minutes so i've got time schedule i think others we can run long but for anybody that does have to leave i just want to say thank you to those to everybody that attended um you can join us on the anchor side on slack we have our own slack community i know there's a six door slack community as well uh we have some some links up here uh to get for for the sig store and cosign stuff on github as well as anchors gripe and sift um yeah we're always looking to have contributors and collaborators on sift and gripe we'd love to to work through you know if there's features that are really important to use i've always opened to work through those with folks you know always appreciate a github star if you want to keep keep up with the projects and if you're interested in speaking have an interesting project or a use case you know that around sift and gripe or you know supply chain and vulnerability stuff we're open to other speakers as well so always open there reach out if you'd like to present at a follow up happy to have you with that i think we can formally end the agenda and then continue on to more q a and discussion kind of after the fact and we'll um [Music] i guess we can continue the recording if we want to record the the q a if you want to follow up with the q a afterward but our formal program has ended and so we can go back to some questions uh with with the dans uh if we have any anything we need to follow up on so thanks all for attending if you have to drop no worries otherwise we'll be here for another half hour or so you know availability is as as um as we can and we can keep the conversation going okay so now that we're kind of formally at time you want to follow up on some of those questions dan lawrence i think we had a couple extra ones um yeah so the next one was about revocation i saw here um dennis um how does key revocation work yeah so that's where there was a lot of those boxes and arrows on that slide that i jumped over really quickly come in if you're just using uh you know the fixed keys generated with generic key pair and you're not using a transparency lodge or anything like that then you're on your own for revocation you know you generally rotate your key and stop trusting the old one and try to resign things that's not a great model though um so that's where some of the the other components come in so six store has a timestamp server and a transparency log which you can use to help out with revocation if you get your signatures timestamped by another party or published into a transparency log then you kind of have a record of all the signatures that you need to worry about and if a key gets compromised you can track down exactly which signatures don't need to be trusted and which artifacts those apply to so it's a little bit easier to revoke the key and move on to another one because you have a record of all of the bad things and good things that happened so a couple different models there depending on uh exactly um which other components you're using um next which i think that's it other questions just about the blog i don't see anything else okay anything else any other any other topics or anything folks want to bring up is it getting supply chain processes in place in your own org that's a whole other discussion topic i'm sure okay but but the tools the tools are coming a long way and it's great to see this like get the proper attention that we think it deserves and we're excited to be working with with the cosign and six door projects to kind of combine our our uh expertise together it's going to be it's going to be an interesting journey looking forward to more collaboration between our groups thanks for coming and joining us dan and dan thanks um any other questions or following you can people can feel free to speak directly we don't have to do just chat um otherwise we'll give it a last call and see if we should wrap up hey uh other dan i have a question um where are you seeing uh the most uh use of of what six stores providing value-wise in tools that are adjacent to stick stores so like i'm curious you mentioned the admission controller example given that you have the stake store infrastructure in place maybe you're using the public record or whatever you're using cosign in your workflows where are you starting to see folks consume those signatures to affect real change in their uh their workflows um yeah the easiest one is uh some of the linux distros that do stuff like reproducible builds um so i got arch linux and debian and a few of the other ones are producing attestations i showed here um that say yeah i rebuilt this and i got the same exact hash and here's the machine that i rebuilt it on i'm sticking those into the transparency log so all the work that people have done over the years to make builds reproducible um now they're using these components to kind of publish and show that they actually reproduce things because if nobody ever checks the reproduction of a build it doesn't really help too much so yeah there's a community of people that are like rebuilding all the debian packages and flagging if something happened and it didn't reproduce anymore they got a different hash from what actually made it upstream so i can use those as an example because it's a way of protecting anybody that's installing debian packages without them even knowing about it just because all these people are writing things to the transparency log and checking each other the debian repo ever had something in there a binary that didn't reproduce from source it would get flagged and we'd figure it out without you having to even know that stuff is happening that's one of the coolest ones that i've seen recently when you say you wouldn't have to know what's happening there's some point where you need to check the log is that right uh no so you're kind of relying on a couple of these other people checking the log um so if you're just grabbing stuff from like deb.debian.org but there's you know three people that are checking all those binaries and rebuilding and making sure that they match and uh putting those in the log you'd be relying on them to kind of flag it and publish it somewhere basically that hey hey something doesn't look right in this binary and you can check the log yourself if you want to as well some people have some automated systems that block updates on debian until the build has been reproduced x times or something like that cool great excellent stuff all right and i'll do a last call for questions and then we'll we'll wrap up any any more questions anybody all right um see i think mirko we had a follow-up for you uh do you wanna we can follow up with that in slack before we end the session um around it was around a question around angkor and the the commercial opera uh offering um is that something you you have you could join us on our slack community kind of follow up on just want to make sure we don't use the connection here yeah no no yeah i can try that later i'm currently on pto in austria but uh yeah i will i will i will i'll come back to you later because i'm working for gitlab and i uh try to understand you know the the differences in those things customers ask me right so yeah i'll come back to you thank you great i put my email address in the in the chat here so you can ping me there if you need to connect another way just want to make sure we didn't lose anything perfect thank you great um well thanks everybody thanks for attending uh much appreciated thanks to our presenters for taking for giving uh some great uh talks today uh we'll be following up again we'll get an email with uh with the details on on the recording as well as you know we'll have the in real life that will be our next meet up at kubecon and we'll then we'll have you know follow-up after that so we're getting into a regular cadence of these meetups so looking forward to seeing you again hopefully and some of you physically if we can that'd be great otherwise have a great rest of your day and thanks for joining us today thanks all thanks everyone

Channel: Anchore

Views: 162

Keywords: open source software, open source, software supply chain, software supply chain security

Id: yUAX58pivB4

Length: 63min 40sec (3820 seconds)

Published: Tue Sep 07 2021