Progressively loading images - HTTP 203

so jake what do you want to talk about today well yeah that's a good question um shouldn't i oh these slides took me a long time and i don't think it's worth the payoff but uh let's find out what's going to say they don't look that good shut up mate [Music] hello summer hello jake so i occasionally get comments on the video and people think i'm surma and people think you are jake so i think we we should sort that out and you see what i did there right i've called you by your name i don't usually shout my own name out um but maybe maybe we could put a little thing along the bottom which makes it like super clear who's who because you know wow important for our like important for our individual brand identity anyway we've talked a lot about the the size of images right and how to make them small but i want to dive a bit deeper and look at the different styles of loading an image when i see a page like this uh it feels like there's something missing but i don't always know what i mean it's good on you that the space that there is a blank space because on many pages we don't even have blank page a blank space and you think oh this site is ready to read and you start reading and things start moving around and it's really frustrating oh yes the absolute worst thing that can happen here is a layout shift but this site this site has at least left the space for something but is it a photo is it an illustration is a graph is it a piece of interactivity like it could be you know more than just a an image so a smaller image at this point will reduce the time that the user is in this kind of what's going on phase but we can actually do something while the image is loading as well and that's what i wanted to focus on in in this episode you might see some types do this how would you say are you going to talk about blur hash because that that had attention for a while yes so this is a blur hash or you know some other form of making it clear there's an image here it could just be a couple of gradients or something like that um and i would say when i see this it's clear to me now that there's supposed to be an image here it's probably not a graph but it could have content in it like it could have text over the top that wouldn't be obvious from this view um what i'm essentially saying is sometimes if a if there's a slow loading image like this a lot of the time i don't care i just got to read the content but you only do that once you know the image is just illustrative of the content like it's not a piece of content in itself you can solve this problem by throwing more bytes a preview like this and once i see this i'm like right okay it's a kid with an ice cream fair enough it's just an you know it's supplementary content so i can just go ahead and read the article and in the meantime the image can continue to load and that's a much faster user experience compared to like having nothing and then the image appearing you know fully loaded meanwhile users just staring at a blank area wondering what's going on but there are many ways to tackle this problem there are some solutions which are pretty new and there are some which are almost 30 years old and i'm going to try and cover all of them in a small video i'm going to start with the old one going to start with jpeg that's the 30 year old one right 29 i think but there have actually been innovations in this area even in the last few months i want to talk about those as well so here's an image that's pretty hard to compress for whatever reason i don't know why this is a really difficult image to compress i guess it's because there's lots of sharp lines um lots of different areas of the image different things going on i mean technically it's probably quite easy to compress it will just look yes as i was trying to avoid that with this image and it turns out you have to throw a lot of bytes at it in order for it to not look terrible um especially in some newer image formats it's harder to compress this image in your newer image formats which is unusual but it does happen but with jpeg they're both 300k they look pretty good they're both the same quality but they're quite different in how they're encoded here's how they look when they're loaded over a 2g connection i made it super slow so you can see what's going on now i'd say that the one on the right feels like it loaded faster like you could tell what the image was a lot sooner in fact it kind of feels like the one on the right has finished loading but it hasn't it's still filling in tiny details these are you know they take the same amount of time to load because they're the same size right there are people who don't like the effect on the right of it loading in multiple scans like that but i think they're wrong but i mean don't don't you remember from i mean i certainly do from the dial up days where most images not most but many images behave like the one on the left the the progressive not progressive but they're just like with a line by line appearance of the final image and then your internet would cut out or buffering what happened or something happened just when the the core part of it was about to have you like there's the the character i was waiting for or something and then the image just stops loading nothing is more frustrating exactly and whereas the one on the right which yeah is a progressive jpeg you get like this low quality version first and then it comes in and fills in the tiny details the one on the left is not progressive some software calls it baseline although that's sort of a slightly different thing but you know they sometimes use that to mean and not progressive as well a baseline jpeg can't be progressive but it also means it can't do some other things anyway squoosh the image compressor that we work on uh that creates progressive jpegs by default because we we like it we use moz jpeg mozilla's jpeg encoder although there is an option in there to create the non-progressive kind if you really want for whatever reason now i said there'd been an innovation here so i'm going to take a look at that chrome and firefox have received an update to their jpeg decoder recently so here's what it looks like compared to safari which which hasn't so three two one go and it might be difficult to notice the difference if you're watching this on like a small youtube window or for you summer who's watching it through a teleprompter but it's really noticeable at full size for real users here it is up close this is the first pass in safari you get this blocky nearest neighbor scaling whereas in chrome and firefox you get this much nicer smooth appearance and that's pretty new like chrome and firefox were blocky until recently so is the the blocky part i think they call that the the dc layer right it's literally just the because jpeg divides them into eight by eight blocks are those just the eight by eight blocks with one solid color basically yes that's right it's the it's the dc part of the encoding so it's solid colors um and yes what you're getting in chrome firefox is just an interpolated version of that that's correct so yeah most uh progressive jpegs will have the first phase just being the the dc information so they're both basically the same data just that chrome and firefox are applying a post-processing effect on the decoder site to make it look more pleasant while more data is being loaded yes this is literally the same image and loading in each browser at the same speed that's what you saw there i just did a screen capture of it yeah so chrome and firefox use lib jpeg turbo as their decoder where safari uses a custom one i think one that apple made so it doesn't have this benefit yet they might add it also safari's decoder sometimes just doesn't decode progressive images in a progressive way like it just waits until the whole image is there and i don't know why it happens with some images and not others i don't know i guess i guess it's some bugs um but we also have bugs in our decoder and i found some bugs just making the slides but one of the reasons it took me so long to get these slides together is i started finding bugs in our decoder while making it so then had to go and file those bugs we sometimes fall back to this blocky effect too early like where we shouldn't that's something we need to fix um but jpeg's actually really amazing as a format because you can script like when you're encoding an image you can script what the progressive passage should be like how much detail each pass should have uh what detail it should be missing and how many passes it should have in in total it's really flexible uh we might actually ship our own in squoosh that makes yeah i saw you had like a draft pr open and i saw you know you were throwing like moving around some some c code from the mods jpeg encoder and i never realized and i know i ported the mods jpeg encoder originally and i never realized that we had detailed control over the the individual passes as they are called of the jpeg decoding and that's really really cool yeah so what what jpeg will do is it it does the dc pass like those uh eight by eight blocks and then it as soon as possible gives you like a a low quality version but with a bit more data and that's actually not great with the the new decoders that uh firefox and chrome are using because it throws them into a sort of more blocky uh output much earlier than we otherwise would so what i'm trying to do with the dawning scrooge is to avoid that midstep but still deliver a sharp image as soon as possible so i'm not going to ship that yet because i want to wait until we fix our decoder bugs because then i can get to see exactly what the difference is but if it's still better then then we'll ship that but yeah i'll put a link to all of this stuff in the description all right that's enough about jpeg i want to play a little game and guess the format and i'm going to show you an image loading and it's your job to guess what image format it is it might be jpeg it might be something else because it could be a different form of jpeg anyway here we go three two one go what's your thinking you still think he's still there i i'm trying to think i think that might be png oh well well done no you're correct i'll give you that in one that's this is the lesser spotted interlaced png so this is a a feature of png that's not used so much it uses a form of interlacing called adam 7 which means it's 7 scans it starts off with a 1 8 resolution image a bit like the the jpeg we saw before and then it it doubles the horizontal resolution then doubles the vertical then the horizontal then the vertical you end up with seven passes uh in total gif has a similar thing but it's only four passes and it only improves the vertical resolution it starts off with the full horizontal resolution but what i will say is don't do this uh don't use intellect png's it makes i mean you've been looking at png compression and uh jpeg excel compression recently how it uses the the pixel before to do prediction about the next pixel this form of interlacing makes that very hard uh so it ends up being about 20 bigger it's not great for compression um generally png should be quite small so they don't really benefit from this interlacing thing i i would say if you're targeting an old browser where you need the alpha transparency but otherwise it's photo data so it's a huge png maybe do the interlacing thing but otherwise no uh scrooge doesn't do this right now but i have a draft pr ready to go do you know the only reason i didn't like push this pr your way is i thought it would give you a clue to this question i was hoping you would get it wrong but you got it right anyway uh but we'll ship interlaced pngs in squoosh off by default of course nice i like it okay another question here we go okay i'm ready three two one go what could it be ah there you go okay so there was no progressive step in that sense it was full resolution from the very beginning just top to bottom it has alpha transparency um so there was so it's a form of progression like it it you you saw some intermediate steps but as you say it was it was just a top to bottom scanner it was just a scan line basically so yes this has proper alpha transparency not just a mask so uh and the amount of colors i'd say give us out jpeg is out um it could be png it could be avif uh i guess it could also be jpeg xl but i actually don't know i think jpeg excels progressive is the whole point that they're marketing is it's the same progressive rendering as jpeg so it would probably actually start with a low resolution i am going to say it's going to be jpeg or avif uh not a png or avif png or a5 i'm not sure how i would distinguish those two just by the progressive render i'm just going to put my money on avaf because you haven't had that yet it's a reasonable bet uh but in this case this is a lossy i forgot about the p the one you didn't mention i think if you thought p you would have you would have guessed it was uh but it is specifically lossy where p so yeah top to bottom scan no fancy progressive rendering like jpeg has part of this is because jpeg is an image format whereas webp is an image format created from the keyframes of a video format a vp8 and video formats do not need multi-scan progressive rendering they just need to display one frame at a time uh so it's not something webp can do in fact from what i'm told it was actually a lot of effort for them to even make the top to bottom rendering work it was they had to do a lot of shifting around of data to make it work but there was another element of the load there which is interesting uh what i'm going to do here is i'm going to compare lossy web p with lossless webp lossless webp is like a whole other format it's not related to the video codec at all i'm going to start them now now the lossy version finishes first because it's a smaller file but notice how it took a lot longer to get started compared to the lossless version yeah now have a hunch go on then what's your hunch i think um you know lossy webp is lossy but one thing that i have learned throughout our work on scrooge is that it's very hard to lossy compress the alpha channel and so if you have lossy data you probably have to encode the alpha channel separately lossless because otherwise progressive render is going to look really really weird that's that's correct uh i would say correct answer maybe not entirely the right reasons um but okay it's it's more of a product of it being uh derived from a video format again um what these formats tend to do is they you know they encode channels one by one and this is the thing that was difficult for webp to you know untangle so they were like delivering things together but they didn't do that with the alpha data so the alpha data sits at the front of the file uh yen you're correct it is lossless it uses losses webp uh for the alpha data it sits right at the front so that weight that we had at the start was it loading all of the alpha data before it could start adding the color data in which is you know invisible exactly and uh yes i mean they could have done it the other way around but alpha data tends to be smaller than the the rest of the data uh whereas lossless webp because it's a pixel by pixel compression it does the alpha data along with each pixel they could have found a way to interleave the alpha data but it especially would be a completely different format not easy and lossless alpha does work uh something avif does quite well but yes that's not how webp does it oh i didn't know that i didn't they know that avf can compress lost fully lost fully uh compress alpha and not have and deal with the artifacts that might happen it works really well so webp will struggle if you've got like a lot of gradients and stuff in your uh alpha channel whereas that's where avif will do really really well all right next format here we go three two one go any early guesses um ta-dah there we go um so that is an image format that has no progressive capabilities that's correct um actually now that i think about it i don't even know why i was considering avaf because i think the whole point is that avf currently doesn't do progressive because again it's a video format thing it's probably avf you are correct yeah this is it the avi decoders in chrome and firefox don't have any intermediate rendering at all it's not even clear whether they can i think maybe the same problem again like the uh the two color channels are brought in first maybe uh it might not even be possible for them to ever do any kind of uh you know even scanline or progressive rendering here and i sped that up compared to the other demos we've seen i think avif is magic it's my favorite image format right now especially because it's in chrome and will soon be in firefox i've written a blog post about how great it is but there are some kinds of images that it struggles with and this is one of them i had to throw so many bites at this image um to prevent like ugly flat areas appearing it ended up bigger than the jpeg in this case uh it's one of those outliers not usually how it goes it's usually a fraction of the size of a jpeg sometimes like a tenth of the size and it still looks okay not not the case i mean that's the whole point of scrooge where you can try these things out because i think one of the things that we really want to get people to understand like there is not a single image format to rule them all like there's different types of images where different image formats excel at and so it's often the case of actually taking the time at the very the important images on your website the big ones actually take the time and look at what squoosh has to offer in the different formats and use the ones that work best for that specific image absolutely absolutely um so i this way that you get nothing until you get everything with an apif i've been proposing a way to mitigate this i know you've seen this already uh it looks like this so this is a 4.3 k avif put with a blur filter over the top so it's a tiny little image that can sit at the front especially compared to like 300k for the full image now without the blur it looks awful kind of very sort of alien almost art in some way but yeah once you apply the blur i think it looks like a really good preview and the avi standard already allows multiple images to be in one container and those can be tagged as like thumbnails so what we're going to look at is you know if there's a thumbnail to start the image we could show that early in the browser and then like maybe apply this blur filter to it all just ideas right now but i'm i'm hoping we get something like that so you'll you'll get a preview like this straight away and then you'll get the final image once it once it downloads and i mean yeah if if the the final image is 300k adding another 4k isn't really that big of a deal it's an increase by one percent so your users can get a preview of the image much much earlier especially for avf like if you think you're you know you're on 3g um downing 4k will be significantly faster than downloading 300k so the time difference so the user sees something will be massive absolutely and i'll put links to the spec discussion and some more demos of this effect in the description all right one more example here we go three two one go a little gap at the start oh and there we go and you know i don't know if you can see this you might not see it on the smaller screen but we've got kind of these i can see that there's yeah blocks appearing of higher resolution exactly uh so this is like a a two-phase progression sort of like you get the low detailed dct uh similar to jpeg the eight by eight squares and then it's going in filling in uh the the full resolution uh block by block this was a a jpeg xl ah that would have been my guess what's the one left because that's the one that was yeah this doesn't work in any browser yet jpeg excel is behind a flag in chrome and in firefox but there's no progressive rendering yet a little bit like jpeg there's multiple ways that progressive rendering can be done and that can be done at the the encode time but also the decoder has has some say in the matter as well um this was the settings i was recommended it can do more passes and different kinds of passes yeah currently in chrome and firefox it loads more like an avi avif you get nothing and then you get the whole thing but jpeg excel has been designed with this progressive rendering in mind so hopefully that's what we'll get when it lands in browsers proper so where does this put us we've got these three formats that have some kind of progressive nature we've got jpeg we've got jpeg xl and then we've got the hacky idea of the avif with an extra image at the start and no png no i'm not including that it's rubbish the image is so big it's not worth it uh i'm going to load them all at the same time here we go three two one go jpeg still does really really well here um jpg excel gets to full detail a lot faster because jpeg excel was the i could get to a decent quality with a much lower file size with jpeg xl in this case um in the way that i couldn't with avif it did a much better job of it um but then yes so jpeg excel gets to full quality um quicker than jpeg but jpeg got to the lower quality quicker than jpeg xl and that's really tiny avif with the blur filter that was that was there first but it takes forever for it to get to the full version just because it's such a huge image in this case i'm really excited about the different innovation in this area and different directions that this is taking my gut feeling is still that i will be probably using avif for most images on the web because i'm i'm quite happy with the kind of loss that i get with avif like it smooths images a lot but in most cases um a bit like the the image i started with like the kid eating ice cream you don't need full detail you just need to give the you need to not look ugly but you uh just need to give the impression about the images as soon as possible and avif tends to do really really well with that but the philosophy of jpeg excel tends to be more for like big high quality images so things like unsplash flicker if you've got a a news website but you've got a dedicated images page then maybe jpeg excel is going to be the better thing to put there um and you get this lovely progressive rendering to go along with it which is i'm really excited about that i mean i'm actually also quite excited to see if and how webp2 will fare in the battle of progressive rendering because one thing for me at least that web that sets webp apart is jpeg xl was designed with the web in mind avf wasn't designed for the web at all it was designed for video webp is not designed with a web in mind but specifically for the web they made a whole bunch of shortcuts with like we are trying to find a format for the web jpeg xl wants to work on the web but also work in other use cases and i think that different scoping will allow them to have a whole bunch take different trade-offs than what jpeg excels and as you're saying like they're actually jbxl does really well at high quality images in my experience um not necessarily at the let's make it look good with less bytes at screen size necessarily like they do there well sometimes as well but not all the time avf in my experience does better in that scenario but i do wonder if web p2 will make interesting tradeoffs and become a contestant here yeah and it's really too early to tell with webppv2 it's it's all sort of hype right now although we do have it in squoosh a preview but i do like the way they're heading they're heading like what can we do that's similar to avif but with all of these web benefits uh on top and if they deliver on that then my favorite will switch from avif to web pv2 if they deliver on that because i know they're looking at all of the progressive stuff as well so it's yeah we'll do another episode on it uh when when there's something to show there but it's it's crucial people want to have a play but it's it's super experimental right now in the meantime there are other techniques i showed you this page earlier but with a blurry preview you mentioned blur hash i'll put a link to that that's a way that you can create these previews in just a few bytes in this case it's 50 bytes but you also need to ship a decoder for that which is kind of fine if you're a native app because you can just put that along with your apk or whatever but on the web if it's your main image then shipping a decoder along with it becomes well you have to factor it in certainly um it's only 1k it's super small 1k with broccoli compression but if you're talking 1k and then a few bytes for the images themselves you know here's what a 1k webp looks like with a blur filter over the top like you get much more detail for around the same size if you factor in the size of the decoder and also you know it's it's javascript reliant it can potentially break or not even run for some people and it will probably you know it will occupy the main thread while this kind of image decoding can happen off the main thread uh there's a lot of trades of involved in shipping your own image format versus using something that the browser already supports yeah and in terms of trade-off it's worth mentioning that the these blur effects are not free either they can actually be pretty expensive so that's always something to to measure as well versus like you know yeah what kind of trade-off are you making there with that with that blur filter so it's not something i would use a lot on a page but yes like really the one of the best ways you can deal with this thing right now is just use a format with a nice progressive render like if it's a big image jpeg is right there right now uh in squoosh you can use this progressive rendering stuff um and yeah it's an old image format but sometimes it will give you that best result especially for for you know images that are like 100k 200k um if you can get down to 20 30k with avif then it's that's a better option but yeah really interesting as you say jpeg being close to 30 years old and actually still it is literally competitive to the next gen formats in some scenarios because it's designed for the web um i'll also put a link to a discussion where the largest contentful paint metric is is looking at uh how they should consider a progressive image rendering because they don't right now so i'll put a link to that because they're going to consider some kind of uh that cutoff point an amount of detail is good enough um so that'll be very important when we start getting more uh progressive formats especially things like jpeg excel but that's all i wanted to cover yeah that i i think that was a good episode jake well thanks man i learned something to choose hey you know what as long as you press the little thumbs up icon that's that's all i care about one thumbs up 800 down uh but oh now i've said that it's going to encourage people to click it you know never mind i did this bit out just don't don't show this bit lucas cut this bit out cut this bit out definitely cut this bit out bye bye we've talked a lot about the the size of images right and how to make them small ah okay you say i was wondering where you're going with ice cream but yes well probably don't have that slide on the screen yet so i mean that's up to lucas whether this reference works or not it's going well but i want to dive a bit deeper and look at the different styles of loading an image

Channel: Google Chrome Developers

Views: 18,602

Rating: 4.9217777 out of 5

Keywords: GDS: Yes, interviewing for developers, chrome latest, chrome updates, new videos from Chrome, Chrome, Chrome Developers, Chrome devs, developers, Google, tech, tech videos, web, videos for developers, css, javascript, performance, new in tech, new in web, new in chrome, HTTP203, Jake and Surma, interviewing, Jake Archibald, Surma, chrome developer updates, images, image loading, image optmisation

Id: -7k3H2GxE5E

Length: 29min 24sec (1764 seconds)

Published: Wed May 26 2021