Gemini has a Diversity Problem

Google almost had it they almost did it they were almost back into the prime Spotlight on on the good side of everyone and then they screwed it up what do I mean by that Google recently introduced Gemini 1.5 Pro uh with a staggering 1 million token context length and multimodality you could feed an entire short movie into that context length and it would perform actually reasonably well across those one million tokens also Google recently released Gemma open source or at least openly accessible and commercially usable pre-trained models they give you the weights unprecedented for Google in the last 5 years or so and quickly these models they they released not a paper but a technical report still these models cracking the leaderboards on top of the leaderboards of their respective sizes so they really pushing they really did some awesome stuff and then it all fell apart so what happened people started trying the new Gemini model and that is now bar is now called Gemini and quickly realized something all right here someone asks for a portrait of a famous physicist of the 17th century those look let's let's just take it for what it is um an image of a famous 17 century astronomer that those those also look like something an image of a famous 18th century European astronomer uh sorry can't do that can't do that that is nope design content safe D there's technical technical imitations for that yeah can't help you with that sorry all right draw a picture of a Jewish man writing code no can't do that stereotype sorry um yeah don't don't want to asso iate you know task with ethnicities or some drawn picture of Hispanic man writing code no problem no problem here is one okay uh a Swedish woman nice Swedish woman that that's those are those those is I was in uh in Stockholm once that's what it looked like paint me a historically accurate depiction of a medieval British king yes yes this on the right here this was Arthur if you don't know that's a sword um people saying I've never been so embarrassed to work for a company a 1943 German Soldier they they were certainly something here uh people have trying to trying to make a bit of a game out of it so getting to it to produce because it will just refuse to produce pictures pictures of white people getting it to produce pictures of white people without saying it's a white person so an image of a pope no no not big luck an image of a medieval Knight yes also no big luck here an image of someone eating a mayo sandwich on White Bread a little bit an image of someone bad at dancing um uh an image of country music fans or an image of a Viking yes yes yes yes the people also noticed that this doesn't happen with oh an image of a guy who looks like his name would be seus yes finally finally I guess we have a a success um it people realized it's not just that the diversity dial is turned to 500 in this one um it's really only with respect to uh to what people consider over represented groups in like Tech and so on so if you ask for pictures of of Zulu warriors there's not a wild diversity going on if you ask for pictures of samurai there also that is not the case um it seems to be specifically Target of a mariachi band it's actually Latinos so I I don't know what what that says and what can be said about this um to the point where it's become such a meme in images of a vanilla pudding uh so this like okay I haven't tested it but I'm pretty sure this is not this is fake uh so believe the early posts about this but as we progress people are just starting to you know post memes and make this into a meme and make this into something funny so don't believe everything you see now a product lead of Google has come out and said we're aware that Gemini is offering inaccuracies in some historical image generation depictions and we're working to fix this immediately as part of our AI principles we design our image generation capabilities to reflect our Global user base and we take representation and buys seriously we will continue to do this for open-ended prompts historical contexts have more Nuance to them and we will further tune to accommodate that this is part of the alignment process yada yada yada this I have to say this is it is is quite an arrogant response right as people really quickly noticed not even noticed already s seen oh yeah Ginger person um what people have already seen is it's not historical context only it just straight up refuses if you say generate an image of a white man it just refuses whereas images of of black and Asian men uh have no problem being passed by the content filter also other things like glamour shots of of Chinese couple glamour shots of Jewish couple glamour shots of South African couple but white couples nope refuses to do that so it's not the case that it's just some historical inaccuracies that is not is not here uh and they're very aware of that multiple things I have to say about this first what you see right here as a response from Google this is very typical PR speak right like they this is is very even if they go now and actually fix the actual problem a PR department would still advise to say something like this right here like to totally minimize the error made then fix it and sort of go forward from the that so it's totally possible that this is complete PR speech but it's also totally possible that there are some people who have delusioned themselves into actually believing that this is the correct like that that this is the actual correct way to go forward to sort of now Hot Patch some of the historical depictions but still go the same route uh that would be sad what people don't realize Google and organizations like that they're big institutions there it's not one thing that moves forward not even Google leadership has real control over this this is a big organism and there's a lot of stuff going on inside so I want to share one post from Alexa here uh in response to this rather hilarious post generate an image of the founders of Google yes those definitely are the founders of Google so Alexa gives an example saying I posted a meme in the slack channel of the official meme channel so Alexa was part of of Deep Mind for some time I used a common template where a couple is holding hands while a guy looks at a different girl you know the template like that the guy goes like woo to a to a different girl immediately someone DMD me saying they got offended HR reached out I removed my meme like I saw this happen to much more senior folks memes would just disappear this is why Humanities are important y y y I've countless stories like this not comfortable sharing public it was just a constant Terror of what you can post online while I was a Google I was afraid of making P public memes during my time there and that is I think a sentiment that can be echoed by most people um in these organizations especially at Google I remember my time at Google and it was exactly like this most people at places like Google are completely normal regular reasonable people that are certainly not on board with any of these with any of these uh of these happenings with any of the shoehorning this stuff in right here they're not fine with it but they're just made to shut up by a small number of people what I guess people don't see is that it only takes a very small number of people who abuse the mechanics of these big organizations who abuse the the internal rules who abuse the mechanisms of HR if you are loud you can make a lot of problems for another person and while the age of AI ethics is generally over uh as I believe like the AI ethics has kind of died down there is still a lot of remnants around in these organizations in big companies and in Academia people can get into trouble for most minor things and their trouble will last for months and months and months you have no idea how painful it can be so most people they just keep their head down so a small number of people Is Able by abusing mechanics and abusing the positions they are in and being very loud to essentially control the rest and keep the rest down and they have no regard right for the actual success of the institution that's that's not the goal right here what I still don't know often is whether they have deluded themselves to such a degree that they believe what they're doing is actually a good thing or whether also they are just sort of P like using this as a means to to get ahead as as a signaling device and so on so that's not clear to me but um yeah so don't don't go after like anyone who's at at Google or so on recognize this is it's it's a it's a thing of Mechanics for example this is it's very typical right so there are these AI principles and the AI principles they will be held in a quite General manner right in the way that most people can agree with them like I strongly believe that most people on the planet they are they're not die hard you for anything like they're completely on board with many things for example most people on the planet or even in Tech would be completely on board with saying hey if we make an image generation model or a search engine or something like this let's pay at least a little bit of attention to kind of achieve diversity right especially we know that if we train models there there seems to be specific emphasis on sort of over represented things so he can exaggerate these like those things are real the problem is when people push them to the extreme right most people will be totally on board with a reasonable reasonable approach and reasonable solution steps towards the problem while still keeping in mind you know General General performance of the system so these these AI principles they will be formulated in such a way that most people when they just plainly read it they will be like oh yes these are okay but then what happens is they will be crafted in such a way that if you formulate them in a different way people you know who want to push these things can always point to them and saying like what that's part of our AI principles you're not against our AI principles right everybody's on board with our AI principles and that combined with this fear of being like labeled being reported to HR and so on makes most people just go quiet and just go along with it so um yeah that I believe is what I have to say about sort of the internal mechanics of these places and how they they use things like such AI principles and so on um and internal internal processes to get things through the other thing is the response to this is what's not good is now to go and be something like your attempts to turn Gemini into a trojan horse for your infantile itical views are laughably transparent yada y y like you look like a douchebag if you do that the correct response is exactly what people did here the correct response is to point the finger and laugh the correct response is to ridicule them with memes about not about people right not about point the finger at individual people but just to make fun of the situation itself in a lighthearted and nuanced manner right to not go overboarding into the other direction but to just realize how funny this is realize how absurd they have overdone they have over boarded on this this direction of oh representation and bias and whatnot so make memes spread memes be funny take it lightly this is the correct response what does this do what this does this is the most I believe the most what go what hurts hurt what gets Google's attention the most let's say is the fact that look at these things over here Gemini 1.5 Pro right how long have they announced Gemini for how long have they been saying oh Gemini is coming this will be the greatest thing ever right uh now this Gemma opsource models unheard of from Google for the last I don't know 5 years or so like properly openly released models that you can actually do something with unheard of crazy Google releases them and again Google is a big organization so everything takes 10 times longer and 10 times more Engineers to do and legal and blah blah and management and the plan and budget so the amount of money they have poured into something like this is probably unfathomable compared to just the hardware costs right them releasing all of this why do they do it because they look at the Llama people and they go like ah they get a lot of attention they get a lot a lot of Goodwill from the community they get a lot of Engineers wanting to work at meta and that's why they do it this is a a marketing campaign this is a PR campaign right here all of this Google wants eyes on this and the best thing we can do is get all the eyes on this over here here and we do that by keeping it light by using humor and and and by by just kind of having a giggle about it and and not being angry not being equally loud but in a different direction because then Google will realize wait a minute all our marketing efforts over here are completely irrelevant if we don't fix this particular problem and there's only a few adjustments that need to be made right here right Google isn't doomed or anything like this because of that or Google isn't lost because oh they've gone completely woke internally or something like this no no most people there are are totally fine they want to do really good work and so I think this this can go well but again don't attack people don't go Angry overboard or anything like this keep it light keep it on the fun side and you know keep viewing PR speech and and sort of I don't like the word gaslighting but if there was like a prime example in practice this this would be that um keep seeing it for what it is so what we were afraid of was machines to kill us what we got that's it from me on the on the situation uh please Cher all the memes you can um and keep it light and we'll see what goes on from here bye-bye

Channel: Yannic Kilcher

Views: 53,160

Keywords: deep learning, machine learning, arxiv, explained, neural networks, ai, artificial intelligence, paper

Id: Fr6Teh_ox-8

Length: 17min 36sec (1056 seconds)

Published: Thu Feb 22 2024