[Day 2 - Morning] - AI Day 2021 - Empowering Innovations

[Music] [Music] [Music] i was working on my phd thesis and this is right in the 90s and at a time when ai was mostly a academic curiosity i never thought that you know one day i'd have a chance to come back to vietnam and file an ai research lab right here [Music] the creation of vinaya research funded by wing group which is the largest company in vietnam is itself a very intriguing proposition [Music] and right from the start i want to be able to create a world-class research lab in vietnam and i want to focus on research excellence and also the commercialization of the best research ideas and basically we want to challenge the presumption that such top institutions can only be filed in the first [Music] world i think doing top-notch scientific research in vietnam is like a promised land that hasn't been much explored it needs brave pioneers and financial backing to make it happen when you come here you know you work with the best minds the best tools and the best infrastructure [Music] i enjoy the discussion with our residents the brightest student in the country i also enjoy the interaction with the talented engineer and scientists the discussion and interaction have always lead to many interesting direction and ideas from the problems i'm working on [Music] one of the greatest things that being a research that i cannot do at other research labs is that aside from doing research i can make jobs and write problems in vietnamese on some days i focus on programming on other days i focus on paper reading writing or project planning still i spend few hours per day to mentor my residents so that i can keep check the progress and provide them with advices if needed back then when i was in my university i had never actually changed a gypno network but after joining video i became one of the most active users of tessera on gpu's management platform however i kept working very hard and things got better so i submitted a paper to icml 2020 and my paper was accepted and it was i was very happy because it was one of my biggest achievements ever delivering world-class ai products making vietnam is an inspiring and meaningful mission that drives our team's commitment together regardless of what role they play vinai is a mix of four to five startups sharing the same mindset who dare to face the more challenging problems in its industry with disruptive approaches for more than a decade away from home the biggest lesson i've learned is living in a first world country will not give me the sufficient perspective or insight to solve vietnam problems nor help grow the next part of vietnamese scientists and engineers so here i am in vietnam at vnai to help build more great products our current project combines state of the art from computer vision energy space and machine learning is run across industry from automatic healthcare services and consumer electronics we will continue to explore the challenging and promising application for ai such as ai and iot with advanced technology like ai big data and our top talent resource and to develop the top quality products not only for wing group and vietnam but also for global market so humans are always the chief architect in any major changes in our society and the ai revolution in vietnam will be the work of many world chain engineers and research scientists and at vinai one of the goal is to create a high quality ai workforce for vietnam and we're talking about people who can actually work on difficult research solutions for key societal problems and then turn those solutions into useful products and this will go a long way towards realizing our vision of being a world leading research institute advancing the state-of-the-art research and also isolating the creation and adoption of innovative ai applications to improve the quality of life of people in vietnam [Music] [Music] vietnam [Music] my [Music] [Music] toy language [Music] foreign [Music] [Music] foreign ai [Music] good morning ladies and gentlemen welcome back to the second day of ai day 2021 empowering innovations the hosted ai event live stream from vietnam organized by vinaya research we would like to start our today event with the theme of ai for innovations global ai products to open the program we would like to introduce professor ming hwang win who is the principal research scientist and head a applied perception group at vinaya professor why has won numerous awards notable the each ieee cvpr best student paper awards in 2012. today professor huai will tell us about perception technologies that vinaya is working on for safe and automotive driving please welcome professor thank you zhang uh can you guys can you hear me and see my slice yes it's working all right great thanks okay so um so good morning everyone and thank you for tuning in today i have the honor to represent the applied perception group of v9ai and i will tell you about some perception technology for safe and autonomous driving that we have been working on so first of all i must say that i feel extremely fortunate to be sitting here presenting my work why billions of people out there are struggling with the current pandemic now in vietnam alone we have already recorded more than 10 000 deaths and a very serious problem and will probably get worse but i'd like to point out also that in vietnam we have many other problems as well and one of those problems is traffic leasing and traffic accident now during roughly the same period of time we lost more than 11 000 people to traffic accidents so vietnam is not the only country that suffers from this problem in the world there are more than 1.3 million traffic related deaths every year and that means that one more person will die due to traffic accident when the countdown clock runs out okay now so most of the accidents are due to human errors from rugged behavior such as drunk driving and speeding to causes due to human limitations such as drowsiness fatigue and inability to react in time so reducing the reliance of on human driver will definitely save many lives and it will also save time and money for example in vietnam i think think about how many times we lose due to traffic jams it would be nice to have a car that can drive itself using traffic jam and give you the freedom to do other work in other countries traffic might not be a big problem but people actually waste so much time fighting parking spots it would be nice to have a car that can go and park itself and return to pickup when you are done right so at being ai our applied perception group has been working on perception technology for said and autonomous driving and we cover all aspects of driving from driver monitoring vehicle monitoring and road and traffic monitoring so this will also be the outline of my talk today first i will be talking about our driver monitoring system well so this is a system that is primarily based on a camera monitoring monitoring the driver face so this system can detect facial feature landmarks register and recognize a driver track the driver gaze detect drowsiness destruction smoking and phone uses maybe it's best that you know i show you how it works [Music] so the system can detect the driver phase and monitor for destruction and easy warnings here you can see that the number of warning has been pop-up to alert the driver the system can detect mobile phone usage this system will do the day night and even work when the driver wear sunglasses now now the our system will monitor the driver and issues warnings to driver if it detects some unsafe behavior and if the driver doesn't respond to escalating level of warnings then you know it can it can signal the car so the car can pull to a full stop and call for emergency support if necessary [Music] okay so um so our system is um it primarily is on an infrared camera so this figure shows the location of the camera and the infrared light uh setup on the wind fast car our system also capable of increa integrating other information from other type of sensors such as the speed the speed of the vehicle is traveling and also the door sensor and some other sensor to enhance the performance of the system now in terms of perception tasks these are the underlying perception tasks that we need to solve in order to enable uh you know for the driver monitoring systems and um and our system achieve high level accuracy for all of these features uh it's in all in the above like 90 range or higher and we have already validated this on uh you know a number of data set both public and cell collected data sets uh and sometimes on millions of images in fact our face identification system has been benchmarked again many other methods by the united states national institute of standard and technology and it ranked number six among several dozen of submitted entries we also compare it with a commercial vendor that we know uh you know most of the most of the times we don't know about the information about that vendor but for the things that we do know i think the performance of our system is very competitive or even better than what they reported so this is the performance in terms of accuracy we also benchmark the performance in terms of resource consumptions so our system can run on multiple hardware platforms and it runs like 40 frames per second on the media head unit which is very small you know chipset and it only consumed 20 of the cpu okay so uh that's it and now let me move on and talk about our vehicle monitoring system well uh first this is a typical cars is they contain many sensors including camera radars and ultrasonic sensors but the best sensor is still the driver but there are two problems with this sensor first the driver can be tired and distracted second the driver cannot see the car itself now for the first problem uh we already introduced our dms system now let me tell you about the solution for the second problem our solution is to develop a 360 degree surround view monitoring system it allows the driver to see the car and it's surrounding from any angle so this is how it works so this is the car and we installed some fixed eye camera around uh you know four-phase eye camera around the car so this app these are physical camera which have very big field of view 198 degree horizontal field of view and the input to our system are four phase eye images and the output is uh basically uh it allows the driver to look at the car from multiple perspective we can look it from the front the side from the back on or from even from the both right now so you might wonder what what would be the benefit of seeing the surrounding well uh first of all we developed this uh you know our first customer even fast and vin fast is a piece of art now if you own a win fast you will want to stare at it and to admire its beauty the second thing is maybe many of you might already be familiar with the bike view is useful for reversal and parking the top wheel provides a big a big picture and is useful for better for perceptions and the side view of the car is useful for navigating through a narrow alleys right this is very important if you want to travel in in vietnam you know how narrow the road could be let me show you how it works you know to surround you why why the car is in motion so the car uh the driver can look at the car from multiple angles here's from you know from the side the car is approaching an intersection and so yeah the driver can also look at the car from the front let's and you see in this case the driver can also uh seeing a motorbike approaching from behind so this is here now there's many technical challenges for this system uh first of all uh we want a system that guarantees the exact visual perception for drivers and also we must guarantee you know uh the system must run real-time have low cpu load and use more amount of memory and i'm very happy to say that we achieve all of this criteria our system can run real time 30 frames per second at even more so 30 frames per second is the limit of the is the frame rate of the camera it only yield 10 percent of the cpu and it's you only very small amount of memory and this is measured on a very small chipset it's a win-fast media head unit so let me show you how surround view can be used for uh parking and navigating through narrow roads so here the driver can select the 360 surround view app to see the car from any angle [Music] most of the cars on the market is they do not have this feature only the last luxury uh cars like all the mexico and now we are you know we bring it to win fast [Music] so [Music] now after after parking the car you can look at the car from multiple angles to make sure that the car is the right spot [Music] here the surrounding is also very important if you want to get through narrow roads avoiding potential collisions uh if you if you paid attention on the right side uh you know on on the on the screen on the right side of the driver there's a little window showing oh here it is showing the side view of the vehicle and this basically shows you how much clearance the car has so it can go through the narrow strip [Music] right and then after parking again you can verify if the car is indeed parallel to the parking lot okay now uh let me let me talk about some perception technology float and traffic monitoring first i will talk about the lane detection problem the lane detection problem as the task is to detect lights lane boundaries and also drive drivable uh free space now lane detection is actually very very important problem it's very fundamental and it's a basic input to almost every autonomous driving or advanced driver monitor assistance system called features uh some of you know it is used for lane centering control it is used for lane departure warning lane keeping assist emergency lane keeping assist how to lane changing on-ramp and off-ramp all the observed feature is also used for forward collision warning adaptive crew control uh blind spot detection and automatic emergency steering as well right so all of this feature depends on lane detection but land detection is so challenging uh it's so hard and pop islands are due to the weather condition uh you can have very complex topology adverse weather condition uh you know lane boundary come in various types you know it could be heart divider subdivided it could be tree and bushes in vietnam most a lot of the streets have no lane marking at all or even when they are landmarking there's so much traffic there's so much occlusion that you don't see you know the lane lines and the legs and in many cases the road get you know the surface we have to degrade the draw services and then it also make the problem much more challenging now the second source of challenge is due to the sensor here with your camera and camera are much worse than humans eyes in terms of resolution okay even though we use high resolution camera you know it's still far behind human eyes the second problem is you know we might have camera installation or calibration error so the estimated physical location of the might be wrong and and also camera is optical distortion and and homogeneous mapping so a far away object only occupies a few pixels why a close by object you know you know may occupy you know half of the image so this leads to some some issues called stability uh how you convert from the error in the pixel space to the physical errors localization error in the image space may correspond to a large error in the physical space for example a lateral error one pixel in lateral error might correspond to 20 to 40 centimeter vertically you know one pixel could correspond to like one to two meter right error in the physical space and this one is non-homogeneous because you know the nearby pixel or the further away pixel corresponds to higher level errors in the physical space so our technical approach here is we based on a mod a cell redundant model we combine two different approaches in deep vision detection and segmentation and this provides internal redundancy inside division model to implant both detection and accuracy and reliability of our system we already performed some quantitative measure and our our method achieved very high precision and recall both for lane detection and boundary uh in terms of localization error obviously it could be higher error uh for for further distance but you know right now for lane within uh with the distance between zero to 50 meters the area is around 8.8 centimeter we also benchmark it with one of the commercial system called mumbai eye for localization error and here's a lower e beta and our method we have a lower localization error on vietnamese condition let me show you some detection results uh so the white line is you know the detected land lines and the green is the detected rival service uh i believe this is the road to uh not bay airport okay also here see how it works on some other uh you know condition like you see ocean park you see on one highway at night yeah mobile is a commercial system but one of the advantages of our system it worked well for our condition and one of the reasons we also have much more training data than they do anyway all right so anyway let me move on and talk about the next perception traffic sign and traffic light detects uh so we view a customized model for vietnamese traffic sign and traffic light and our system can detect a classified 392 types and it is particularly accurate for science needed for intelligent speed assistance such as you know speed limit slow down or things that one would need for controlling the speed of an autonomous vehicle our system can detect size as small at 12 by 12. it's very tiny if you look at the size of the original input image 1900 by 1200 we can detect traffic light and also the countdown numbers on traffic for traffic light this our system runs 25 frames per second on the drive agx and right now we are working on uh do some added feature to provide the depth for the design and also so that the system can understand the text information on information size as well now quantitatively our system can detect all the type of site from no entry site to danger size uh traffic lights most of these have very high detection performance measured in term of every precision for traffic size between zero to a hundred meters the detection we call a hundred percent if you go beyond a hundred meter it becomes ninety percent but beyond that it's uh the site are tiny so the detection recall dropped to forty one percent and we're working to improve that so even that uh and i also said you know our you know uh some sites are more important than the other and for the site for intelligent speed assistance our system can detect with 99 uh recall we can also detect traffic light uh the type of color the lifetime the number and also the dry directions that you'd like is supposed to apply for okay so let me show you some detection result as you can see you can see that you can detect something even very fast look at this limit i can't even detect it [Music] it also work at night as well okay all right so let me move on and talk about the next uh perception module for 3d object detection so this is a component that the input to this system is images from multiple cameras so this is the multiview [Music] multiview system and the output is the 3d information of the object from the location of the object the size the heading angle of the object right now our system can detect at least eight classes from car truck bus to ambulance and fire truck and the detection range uh up to 100 meter and right now our system can run like 40 frames per second on the nvidia agx drive um in terms of detection accuracy uh the recall is around 75 percent with the localization error of 0.3 meter in range of 20 meter and about 1.2 meter if the object is further than a 50 meter and we test it is on nvidia agi drive and depending how many cameras you want to use at the same time the frame rate will vary so let me show you how you know how it works so this is our system running together with also land detection and traffic side detection so it it is able to detect you know car motorbike [Music] okay all right so uh you know so another component that's uh uh i like to talk about is uh so the parking spot detection now so our goal is that you know for the car to scan and detect uh various higher parking slot uh and output the slot coordinate and its output uh which is whether the parking spot is occupied or not using four fish eye camera for the surround view monitoring systems that i mentioned earlier so these are several types of parking slot we actually very challenging problem first of all uh there's a lot of the skill variants in the parking uh type now here i show some of them it also had huge variation in internal appearance you know the surface could be grass or brick uh we have parking light damages uh you know the low light condition adverse weather condition and you know it it might be occupied by people and object it might have shadows so this is a very challenging problem to solve in general having says that i i'm very happy to say that our system has a detection performance uh like 89.6 every season and it can run 64.8 frame per second on the cpu uh or if you want to run on android os then it runs about 11 frames per second i think very very fast or at least faster so let me show you some detection output uh so here as the car uh drive through the parking uh you know a parking garage it automatically detect the parking spot the blue box means the occupy spot and the green one is unoccupied freeze parking spots that the car can can pull in okay now in addition to working on individual perception modules that is mentioned we have been also working on mapping and sensor fusing that combine all of this output together and provide useful information for motion planning and control so we have successfully developed some demo that using sensor fusion for control and here we uh which i'm going to show two applications that we have one is for lens centering control and one is for auto party assist so this is for lane centering control so so this is the car drive it's uh by itself uh on the road without uh then the divider and lane marking it can automatically control the speed and [Music] when it can come to a curve it's automatically slowed down uh and then it can run with 50 kilometer per hour and here's another features that we have we have developed it for auto parking assistant so as the car uh pulling a parking lot uh you know the system will automatically detect the empty parking spot and then it choose one of the parking spot here's the number two it chosen and the car could you know control itself to drive it to that parking slot yes all right so i hope you enjoy uh what i just showed you in summary uh we had the applied perception group of vinai and we have been working on some perception technology for safe and autonomous driving and today i show you some overview about what we've been working on from driver monitoring vehicle monitoring to road and traffic monitoring and uh thank you very much for listening thank you so much dr hyun for your presentation very interesting so we received so many questions from participants and i want to share with you one question first yes okay oh right so sorry i would say anyway uh hi hun yes uh right now our system the aim for um uh you know we do not you like that uh we aim for systems that you know i economically uh uh enough that to uh to reach to a big market as well so that also uh you know raised some challenges in terms of perception because you know it's very difficult um if we open cameras and could be also it makes our life much more challenging too but at the same time yes that's what that's the problem that we need to address thank you for your answer and the next question is waiting for you a good vinai dms solution work on system without dla gpu such as raspberry pi yes uh yes i think so we so far we have been we have demonstrated this on multiple hardware platforms and we we are currently also working with several hardware vendors to port our system on other platform as well so uh it will be uh yeah so the answer is yes yes and um the third question very interesting um he wondered to professor huang most videos you showed under nice weather conditions does your solution work under rainy weather or night driving condition good question so you know so so we we also tested this on uh you know adverse weather condition uh right now uh i would say you know we we have tested in terms of the software level we haven't done it in terms of the whole system yet but uh that that is on our you know roadmap for validation and testing through so that's a he should support it and we're also collecting more data to validate and also train our system wow the nice answer and um one more question is waiting for you to dr ming hwang i saw there are many traffic signs contain a lot of text so can you deal with them uh that is something that we are actively working on right now so we want to not only right now what we can do we can detect the sign we can recognize the site type the system cannot read the text yet but that is something that we are actively working on so the last question for you to professor why may i know if i wear black sunglasses will your system still can recognize my face and monitor my sleepiness thank you professor yeah the answer is yes we you infrared camera so uh so in in the video that i show you know it can be a detect distraction the track that i would even when the driver wear sunglasses and right now the system worker in the you know nighttime daytime wearing sunglasses is no problem uh and i i should also say that our system also incorporates some sensor-based fusing so even when the driver is not in the view of the camera we can also detect some destruction or some dangerous behavior as well thank you so much all the participants who sent that a lot of questions very meaningful and thank you so much uh professor why wouldn't for your presentation and especially your nice answer is meaning so much to us thank you so much thank you very much and now please enjoy the video [Music] [Music] dear participants as you can see autonomous driving has become one of the global most medicine phase and it has ignited a manufacturing rate for smart vehicles that's what the the future holds for autonomous driving drop in the following talk and hear from one of the most influential fingers in this generation defining field dr ching yao chan dr chen received his phd in mechanical engineering from the university of california berkeley in 1988 and he has been with the university of california berkeley since 1994. he currently co-director of berkeley deep driver and he's the researcher at california partners for sven transfer transportation technology in 2003 dr cheng led a team of researchers and engineers in demonstration of best automation technology in san diego which won the best its research award from its america in 2004. in 2020 dr chen received the team leadership award from berkeley institute and transportation studies talking about the intersection of artificial intelligence at autonomous vehicles dr chen will include his observations on the potential of these two technologies through his team's research works welcome to our vitoria stage dr ting yeo-chan co-director of berkeley deep driver please welcome you hello uh thank you miss hong and thanks uh for having me yes my pleasure so should i begin yes please and check your slideshow okay can you see my screen yes it's working okay great i'll start well wait thank you very much uh everyone uh for joining us and i particularly want to send thank the organizer for having me i'm uh ching yao chan the co-director of berkeley deep drive from uc berkeley and today i will talk about uh when ai meets av and ai everybody knows it's a very active research and scientific area and av standing for autonomous vehicle is also very active in the last 10 years so i will briefly talk about ai for av and [Music] people often ask how do we use ai and for particularly uh autonomous driving it's a it's a very interesting but also very challenging topic so how do we use it and where do we use it and then i'll talk a little bit about our organization berkeley deep dry and i'll use uh some of the research uh project in my team uh to highlight uh the re the type of research that we do before i conclude so everybody know does the enable very powerful ai and a very fitting challenge alternate driving the question that we would like to explore is where and how we should use it so we start with the functional block diagram of automated driving system we start with the driving environment and then use sensing to detect things around us we do detection tracking perception and then we also use mapping and organization to decide where we should go joined together we will conduct prediction behavior and planning for the autonomous vehicle then we feed that into a vehicle dynamic kinematic and dynamic model to decide what kind of control command that we should use now if the driver is in the loop it will join with the automatic control system to issue a command for actuation which will lead to the change in the recall ego vehicle state which complete the group and going back to the driving environment now we can roughly separate or categorize the kind of functionality that we just described into three different stages the first stage is sense so for example if you look at this video we use a machine learning methodology to detect vehicle pedestrian uh the surrounding the lane uh marking and so on to help us going through the scene the next stage is plan so you probably have seen a lot of publication and uh media article about how google crews and many other companies company use machine learning approach to do prediction behavior and trajectory and here's a few selected uh images to highlight the progress that has been made the third stage is control so i use one of my teen researchers research for as an example uh we apply deep learning for intelligent power trend of a car and then for this uh machine learning uh approach we found that a dqm controller can reasonably do gear shifting and it can do uh control hybrid vehicle at low high speed speed and then uh convert to ice mode at high vehicle speed and then you can also do braking degeneration and due to the time limitation i wouldn't go into the graphic uh to the right but you can probably see uh one of our paper archive that explains all the detail so as i explained there are three stages stance plan and control in the automatic driving system in general the timing and precision criticality is more important to the right as i use a wage to represent the kind of requirement that we may have and in terms of industrial automation maturity uh as many many people know that in uh breaking all anti-lock braking and traction controls and many other functionality that's uh available in almost every car today uh the room the more to the right is the more mature that technology has uh already progressed so therefore if we were looking for area for to focus on innovation the further to the lab will probably give you uh more opportunity and therefore the potential for machine learning and ai will be shown at the purple wedge uh to the tool there will be more application on the left uh than on the right but nevertheless it can be applied across the spectrum of the functionality so let me move on to the next topic which is the ai research and industrial corporation activity at berkeley berkeley has a very long history and very strong reputation for ai research uh so one of the group at berkeley is the so-called berkeley artificial intelligence research bear which has been existent for many years and has almost 30 faculty members associated with bail group about six seven years ago we started berkeley deep dry which is we abbreviate it as bdd and then about three years ago we started another program called bear open research common and then as you see a few company logo highlighted here below in the slide is the company that we have very strong collaboration relationship with so as i say we started berkeley deep dry because the industry was moving very quickly into autonomous driving and a very strong interest which is aligned with our mission to merge machine learning and computer vision to enable intelligent autonomy intelligent autonomy could be applied to many systems but the application of focus for the last few years has been autonomous driving and robotic so the if you go to our website deepdrive.berkeley.edu you can see a lot of research activity description and roughly is separated into four area automatic driving robotic computer vision and perception and machine learning and ai methodology so if you think if you see any research project on our website it's almost is uh pieces of big puzzle and every piece is of research effort to explore the possibility for intelligent autonomy so let me uh move on to the next topic which i will introduce some of the research activity and this is only a highlight and not a full description of what we have but i will cover four research topics the first topic is by dr honor who's now with mice paris tech in france the question that we would like to ask is everybody know data is very important but what if you have too much data for example if you look up the kind of data collection performed by all the leader or big company in the autonomous driving domain you will see them collecting millions and millions of driving miles on the road now the interesting thing is these uh databases are so gigantic is also very overwhelming so the question is if we have so much data how do we streamline it or how do we identify the one that are really important so we develop a methodology mathematically uh if you look at the picture on the left on the lower left at the upper corner uh there's an ellipse which define the distribution of the data and then when we detect a new piece of data we decide mathematically what's the difference of that new data relative to the distribution of the data that we already look at so for example we these are the pictures that we identified to be unique or very rarely seen and has a long distance to the original distribution of data set so for example refraction on the window window construction side uh tunnel uh unusual vehicle and road construction and so on similarly using the same technique we can find a lot of similar sin which means that we already have very large volume or very frequent encounters of these data set uh in the data set we already evaluated so you can use these kind of technique to filter out the data that you don't need to overemphasize or overtrain your model the second topic i wish to introduce uh is by um ishikaw a phd student and he's working on pedestrian trajectory prediction so for example we took a data set that's recorded on a street next to the campus and then we identify the so-called posture of the pedestrian including the head the shoulder the body uh and the heel and see how these posture variable can be applied or utilized to improve the prediction of pedestrian trajectory so we use uh several messer including cnn including long short-term memory and some other traditional machine learning technique and then we built a model fitted with two seconds of data at 15 frames per second and we want to produce a prediction one second forward which include the angle and the scalar distance of the pedestrian movement so this is a quick look at uh the kind of model uh the training neural network that we build and utilize now here is a video to highlight some of the results um you will begin to see a green line for example now to the right of the picture moving across the street and the the green line is the prediction result and the red line is the ground truth and in in this example because our prediction result was so good that the green light actually overlay the red line and you cannot see it so this uh just illustrates the effectiveness of our uh model in predicting the project uh the trajectory of these uh pedestrian movement the third project i want to highlight is we use a methodological method learning for autonomous lane change and this is by dr faye e feiyen and then dr pinwa so the meta learning case in our application is we want to know whether we can train a model that can learn to learning learning to learn fast so we have a test learner which is the decision making strategy to whether to make a land change and then we have a major learner which enables efficient efficient adaption of the lane change so let me show you the data set and the application case so imagine that we are moving a red car to the exit in light traffic on the top graphic that you can see the car moving in very light traffic condition so let's say we have data for light traffic and then we have data for moderate traffic and we have trainer model so the question we want to explore it if we put the same model in the dense traffic such as the one on below can the model learn very quickly uh and then make the right decision uh for lane change so here's some result the testing results show that on the left we have two measure one is collision rate in other words if the model make a mistake it will collide with the car nearby the second on the lower part is the success rate of exit from the freeway and then the higher success rate mean that it will completely maneuver within the distance that was is given and there are two lines one is the green which is the meta learning agent and the orange line is the pre-trained agent using model that did not adopt the meta learning methodology so on the right we showed that in 5 step 20 step and 40 step how quickly uh our meta agent adapt to the new traffic condition and you can see in different time steps that it has successfully uh given the model the advantage of learning very fast the last research highlight i want to introduce is exploring how do we use it on a real vehicle uh i call the project a dual controller for tracking and this is done by dr eamon chan so here's the illustration let's say we want to have a call follow a reference pass on the lab now the concept is very simple can we use a deep learning model deep reinforced learning model to trend the model so that it would follow the path successfully what we found is deep research deeply enforced learning dil even though it's very powerful it's not as straightforward because it's a very complicated system so if during training you will find erected action which uh make the training or the learning very inefficient and the the controller the dio controller become unstable so we develop a scheme which has in parallel a dual controller a neural network and a deep reinforced learning learner uh controller coupled with a traditional controller so for example for steering it could be a very simple pid controller and then we use the traditional controller to compare with the machine learning controller and then we found that the machine learning controller can optimize and provide supplementary input or enhancing the traditional control very effectively and it also can be trained very stable in a very stable and efficient manner so here's the example let's say we want a car to follow this peculiar very challenging curve and we found that using our approach we can execute the controller very february over a p npc model prediction control controller and then it can be done in a production uh uh processor uh for real-time control so um even though i highlighted a few uh research is actually not a very good coverage of all the research that we do and so i do encourage everyone to go to our website deepdrive.berkeley.edu to look at more research at bdd so let me uh come back here and conclude my talk as everybody see the authors vehicle industry is moving very quickly you probably saw a lot of media articles about waymo robot taxi you also see many autonomously people mover type of shuttle around the world and you also heard about amazon and walmart and many other companies working on the delivery robot and then you also have seen many uh particularly recently a lot of uh [Music] investment and uh research on uh automated truck which will move cargo for long distance for example so this is a very exciting era for ai and av uh i think this is uh for people who are in this field which we are very lucky to witness this exploding movement in the areas of ai and av and particularly the combination of ai and av so in conclusion as we see the ai evolving with av i think we can confidently say that av when realized is a very disruptive transformation it will change the landscape of transportation and mobility in the meantime ai is a very powerful enabler and it will help us accomplish the goal of making av a reality and potentially they can jointly lead to a technology ship in transportation landscape so with that i conclude my very brief talk today and for those who have question again i encourage you to go to deepdry.berkeley.edu to see more of our research and then if you have any question you can reach me at the email listed here thank you very much and i appreciate the opportunity to talk to you and i wish you the best in pursuing your your research thank you yes thank you so much for having here and especially for your very engaging presentation so organizing committee have received so many questions from participants and i will send you some question the first question when an ai break the law like making accident who will be responsible could ai system be held crimeanly liable first action okay uh this is a question many people are interested in uh i think that we should we should not uh just look at ai ai become part of the vehicle system so uh when when some accident happened it usually it's not as straightforward as the fall of one piece of uh software or something like that but more likely the system uh the system had to be exam um and so i think that uh a simple answer is of course when we implement ai or any other software on a car we should go through the very thorough systematic approach to make sure that safety is guarded but if if ai it has difficulty achieving some very rare incidents of course uh the the challenges for the uh people working on the vehicle system to improve that uh and then i think the broader question and it's probably gonna take another whole day to discuss is uh when we when we take the driver away uh not necessary ai right because we could use a rule based uh algorithm controller car which let's just say it is an automated driving system replacing the driver so we take the driver away out of the responsibility certainly fall to the shoulder of the automatic system so that's why it's very important for the automated system uh to be as good or even much much higher better at a very with a very stringent uh requirement for them to uh perform well beyond the capability of a driver so that's uh that's a quick and uh maybe a too long answer for some of you but uh it's uh you can probably find a lot of discussion of this topic online and i'm happy to chat it with you later uh if we have a chance such a clear answer and now right here is some more question for you you what is your opinion in developing the ai tasks which are even hard for human perception and please uh okay so um i think that did we we have to be realistic right and so ai can do a lot of things and so for example ai uh can process uh much much greater amount of data than human but on the other hand uh human is very good in perceiving the overall concept you know for example let's say you have multiple sign on the side of a street now if you don't tell the car or ai or not what the difference or what the distinction and uh meaning of multiple sign on the side of a three is very hard for ai to interpret but only on hand when human having driven or having had having the knowledge for the for 18 years before you get on the road can very quickly interpret these uh ambiguous uh situation uh so so in this regard in some situation the humans are still better than a.i but move on if in the future ai becomes so powerful and can emulate what the human can do certainly it will take a lot of progress in ai methodology as well as the method that we use to train and the data set that we use to uh evaluate and test the ai system so uh i don't think there's a there's an answer right now for the for this question but it's uh it's that it's uh also very active research area which is why it make it very exciting and interesting for all of us yes so enthusiastically yes um and now would you might help us answer this question one more yeah sure a two professor ching yao do you think deep reinforcement learning could complete compete with the traditional methods in path planning and control for autonomous vehicle well i think that like uh my the research project that i highlighted in my slide i think it's certainly possible uh to have deep reinforced learning accomplishing the kind of traditional controller uh it's designed for however as i highlighted in my last uh research topic sometimes the training it's not very efficient or or can generate very erratic action so therefore at this stage i'm talking about the technology in 2020 and 2021. uh we we cannot say this is a 100 percent achieve however i think that with with the so many bright people and so many people in the industry working toward this uh task i think it's uh it's very promising that we can do that thank you so much and on behalf of organizers once again we like to express our deep appreciation for your very thoughtful preparation and especially very nice answers thank you and we wish you good health thank you very much and thank to everybody who listened and thank for having me yes my great honor thank you and now ladies and gentlemen please enjoy the video clip [Music] [Music] by [Music] foreign [Music] [Music] [Music] foreign be [Music] ai [Music] it's such a video [Music] do [Music] [Music] bye [Music] this [Music] [Laughter] [Music] speaking how may i help you ladies and gentlemen is clearer than ever then ai has been first lighting the ways of our life of race this reminds me of the court artificial intelligence will empower us not exterminators from the ai advocate dr orrin axiony in his popular ted talk making a case for the life-saving benefits of aius wisely to improve our way of life dr arien scieni is currently the ceo of ireland in new talk for the ai one of the leading global ai institute previously dr azir lee with the the washington research federation entrepreneurship professor in the department of computer science and engineering at the university of washington dr essioni was elected as trooper ai fellow in 2003 and he received the robert as ingomor memorial award in 2007. dr sciony is also entrepreneur who has found it and co-founded several business ventures dr ari essioni and counters first hand the growing fears about ai potential for fbi a power and it's such a great honor for us that have companion with ai day 2021 today to share about ai research engineering and how to build human-centric companies and anything ai related introduce to you all dr rayne ascioni for the talk on semantic scholar nlp and the fight against kobe 19 please welcome to vitro stage thank you so much for the kind introduction and i beg your pardon yes it's okay signal yeah okay um i before i start my talk i want to say that a few years back i had the opportunity to visit vietnam it's just a wonderful visit i was really impressed by uh the natural beauty and the the people who i met and the industriousness hard work and uh intelligence so the only thing i'm sad about today is that i'm giving this talk uh remotely as opposed to being in vietnam i hope once covet is over we can we can meet in person my talk as it says here is uh about semantic scholar and how we attack covenanting but i did want to start uh with the point about how ai evokes fear in a lot of people that i talk to are very scared of ai and i think that is true across the world and even for people who are computer science experts we have folks like elon musk warning that with ai we're summoning the demon very uh negative language very scary picture as you can see at the same time we have folks like sorry can you check your slash show it didn't run oh dear um okay so it is running on my uh screen but not it's not showing on yours um is it um or maybe i need to i'm sorry i need to share the screen yes so on behalf of organizers once again express our deep appreciation for your value eternity here at this very special event today and wish you have a good time at this event thank you um let me just put this aside um so yeah can you see my screen now yes i am so sorry about that i had shared it before in the test room uh but not in the real case so as i was saying ai evokes fear elon musk uh is a big uh big part of that and then we have more reasonable voices like roboticist rod brooks who says if you're worried about the terminator the robot from that hollywood film just keep the door closed and even if the robot eventually learns how to open the door he'll find there's a staircase behind it and robots have difficulty managing stairs and even if it climbs the stairs there's the next challenge and the next challenge so worries and fears of ai may be overblown the point that i like to highlight is that the success of ai these days as all of you probably know is due to advances in machine learning and deep learning but a point that people don't always emphasize is the machine learning is still 99 human work what i mean by that is if you think about the formulation of the machine learning problem defining the inputs like the target concept or function that's being learned like the data set and the labels on it and the particular algorithm or architecture that's being used all that is done by hard-working humans and of course machine learning is typically an iterative process where you do that and then you look at the output and you see that it went wrong and you iterate again and again and again all that is very much a human in the loop so my belief is that ai is a tool that we use it's not a being and of course it's up to us to use the tool to benefit humanity we see actors you know countries like china using ai for surveillance or to suppress democracy in various ways clearly that's not a good thing but again it's not ai that's being bad is how it's used and the mission of my institute the allen institute for ai or ai2 as we call ourselves is to use ai for the common good we're a non-profit research institute founded in 2014 by the late paul allen who's was a co-founder of microsoft and has dedicated his wealth to advancing research and science to make the world a better place over the years we've become leaders in computer vision natural language processing deep learning this slide is a little bit old we now have more than 500 papers and 18 best paper awards and most importantly we have more than 100 phds researchers and engineers and we're hiring we're hiring from all over the world so if you want to join us uh please look on our website lnai.org uh open positions we have the ability to uh to grant visas and we're very much looking for talented people in fact one of our most talented superstars uh the person who launched some anti-scholar with me is uh is vuha he is uh from vietnam and he is i guess uh fourth from the left on my screen uh with uh with the sunglasses uh anyway um so let's talk about semantic scholar what did uh vu and i do we noticed that there's a huge problem with academic information overload there's actually more than eight million new papers uh per year and yet even the most diligent scientists read around 200 papers per year so there's a huge gap there and we need tools to help us decide which paper to read what to remember how to extract the key results and so on so we built this tool semanticscholar.org it's a free service that helps you cut through the clutter and hone in on the key papers and the key results this really if you think about it three components three steps of how we engage with scientific papers we have to discover them from social media like twitter or doing a search once we discover a paper we have to decide whether to ignore it to skim it or to read it and once we read it we want to be able to understand the content retain it and maybe even cite it and semantics of caller has support for all uh three of these stages probably the place where we have the most uh impressive features is around helping you decide whether to read a paper how do you choose between the 8 million down to the 200 or fewer you're going to read and so of course papers have abstracts but we highlight in the paper the key sentences we automatically extract the figures and tables so that for example if you're on your phone you can just go to semanticscaller.org and go to a paper and quickly zoom and look at the different figures and tables to get a sense if this is something i want to read many other things i won't mention but i will mention that something we've introduced recently is tldrs tldr is the english phrase too long don't read and often when we're trying to decide whether to read the paper it's too long to read the paper to make the decision so we automatically generate one sentence summaries single sentence summaries of the paper to give you a sense here's what's in here here are the results and you can read that and again if you go to semanticscaller.org and do a search in computer science you will see these one one line summaries they're very useful over the last five years since we created semantic scholar we've expanded to all academic disciplines and not just computer science also biomedicine and really all fields astronomy economics etc so we have more than 190 million papers and we have more than 8 million users per month and some from vietnam but we definitely need more so i hope you will try it out and spread the word um because we had this uh system in march 6 of last year when uh news of covid was spreading rapidly and the disease was spreading rapidly the american white house contacted us and asked us to build a machine-readable corpus of all the relevant papers about covid about sars about mares about coronavirus they wanted to create a corpus that was machine readable so that other people could use systems uh to analyze uh this information and i'm very proud of our team we work day and night and within 10 days we were able to release this corpus called core 19 and we had quite a few partners here including ibm they gave us table processing software various pre-print servers like med archive and bioarchive and the national institute of health and so on it was a big partnership and we announced that we had these uh 24 000 coronavius research papers all in one place um i'm proud to say that now uh more than a year later we have more than 500 000 papers available on this topic in a resource that's updated daily and it's not just that the text is machine readable also the the the tables are available as well and google's kaggle competition launched a special competition uh to answer questions about kovid based on this on this corpus of papers and it became the most uh popular google kaggle competition i'm sorry the most popular cargo competition ever with uh millions of views of our data and by now almost a thousand systems that use our our information and again from a wide variety of places all over the world governments and universities and clinics and so on let me just give you a couple of examples so one of the things that people built was question answering systems so this is one that was built by amazon web services to show their capabilities and you can ask simple questions like uh how infectious is covid19 or you can ask quite technical questions like are il-6 inhibitors key to kova 19 and when you click you type in the question you click and out pops the answer and to show an example what the answers look like i actually went to a question answering system from korea university and i don't know if you can see this but the question was what temperature kills h code 19 and you see the answer up top highlighted is 56 degrees celsius and we don't just give the answer because the systems can be wrong we also give the sentence in bold and of course you can click through and and see the paper we didn't just have question answering systems there were search systems there were systems looking for patterns here's an example of a system that we built that does a visualization of different concepts in the paper and connections between them on this graph and when you identify several concepts like the ones i have in color here and you're interested in their connections you can click on the link the blue link between them and that'll instantly take you to the right paper so very much cut through the clutter we also built the ability to look at the work the different groups are doing and if you look at the bottom left uh scissite is up and running at lnai.org so you're welcome to check that out we have a lot of demos and also open source data like court 19 open source code and so on and so on our vision here is very much to connect the ai community and the medical community um and this is true uh for covet 19 but not just there what about the next epidemic what about finding a cure for cancer there's just too many papers too many experiments too many clinical trials and scientists and doctors and medical researchers need ai and systems like semantic scholar to help figure out uh what's going on to help them be better better researchers so our vision for the future is to not only help them doing search or deciding which paper to read but to even answer questions and formulate hypotheses that uh scientists can then test out uh in the lab and we have active efforts to analyze these um hundreds of thousands or millions of papers and find connections that scientists may have missed so with that i want to go back to the original point the original uh comment by elon musk where he said uh that ai is summoning the demon and i like to to quote my colleague eric horvitz from microsoft who says no it's the other way around it's the absence of ai technologies that is already killing people and he means that if we don't find cures to diseases using ai we're killing people and of course right before my talk you saw uh the uh the the work on self-driving cars and there are so many accidents from cars and motorcycles and so on uh deaths and injuries and most of these could be prevented once we perfect that technology so my view of work on ai is not it's not something we should be afraid of it's something we should we use uh to make the world a better place i um i'm going to keep my talk short because i hope we have a chance to answer questions i'm very happy to tell you more about the technical details more about the allen institute for ai or really any topic you like so i will stop here turn off the screen share and hopefully can answer questions i see oni and now we're waiting for the question from participants uh to dr essie it took you only two days to hover the cult 19 initial version ring from a manager perspective and you think they are key factor reason to this feeding result apart from no speed sleeping i'm sorry can you can you repeat the question just a little bit more slowly i couldn't quite hear uh it's uh show on the screen and i repeat off uh it took you only two days to have the cult 19 initial version ready for a manager perspective what do you think they are key factors reasons to this speedy result apart from no sleeping thank you thank you yeah it's a little bit small on my screen so i appreciate you reading it um so the reason that the white house contacted us i wanna i would love to say that we are the smartest and most hardworking people in the world that's not true uh we were working very hard it's an important thing but we already had the infrastructure set up so semantic scholar takes in papers from remote sites and ingests them and builds this graph of machine readable information so through contacts the cto the chief technology officer of the united states had heard about our work and knew that we would have the infrastructure available to move quickly so we just have to take existing infrastructure and pivot it uh to to to do this uh and that was smart of him so that that's how we were able uh to do it thank you for your nice answer and the second question is waiting for you how to apply ai to detection and treatment of patients with covenanting so um one of the um interesting thing is actually right i think we're all now aware that china delayed the release of information about kovit 19 but there was actually an ai system monitoring information on the internet i think it was called blue dot that actually um identified uh the fact that some uh potential pandemic was happening and alerted us uh easily so or earlier so the first thing is we can build alert systems the second point is ai systems are very widely used in diagnosis particularly uh in analysis of images so what's called radiology we take pictures of uh cancer tumors pictures of lungs and the ai systems are very good at classifying these so i think that there's there's big opportunities there i told my son he was here helping me with the tech a little while ago he's 11 years old i tell him don't become a radiologist it used to be a very good profession but by the time you finish medical school uh the the field of radiology will be uh largely automated well thank you so much and uh hopefully you will answer one more last question covet is a global problem however the very important corpus a trial build is for english only is there any any effort to build a similar munti lingual corpus yes thank you for that question that is a very fair question and i have to confess as you probably know that the united states is very english centric because we have limited resources we have chosen to focus on english but of course there are machine translation efforts and the possibility of taking that and translating those resources elsewhere they're available under free license and so somebody with expertise in translation whether it is google or somebody in vietnam could take that effort we make all the tools and the data freely available so i apologize that our work is just in english that is a big limitation i recognize that but i hope that you will join me and make it better so once again on behalf of organizers we would like to express our give thanks to you for the very meaningful presentation and especially your nice answers thank you and hopefully you will support it in the future thank you for inviting me and a real pleasure yes my pleasure thank you ladies and gentlemen please enjoy the next video clip [Music] [Music] [Music] ladies and gentlemen another partner has wholeheartedly supported our ai day 2021 so far that you all must have learned of is google today's event we have been receiving interest from top-notch experts at today's such a great honor for us to welcome dr marion crock vice president of engineering at google she is crea accredited as developer voice over ip the technology that has enabled entire workforces to come continue to communicate and familiars and friends to remain in touch despite of the current global situation dr marion clark has spent the decades working on groundbreaking technologies with over 200 patients 100 of them for the voice over ip for the past six years there's been vp at google working on everything from such real-life ability engineering to bring to public wi-fi to india's railroads currently dr marion crock overseas google center fall responsible ai and human center technology her team helps ensure that ai is developed responsibly at google and also has a positive impact recently dr marion crock has been honored in forbes 50 over 50 visit list which highlights the examinable women who are shaping the future of science technologies and art so i'm in conversation with dr marion today let's also welcome mr d v taka his program manager lead at google research where the lead strategy and programs for critical functions edu including academic engagement cs education at k-12 and serves as operations lead for the ai for social good research team he's one of the founding members of google research india and also an active human computer interaction researcher with interest in examining instruction of hci and ai for muggy knowledge communities let's join us to hear directly from the incredible women welcome dr marion all right uh we'll get started hi mariam uh welcome and thank you for being here at the idea empowering innovations 2021 first question to you is this your first time in vietnam or southeast asia or have you been here before well fortunately i have um i when i first joined google i worked for a pa called access and energy and my role was to understand the internet connectivity needs of emerging markets that that role transitioned into nvu and in it i was fortunate enough to visit vietnam and indonesia and we also went to thailand as well in miramar then it was lovely i remember that within vietnam we were in saigon and i don't know if it was just you know specific to saigon but it had the best internet access it was very powerful everywhere we went we were able to get it is that still true today i am not sure i haven't been there in a while uh but that's so to know that you were in the region and we can't wait to host you back again in person hopefully after code is not a thing in the world i would love that thank you so much it's amazing um so maybe you took on this new role it sounds super exciting you're the head of this new center at google called responsibility human centered technology can you tell us a bit about like one the origin story like what was the idea behind creating this new center and two what excites you about this place particularly to believe this yes yeah so um you know i did i did research um people had been nudging me to to get into the area and take on leadership for quite a few years because they knew my background and they knew that i would have a passion for it but i was not so sure so i did research both externally and internally and externally i found that you know most corporations had very high level ai principles similar to google i think we were one of the first corporations to actually institute them but they were very high level and abstract and they they lacked normative standards um for making them measurable and verifiable and they also hit a lot of the tensions that were in the field that were that were quite ripe and so you know that was my my view of the external field and then internal to google we had many teams working on very very um excellent initiatives but they were somewhat siloed and they lacked any single senior leader that could advocate for them and that could you know help them to to better their careers and have a real impact on google and they were kind of working on their own in some sense um so i made a proposal to sundar and jeff dean who's the head of research as well as the other google leads to bring together these siloed groups and we're still in the process of doing that and to have a senior leader there and they accepted the proposal and i was so honored that they you know went along with it and selected me as a leader um i volunteered to to actually be the leader and i was happy that they um accepted that it's it's been an honor to really support the team itself this new organization they are they're doing amazing excellent work and you know my desire is just to amplify that work that's incredible and that sounds like an incredibly tough job as well to bring together so many people so yeah yes you you asked me what excites me personally about the work and i should tell you that i have a background in quantitative analysis and social psychology and for most of my career which has been quite long almost 40 years now i hate to tell you but i'm kind of proud of that now you know and most of it has been in network engineering and reliability engineering and even when i've done that work i've taken it on from a user's point of view because as you probably know you can have all your systems behaving in a very reliable way from all the inspections that you've done as an engineer yet from the end user's point of view they're not having a good experience so you have to think of of engineering from an end-to-end point of view and include the user in this and i saw responsible ai as like such a melding of technology and the social impact of it and and the focus that needed to be on users and end users and i loved it so and i've always had this crazy desire to to really change the world and the work that i that i do and so this is like the perfect opportunity for me yeah i i love that both like in terms of the intersectionality and backgrounds and also from where your users come from right because as google you have users literally from all parts of the world so exactly responsibility exactly and you know my own life experiences living in america as a black woman it's taught me about you know the way that you can become marginalized and how decisions can be made unfairly so i think that also helps motivate me absolutely thank you so much for sharing that personal landing uh anecdote as well uh like this is a new field uh for ai but ethics fairness these notions have been around for a while so a lot of our viewers might not know like how do you think of product inclusion or ethics or fairness like what is the difference between all of these firstly uh so how would you explain it to like a person who has not heard of these terms and how do you explain it in a way that they can also use these in their daily lives as they think about building technology yes we'll think of fairness and equity as a part of responsible ai it also includes things like robustness which the way that word you know the various definitions as i said before but the way that i'm defining it and and for this purpose is like misinformation or information that cannot be verified right and you're using that to make decisions which is quite you know disturbing um and then there's transparency and interpretability that is also part of ai and that just means understanding fully what the model is what the data set is who's who's actually um part of that data set and how is it labeled and what is the model being used for and and what what is it what are some of the proxies that are being used to make predictions like where people live could be a proxy for whether or not they're able to gain access to loans or other monetary benefits so transparency and interpretability are very important parts of it and then there's also privacy you know and making sure that no personal information that is sensitive is being leaked as well so there are many dimensions to responsible ai that and it includes fairness and equity and product inclusion it goes far beyond ai or responsible ai and it looks at every aspect of a product in the design of the product and making sure that all its features and its physical kind of dimensions are applicable to everyone and that so you know that's what product inclusion means so you could think of ai as being somewhat embedded in that but as a small part of responsible ai is part of that yeah that's extremely helpful and i love how uh there was this constant use of the user and like centering all of these terms as well around the user and that's so important for ai researchers right like the thing that we're creating is eventually for an end person and we need to think about them which is just amazing how you explain this oh yes yes it's extremely important it's a it's a um a science that is based on you know technology and you know engineering but so much of it from the beginning to the end you need to have kind of an understanding of the social context and the people involved many people would make the claim that no technology is independent of the people that are designing it as well as the people that use it that's a very profound thought thank you so much so i guess like our viewers would understand a lot of this better by knowing a bit about the projects that your team is doing so i know we can't cover this possibly in this fireside chat but if you could give us a few examples of the exciting work that's going on in this new center and just explain some of the projects that are very exciting to you i'm sure all of them are but a few examples would be very helpful to visualize the work sure so as you can imagine there are teams that are looking at the genealogy of of data sets right where did these data sets that we're using to make these very profound predictions and responses where do they come from and who are the who are the people that are labeling the data we often call them crowd workers but who really are they and do they represent most of the world you know or all of the world um are they being paid are the images in a data set collected in a in a equitable way and are people consenting to have their images used so we're looking at the genealogy of data sets which i find fascinating we're also doing a lot of work on transparency and interpretability so making it clear to many different types of stakeholders the researchers and engineers as well as policy makers and end users which are all very distinct making it clear to them you know what the model is doing and how it's doing it and what all the innards of that model are um so that's very important work that we're doing we're doing a lot a lot of work in crisis response as well and and responding to things like being able to predict an oncoming flood in different countries especially within india and that is exciting work we're doing work in health care as well um i could i could keep going on but it's it's like it covers almost anything that you could think of that involves society you know ai is is a can be it let's say it is a very powerful technique for being able to forecast and predict and then respond to things so if you can think of any aspect of society that involves that we're trying to make sure that our team covers that that sounds so excellent and so exciting and median also has a center which is central to google right in a research arm i'm curious if i'm a user of youtube would i get to know if your team also worked on that because so many of the products that google has touch users directly so how would any of our viewer or myself know that it was marion's team that worked on youtube or something like that i'm curious if you have any thoughts sure that's a that's a great question and i think of it almost like reliability engineering if you have to think about it then we've done something wrong right because that means the system is not working well and you start complaining about the reliability of it so it's almost the same thing with ethical ai is that you should just have a very useful and beneficial experience with any of the systems so with youtube you should not be offended by anything that you're seeing you should not feel like whichever segment of society that you belong to is not represented on youtube that you're not seeing a lot of hate on youtube so we've done we're done a lot of work across research on responsibility and youtube but there's so much more to do and something like say the service that we're using now me at one point in time i actually had um a colleague of mine who who like me has darker skin have to walk around with a lamp when whenever he would go into a conference room because it wouldn't be able to pick up his skin color right so you know you should not have to experience anything like that in any other user-facing camera work that we do so if you have a bad experience then you blame me but if you have a good experience just be satisfied and probably someone working on responsible e.i has done something in the background to ensure that i love that i love that seamless integration piece that you're hinting to and your willingness to take the blame but not take the praise so if it works well also we uh praise your team for it um switching gears a bit uh into words we spoke about inclusion a bit uh in different facets that you shared as well right and there's also this part around regional inclusion and just how ai is distributed across the globe and it's developing at different paces at uh different locations so we're curious to know like if your team were to approach or if you were to approach a responsible ai product or research agenda in the united states versus in india where i'm based or say in nigeria like what would be those nuances that you'd think of when you're building responsibility for you know different places but uh and have responsibility towards different uses in the regional context yeah first of all that's that's a great question and it's an important question and we're going to have more investment in this coming year in in globalizing a lot of the work that we're doing and regionalizing it and i think to start off with what you want to make sure of is that you're not using a westernized data set and model and just bringing it into different regions of the world and expecting it to perform the same way it would in northern america um i think you have to be very well equipped to understand the region that you're in and i think you have to have this participatory model where you're having the people within the region work on the model and you have to make sure that it includes all the people within the region from my work in nbu and i'm sure as you know many many people still lack internet access and are participating in the internet so it's important to include those people so that they don't get you know left out or even further marginalized and we amplify some of the um inequities involved so you know you need community workers who can represent those people also involved it's also important to recognize that there's no such thing as universal values you know um other some may argue that there are but what is right and wrong is very dependent on what society and what culture you're in eating within a culture you may have very you know lots of variations depending on your ethnicity or your the tribe that you belong to the region that you live in um the gender that you are um so you have to be quite nuanced in making sure that you're not amplifying the worst of that society and that you're understanding the various values within the society so for example if you wanted to say who's the most beautiful person in india you would try to have perhaps you could answer it by having a diverse set of you know images of people that are thought of as beautiful depending on which you know region they lived in or maybe even in a more sophisticated way you could say that depends on who you ask and it's very dependent on you know where you're from and what you believe in or you know that that it that this is a a sensitive subject and it really depends on who you're asking um you also need to make sure that as i said before and i'm going to just say it again for emphasis that you're including as many people in some way you know having representatives of people a country is huge as say india or you know all the different regions and countries within africa um you just have such wide variations you know and you have your each country has their particular set of biases and inequities that you have to take account of when you're trying to make sure that your your model is fair and equitable that's a very long-winded uh statement but i hope it's it's understood yeah i know that's extremely helpful thank you so much like keeping in mind those regional nuances being willing to learn through communities i think that's perfect right and that's what our viewers and everyone who's developing here would keep in mind while they're doing that i mean just shifting gears a bit towards where we are in the world that we live in right now which is a world with pandemic and it has brought focus to health and public health in in a way that it did not earlier right so i'm curious to learn your thoughts on the role of ai in public health or just health broadly and what can we do responsibly in this space right because with health and the opportunities that we provide through it it's just imperative that we do it responsibly so i'm curious to learn one is your team doing any of this work and two uh how do you think ai will have a role to play in this not just right now in the pandemic world that we are in but also in the future yeah so i think ai and in particular responsible ai has a large role to play in public health because what you want to make sure is that there's health equity across the world and that people are receiving the very best care that they can at the lowest cost and that you're not depending too much on highly trained specialists which are few in numbers to achieve those results so for example some of the work that we're doing in diabetic retinopathy which is the fastest growing disease that is preventable in terms of vision issues it can lead to blindness and diabetes is really growing across the world and it relies on very trained you know highly educated specialists that are rare in the world to to be able to diagnose it by looking at a picture and an image of a person's of retina and so we're using ai you know models that have been trained on many of these images to look you know to to compare to have a technician compare the image that he or she has received from someone and then if it looks suspicious to then refer them to a specialist so you don't have to have that specialist you know right in the loop immediately but it can come later you can also have and we have been working on being able to diagnose tb from looking at chess x-rays and then as you may be aware of nythia's work in training mothers and coaching mothers in terms of prenatal as well as postnatal care for their infants as well as for themselves um nithia has done amazing work in in in india in in this area in general so um i'm sure you're familiar with that um you have to be careful with health care you know because you health care is probably one of the biggest divides that we have in in the world in terms of who gets it and who doesn't and who receives the best care and who doesn't and you know say if it's based on a race or or skin color sometimes you can look people have found you can look at x-rays and be able to detect that and that may not be the best thing in the world right so you want to take out any variable that would cause someone to mistreat a person whether it's intentional or unintentional based on these dimensions of i would say out group kind of marginalization and we hope through the responsible ai work that we do that we uncover many of those things and make sure that people are as blind as possible to to those inequities and that well i shouldn't say to the inequities but to the markers that make things inequitable so that it doesn't interfere with their ability to provide help for people that's super impressive and we love the focus on equity throughout and making sure to be aware of these markers of marginalization that's just incredibly important uh i know we're just coming on time but we have a few student questions for you because a lot of our viewers are students so one of them is and this is probably a student who has hit a wall with their research so what they're asking is what do researchers usually do when they run out of ideas how do you find inspiration for new research ideas and especially when the idea that you were working on that is not giving results which are competitive so do you have any advice for the student yeah so what i would say is just look around you you know look around you and see where things are broken and not working well where the need is and that alone should inspire you you know unfortunately we live in a world that is just you know it sometimes you can look at it like um in a place right now where it's a beautiful sunny day and it looks all fine but there are so many issues in the world you know and sometimes you have to go outside of your neighborhood or just talk to someone else who's you know not maybe in at the same fortunate position that you may be in and understand what their struggles are and let that inspire you to try to work in a way that could actually address those needs so as long as you're you know motivated to change the world and and and provide benefit to the world you're not going to run out of ideas you're not that's an incredible thought to also end this uh chat on thank you so much for sharing your wisdom all of your experiences and telling us so much about the work uh maybe like one final parting question so uh when people are of course going to be very excited to you know think through careers at your team uh is there a way that they could think about it or any backgrounds that you might be specifically looking for if people want to apply to your team sure so you know of course um we're looking for people who are familiar and and okay with technology um [Music] so that they can work with the engineers at google to improve their designs and their models but at the same time we're also we also have people who are human rights specialists who are social psychologists who are anthropologists who are sociologists who you know blend in with the team so that they can provide the input that i'm talking about in understanding the human perspective so it's a wide range of specialties that we're looking for um and if you do know of you know students who would be interested let me know um i should also say that in in my work in access i was able to visit india quite a number of times and it was it was amazing it was amazing and um i was part of the team that brought wifi to the to the railroad stations and that was like an incredibly exciting experience awesome thank you for sharing and we can't wait to host you back in the region all right this is uh the vita from google research india signing off with marian thank you marian for taking this time and sharing all of your wisdom with us thank you it's been a pleasure thank you so much bye everyone thank you so much dr marion and mr divi for the inspiring sharing and now here comes a video i was working on my phd thesis and this is right in the 90s and at a time when ai was mostly a academic curiosity i never thought that you know one day i'd have a chance to come back to vietnam and file an ai research lab right here [Music] the creation of vinaya research funded by wing group which is the largest company in vietnam is itself a very intriguing proposition [Music] and right from the start i want to be able to create a world-class research lab in vietnam and i want to focus on research excellence and also commercialization of the best research ideas and basically we want to challenge the presumption that such top institutions can only be found in the first world [Music] i think doing top-notch scientific research in vietnam is like a promised land that hasn't been much explored it needs brave pioneers and financial backing to make it happen when you come here you know you work with the best minds the best tools and the best infrastructure [Music] i enjoy the discussion with our residents the brightest student in the country i also enjoy the interaction with the talented engineer and scientist the discussion and interaction have always lead to many interesting directions and ideas from the problems i'm working on [Music] one of the great things that being a resource that i cannot do at other research labs is that aside from doing research i can make jobs and write problems in vietnamese on some days i focus on programming on other days i focus on paper reading writing or project planning still i spend few hours to mentor my residents so that i can keep check the progress and provide them with advices if needed back then when i was in my university i had never actually changed the gypno network but after joining vinaya i became one of the most active users of tessera and gpu's management platform however i kept working very hard and things got better so i submitted a paper to icml 2020 and my paper was accepted and it was i was very happy because it was one of my biggest achievements ever delivering world-class ai products making vietnam is an inspiring and meaningful mission that drive our team's commitment together regardless of what role they play mine ai is a mix of four to five startups sharing the same mindset who dare to face the more challenging problems in this industry with disruptive approaches the more than a decade away from home the biggest lesson i've learned is living in a first world country will not give me the sufficient perspective or insight to solve vietnam problems nor help grow the next part of vietnamese scientists and engineers so here i am in vietnam at dnai to help build more great products our current project combined state of the art from computer vision energy space and machine learning is run across industry from automated healthcare services and consumer electronics we will continue to explore the challenging and promising application for ai such as ai and iot with advanced technology like ai big data and our top talent resource and to develop the top quality products not only for wing group and vietnam but also for global market so humans are always the chief architect in any major changes in our society and the ai revolution in vietnam will be the work of many world well-chained engineers and research scientists and at vinai one of the goal is to create a high-quality ai workforce for vietnam and we're talking about people who can actually work on difficult research solutions for key societal problems and then turn those solutions into useful products and this will go a long way towards realizing our vision of being a world leading research institute advancing the state-of-the-art research and also isolating the creation and adoption of innovative ai applications to improve the quality of life of people in vietnam [Music] i love cycling to work early in the morning hanoi streets before the rush hours give me a small zen moment to clear my mind cutting through the wind always gives me a lot of energy to start the day i like to come to the office half an hour earlier to have enough time for my favorite coffee [Music] our team is a very young and motivated group not only are they the brightest minds amongst the peers in their field they're also multi-talented [Music] one of the biggest challenges for us is that every single product that we're building is so new to the market very few other companies are even attempting the same things that we do and this is a completely new frontier for all of us there'll be nobody there to teach you how to do it i'll walk you through it step by step therefore an innovative mindset and courage to face difficulties heads on is a must-have for each member of vnai's applied team [Music] i spend a lot of time during the day working with both sides of the code research and application sometimes we need to know what kind of experiments and modifications a research scientist comes up with and give him or her the inputs from an applied perspective other times i act as the bridge from the engineers delivering our internal cure results and customer feedback to the core team it's all about communication qa test report and customer support feedback are great but also can keep you up all night sometimes we share the enormous motivation and pressure of making many brand new products at the same time achieve world-class quality but need to be done in a very short amount of time what we're doing is challenging but also very inspiring i feel grateful seeing our product making great impact and appreciate every moment that our team share together making air products especially novel products on the market is a thorny and endless journey with advances in ai technology and data along with customer demands you will never be satisfied with what you can do today you dare to challenge yourself and challenge the world come to join us be part of the extraordinary this is a true race [Music] [Music] so ladies and gentlemen it is irrefutable that ai is no longer something new something superior that not everybody can access and understand we all admit that ai has not been increasingly embedded into our daily life starting from several high-tech products first but is that all we can talk about ai applications to get us closer into trends in ai innovations and what makes remarkable humankind have achieved achieved so far in application field you are invited to follow our discussion regarding ai for innovations and vietnam and global ai products this panel is moderated by ms henwin senior product manager at adap along with the presences of four panelists including mr alexander feiner head of autopilot software at vinaya research miss mido product product manager at waymo dr long fun chief technology officer at knowledge ai and dr win-win founder ceo of trusting social hopefully the organizer will receive a lot of interaction questions with us and now please welcome all the panelists and moderators thank you so much for the great introduction hi everyone hope everybody is doing very well and we are the last panel before your lunch break so uh my name is henwin and i am the senior product manager at adobe i am very honored to lead this panel today and uh you have seen for sure in the past one and a half days um you know you have seen and heard many of the different you know latest cutting-edge technology in ai however i'm sure everybody knows that the technology itself doesn't guarantee a successful product in real life so we are here in this panel to really uh tackle the challenge of launching an iei product and what other lessons learned that we can learn from the real ai practitioner product managers and entrepreneurs in the real world not only in vietnam but also in the global stage so it is my very honor to introduce again our four distinguished guest speakers um uh and each of them will have a few minutes to introduce themselves um and then we will go dive right in in the panel so without further ado let's welcome everyone into the panel about ai product and innovation so first of all let me introduce um lady first um my dog can you please tell us a little bit about your background and experience hi everyone uh this is my here calling from us uh in texas right now um i'm a product manager right now currently in waymo before that is with twitter and aws have a quick spring in apple when i went to singapore has always been a product manager engineering my training but get attracted to product management as soon as i finish my master and very happy to be here uh and being part of sharing my experience about ai to everyone thank you my so much for your introduction and next up we have alexander finder can you please speak hi sonny my name is alex yeah i'm i'm a computer scientist as a background studying at the university of bremen um afterwards i was working in a group of reliable embedded systems for a couple of years and then i switched to autonomous driving so since more than eight years now i'm working in that area uh in different stations on my career path uh starting with mercedes as a software architect and uh later also as a product owner um in in level three and level four of automated driving and then i also switched to tier one was working there couple of years for adaptive leading teams doing level zero to level four ad feature development and production programs and yeah since uh half and a year i'm happy to to work with vinnie i i'm here i'm leading the team for ada sad software development and so far it's a very exciting and definitely also different ada's experience what i've seen so far thanks thanks alex we can't wait to hear your stories and experience um next i would like to introduce dr long fun hello everyone my name is long fan i'm the cto of a company called knowledge ai and currently i'm calling from boston massachusetts so um i've been in the field of robotics and autonomous systems now for well over two decades uh in the 90s when i went to mit i developed some of the very first autonomous helicopter systems in the world we made the very first one back in 1996. i also led development through my master's thesis and early lidar technology then i went to work in finance for 10 years where in finance i developed one of the first high-frequency trading systems on wall street and then i went back to mit i got my phd in mechanical engineering in the field of infrared imaging so kind of citywide infrared imaging and then in 2014 i started pioneering efforts in hybrid electric aviation most recently last year i started my new job as cto of a company called knowledge ai and basically we're developing one of the first hybrid learning platforms in the world so students can learn both uh at home and in school and leverage ai thank you so much and long and next may i introduce dr wynwyn hey um i'm the founder and ceo of tracing social we are a ai company in fintech what we do is we partner with the data owners like tickle marketers e-commerce mobile wallets in emerging markets to make sense of their data in order to provide credit score to consumers we are now providing credit score to about 1 billion consumers in india indonesia vietnam and the philippines i have been in this journey for last 15 years starting from my phd to six years in big data in banking in new york and then started tracing social intolerance a team and mission that tracings also has is to leverage big data and ai to bring financial inclusion to everybody in the world thank you so much and um as you can see in this panel we have a group of very diverse background and experience and i can't wait to explore their what their uh what they have to bring into um and bringing their own very unique perspective from different industries different verticals and also as you you might know that they come from various uh background also experiment have experience in research and then entrepreneurship and then also um leading you know big tech company in the world and so let's dive right in our panel discussion today the very first question i like to ask the panel and i know this is very related to our audience over here is what is the fundamental difference between ai research and ai product so can you tell a little bit of the similarities or end differences um may interview um introduce and along first do you want to give us your perspective okay um my perspective is uh coming from a very uh research focused university like mit and then also coming from industries like finance or or robotics industry so um academically when we're thinking about um research and ai i think that the the topic typically is is very wide and very open to discovery um it's not as business driven and focused as we typically have to be in industry and the primary drivers around that is really uh resources and time management right so in industry you may be running um you know a team of 20 30 50 or hundreds of people or thousands of people and universities you know you typically do research in smaller groups you know maybe three four five people would be a big group already at the arena university level and also the time difference right at the universities we have to write grants it may take a year or two before the project gets started in the real world your bosses always say okay deliver this to me tomorrow you know it's like you're looking at very different time frames and so on the business side we have to be very focused when we're looking at using ai we have to be okay how does ai solve our business problem very quickly and how do we you know it has to be very narrow it has to be very defined in research okay you know let's explore more you know what can ai do for the world how can it help you know feed people how can it help develop new technology we have more time to think about that so that's my perspective very very uh thoughtful thank you so much for your opinion and and um um for other uh guest speakers feel free to share your thoughts as well what is the difference between ai research and product sure so in my opinion if we look at the difference for what is research what is what is applied ai or what is a product is in research you have to be fast also yeah you have to be fast and presenting new algorithms new papers publishing them on the top conferences and and so on so you you also have uh often infrastructure that is a super computer like we have for instance also the super part yeah and then for videoi you have you know i love have a lot of performance yeah hardware existing where you can do your research maybe not all universities but in general um you have that but if you go into the product yeah and you go into embedded yeah then you have a drop-off uh performance yeah and in your smartphone in your ecu on the car yeah so when you need to cope with that yeah what you have there yeah so and the major thing there is make it scalable and robust yeah if you have a product and you you want to achieve a big market many users then that product should work right so you cannot just like give something to a user and after two three attempts uh trying to play around with it it's not working anymore so i give you an example so i have a car where i have an ada system and um it's has so many false positives that i just switch it off because i say like okay this is nothing i want to use so and the same may happen with any other product you you develop an app yeah with maybe whatsoever technology behind it can be ai or whatsoever but if it's not working your rating will be bad and no one will use it so um making it robust making it efficient to run and embed it are things that that come in into into product development yeah so that's like kind of fundamental distinction in my eyes also and the research being fast publishing things fast bringing it to a patent are things that that you will have to see there or need to need to be ready for yeah yeah and so you know a lot of people coming and then ask a question you know there are so much you know innovation and technologies out there in ai and you can see they're you know it just didn't every day but bringing the innovation into the real world require us to really define the right problem to tackle the right problem for ai products to tackle so in your mind how do we go about and define the right problem for ai what is the value of ai when it comes to solving the real human being's problems here um uh i would love to hear from from my um what is your perspective yeah thank you um maybe you just go straight to some examples um the i think that the um the thoughts of defining ai and whether it's the right fit uh like the right technology for a product comes with a lot of trial and error um one examples in front end um like when i first started with um um amazon um amazon back then as the at least for the supply chain or how part of the of amazon is it's a very hard place to apply ai um although we all think that ai can be like help with automations and making people more efficient but the the requirement in terms of safety accuracy is almost like 99.99 requirement is very hard to achieve back then and the uncertainty in ai is something that cannot be like understand by system engineering industrial engineering and simply understandable so as amazon has other area of amazon can adopt air very quickly but defining the ai product and that part of the voice was pretty challenging um when we came to twitter uh surprisingly to many of you twitter has recently uploaded and i'm halfway relatively new uh ed has always been with ai but recommended system is pretty new and deep learning in particular is less than four years old in twitter um so when i was there like the the fact of us like picking the right problem to choose with ai's is is the real challenge um one example is that i don't know whether that's right or wrong but we used to come with a problem of people we used to be very aggressive um in pre-blind each other so um so then you know the product team come together and we have a very wild idea that patience we can look at what people are composing as they composing their reply can we just like detect uh early detect their their mode and nudge them toward a battery life it turns out to be a pretty hard problem and the the mistake that we met was that although we have tuned the model to be super accurate like very conservative you we can we're going to only like not you if you do something very bad um but because of the quality of data and a lot of new training during the black lives matter movement we we the models only associate black lives matter or the blm carb hashtag um with something that they need it needs to be flagged soccer is come up with a backfire for twitter right the the intention of an ai product was like good in purpose but then the implementations um can give like big black plus to people um when it's come to way more again the online twitter and amazon the product was built without ai and then we're trying to use ai to make more efficient replacement automation uh waste mobile build with an ai first approach um then the the challenge or different that we have is totally different but i think that in general what i'm trying to say is that there's so many different paths to define an ai and instead of spending too much time to think about what is the right way i would encourage on the entrepreneurship and startup to just do it a big company small company everyone make mistake and the more that we can foster or share those mistakes very open within the community and learn from each other um the better we can make ai to own the product well thank you my and let's uh also listen to uh uh advice from a startup uh startup startups powder so and and when i'm wondering in trusting social how do you go about and define the problem to solve for um you know the finance industry sure um for ai business and ai product right the purpose of it is actually to solve a problem for consumer you know it can be enterprise consumer or individual consumers but we have to solve it at the right price and an acceptable timeline so being a ai company in an emerging market actually constrain us a loss because we provide pretty much the same quality of service for example in facial recognition at a fraction of a price compared to a u.s market we have to be very careful on what we should invest in and what we should not invest in and actually even though the price that marketplace for ai is hugely different the cost of ai production or development is about the same across the world so it's double hard for emerging markets to to to bring ai to to life right so our framework is basically to value evaluate an idea in two dimensions number one what is the delta or the marginal improvement that ai can do compared to existing solutions mostly by human and the second dimension is how frequent is this problem right and being in emerging markets while willing is to buy an ability to pay solo and targeting the undermined consumers that will need some ability to place even lower we have to choose a very very big problem for example we choose to provide credit score to 1 billion consumers and less than 100 million it actually start not making sense from the business standpoint so for us ai product is to solve the problem within a price and time constraint and the better the leverage the ai provides on top of existing solution the better and different from from my who comes from a very big companies right resources in emerging markets is extremely uh scarce when it comes to ai talents so picking the right problem is probably very very critical for all startups like us so now we have about 200 people probably 100 people with phd in muscle but that's still very very small compared to waymo or google or facebook so evaluating the delta and evaluating the the scalability of the problem is crucial for us right um i i really like what your your share you want to share about like there are so many problems out there but pick the problem that you and only you want to sew or the best feature so not every company can attack everything so we need to be very selective there and very thoughtful there um and long i know that when you know when it comes to ai you also have very very great framework to define because you also you know moving from industry to another industry and kind of like being a fresh eye fresh eye in you know a new industry so how do you go about to explore new domain and find the problem that you want to solve with ai technology can you share from your experience um yeah this is uh uh it's strange um i've had this uh very fortunate opportunity to go from different industries from robotics uh to finance to aviation and now to education right so that's that's quite a kind of big jump each time um so the way i look at it really is every time um every time i do something i build a set of tools that goes into my toolbox that i learned whether it's engineering tools mathematical tools uh finance tool financial modeling uh you know how to build motors you know like different different tools goes into our toolbox and then when i go into a new industry my toolbox comes with me and i always think okay how can i take my toolbox everything i've learned and then learn something new because you always have to learn something new and you kind of put it together and sort of the usually the the top feedback i get is i am the out of the box thinker i'm the out of the box thinker because when i go there and i have my toolbox i'm saying hey we can apply this tool to solve this problem you can apply something else to solve this and the people that have been there they're like they keep using those same tools right and then also you come in with your toolbox and you're like wow you know look at all these other tools have you thought about using this equation here because i think this might solve it and then all of a sudden like you do that and people are like wow that's incredible you're so smart but you're not it's just like you did something somewhere else and then you applied it here and then people think you're really smart but really what you are is experienced um so i would approach ai similarly right there are many ways to solve a problem the question is what is the best most efficient way uh like dr ewen just said that it's talent is very scarce you know and so when you do have access to talent talent gives you possibly possibly a different way to solve a problem but but it may not be the best way it may not be the most economic way it may not be the most efficient way but it's a way where ai is very powerful is when it can solve a problem in an efficient economic and timely way right and then so you apply your toolbox and every time you go you bring your toolbox you bring your toolbox with you and from that perspective you will continue to learn and build your toolbox in my opinion until the day you die lifelong learner right um alex do you want to add anything for this topic i think it's not too much about define the right problem for my product i say especially i feel like it's more about the right team also to solve them problems you're facing yeah but in general problems are out there enough yeah so if you just like go out there in the world and open your eyes you'll find enough problems where you say like oh hi pencil uh what would be a product to solve that um and you can just like tackle many of them with the eye and can have very fast solutions yeah but um it's often also not just inventing new things yeah it's also about um how how do i reach the right market yeah so how do i reach the right customers yeah or a bigger customer um market um how do i make things faster more usable or sometimes even just cheaper and this is i think something that we we have to think about as well if you think about how to define the right problem for an ai product it's also about how to make things maybe better than existing so far yeah so there is not just always markets for just one solution out there the other markets for multiple solutions and you just need to find find a way how to distinguish yeah from the other state yeah and there may be many many factors playing a role so other than that i think the colleagues said already a lot about um how to how to define the right problem yeah right and so let's let's move into the next part of after define a problem let's say we we choose a problem to solve to tackle with ai but the messy middle is when you're actually working on the problem the solution and launching it so throughout your i don't know collective almost 60 years of experience in this group can you tell a little bit about you know what other challenges that are the top challenges that you have seen when it comes to bringing an ai product from lab into the real world what are the top challenges um from your view who want to go we're going to go first who have a burning problems to you to share with us i can go first yeah for the banking industry and for tracing social even after you produce a credit score that is so powerful that basically is better than any solution out there it typically takes us about two to three and in some market four years to convince the banks that even though the back test results show this you should actually use us which turns out to be a huge challenge because in order to use the ai solution where if previously hundreds of underwriters you know shipping through um the pile of application forms and supporting documents it's a it's a disruption to their business existing business and it's a threat to a lot of people in that organization so um so that is actually the biggest challenge that we have once we are ready to launch in the market right um another challenge that we have is that because we so today about two to three million people get loans every month based on increasing social credit score so if we do something wrong maybe half a million will not get loans right so it's such a mission-critical product we have to guarantee a almost 100 percent accuracy all the time based on a very flaky data input and i think a lot of companies face the same problem but you're required to provide very high level of consistency uh in underlying terrible data injection so so a lot of technology has to be developed a lot of lessons learned from that in order to basically push the industry uh forward um and then i think for us once we convince the bank it's actually much easier to convince the the consumers consumers what they all see is that now i have to do less but i receive better results then the benefit for consumer is obvious but the benefit for enterprises that work with us is actually not that simple yeah yeah totally related to that so my work at adobe is also we work with enterprise and speaking to enterprise you are dealing with um so many different stakeholders um and everyone has different goals and agenda and even convinced you know one of them can be a blocker you know fail to convince to them it can be a blocker or can be adding risk to the project um so really can can relate to that as well um so in your in your experience uh alex i know that you have experience working in germany and then in the u.s market and then also now in vietnam what are the challenges of working in different geo and especially in your industry automobile in autonomous vehicle the challenges of working in different environments is for sure different cultures but the challenges for for bringing an ai product yeah to life in our industry in automotive industry is um it's really about do we have enough data did we do enough testing yeah how do i know that i can already release really um into the market and launch something that is autonomous driving yeah so the first the first uh oems now bringing products onto the market yeah so with the level three system level four systems out there testing for long and also rainbow for instance doing now with customers yeah but the small number of of customers who can use it right now yeah so but but there's always the question did i really validate and test enough and and i faced these problems going from one market to the other one yeah so i'm in vietnam i have a completely different traffic environment right so i have many many bikes i have traffic participants acting uh completely different than you may be used to in the u.s or in the in europe yeah so people just ignoring zebra crossings yeah or sometimes even even traffic lights yeah so um that's kind of completely different way of driving and traffic flow you have in in the different regions and markets and you need to have your product yeah if you're going like worldwide with your products you have to expose it as you have to test it on these uh different markets yeah and make it robust working there yeah so and the challenge there is really to to to get enough data in the validation yeah so if you're talking about about the level three automated driving we're talking about 10 to the power of minus seven of hours being fault free yeah and if you're going level four we're even talking about 10 to the power of minus nine so it's it's a huge challenge to to achieve that level and to find a way how to do so yeah by by actually defining also what are my corner cases what are my edge cases i can do millions of kilometers of driving on on free roads but what did i gain from that not much so i need to find these corner cases which maybe maybe only occur every thousands of kilometers and if i'm able to prove i'm working in these scenarios and these edge cases yeah then i also reduce the mileage maybe that i need to drive yeah so um and and another big challenge is in our industry is to to uh the coast of failure is really high yeah if you fail with a level three level four system your system yeah will really harm human beings right so that's the reason you have all these safety measures and safety analyzes to do and so on so um so that's really hard i think in the automotive industry to bring something from from an idea into a product because you have to go through these so through these processes and unfortunately right now still at least what i see is that the simulation is is not not still good enough in comparison to the real world yeah so there's still a little bit like the gap to close between simulation simulated data and real-world data such that you can reduce also more the mileage you need to do on public roads so yeah these are i think the biggest challenges we are facing in the automotive industry there are different markets with different way of behavior of traffic participants and then finding these corner cases and and tackling them in a way that that you make your your product safe yeah and reliable and robust such that you can hand over to the customers all right right thank you for sharing that that's and and by the way i've seen the uh um announcement yesterday that we're launching soon in vietnam the very first ever um auto um autonomous driving system so congratulate on the first milestone of the team on that and later on would love to hear how did you overcome those challenges um um but i would like to you know continue the conversation and kind of hear from you know my and and and and long fine how do you you know experience these challenges in your specific industry yeah thanks all for uh um i think maybe it's good from um the product manager to at least be aware or have a good systematic framework to detect the problems and so far like my the personal framework that i developed um for me to think about it is basically is thinking about product and ai as an onion and you have the most and outer part of the problems is social political culture um like what kind of courses regulations opinions that you're going to get so like self-driving car extremely hard um internal like at least in the us you might have a better chance in china um at something like uphill battle right now uh but maybe some other technology is not facing the same thing so that's the most likely understood and the pm need to be able to check that box or you see that there's a huge problem then what would your pr what would your product marketing work with your engineer to manage that upfront right and build that images of company um the second the next layer is um if you know many speakers touch here it's not coming to integrate the demand um is the is the customer ready is the market ready um and and even though maybe there's no social pressure there's no culture but then because the market is not ready um you need to spend a lot of effort to convince and another thing that a pm need to worry about and um if you can find a very niche market that there's a demand you can convince people then let's check another box so let's assume that you're landing a product you check the first two blocks there's no regulation cultural pressure and to overcome uh the market is from i'm ready the next thing is like within your team within your company so i think i like to explain a little bit about data infrastructure like um women talk about talents the the hard part i think that a lot of people don't understand is that uh one single talent in ml required five engineers to support him um right from data engineering if you're asking a phd to try to clear their own data create an infrastructure it's just going to get crazy they say maybe like five minutes doing their actual work and the rest of the day building infrastructure and that happened with twitter um when i was there that we're trying to implement with deep craft learning and um it took almost like two years still haven't figured out infrastructure um so you know like thinking about data team infrastructure those are all the things that i think it's like a little bit beyond the scope of product manager but then you need to be able to partner with engineering team to understand their constraints and and help to escalate our school funding resource for those and um the last thing that i think is very important for anyone doing pm is that sometimes it's like us um this question is you need to control um one one thing that um in twitter that during the improvement of recommendation i recognize that we have a very unbalanced uh distribution of um power in twitter so the the pareto rules here right like 80 percent of the content from platform coming from 20 of users sometimes even like top 5 percent the question for product manager and you need to answer is is it okay is it the right thing to do or you want to come with like equity you want to be like more responsible you want to steal your product in a different direction uh so that question nobody's going to be able to answer maybe the the entire world will know your product as well as you um but the product manager in the ai need to understand that ai is beyond just technology sometimes it is the spirit or the care that you put into your product so i think that that's all the challenge that you know like from the this giant onion you walk all the way to the core and and every single step it's very ambiguous sometimes you don't know whether um it's it's the right or wrong answer um but so far one of the time whenever i have some very ambiguous problem my um the way that has been helping me is two way number one i'm trying to have a debate with the as diverse team as possible um even the team someone who have no understanding of ai within my team secondly i'm just imagine that if one day um the internet or some paper know about this decision and going to write a paper outfit would i be proud of that decision um so those as you know how we we should think about challenges and what is the the guiding principle to make those decisions as ai leaders very very great to hear that um yeah i i have heard from day one that i started pm is would you build a just that your grandma are going to use it are you going to build a product that your sister you know your family going to use it right so think about a value of the ai not just a business perspective but a human being how do they perceive that and um and think through how to tackle those problems with that in mind so this is this is very great um and now i would love to hear your thoughts as well um i guess uh just to answer the question directly i think one of the greatest challenge to uh bringing a ai product to life is really it's actually people in different forms and different you know end of the day it's kind of people whether it's political decision management decision customer decision whatever form are the barriers and challenges to bringing ai to life let me give some examples so i'm sure maybe some of you have seen boston dynamics um right actually it's only a few miles from where i am right now and i drove by yesterday and actually know the team at boston dynamic so really amazing robots right why don't you see their robots outside right now why why can't you see a boston dynamic robot walking around new york city or san francisco or vietnam or anywhere right why because people can't really truly accept that right now even though that robot can do parkour right a few days ago right wow i mean it can do flips and jump and even better than better than you know me and and and and people right why can't we have something like that because people can't really accept it right now um so that's one challenge um another challenge i think um coming on the development side i think a big challenge that i see often is uh this word why don't you just use ai to solve that for some reason in humans mind ai solves everything right it's like just you it's like a very open term very loosely you use ai and ai will solve it for you um ai isn't actually built by computers it's actually built by humans so humans have to build ai to solve problems for other humans and sometimes it's not possible to do that sometimes what they're asking to build an ai system to do something maybe that's not possible but but people think oh it's easy and and you know just a flip and magically it's done like flip a switch magically alex will be able to have an autonomous car that's driving around saigon dodging all the crazy traffic and all the all the crazy things and uh deal with all the edge case scenarios right just oh that's easy just do it it's actually not easy and i can tell you as a practitioner of robotics and autonomous systems for over two decades most of the time we're wrong most of the time we cannot achieve that sometimes we can after a lot of failures and learnt lessons and going back to the drawing board and redesigning and rethinking about how to do it and bringing in new tools and new thinking maybe if we're lucky we can solve that with ai but most of the time most projects i would say if i was to work on four a.i project probably only one will actually come to fruition the other three will probably not yield much positive results yeah i i i'm sorry so that lead me to uh the final but also i think the most important question of today is what is your practical advice or um learn lesson learn that you want to share with the audience in terms of you know overcoming all these challenges in your real world in your you know industry and so what are one or two things you want to share with us so that we can take home and kind of apply into our day-to-day job as an ai practitioner um so this one i want to open for everyone in the panel and feel free to share your perspective and one of the one or two advice so let me go first um and i'm gonna give my advice to um ai entrepreneurs uh i spent the first three years just bootstrapping between data equation uh modeling and product design and then value creation right the first data partner that we have uh give us just tiny bit of data right and we build model that actually never comes to market but we use that as a credential with a second one which is a you know have a bigger laser set and be a quite successful model than that when almost went live but but never see the the water advice because some you know unexpected change but then we use that for the third one which is the a telecom operator in asia and it turns out to be a very big success so the pain of um ai entrepreneur in collecting the data enough data to make a difference in the early days is huge right and and it's basically your talent your business savviness everything goes in there to make sure that you're going to have a defensive data set that allows you to push track once you you go live in the market people trust you then people open up uh data with you then the path will be slightly easier so focusing on bootstrapping to get enough data so that you can make a difference um that's for the early stage ai entrepreneurs right for like a little bit more mature stage what i learned in the last eight years interesting social is that it's best to sell a business opportunity versus selling a technical solution so for example uh our credit score right uh i will loan a thousand dollars a question cost like a dollar it is extremely hard to sell but if i go to the same bank and say that will help you to do to acquire unbind consumers and we get 50 of that would be much more happy to give us 500 instead of one right so selling something that brings immediate business value is much easier than selling a piece of a a puzzle so those are the two that i have you so much and thank you yeah uh who would like to continue this is this is great i think it's well said from from knowing and i also like what long said before about the toolbox or the tool suites yeah and for me it's kind of experience also yeah that's like what what you're talking there so in in our industry like having the experience what can go wrong yeah with within the vehicle with the sensor yeah so what are the capabilities of a sensor in high slopes in in in narrow curves or high curves yeah um what may go wrong in the electrical network all of that is kind of things the team should also learn yeah we learned that also now with the with the autonomous ride we did on on that beautiful island and entree yeah so so many things you're facing yeah from one day to the other you have different weather and you don't have collected data for that yet and so yeah it may have an impact on on the way how how the driving is maybe and all of that is like kind of learning your product knowing your product what can it really do and defining then also the operation domain design so what are really use cases we can handle and we can trust into that product or we can also release them too so and and this is where the team that is developing a product is also needs to learn this this product how to use it what may happen with it developing their tools that what what long step like having that experience in a team yeah so i think that's that's kind of of uh if something that we have as lessons learned uh for how to how to overcome challenges and an ai system is similar right it's it only knows what it has seen so far uh and we see now all these these fancy uh situations from tesla which is great but also the rest of the industry uh a moon being detected as a traffic light and whatsoever it's interesting but that are real music that may happen and you only can overcome many of them by gathering by acquiring the tyrants for the product yeah so that's really key also awesome yep yeah thanks long again um i think i just really want to react to what alex mentioned earlier about like the um like learning and failing um two things that i i want to bring up is building really building an ecosystem around ai and and don't be intimidated by technology so on the first point about building an ecosystem around ai now the pm is that a lot of times we think about a model as its entity and this is the only thing that we care about the fact is not true a lot of times when i ship a model to team representing machine learning team i'm asking the other product owner what is your requirement and the requirement come back from legal 99.99 percent position impossible you know and and i would like but then this doesn't mean that we cannot launch right china smoking but can we wrap our model into something else we have so many different system manual processes um alert right instead of hard blocking people creatively think about a way to wrap your solution in an ecosystem so that the ultimate um solution like we said right you deliver a solution another technology and as long as the solution work ai can just be a component you know don't over value it and and think about the ecosystem and in that ecosystem if you're fortunate enough of not working in self-driving car um the reason why i'm saying that is because terrorizing cars right now do not have a feedback loop at least i think that both winning big mo we we have yet to have like a very robust feedback loop but if you're working on an online product explicitly designed for feedback loop in your ai how you can your model increase bit by bit data by data because that's to win set right that is how you make improvement if your data model is just there and do not receive any feedback loop in the world cannot improve um it is it's not work you need to create that like snowball effect that you do nothing and suddenly one really gets much better so that is the kind of lighting ecosystem and the feedback loop and the next thing that i need i want to warn everyone is that we have just built a hype around industry that deep learning is key technology is the best the fact is not true my team has spent in twitter we spent almost i think a year six months using deep learning and trying to beat logistic regression that has been tuned very well kind of like new models barely beat the um the logistic regression system when we have like bird coming out we were like hey unless you brought them c like three times improvement model doesn't happen so so i would say that you know don't be intimidating saying that the most technology that you don't have access to the talent you don't have start with something simple and notice that right now i noticed that there's a train of ai that thing needs to be explainable um to create trust and the fact that your model is simple by decision tree also machine learning uh before you get into deep learning it is already valuable enough and maybe you can say that right hey our motto is explainable um and that's become a feature not an efficiency of products so that is my to take away that when you look at your product think about it at the ecosystem ask yourself have you get your model of feedback loop that you need and and if you someone put it in front of you an ai problem start with something the most simple one like start with a classifier a binary classifier a decision tree logistic regression don't jump into deep learning because then you don't have data and deep learning might not be able to beat electricity question real example show and now like um one said we're wrong many times uh i myself wrong every single time so it's good for pm to make a hypothesis don't be so confident um and try a way to prove yourself wrong the faster that prove yourself wrong great and long you are the first one to answer and i'd love to hear uh conclude these final questions yeah i have two advice um as a student of ai and even though i'm experienced i'm still a student i think my my first advice is that um everyone should continue to learn ai is not like it's not like algebra where it's fixed right i mean algebra is a very very well studied subject that taught for hundreds of years and there's only so many things you can do with algebra ai is something that is forever that requires skill sets that is beyond just taking a few classes beyond just getting a degree even a phd it is something that is evolving like an organic organism that will require nutrients and the nutrient is knowledge the more tools you bring to solving an ai problem the more likely the probability of you solving it increases that's my first advice so be a student continue to learn the second is i think this is very difficult for a lot of people particularly phds because i work with a lot of them most problems in this world most likely someone else has worked on it all right most problems i'd say like over 95 of the things the hardest things that someone can think of and you think oh this is so new so cool and i want to come up with a new way to solve that most likely if you just go on google and do some search someone's already thought about it someone's already worked on it and all you need to do is learn all right it can save you a lot of pain a lot of time a lot of money go and learn learn what they did learn the strong the strength the weaknesses where they failed they typically write that in research paper right we did this with these condition and it only works in these conditions okay learn read um and keep yourself educated constantly even the stuff they're doing at tesla most likely some phd student at stanford just did his phd on the latest thing that tesla did go search for that guy search for his phd thesis read it i know it's like okay i don't want to read it but most likely what you're trying to solve is probably somewhere there if you're willing to put the effort to find it so before you go charging off thinking that you can solve something you're this new cool guy and you're the only one that can solve that do your research and save yourself a lot of time yeah i i i would like to just just really conclude what what you just mentioned in in the piano i think ai products in order to be successful it takes a village it takes a team it takes time and also it takes a lot of learning in the process so start being very humble and start being very open-minded and curious um what are the best way what a better way to solve the current problem that we've seen and so with that i would like to uh conclude the um the first part and i know that we have only a few minutes left so let's go into the questions submitted on slido and trying to see if we can maybe answer a few of them um so could you please a technical team can you help show the questions all right and uh since we don't have a lot of time so maybe let's do one speaker for maybe one questions um so that we can um go into a few of them um so yeah let's go down here oh okay so um one of the questions i think from from ha to do can you highlight that one now we have seen so many benefits of ai applications in our lives so let's think the difference what are the cons and limitations of ai at this time so that was a very very broad question um i yeah uh let me share one perspective here it was a note that i i took uh in preparation so um i think i'll reframe this question the value you know how how you know where is ai valuable right and i have my own framework to look in this where is ai valuable in this world so um after many years i've kind of very generically summarized ai into doing two things first is ai can either augment or it can replace and in the case where it augments it makes something better it makes better decision it can help a person it can help a machine it makes that function feature functionality better and the second part of ai is it can replace i'm trying to replace this very tedious manual labor it's it's simple this guy is just sitting here doing a switch i'm going to replace that with a robotic ai system that can flip the switch automatically something like that so when you when you break up most of the problems you'll say okay is it here to augment or is it here to replace and then in each case how you approach ai will be different got it all right i think we can go to the next question um can you please help me to scroll down a little bit all right so um i don't know the name okay so there's another question about covet 19 pandemic has got some significant challenge for so that's one if i'm too in i hope i get your names right um the kobe 19 pandemic has caused significant challenge for supply chains globally can we solve this with ai and ml especially matching the supply to the demand so this is not a right or wrong answer but maybe some of you who have been maybe want to share your thoughts on this one the question is can we solve the problems with ai and ml especially matching the supply to the demand of the global intelligent business network and platform so i can give my experience here so during the pandemic especially in kochiming city tracing social offered a solution to match the contribution from donors to the poor and jobless workers in ho chi minh right who don't have any documents it doesn't require a lot of ai it does require some technology on identity verification and so on um but actually it is the same challenge as i mentioned earlier of how we can actually transform the the whole ecosystem to fit into a disruptive product right so in in case of consuming right now it is the military who deliver the food and and even the cash to to those who did have right we don't have to do it that way but it's just the way that we have done that for a long time so even when pressing social with you know the best intent and the best technology sometimes we just fail in doing so i think when we as as ai practitioners have to always think about how the society will react and adapt and therefore hopefully select our solutions it's just right to say yeah like i said i mean the the the pandemic itself is a very complex problem and and i think there's no one has one answer a perfect answer for that i think again this one require so much teamwork and collaborations among different people from different industries and maybe you know ai and non-ai together and so um with regard to the time i am very very um you know really enjoy listen to their uh conversation today and myself i learned a lot i definitely want to rewatch this session but um since we are running out of time i just want to spend the next you know um you know my gratitude to this panel and thank you so much for your time um working around the uh the whole different time zone here and i know and long this is almost 2am your time so for everyone thank you so much for spending your special time with us and thank you the audience to submit the questions and uh will we we really hope this panel is useful somewhat useful to your day-to-day practice as an ai research or even product or entrepreneur in vietnam or around the world um whoever interested in this topic please let us know and also please also participate in other finals going this afternoon thank you so much for your participation and again uh i wish you um all the panelists your very best health and uh great you know success in your career as well so thank you and look forward to you seeing your products in life you know very soon with that thank you very much bye-bye thank you everyone thank you thank you a lot for the interesting penalty discussion and thank you all the panelists and charming moderators especially all the high quality questions sent from the audiences all over the world and we do hope that the previous panel discussion has brought to you useful information and inspired those on the journey to build great ai products this also marks the ending of our day tuesday morning section and thank you for your attention we will come back at 1 30 pm vietnam time this afternoon with series of discussions and talks delivered in vietnamese and i know many young enthusiasts are waiting for please stay soon thank you and we do hope that to see you later [Music] so [Music] [Music] [Music] do [Music] you

Channel: VinAI Research

Views: 53,834

Rating: 4.6407185 out of 5

Keywords:

Id: 1IyAHDXY6xo

Length: 233min 17sec (13997 seconds)

Published: Fri Aug 27 2021