How to Read a Research Paper

Hello world it's Siraj and I'm gonna show you how I read research papers and give you some additional tips on how you can consume them more efficiently reading research papers is an art whether the topic is machine learning or cryptography distributed consensus or networking in order to truly have an educated opinion on a particular topic in computer science you've got to get yourself acquainted with current research in that subfield it's easy to agree with a claim if it's got enough hype behind it but being critical and balanced in your assessment is a skill that can be learned PhD students are taught how to do this in grad school but you too can learn how to do this it just takes patience and practice and coffee lots of coffee every single week I read between 10 to 20 research papers in order to keep up with the field and I've gotten better at it over time and I don't have any graduate degrees I'm just a guy who really loves this stuff and I teach myself everything using our new collective University the Internet one of my favorite resources to find papers on machine learning is the machine learning subreddit people post papers they find interesting every day and they've also got this cool weekly what are you reading thread where people post the papers that interest them the most currently additionally there is this web app called archived sanity com created by Andrey Karpov II which basically goes through archive and finds the papers that are most relevant you can filter them by what interests you by which ones are most popular or by the ones that are most cited lately Google and deepmind respectively publish their work on their websites for easy access and there are of course journals like Nature that you can find some top papers in easily the pace of research is accelerating in machine learning because of a few reasons not including Smith you in academia and in the public sphere the democratization of data computing power education and algorithms is all steadily happening over the internet because of this more people are able to make their own insights into this field in the industry the big tech companies profit more when their own teams discover new machine learning methods so there's this race to create faster more intelligent algorithms all that is to say that there are a lot of papers you could be reading right now so how are you supposed to know what to read well what I found is that every week there are maybe two or three papers that are getting the most attention in machine learning and the tools I've mentioned helped me find them and read them but most of my reading is a result of me having a goal that goal could be to learn more about activation functions or perhaps probabilistic models that use attention mechanisms once I've got that goal it makes it much easier to create a reading strategy that points towards that goal just being a good math heavy machine learning paper reader is not a goal to aspire to your stamina is more of a function of human motivation which is a function of the goals you're trying to accomplish I found that I can crush through and understand the most difficult papers much more when I have a real reason to do so so let's take the landmark paper by a friend of mine Ian good fellow on generative adversarial networks as an example there is a lot in this paper he synthesizes some ideas here that made Yamla kun say that this concept was the coolest idea in deep learning in the last ten years the way I read papers is by performing a three pass approach on the first pass I'll just skim through the paper to get it just of it meaning I'll first read the title if the title sounds interesting and relevant generative adversarial networks yo let's go I'll read the abstract the abstract acts as a short standalone summary of the work of the paper that people can use as an overview if the abstract is compelling an adversarial process between two neural networks that were temples a game all right this is lit then I'll skim through the rest of the paper by that I mean I'll carefully read the introduction then read the section and subsection headings but ignore everything else mainly ignore the math I never read the math on the first pass I'll read the conclusion at the end and maybe glance over the references mentally ticking off the ones I've already read if there are any I just assume the math is correct on the first pass my goal for this first pass is to just be able to understand the aims of the author what are the papers main contributions here what problems does it attempt to solve is this a paper I'm actually interested in reading more of once I've done the first pass I'll go back to see what other people are saying about this paper and compare my initial observations to theirs basically the aim of this first pass is to ensure that it's worth my time to continue analyzing this paper live short and there are too many things to read if it does pique my interest then I'll reread it a second time on the second pass I'll read it again this time more critically and I'll also take notes as I go I'll actually read all the English text and I'll try to get a high level understanding of the math that's happening in the paper so it's a minimax game that looks to optimize a Nash equilibrium okay I kind of get that eventually the generator Network creates fake samples that are indistinguishable from the real thing so the discriminator is powerless cool I'll read the figure descriptions any plots and graphs that are available and try to understand the algorithm at a high level a lot of times the author will break down an equation by factoring it out I avoid trying to analyze this on the second pass I see that it's using a loss function called the kullback-leibler divergence never heard of that one but I do get the concept of minimizing a loss function when I read the experiments I'll try to evaluate the results are they repeatable are the findings well supported by evidence once I've done that hopefully there is some associated code with the repository available on github I'll download the code and start reading it myself I'll try to compile and run the code locally to replicate the results as well usually comments in the code help further my understanding I'll also look for any additional resources on the web that help further explain the text articles summaries tutorials usually a popular paper will have a breakdown that someone else has done online that will help drive the key points home for me after this second pass I'll have a Jupiter notebook full of notes and associated helper images since I teach this stuff on YouTube teaching is really the best way to fully understand any topic when it comes to the third pass it's all about the math my focus on the third pass is to really understand every detail of the math I might just use a pen and paper and break down the equations in the paper myself I'll use Wikipedia to help me understand any of the more formal math concepts fully alike the KL divergence and if I'm feeling really ambitious I'll try to replicate the paper programmatically using the hyper parameter settings and equations that it describes after all of this I'll feel confident enough to discuss it with other people greeting papers is not easy and nobody can read long manipulations of complicated equations fast the key is to never give up turn your frustrations into fuel to get better you will understand this paper you will master this subject you will become awesome at this it gets easier every time as you build your merkel dag of knowledge see what I did there if you don't get a math concept guess what Khan Academy will teach you anything you need to know for free and lastly do not hesitate to ask for help there are study groups and communities online that are centered around the latest research in machine learning that you can post your questions to don't be afraid to reach out to researchers as well you're actually doing them a favor by having them explain to you in terms you understand all scientists need more experience translating complex topics I've got lots of great links for you in the description and I hope you found this video useful if you want to learn more about machine learning AI and block technology hit the subscribe button and for now I've got to reread the capsule Network paper so thanks for watching

Channel: Siraj Raval

Views: 345,158

Keywords: How to Read a Research Paper, how to read research papers efficiently, how to read a paper, research papers, How to Read Research Papers, how to read a research paper computer science, how to read research articles efficiently, how to read a research article, how to read a research paper quickly, research paper, how to read journal articles efficiently, how to read a scientific paper, how to read journal articles, research, scientific article

Id: SHTOI0KtZnU

Length: 8min 44sec (524 seconds)

Published: Fri Jan 26 2018