Geometric Deep Learning

deep learning on 3d objects finally hello world it's suraj and geometric deep learning is an emerging field of machine learning that's able to learn from complex types of data like graphs and 3d objects in this tutorial video we'll learn how to use geometric deep learning to classify groups with similar interests in a social network and apply it to 3d object classification as well in the last decade deep learning algorithms like convolutional neural networks and recurrent neural networks have allowed us to achieve unprecedented performance on a broad range of problems we can now classify images recognize speech translate languages generate pickup lines and some of these metrics have incredibly already surpassed human capability it's been incredible to see the progress that's happened but a lot of the algorithmic techniques that have been used to achieve these breakthroughs are actually pretty old what's been the real catalyst for the deep learning revolution so far has been the availability of new types of datasets that have been generated whether it be the Atari data set for deep minds deep queue learning algorithm or the chess game data set for IBM's deep blue victory also GPUs and Hinton's blessings the data set point is important because in the past few years we're seeing more and more of a special type of data set 3d objects thanks to companies that have democratized tools like the Kinect that are able to capture 3d point clouds of objects not just 2d images we now have a wide variety of 3d objects available to train models on once we do that we can classify them however we'd like or even generate new objects graphs are another type of emerging datasets we can consider social networks as graphs where the characteristics of users can be modeled as signals on its vertices the sensor networks are another example distributed interconnected sensors have readings that can be modeled as time dependent signals on graph vertices in genetics gene expression data is MA signals defined in the regulatory network graph models are also used to represent anatomical and functional structure in the brain you would think that we could just feed this type of data to a deep neural network and assume that it would be able to properly parse it right ah ah neural networks aren't that great at interpreting this type of data the reason being deep learning generally works well on what's called Euclidean data graphs and 3d objects those are considered non Euclidean data sets let's break down the difference Euclid was a Greek mathematician who wrote a book called the elements over 2,000 years ago where he outlined the geometric properties of objects that exist in flat two-dimensional planes that's why euclidean geometry is also known as plane geometry euclidean geometry is pretty straightforward pun intended lol it has its own set of rules like the interior angles of a triangle add up to 180 degrees and two parallel lines will never cross and the shortest distance between two points will always be a straight line Euclidean geometry is still practical and has been used by modern engineers to design buildings predict the location of moving objects not die and surveil and we can also use it to help us define data for example we can consider an image as a function over a two dimensional plane this function can help us define the image intensity at a specific coordinate on the 2d image plane and if we need the value of the image intensity at another pixel a certain number of units along the x direction we can just plug that new coordinate set into our function this property is very useful if we want to define a convolution convolutions are one of the main building blocks of convolutional neural networks the type of network as suited for image processing the term convolution refers to the mathematical combination of two functions to produce a third function it merges two sets of information in the case of a CN n the convolution is performed on the input data with the use of a filter to then produce a feature map we execute this convolution by sliding the filter over the input at every location a may it's multiplication is performed and sums the results on to the feature map it's one of several operations that make these networks work really well on image data but if we curve the image so it becomes a 3d object we can't just convolve around a 3d shape using vectors like we did for 3d images that means we need to redefine the whole notion of a convolution non-euclidean geometry encompasses this type of data whether it's a sphere a shapeless 3d mass like ditto and Pokemon or a graph of some kind a lot of the most interesting data types lie in this category from protein interaction networks to knowledge graphs to the entire world wide web the non Euclidean nature of this data implies that there are no familiar properties like a common system of coordinates vector space structure or shift invariance we need a neural architecture that can learn from non Euclidean data with the kind of accuracy that CNN's give us for Euclidian data enter the graph convolutional Network or GC n GCN's are very powerful models so powerful in fact that even a randomly initialize two layer GCN can produce useful feature representations of nodes in a given network we can think of a general graph based learning problem in the following way we're given a set of nodes each with an observed number of numeric attributes for each node we like to predict an output label we have labels for some nodes but not all nodes we're also given a set of weighted edges summarized by an adjacency matrix the main assumption is that when predicting the output for a know the attributes and connectivity of nearby nodes provide useful contextual information GCN's can solve this problem they are neural networks that can operate on graphs a good way to imagine what's happening is to consider a neural network that receives as input a set of features from all nodes in the local neighborhood around a node and outputs an estimate of the Associated label the information from the local neighborhood gets combined over the layers via the concept of graph convolutions the deeper the network the larger the local neighborhood we can think of it as the generalization of the receptive field of a neuron normal CNN this network is applied convolutional e across the entire graph always receiving features from the relevant neighborhood around each node let's talk more about how these graph convolutions are defined by applying a GCN to a real-world graph we have a social network data set here of football enthusiasts that are divided into two groups FC Barcelona fans and Real Madrid fans and this is represented as a graph Barcelona is the better team but that has nothing to do with this nodes represent members of the network and the edges are there at mutual relations the leader of both groups is denoted by a respective letter we can build our graph convolutional network by first initializing it at random to produce feature representations then we'll stack our GCM layers using the identity matrix as a feature representation so each node is represented as a one hot encoded categoric variable our GC n will take as input a feature matrix that includes a number of nodes and the number of input features for each node it also takes as input the matrix representation of the graph structure each hidden layer of the network corresponds to a feature matrix where each row is a feature representation of a node at each layer these features are aggregated to form the next layers features using a propagation rule so features become increasingly more abstract at each consecutive layer our simple propagation rule consists of the weight matrix for a layer and a nonlinear activation function riilu so the size of the second dimension of the weight matrix determines the number of features in the next layer this is reminiscence of the convolutional filtering operation from regular convolutional networks in that the weights are shared across nodes in the graph we can easily extract the feature representations from the graph and plot it in a few lines it cleanly separated both sides and we haven't even started the training process yet if we start training our output graph will get way more accurate and this demonstrates the power of GCS pretty dope right but social network graphs aren't the only type of Euclidean datasets we can also use GCN's on 3d objects if consider them as point cloud data point clouds are just a set of points represented in 3d by their XYZ locations it's what hardware like the Kinect generates it's computing not just pixel data but depth as well so there's a third dimension involved if we consider the points in a point cloud has nodes in a directed graph we can apply GCN's to them there has been several work in this space just this year a paper titled dynamic graph cnn's use a GCN to improve feature extraction from point cloud data there are several variations here Splatt net point net there are different types of graph convolutions and different pooling architectures designed to work well with point cloud data this space is moving fast so expect to see even more improvements in the next few months but we can already use several of the freely available models on github ourselves to build products and services there are three things to remember from this video deep learning generally does really well at extracting useful feature representations from euclidean datasets euclidean data follows the laws of Euclidean geometry its grid like and follows rules involving straight lines and points geometric deep learning models like graph convolutional networks help learn from non Euclidean data like graphs and 3d objects by introducing an ordering of mathematical operators that are different than in classical convolutional layers in this video give you an idea for a start-up share it with us in the comment section and please subscribe for more programming videos for now I've got a visit the fifth dimension so thanks for watching

Channel: Siraj Raval

Views: 71,993

Rating: 4.8147817 out of 5

Keywords: geometric deep learning, deep learning, neural network, convolutional network, social graph, graph data, 3D object classification, 3d Object, euclidean, geometry, math, mathematics, siraj raval, python, programming, classification, machine learning, artificial intelligence, coding, tutorial

Id: D3fnGG7cdjY

Length: 10min 25sec (625 seconds)

Published: Wed Dec 12 2018