Support Vector Machines Part 2: The Polynomial Kernel (Part 2 of 3)

a once knew a colonel its name was Fred the stat quest isn't about that Colonel stat quest hello I'm Josh stormer and welcome to stat quest today we're going to talk about support vector machines part two the polynomial kernel specifically we're going to talk about the polynomial kernels parameters and how the polynomial kernel calculates high-dimensional relationships note this stat quest assumes that you are already familiar with support vector machines if not check out the quest the link is in the description below in the stack quest on support vector machines we had a training data set based on drug dosages measured in a bunch of patients the red dots represented patients that were not cured and the green dots represented patients that were cured in other words the drug doesn't work if the dosage is too small or too large it only works when the dosage is just right because this training data set had so much overlap we were unable to find a satisfying support vector classifier to separate the patients that were cured from the patients that were not cured however when we gave each point a Y access coordinate by squaring the original dosage measurements we could draw a line that separated the two categories of patients so we used a support vector machine with a polynomial kernel to compute the relationships between the observations in a higher dimension and then found a good support vector classifier based on the high dimensional relationships the polynomial kernel that I used looks like this a and B refer to two different observations in the dataset are determines the coefficient of the polynomial and like I mentioned in the earlier stat quest D sets the degree of the polynomial in my example I set R equals 1/2 and D equals 2 since we are squaring the term we can expand it to be the product of two terms now we just do the multiplication b.p boo peepee poo-poo and combine these two terms and just because it will make things look better later let's flip the order of these two terms finally this polynomial is equal to this dot product a dot product sounds fancy but all it is is the first terms multiplied together plus the second terms multiplied together plus the third terms multiplied together the dot product gives us the high dimensional coordinates for the data the first terms are the x-axis coordinates and the second terms are the y-axis coordinates the third terms are z axis coordinates but since they are the same for both points we can ignore them thus we have X and y-axis coordinates for the data in the higher dimension BAM alternatively we could have set R equals 1 and D equals 2 now when we do the math we get this polynomial and this dot product we can verify that the dot product is correct by multiplying each term together and then add everything up and the result should be equal to the polynomial using this dot product the new x-axis coordinates are the square root of two times the original dosage values so we move the points on the x-axis over by a factor of the square root of two the new y-axis coordinates are the same as before the original dosage values squared and just like before we can ignore the Z access coordinate since it is a constant value now just like before we can use the high dimensional relationships to find a support vector classifier double bam now brace yourself things are about to get a little crazy going back to the polynomial kernel with R equals 1/2 and D equals 2 it turns out that all we need to do to calculate the high-dimensional relationships is calculate the dot products between each pair of points and since this kernel is equal to this dot product all we need to do is plug values into the kernel to get the high-dimensional relationships for example if we wanted to know the high-dimensional relationships between these two observations then we plug the dosages into the kernel do the math and 16000 2.25 is one of the two dimensional relationships that we need to solve for the support vector classifier even though we didn't actually transform the data to two dimensions triple bam fortunately why we only need to compute the dot product is out of the scope of this stat quest wah-wah to review the polynomial kernel computes relationships between pairs of observations a and B refer to the two observations that we want to calculate the high dimensional relationships for our determines the polynomials coefficient and D determines the degree of the polynomial note R and D are determined using cross-validation once we decide on values for R and D we just plug in the observations and do the math to get the high dimensional relationships hooray we've made it to the end of another exciting stat quest if you like this stat quest and want to see more please subscribe and if you want to support stack quest consider contributing to my patreon campaign becoming a channel member buying one or two of my original songs or a t-shirt or a hoodie or just donate the links are in the description below alright until next time quest on

Channel: StatQuest with Josh Starmer

Views: 149,054

Keywords: StatQuest, Josh Starmer, Machine Learning, Support Vector Machines, SVM, Polynomial Kernel, Kernel Trick, Statistics, Data Science

Id: Toet3EiSFcM

Length: 7min 15sec (435 seconds)

Published: Mon Nov 04 2019