Convex Optimization and SVM (Support Vector Machines)
Well, SVM is not a new name in the world of Machine Learning. From beginners to professionals, everyone is a fan of this classification module. Because of its extensive fitting modules and flexibility, it rules in the hearts of several machine learning enthusiasts and data scientists. But wait, did you know what happened inside?
Okay, I am going to show you how to CODE the SVM. There are plenty of videos and websites showing the complex mathematics behind the functioning of the SVM. It’s pretty complicated, right? Well, we will see about that…
Know the modules??
There are some important modules which are needed to be considered while modeling the SVM code. There can be many more modules. It depends on how creatively you write the code for yourself. The more creative you are, the more efficient and beautiful your implementation becomes. Here’s my way.
Here, I have implemented the ‘cvxopt’ for the ideation of the methods. cvxopt stands for ‘Convex Optimization’. It is one of the methods used for solving the Langrangian problems in SVM. It makes out job easier. The rest would be pretty obvious to you…
Let’s Define the CLASS
Okay, obtaining the features from the dataset is the most important part of modeling the SVM code. Everything in machine learning works on datasets and features. So taking the features and extracting the important features (means all the features and samples!) we can compute the mathematics for the SVM.
Well, SVM works on the principles of kernelization. There are some kernel methods defined within the class of SVM which makes it easier for the SVM to model itself to the closest possible fitting method from the perspective of a particular dataset. I think there is a requirement for some instances of mathematics to show why I’m doing this.
Some MATHEMATICS…
This is the standard form of quadratic programming (QP) which we would be implementing in the blog. The CVXOPT QP framework expects a problem of the above form by the parameters {P,q,G,h,A,b}; P and q are required, the others are optional. So, how to apply in terms of programming?
Now, since we have defined the program parameters, we have to solve for the generated equations and hence we need some sort of solver to solve for the equations. (Mechanical Joke: This is like ANSYS, you would be magically getting all the solutions to your problems!). Several functions work as solvers. A comprehensive list has been depicted in the cvxopt documentation. But for the ease of the tutorial, I would be implementing the simple ‘QP’(Quadratic Programming).
Choosing the boundaries and other parameters…
The solutions for the same would come in the form of a python dictionary with a lot of options to choose from based on the work we want to do. Since we are implementing the SVM, we would choose the ‘X’ from the dictionary. Now, using the NumPy ‘ravel’ function, I would be making a linear matrix of the solutions.
Biases and Weights…
Now, you must know the basic function used in the definition of the SVM. It should be pretty obvious by now. If you don’t remember, then please invest some time here. You won’t regret it, I promise.
Here, we have instantiated the weights and the biases for the entire classes to be used upon. The usage of biases and weights are prevalent in the SVM studies because we have to take care of the dataset and the noise in that otherwise, a backlash might come because of ‘underfitting’ or ‘overfitting’. Based on the parameters, we have defined the parameters and have updated them over a loop.
Predict FUNCTION
Now, the most important function. ‘predict’ function is defined for obtaining the classes in the ‘SVM’ and hence requires to reveal the sign of the function. Remember that we are implementing for a two-variable class and hence would be considering ‘+1’ and ‘-1’ for the classes only.
f FUNCTION
Not a slang! It’s a function of visualization. The visual perspective is also necessary for the success of SVM. Clearly, pointing it out using the algorithm…
Testing PHASE…
Now, everything has been done. Let’s just run the piece of software and see how we can perceive the classification algorithm.
Using the ‘make_blobs’ function, for preparing the model dataset. Since we are generating for the ‘+1’ and ‘-1’ classes, let’s prepare the model based on the same. Till now, we have defined the model for determining accuracy. Well, let’s visualize it.
These were the final steps for the implementation of SVM. Let’s check the results out.
Results
Well, I am not going to show it. Try it yourself and search for better answers. Who knows, maybe you’re the next data-science star. Maybe you’ll get a call from ‘Joma’ this very day? Cheers!
If you need the entire code, please visit this place.
