MLAlgorithms

Some prediction, classification and clustering algorithms on Python (without using Scikit-learn)

1. Gradient descent - one of the most popular optimization algorithms practically used in some regression and classification procedures. References:
   • https://www.coursera.org/learn/machine-learning - Week 1
   • https://ru.wikipedia.org/wiki/%D0%93%D1%80%D0%B0%D0%B4%D0%B8%D0%B5%D0%BD%D1%82%D0%BD%D1%8B%D0%B9_%D1%81%D0%BF%D1%83%D1%81%D0%BA
2. Linear regression - the simplest prediction model. References:
   • https://www.coursera.org/learn/machine-learning - Week 1
   • Bishop's Pattern Recognition and Machine Learning - Chapter 1.1
3. Maximum likelihood estimation - probabilistic approach for estimation the unknown parameters of the model. References:
   • https://en.wikipedia.org/wiki/Maximum_likelihood_estimation
4. Logistic regression - one of the most popular and simplest classification algorithms. References:
   • https://www.coursera.org/learn/machine-learning - Week 3
   • Bishop's Pattern Recognition and Machine Learning - Chapter 4.3
5. Naive Bayes Classifier - the classification algorithm often used in spam detection. It bases on finding bigger posterior probability from Bayes rule. References:
   • Bishop's Pattern Recognition and Machine Learning - Chapter 1.2 (Bayesian probabilities)
   • https://www.youtube.com/watch?v=qRJ3GKMOFrE - Andrew Ng lectures(Stanford)
6. Gaussian discriminant analysis - another classification approach based on posterior probabilities estimation. It uses the assumption of Gaussian distribution. References:
   • https://www.youtube.com/watch?v=qRJ3GKMOFrE - Andrew Ng lectures(Stanford)
   • Bishop's Pattern Recognition and Machine Learning - Chapter 2.1 (some notes about Gaussian distribution)
7. SVM. Kernels - the classification approach providing separating examples of different classes from estimating and optimizing margins between them. References:
   • https://www.coursera.org/learn/machine-learning - Week 7
   • Support Vector Machines Halldor Isak Gylfason Department of EECS University of California, Berkeley Berkeley, CA 94720-1770
   • CS 229, Autumn 2009 The Simplified SMO Algorithm - the algorithm for providing the conditional optimization in SVM
   • Bishop's Pattern Recognition and Machine Learning - Chapter 6(examples of kernels)
   • http://web.cs.iastate.edu/~honavar/smo-svm.pdf - SMO algorithm
8. K-means - one of the most known and simplest clustering algorithms. References:
   • https://en.wikipedia.org/wiki/K-means_clustering
   • Bishop's Pattern Recognition and Machine Learning - Chapter 9.1
9. PCA - the preprocessing procedure for converting a set of observations of possibly correlated variables into a set of values of linearly uncorrelated variables. References:
   • https://en.wikipedia.org/wiki/Principal_component_analysis
   • https://www.coursera.org/learn/machine-learning - Week 8
   • Bishop's Pattern Recognition and Machine Learning - Chapter 12
10. ICA - the algorithm for decomposition a multivariate signal into independent non-Gaussian signals. References:
    • https://en.wikipedia.org/wiki/Independent_component_analysis
    • https://en.wikipedia.org/wiki/FastICA
    • Independent Component Analysis􀀀 A Tutorial Aapo Hyv􀀀rinen and Erkki Oja Helsinki University of Technology Laboratory of Computer and Information Science P�O� Box ����� FIN������ Espoo� Finland