Courses – Fakultet tehničkih nauka u Novom Sadu

Subject: Machine Learning (17.SES203)

Native organizations units: Sub-department for Applied Computer Science and Informatics

Type of studies	Title
Undergraduate Academic Studies	Software Engineering and Information Technologies (Year: 4, Semester: Summer)

General information:

Category	Scientific-professional
Scientific or art field	Applied Computer Science and Informatics
ECTS	6

Mastering basic concepts, elements, and techniques of machine learning. Enabling the students to understand basic machine learning models, as well as their theoretical foundations. Allowing the students to recognize the type of machine learning problem in real-world practical situations, and to apply adequate algorithms.

Upon successful completion of the course the student knows: to define and differentiate between the fundamental problems in machine learning (regression, classification, clustering, and dimensionality reduction); which algorithms and models are applicable for solving each of the fundamental problems; the theory behind machine learning models; practical implications necessary to implement machine learning models; how to design a valid experiment for model evaluation and comparison; how to apply the obtained knowledge to real-world problems.

(1) Supervised learning: Linear regression (single and multi-variable, Gradient Descent and its variants, closed-form solution, the influence of outliers); Non-parametric approach (k-nearest neighbors, kernel regression); Maximum Likelihood; Classification (Logistic regression, Perceptron, Naive Bayes, Support Vector Machines, Ensemble learning). (2) Experimental design and model selection: Performance measures; Experimental procedures (Cross-Validation, train/validation/test split, model selection, and hyper-parameter optimization); Overfitting and regularization (Ridge and Lasso regression, Elastic Net). (3) Semi-Supervised Learning (overview of the basic concepts and algorithms). (4) Unsupervised learning: Clustering (k-means and Gaussian Mixture Models); Dimensionality reduction (Principal Component Analysis); Practical advice for applying machine learning algorithms; (6) Learning theory: Hoeffding's inequality; Vapnik–Chervonenkis dimension; Approximation-generalization tradeoff.

On the lectures, students are introduced to the basic concepts, machine learning algorithms, and their theoretical foundations. During the laboratory exercises, for each learning concept, the students are given a real-world problem to work on, and the possible solutions are discussed. After the laboratory exercises, students try to solve the given problem on their own, and, by applying the obtained knowledge, try to achieve maximal performance. Students are awarded points for solving the problems presented during the laboratory exercises. Additionally, students earn points by working on the class project. For the class project, the students propose the real-world problem from the field of machine learning they would like to work on, as well as the reasonable methodology for its tackling. The final exam (testing theoretical knowledge) is an oral exam.

Authors	Title	Year	Publisher	Language
Hastie, T., Tibshirani, R., Friedman, J.	The Elements of Statistical Learning : Data Mining, Inference, and Prediction	2017	Springer, New York	English
BISHOP, Christopher M.	Pattern Recognition and Machine Learning	2006	Springer, New York	English
Awad, M., Khanna, R.	Efficient Learning Machines	2015	Apress Media	English
Ian Witten, Eibe Frank, Mark Hall, Christopher Pal	Data Mining, 4th Edition	2017	Morgan Kaufmann	English
S. Shalev-Schwartz, S. Ben-David	Understanding Machine Learning: From Theory to Algorithms	2014	U elektronskom izdanju: https://www.cs.huji.ac.il/~shais/UnderstandingMachineLearning/understanding-machine-learning-theory-algorithms.pdf	English