This project focuses on predicting student academic performance using behavioral data and machine learning techniques. The goal is to apply the XGBoost algorithm to uncover patterns in student behavior and evaluate how well these factors can predict academic outcomes.
In this section, we load the dataset from Kaggle and perform initial data exploration. The two available tables are merged into a single dataset, after which we inspect the dimensions to better understand its structure. Finally, we clean the data by removing missing or empty values to ensure consistency and reliability for the modeling phase.
Here, we prepare the data for training and testing. The dataset is divided into input features (X) and the target variable (y). Then, we split the data into training (train_data) and testing (test_data) sets, allowing us to evaluate model performance on unseen data.
In this stage, we build the predictive model using scikit-learn and XGBoost. The model is trained by tuning key hyperparameters such as the number of estimators (n_estimators), learning rate (learning_rate), and tree depth (max_depth). This process helps optimize performance and improve prediction accuracy.
To assess the performance of the model, we compute several evaluation metrics, including Mean Squared Error (MSE), R (correlation coefficient), and Accuracy. The model achieved an R² score of ~0.93, indicating a strong ability to explain the variance in student performance. These metrics provide insights into how well the model predicts student performance and how closely predictions match actual values.
Lastly, we visualize the results to better interpret the model. We generate a table showing the importance of each feature, highlighting the most influential variables. Additionally, we create a plot comparing real values versus predicted values to visually assess model performance. We also represent the first decision tree generated by the model, which shows how XGBoost splits the data based on the most relevant features — in this case, hours_studied and assignments_completed — to make its initial predictions.