Skip to content

Expand "Model Architecture and Training" section with implementation details #4

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
kjappelbaum merged 3 commits into from
Apr 12, 2026
Merged

Expand "Model Architecture and Training" section with implementation details #4

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
68de1bc
Initial plan
Copilot Apr 2, 2026
196ee5b
Expand model architecture and training section with missing details
Copilot Apr 2, 2026
f1a3238
Apply suggestion from @kjappelbaum
kjappelbaum Apr 12, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion src/tex/ms.tex
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -250,7 +250,11 @@ \subsection{Clever Hans Analysis Framework}

\subsection{Model Architecture and Training}

All models used gradient boosting, implemented with \texttt{LightGBM}\autocite{lightgbm} with default hyperparameters.
All models used gradient boosting, implemented with \texttt{LightGBM}\autocite{lightgbm}. All LightGBM hyperparameters were kept at their defaults. No hyperparameter search was performed; the goal was to compare model types under equivalent, untuned conditions rather than to maximise individual task performance.

@sourcery-ai sourcery-ai Bot Apr 12, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

question: Clarify what "equivalent, untuned conditions" means across different model roles.

Since LightGBM appears as base models, proxy sub-models, and the final indirect model, please specify whether the same default settings (e.g., learning rate, num_boost_round, early-stopping configuration) were used for all classifier and regressor instances, or note any systematic differences. Otherwise, readers may infer that some roles benefit from different implicit defaults or training behaviours.


The indirect (proxy) prediction pipeline comprises three sequential sub-models. First, a \texttt{MultiOutputClassifier} wrapping \texttt{LGBMClassifier} predicts binary author-presence features; second, an analogous \texttt{MultiOutputClassifier} predicts binary journal-presence features; and third, an \texttt{LGBMRegressor} predicts publication year. The outputs of all three sub-models are concatenated to form the meta-feature vector that the final indirect model uses to estimate the target property.

Model performance on regression tasks was evaluated using mean absolute error (MAE), mean absolute percentage error (MAPE), and the coefficient of determination ($R^2$). As a baseline, a \texttt{DummyRegressor} predicting the training-set mean was used.

@sourcery-ai sourcery-ai Bot Apr 12, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

issue: Address how MAPE is handled for zero or near-zero targets.

Please briefly state how zero or near-zero target values are handled when computing MAPE (e.g., exclusion, adding an epsilon to the denominator, or a masked/modified MAPE), so readers can correctly interpret the reported values.


\subsection{Cross-Validation Protocol}
I used 10-fold cross-validation with random shuffling unless otherwise mentioned. Individual dots in swarm plots indicate the performance on the individual folds.
Expand Down
Loading