Skip to content

Add ROC-AUC score to compute metrics function in ISSR_AI4MH_Yixing_Fan/pipeline/test_model.py#47

Open
agentksimha wants to merge 7 commits intohumanai-foundation:mainfrom
agentksimha:added/roc-auc
Open

Add ROC-AUC score to compute metrics function in ISSR_AI4MH_Yixing_Fan/pipeline/test_model.py#47
agentksimha wants to merge 7 commits intohumanai-foundation:mainfrom
agentksimha:added/roc-auc

Conversation

@agentksimha
Copy link
Copy Markdown

Added roc_auc_score to compute_metrics using softmax probabilities (class 1) rather than hard predictions, which is required for a valid AUC calculation.

Justification:

  • if Dataset is likely class-imbalanced; AUC is more reliable than accuracy alone
  • Captures model performance across all decision thresholds, useful for tuning the classification cutoff in a safety-critical context
  • Aligns with standard evaluation practice for binary NLP classifiers

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants