Skip to content

NMI/AMI return 0.0 instead of 1.0 for two single-cluster labelings #3469

Description

@AlejandroCoronadoN

Bug

normalized_mutual_info_score and adjusted_mutual_info_score return 0.0 for two identical single-cluster labelings, where scikit-learn (the documented reference for these metrics) returns 1.0, a perfect match.

Repro

import torch, numpy as np
import sklearn.metrics as skm
import torchmetrics.functional.clustering as tmc

a = np.array([0, 0, 0, 0]); ta = torch.tensor(a)
print(float(tmc.normalized_mutual_info_score(ta, ta)), skm.normalized_mutual_info_score(a, a))  # 0.0 vs 1.0
print(float(tmc.adjusted_mutual_info_score(ta, ta)), skm.adjusted_mutual_info_score(a, a))        # 0.0 vs 1.0

Why

Both labelings have zero entropy, so they trivially agree. scikit-learn treats this limit case as a perfect match and returns 1.0. torchmetrics falls through to the mutual_info == 0 short-circuit and returns 0.0. The asymmetric case (one single cluster, the other not) already returns 0.0 in both libraries and is correct.

Environment: torchmetrics 1.9.0 / current master.

I have a fix ready and will open a PR referencing this issue.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions