Behavior for overlapping overload definitions

[glencoe]

Hi,

From what i understood there are still discussions about overload, but I've read here

python/typing#253 (comment)

that overlapping overloads should have compatible return types as in this example from the same comment:

class Animal: pass
class Dog(Animal): pass

@overload
def f(x: int) -> Dog: ...
@overload
def f(x: object) -> Animal: ...

var: object = 1
reveal_type(f(var))  # 'Dog' is a valid subtype of 'Animal':

My impression from zuban (i think ty has similar plans), you decided to
instead choose the first matching type and warn about unreachable overloads.
Which would enable an implementation like this to have matching static and
runtime behaviour

type ImmutableAttr = "MyFrozenDict | tuple[ImmutableAttr] | int | str"

@overload
def fn(a: Iterable[str]) -> tuple[str, ...]: ...

@overload
def fn(a: Mapping[str, Any]) -> MyFrozenDict: ...

def fn(a: Iterable[str]) -> tuple[str, ...] | MyFrozenDict:
    if isinstance(a, Mapping):
      return MyFrozenDict(a)
    else:
      return tuple(*a)

x: tuple[str, ...] = fn(["a", "b"])
y: MyFrozenDict = fn({"a": 1})

Is my impression correct and you're planning to keep that behaviour (opposed to e.g., pyright)?

[davidhalter]

Your observation is generally correct. I'm not exactly sure what you mean by

Which would enable an implementation like this to have matching static and runtime behaviour

though. I think if you're interested how overloads are supposed to work you can read the official documentation here:

https://typing.python.org/en/latest/spec/overload.html

This is the current spec for overloads. Zuban doesn't exactly implement it this way, but it's very similar and should generally follow these rules. Pyright as far as I know is not compatible with the overload conformance tests and hasn't been for a long time.

I'm closing, because I think this answers your initial question, but I'm happy to keep discussing this.

[glencoe]

Thanks for the quick reply. Actually i was aware of the specification, but I think the important part i missed before was

For all arguments, determine whether all possible materializations of the argument’s
type are assignable to the corresponding parameter type for each of the remaining overloads. If
so, eliminate all of the subsequent remaining overloads.

Edit: So if i got that right, there is nothing in the current specs implying mypy needs have its overload-overlap error code. Is that correct?

[davidhalter]

Correct. This check is just "additional" and not mandated by the spec. I'm not sure if similar checks exist for Pyright and other type checkers. But IMO the check is really important, because it shows when an overload member cannot ever be matched, which can easily be the case for complicated definitions with type vars or inheritance.

[glencoe]

Are you talking about overload-cannot-match? Because that's the one pyright does not have afaik. When i tried, i could only provoke overload-cannot-match but no overload-overlap with zuban. overload-overlap just checks for overlapping overloads and issues an error in the case like the one below

@overload
def fn(x: list[str]) -> int: ...

@overload
def fn(x: Iterable[str]) -> str: ...

def fn(x: Iterable[str]) -> int | str:
    if isinstance(x, list):
        return len(x)
    for i in x:
        return i
    return ""

Take as an example a compiled language.
In that case resolution would occur statically, therefore type analysis and runtime behaviour will always align.
There is no way someone could return a non-compatible type based on some reflection based control statement during runtime.
But in python, as soon as i'm using the isinstance check here things become ambiguous.
Therefore an overload like this

class A:
  pass

class B(A):
  pass

@overload
def fn(x: B) -> C: ...

@overload
def fn(x: A) -> D: ...

would only align with runtime behaviour if D was a subtype of C. Alternatively one would have to somehow access the type annotation of the passed argument to decide what type to return during runtime. As that is not possible, there'll always be ambiguity if D is not a subtype of C (be it structural or nominal).

Edit: So i should have looked deeper into the definitions of the actual error codes to begin with. And now that i (at least hope) understand the overload resolution better, what i should have written is: I really like the fact zuban has the overload-cannot-match check, that pyright hasn't. Are you planning to add overload-overlap as well?

[davidhalter]

Sorry, you're right, I confused these two error codes.

overload-overlap is planned, in fact it existed a while ago, but I disabled the check, because it caused too many false positives and issues (especially compared to how Mypy implements it). I'm not sure it will ever happen, but it's definitely something I would like to implement. It's just hard to implement it in a way where it's useful and not annoying. It's really easy for this check to be annoying when it is just slightly wrong with TypeVars/ParamSpec.