mypy override errors for input/output/requires

[hirosassa]

Problem

Luigi master added type annotations to Task.input(), Task.output(), and Task.requires(). This causes mypy to report [override] errors in TaskOnKart:

gokart/task.py: error: Return type "FlattenableItems[TargetOnKart]" of "input" incompatible with return type "Target | list[Target] | dict[str, Target]" in supertype "Task"  [override]
gokart/task.py: error: Return type "FlattenableItems[TargetOnKart]" of "output" incompatible with return type "Target | list[Target] | dict[str, Target]" in supertype "Task"  [override]
gokart/task.py: error: Return type "FlattenableItems[TaskOnKart[Any]]" of "requires" incompatible with return type "Task | list[Task] | dict[str, Task]" in supertype "Task"  [override]

Root Cause

Luigi defines return types as T | list[T] | dict[str, T], while gokart uses FlattenableItems[T]:

FlattenableItems: TypeAlias = T | list['FlattenableItems[T]'] | tuple['FlattenableItems[T]', ...] | dict[str, 'FlattenableItems[T]']

Two differences make FlattenableItems[T] incompatible as a subtype of luigi's type:

1. Includes tuple — not present in luigi's type
2. Recursively nested — e.g. list[list[T]] is valid, but luigi only allows list[T]

The concrete type parameters (TargetOnKart <: Target, TaskOnKart <: Task) are fine (covariant direction).

Options

1. Suppress with # type: ignore[override] — simplest; no runtime issue exists
2. Narrow FlattenableItems to match luigi — would need to audit all usages for tuple/nested patterns
3. Propose widening luigi's return types — upstream PR to support tuple and recursive nesting
4. Combination: suppress now, propose upstream change later

Discussion

Which approach should we take?

[hirosassa]

I think 3 is the best choice because Luigi already implemented such features.
Luigi's own flatten() and getpaths() already handle tuples and recursively nested structures at runtime. However, the current type annotations on Task.input(), Task.output(), and Task.requires() only declare T | list[T] | dict[str, T], which is narrower than what the implementation actually supports.

[hirosassa]

ref: spotify/luigi#3428

[hirosassa]

I found that the PR above does not solve type check errors.
The return types of Task.input(), Task.output(), and Task.requires() use list and dict, which are invariant in Python's type system. This means even though TargetOnKart is a subclass of Target, list[TargetOnKart] is not a subtype of list[Target].

As a result, any subclass that narrows the type parameter (e.g. returning FlattenableItems[TargetOnKart] instead of FlattenableItems[Target]) will always produce mypy [override] errors — regardless of how the type alias is defined.

[hirosassa]

There're a solution using Sequence/Mapping (these are covariant) instead of list/dict for luigi's FlattenableItem, but it is slightly weird change for luigi users.