Skip to content

Specialize type variables of generic functions that are arguments #1471

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
rwbarton wants to merge 1 commit into from
Closed

Specialize type variables of generic functions that are arguments #1471

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 15 additions & 2 deletions mypy/checkexpr.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@
from mypy import messages
from mypy.infer import infer_type_arguments, infer_function_type_arguments
from mypy import join
from mypy.subtypes import is_subtype, is_equivalent
from mypy.subtypes import is_subtype, is_equivalent, unify_generic_callable
from mypy import applytype
from mypy import erasetype
from mypy.checkmember import analyze_member_access, type_object_type
Expand Down Expand Up @@ -327,7 +327,20 @@ def infer_arg_types_in_context2(
for i, actuals in enumerate(formal_to_actual):
for ai in actuals:
if arg_kinds[ai] not in (nodes.ARG_STAR, nodes.ARG_STAR2):
res[ai] = self.accept(args[ai], callee.arg_types[i])
callee_arg_type = callee.arg_types[i]
arg_type = self.accept(args[ai], callee_arg_type)
if (isinstance(arg_type, CallableType) and

@JukkaL JukkaL May 4, 2016

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doesn't seem to fix this for more general argument types. For example, what if the formal argument type is List[Callable[...]]? We could look into the types recursively, but not sure if that's the right thing to do (and it might be inefficient). I thought about this for a bit and this is far from clear to me.

@rwbarton rwbarton May 7, 2016

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A list-of-generic-function isn't a subtype of list-of-specific-function anyways, because List is invariant in its argument. But your concern is valid for covariant type constructors.

I'm going to think some more about why exactly this change is needed and what goes wrong without it...

@JukkaL JukkaL May 8, 2016

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The example I used involved a [...] expression in the argument so the list-of-generic-function could be specialized based on context (but isn't specialized correctly), but you are correct and it's a different case due to invariance.

@rwbarton rwbarton May 9, 2016

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you're talking about a case like

def f(xs: List[Callable[[T], S]], y: T) -> List[S]: pass
def id(x: U) -> U: pass

f([id], 1)

In principle we should be able to type check this as follows:

  • The first pass of type checking finds that T = int.
  • f provides the context List[Callable[[int], S]] to the expression [id], which is itself treated as a call to a generic function "[_]" that produces a list.
  • [_] provides the context Callable[[int], S] to the expression id. Now when id is type checked, the result is a generic callable type, so the code in this change unifies that type with the context, resulting in Callable[[int], int].
  • Now the first argument of f has type List[Callable[[int], int]] so we can infer that S = int.

In practice the third step doesn't work yet because we don't substitute variables in infer_function_type_arguments_using_context when the replacement would involve a type variable (here S).

isinstance(callee_arg_type, CallableType)):
# The argument is a function and might be generic.
# Specialize its type variables as necessary to
# match the expected argument type.
unified_type = unify_generic_callable(
arg_type, callee_arg_type, False)
if unified_type is not None:
arg_type = unified_type
# If unified_type is None, leave the type alone;
# we'll probably produce an error soon.
res[ai] = arg_type

# Fill in the rest of the argument types.
for i, t in enumerate(res):
Expand Down
3 changes: 1 addition & 2 deletions mypy/expandtype.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -57,8 +57,7 @@ def visit_deleted_type(self, t: DeletedType) -> Type:
return t

def visit_erased_type(self, t: ErasedType) -> Type:
# Should not get here.
raise RuntimeError()
return t

def visit_instance(self, t: Instance) -> Type:
args = self.expand_types(t.args)
Expand Down
23 changes: 23 additions & 0 deletions mypy/test/data/check-inference-context.test
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -678,6 +678,29 @@ class A: pass
[builtins fixtures/list.py]
[out]

[case testFunctionReferenceAndHigherOrderFunction]
from typing import TypeVar, Callable, List, Any
t = TypeVar('t')
s = TypeVar('s')
u = TypeVar('u', str, int)
def apply(func: Callable[[t], s], arg: t) -> s: pass
def f(x: t) -> List[t]: pass
def g(x: u) -> List[u]: pass

a = '0'
a1 = apply(f, a)
a1() # E: List[str] not callable
a2 = apply(g, a)
a2() # E: List[str] not callable
b = '0' # type: Any
b1 = apply(f, b)
b1() # E: List[Any] not callable
c = [1]
# The error below is not great--the real issue is that c is not a valid argument to g.
# Important thing is that the call is rejected.
c1 = apply(g, c) # E: Argument 1 to "apply" has incompatible type Callable[[u], List[u]]; expected Callable[[List[int]], List[u]]
[builtins fixtures/list.py]

[case testChainedAssignmentInferenceContexts]
from typing import List
i = None # type: List[int]
Expand Down