Crash when typechecking transformers 4.10.0

Crash Report

I am typechecking a codebase which imports transformers 4.10.0 and hitting a mypy internal error. Also reported in the transformers repo as huggingface/transformers#13390

Traceback

This traceback is generated using the current mypy master (pip install git+https://github.com/python/mypy).

src/transformers/trainer.py:1435: error: INTERNAL ERROR -- Please try using mypy master on Github:
https://mypy.readthedocs.io/en/stable/common_issues.html#using-a-development-mypy-build
Please report a bug at https://github.com/python/mypy/issues
version: 0.920+dev.5e73cd7b0dc9b871716acf0086b2ca0376fcff39
Traceback (most recent call last):
  File "/home/david/dev/transformers/.env/bin/mypy", line 8, in <module>
    sys.exit(console_entry())
  File "/home/david/dev/transformers/.env/lib/python3.9/site-packages/mypy/__main__.py", line 11, in console_entry
    main(None, sys.stdout, sys.stderr)
  File "/home/david/dev/transformers/.env/lib/python3.9/site-packages/mypy/main.py", line 87, in main
    res, messages, blockers = run_build(sources, options, fscache, t0, stdout, stderr)
  File "/home/david/dev/transformers/.env/lib/python3.9/site-packages/mypy/main.py", line 165, in run_build
    res = build.build(sources, options, None, flush_errors, fscache, stdout, stderr)
  File "/home/david/dev/transformers/.env/lib/python3.9/site-packages/mypy/build.py", line 179, in build
    result = _build(
  File "/home/david/dev/transformers/.env/lib/python3.9/site-packages/mypy/build.py", line 254, in _build
    graph = dispatch(sources, manager, stdout)
  File "/home/david/dev/transformers/.env/lib/python3.9/site-packages/mypy/build.py", line 2707, in dispatch
    process_graph(graph, manager)
  File "/home/david/dev/transformers/.env/lib/python3.9/site-packages/mypy/build.py", line 3031, in process_graph
    process_stale_scc(graph, scc, manager)
  File "/home/david/dev/transformers/.env/lib/python3.9/site-packages/mypy/build.py", line 3138, in process_stale_scc
    if not graph[id].type_check_second_pass():
  File "/home/david/dev/transformers/.env/lib/python3.9/site-packages/mypy/build.py", line 2192, in type_check_second_pass
    return self.type_checker().check_second_pass()
  File "/home/david/dev/transformers/.env/lib/python3.9/site-packages/mypy/checker.py", line 354, in check_second_pass
    self.check_partial(node)
  File "/home/david/dev/transformers/.env/lib/python3.9/site-packages/mypy/checker.py", line 366, in check_partial
    self.accept(node)
  File "/home/david/dev/transformers/.env/lib/python3.9/site-packages/mypy/checker.py", line 411, in accept
    stmt.accept(self)
  File "/home/david/dev/transformers/.env/lib/python3.9/site-packages/mypy/nodes.py", line 696, in accept
    return visitor.visit_func_def(self)
  File "/home/david/dev/transformers/.env/lib/python3.9/site-packages/mypy/checker.py", line 743, in visit_func_def
    self._visit_func_def(defn)
  File "/home/david/dev/transformers/.env/lib/python3.9/site-packages/mypy/checker.py", line 747, in _visit_func_def
    self.check_func_item(defn, name=defn.name)
  File "/home/david/dev/transformers/.env/lib/python3.9/site-packages/mypy/checker.py", line 809, in check_func_item
    self.check_func_def(defn, typ, name)
  File "/home/david/dev/transformers/.env/lib/python3.9/site-packages/mypy/checker.py", line 992, in check_func_def
    self.accept(item.body)
  File "/home/david/dev/transformers/.env/lib/python3.9/site-packages/mypy/checker.py", line 411, in accept
    stmt.accept(self)
  File "/home/david/dev/transformers/.env/lib/python3.9/site-packages/mypy/nodes.py", line 1024, in accept
    return visitor.visit_block(self)
  File "/home/david/dev/transformers/.env/lib/python3.9/site-packages/mypy/checker.py", line 2005, in visit_block
    self.accept(s)
  File "/home/david/dev/transformers/.env/lib/python3.9/site-packages/mypy/checker.py", line 411, in accept
    stmt.accept(self)
  File "/home/david/dev/transformers/.env/lib/python3.9/site-packages/mypy/nodes.py", line 1216, in accept
    return visitor.visit_if_stmt(self)
  File "/home/david/dev/transformers/.env/lib/python3.9/site-packages/mypy/checker.py", line 3350, in visit_if_stmt
    self.accept(b)
  File "/home/david/dev/transformers/.env/lib/python3.9/site-packages/mypy/checker.py", line 411, in accept
    stmt.accept(self)
  File "/home/david/dev/transformers/.env/lib/python3.9/site-packages/mypy/nodes.py", line 1024, in accept
    return visitor.visit_block(self)
  File "/home/david/dev/transformers/.env/lib/python3.9/site-packages/mypy/checker.py", line 2005, in visit_block
    self.accept(s)
  File "/home/david/dev/transformers/.env/lib/python3.9/site-packages/mypy/checker.py", line 411, in accept
    stmt.accept(self)
  File "/home/david/dev/transformers/.env/lib/python3.9/site-packages/mypy/nodes.py", line 1083, in accept
    return visitor.visit_assignment_stmt(self)
  File "/home/david/dev/transformers/.env/lib/python3.9/site-packages/mypy/checker.py", line 2047, in visit_assignment_stmt
    self.check_assignment(s.lvalues[-1], s.rvalue, s.type is None, s.new_syntax)
  File "/home/david/dev/transformers/.env/lib/python3.9/site-packages/mypy/checker.py", line 2226, in check_assignment
    self.check_indexed_assignment(index_lvalue, rvalue, lvalue)
  File "/home/david/dev/transformers/.env/lib/python3.9/site-packages/mypy/checker.py", line 3208, in check_indexed_assignment
    self.expr_checker.check_method_call(
  File "/home/david/dev/transformers/.env/lib/python3.9/site-packages/mypy/checkexpr.py", line 2457, in check_method_call
    return self.check_call(method_type, args, arg_kinds,
  File "/home/david/dev/transformers/.env/lib/python3.9/site-packages/mypy/checkexpr.py", line 933, in check_call
    return self.check_callable_call(callee, args, arg_kinds, context, arg_names,
  File "/home/david/dev/transformers/.env/lib/python3.9/site-packages/mypy/checkexpr.py", line 1024, in check_callable_call
    arg_types = self.infer_arg_types_in_context(
  File "/home/david/dev/transformers/.env/lib/python3.9/site-packages/mypy/checkexpr.py", line 1123, in infer_arg_types_in_context
    res[ai] = self.accept(args[ai], callee.arg_types[i])
  File "/home/david/dev/transformers/.env/lib/python3.9/site-packages/mypy/checkexpr.py", line 3912, in accept
    typ = node.accept(self)
  File "/home/david/dev/transformers/.env/lib/python3.9/site-packages/mypy/nodes.py", line 1600, in accept
    return visitor.visit_call_expr(self)
  File "/home/david/dev/transformers/.env/lib/python3.9/site-packages/mypy/checkexpr.py", line 277, in visit_call_expr
    return self.visit_call_expr_inner(e, allow_none_return=allow_none_return)
  File "/home/david/dev/transformers/.env/lib/python3.9/site-packages/mypy/checkexpr.py", line 326, in visit_call_expr_inner
    callee_type = get_proper_type(self.accept(e.callee, type_context, always_allow_any=True))
  File "/home/david/dev/transformers/.env/lib/python3.9/site-packages/mypy/checkexpr.py", line 3912, in accept
    typ = node.accept(self)
  File "/home/david/dev/transformers/.env/lib/python3.9/site-packages/mypy/nodes.py", line 1520, in accept
    return visitor.visit_member_expr(self)
  File "/home/david/dev/transformers/.env/lib/python3.9/site-packages/mypy/checkexpr.py", line 2047, in visit_member_expr
    result = self.analyze_ordinary_member_access(e, is_lvalue)
  File "/home/david/dev/transformers/.env/lib/python3.9/site-packages/mypy/checkexpr.py", line 2065, in analyze_ordinary_member_access
    member_type = analyze_member_access(
  File "/home/david/dev/transformers/.env/lib/python3.9/site-packages/mypy/checkmember.py", line 126, in analyze_member_access
    result = _analyze_member_access(name, typ, mx, override_info)
  File "/home/david/dev/transformers/.env/lib/python3.9/site-packages/mypy/checkmember.py", line 143, in _analyze_member_access
    return analyze_instance_member_access(name, typ, mx, override_info)
  File "/home/david/dev/transformers/.env/lib/python3.9/site-packages/mypy/checkmember.py", line 219, in analyze_instance_member_access
    typ = map_instance_to_supertype(typ, method.info)
  File "/home/david/dev/transformers/.env/lib/python3.9/site-packages/mypy/maptype.py", line 20, in map_instance_to_supertype
    if not superclass.type_vars:
  File "/home/david/dev/transformers/.env/lib/python3.9/site-packages/mypy/nodes.py", line 2656, in __getattribute__
    raise AssertionError(object.__getattribute__(self, 'msg'))
AssertionError: FuncBase for non-methods lack info

To Reproduce

I can reproduce this locally in the following way:

Clone the transformers repo, checkout the v4.10.0 tag
Create a blank virtualenv and install mypy 0.910 (or mypy master)
Add the following mypy.ini file:
```
[mypy]
check_untyped_defs=True
```
Run mypy src/transformers/trainer.py

I'm afraid I tried to reduce to a minimal repro with no success.

Your Environment

Mypy version used: 0.910 and master
Mypy command-line flags: None other than --show-traceback
Mypy configuration options from mypy.ini (and other config files): See repo above (just check_untyped_defs=True)
Python version used: tested on 3.8.11 and 3.9.5
Operating system and version: Ubuntu, tested on 20.04 and 21.04

This may be related to #7045.

https://github.com/huggingface/transformers/blob/b91e65afe0f467e24183928bf57d92b2cef4b69f/src/transformers/trainer.py#L1435

logs["learning_rate"] = self._get_learning_rate()

calls

https://github.com/huggingface/transformers/blob/b91e65afe0f467e24183928bf57d92b2cef4b69f/src/transformers/trainer.py#L256

class Trainer:
    from .trainer_pt_utils import _get_learning_rate, log_metrics, metrics_format, save_metrics, save_state

Is there a way to configure mypy so that it will not follow imports into the transformers library alone?

@matthen

@matthen, yes, there is. Please see huggingface/transformers#13390 (comment).

Fix class scoped imports

Fix crashes in class scoped imports (#12023)

Navigation Menu

Repository navigation

Crash when typechecking transformers 4.10.0 #11045

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Crash when typechecking transformers 4.10.0 #11045

Description

Activity

shirayu commented on Sep 29, 2021

matthen commented on Nov 25, 2021

aphedges commented on Nov 25, 2021

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions