Repository navigation
Description
Hello,
I've encountered a little bug recently in the "Tests" tab. I am not 100% sure that the bug is related to vscode or to the python extension. According to the answer on vscode repo, it is due to the extension, that's why I'm reposting here.
Environment data
- VS Code version: 1.34.0
- Extension version (available under the Extensions sidebar): 2019.6.24221
- OS and version: Windows_NT x64 6.1.7601
- Python version (& distribution if applicable, e.g. Anaconda): 3.6.8 64 bit
- Type of virtual environment used : conda
- Relevant/affected Python packages and their versions: user interface of the test tab, pytest
- Jedi or Language Server? "python.jediEnabled" = True
My file settings.json is :
{
"python.testing.pytestArgs": [
"tests"
],
"python.testing.unittestEnabled": false,
"python.testing.nosetestsEnabled": false,
"python.testing.pytestEnabled": true
}
Bug description :
When a test file contains an upper letter, it is not displayed properly in the test tab. After running if passed or failed, it is displayed with a "grey question mark".
Steps to Reproduce:
- Create an empty project (note : use a conda environment where pytest is installed)
- Create a "tests" folder
2.1 Create a test_working.py file
import pytest
def test_this_one_works():
assert 1==1
2.2 Create a test_NotWorking.py file with the same code
import pytest
def test_this_one_works():
assert 1==1
In the Test tab of vscode, run Discover Tests
If everything ran properly, you should see the following architecture :
- (?) tests
- (?) test_notworking.py (with lower case!!!)
- (?) test_this_one_does_not_work
- (?) test_working.py
- (?) test_this_one_works
- (?) test_notworking.py (with lower case!!!)
In the Test tab of vscode, run Run All Tests.
Actual & expected behaviour
-(green tick ok) tests
- (?) test_notworking.py <- BUG - EXPECTED BEHAVIOUR : should have a green tick and have name in upper case
- (?) test_this_one_does_not_work <- BUG - EXPECTED BEHAVIOUR : should have a green tick and have name in upper case
- (green tick ok) test_working.py (with lower case!!!)
- (green tick ok) test_this_one_works
Logs
The tests seem to be ran whatever is displayed (the bug only concerns the user interface) : indeed in the bottom line (where my python version and my environment are shown), I see an icon "two tests passed". When i run pytest tests in the command line, everything is fine.
Python Test log
python C:\Users\F400411\.vscode\extensions\ms-python.python-2019.6.24221\pythonFiles\testing_tools\run_adapter.py discover pytest -- -s --cache-clear tests
============================= test session starts =============================
platform win32 -- Python 3.6.8, pytest-5.0.1, py-1.8.0, pluggy-0.12.0
rootdir: c:\Users\F400411\Documents\1_Datascience\3_MISSIONS\bug_test_vscode
collected 2 items
tests\test_NotWorking.py . [ 50%]
tests\test_working.py . [100%]
- generated xml file: C:\Users\F400411\AppData\Local\Temp\tmp-5180zo59mydLbZ4a.xml -
========================== 2 passed in 0.03 seconds ===========================
python C:\Users\F400411\.vscode\extensions\ms-python.python-2019.6.24221\pythonFiles\testing_tools\run_adapter.py discover pytest -- -s --cache-clear tests
============================= test session starts =============================
platform win32 -- Python 3.6.8, pytest-5.0.1, py-1.8.0, pluggy-0.12.0
rootdir: c:\Users\F400411\Documents\1_Datascience\3_MISSIONS\bug_test_vscode
collected 2 items
tests\test_NotWorking.py . [ 50%]
tests\test_working.py . [100%]
- generated xml file: C:\Users\F400411\AppData\Local\Temp\tmp-5180SM9GGMumKvVy.xml -
========================== 2 passed in 0.02 seconds ===========================
Conclusion :
This bugs is not critical at all since it only concerns the UI, but in my old projects I can't see the tests in the UI (while it used to work perfectly fine few weeks ago). I definitely do not want to change the names of the scripts (the tests scripts are a mirror of the scripts names of my project).
Any hope this would be easy to fix?
Many thanks for reading and any further help !
Galileo

Activity
karrtikr commentedon Jul 25, 2019
Thanks for reporting this Yolan Honoré-Rougé (@Galileo-Galilei)
PhilMacKay commentedon Jul 26, 2019
This is very likely related to #6410 except in the current issue, upper case letters are in the file name instead of the directory name.
marsfan commentedon Jul 28, 2019
Philippe MacKay (@PhilMacKay) Don Jayamanne (@DonJayamanne) I just stumbled across this trying to fix it on my computer. It is definitely (in my opnion) an extension of #6410.
In settings.json if I put
"python.testing.pytestArgs": [ "tests" ],to set up pytest, it correctly displays the markers as a red x or a green check.
But if I put
"python.testing.pytestArgs": [ "Tests" ],`
in settings .json, the results are all the grey questionmark, even after running.
Hereis the python test log from the a passed test run with all lowercase folder name in settings.json
And here is the failed, where the folder is listed as Tests
For me it appears that the actual folder name does not matter, just the capitalization in settings.json
PhilMacKay commentedon Aug 7, 2019
Gabe R. (@marsfan) I've tried this on my side and it doesn't solve my issue.
Have you tried to see what happens when the python file itself has capital letters in it? What about if it was located inside a sub-folder with a capital letter in it?
Edit: After extensive tests, I have been able to reproduce your issue. Using a Pytest version below 5.0.0 and Python 3.7.3, I get the exact same behaviour as you. It enables me to run tests :D
For a complete fix, we need to change the "NORMCASE" method in ~.vscode\extensions\ms-python.python-2019.8.29288\pythonFiles\testing_tools\adapter\pytest_discovery.py in class TestCollector. I changed
NORMCASE = staticmethod(os.path.normcase)toNORMCASE = staticmethod(os.path.normpath)and it solved the problem!ericsnowcurrently commentedon Aug 13, 2019
This should be resolved now due to #6781 and #6877. Those changes are already in the insiders build of the extension and will be part of the September release.
ericsnowcurrently commentedon Aug 14, 2019
Philippe MacKay (@PhilMacKay) indicated that this is resolved. See #6820 (comment)