You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This rewrite is for the underlying infrastructure of how python handles testing therefore all behavior visible to the user should be pretty much the same (minus maybe some error logs going in a better place or test names shortened). Therefore throughout this testing, you can reference the current testing in Python to compare if you think something doesn't look right (or you can always post here too!).
Steps
Part 0: Setup
make sure you have the setting "python.experiments.optInto": ["All"]
Open the inc_dec_example_repo/inc_dec_example_test_suite testing repo folder in vscode
Use the create environment command from the command palette, select venv and whatever python version you want >=3.7
in the terminal run python -m pip install pytest
From the command palette run python: configure tests select pytest and "." (root directory)
Part 1: UI Nesting & Run
Go to the testing explorer, and confirm all the tests are discovered
Part 2: Debugging
add some break points to the pytest tests and check to make sure these break points are hit
make sure you can do other normal debugging actions (stop, restart, step over, etc)
Part 3: Large Files
In test_parameterized.py, change @pytest.mark.parametrize("num", range(1, 100)) to be much larger like @pytest.mark.parametrize("num", range(1, 10000)) and make sure it both works for discovery and run tests (tests should be displayed on the test explorer after discovery and run).
Part 4: Seeing Dynamic Run
Expand the testing explorer UI to see all the parameterized tests under test_wait_parameterized.py so you can see the nodes 1 - 10.
Click run all and make sure that you are seeing the nodes on the test UI finish at different times (ie they don't all resolve together, like some are still spinning while others are complete with a check or an x)
Eleanor Boyd (@eleanorjboyd), I've added some simple Markdown formatting to the test plan item. Please remember to make this easy for testers to follow.
make sure you have the setting "python.experiments.optInto": ["All"]
Maybe a safer option would be "python.experiments.optInto": ["pythonTestAdapter"], since All might affect testing in unexpected ways, no?
From the command palette run python: configure tests select pytest and "." (root directory)
Is this going to be the final UX? Feels strange to me that the user would have to do these manuals steps just to achieve what should've been discovered automatically in the first place since:
Configure tests should've have to be a user action, but automatic
pytest can be infered by looking into the sources
. can be assumed to be the default, while giving the use a non-prompting option to change
Please ignore this if that's already being considered.
Activity
joaomoreno commentedon Jun 27, 2023
Eleanor Boyd (@eleanorjboyd), I've added some simple Markdown formatting to the test plan item. Please remember to make this easy for testers to follow.
Maybe a safer option would be
"python.experiments.optInto": ["pythonTestAdapter"], sinceAllmight affect testing in unexpected ways, no?joaomoreno commentedon Jun 27, 2023
Unsure if these are all the tests:
A better step description would be "confirm that X tests are discovered" or "here's a screenshot of all the tests which should be discovered".
joaomoreno commentedon Jun 27, 2023
Is this going to be the final UX? Feels strange to me that the user would have to do these manuals steps just to achieve what should've been discovered automatically in the first place since:
pytestcan be infered by looking into the sources.can be assumed to be the default, while giving the use a non-prompting option to changePlease ignore this if that's already being considered.