Consider adding doctest.skip_if decorator

@serhiy-storchaka

Feature or enhancement

Right now there's no way to skip the whole test / class doctest. You can only skip individual statements with # doctest: +SKIP

This causes problems in several use-cases:

You have a doctest that should be skipped based on the platform
We have doctests that should be skipped based on a resource availability
Libraries might want to skip some doctests based on the Python version / any other lib version

Right now we have this hack:

cpython/Lib/test/test_doctest/test_doctest.py

Lines 727 to 730 in 2e9be80

    
               if int.__doc__: # simple check for --without-doc-strings, skip if lacking 
        
                   def non_Python_modules(): r""" 
        
           Finding Doctests in Modules Not Written in Python

cpython/Lib/test/test_doctest/test_doctest.py

Lines 1999 to 2003 in 2e9be80

    
           if not hasattr(sys, 'gettrace') or not sys.gettrace(): 
        
               def test_pdb_set_trace(): 
        
                   """Using pdb.set_trace from a doctest. 
        
                   You can use pdb.set_trace from a doctest.  To do so, you must

cpython/Lib/test/test_doctest/test_doctest.py

Lines 2938 to 2943 in 2e9be80

    
           if supports_unicode: 
        
               def test_unicode(): """ 
        
           Check doctest with a non-ascii filename: 
        
               >>> doc = ''' 
        
               ... >>> raise Exception('clé')

However, this is just test functions. For library functions it would be much harder to do.
You have to jiggle __doc__ attribute around.

Since #117297 we now have correct skipped count for doctest + unittest integration.

So, I propose adding this decorator to doctest.py

def skip_if(condition):
    def decorator(func):
        if condition and HAVE_DOCSTRINGS:
            func.__doc__ = ">>> pass  # doctest: +SKIP"
        return func
    return decorator

It will allow us skipping some tests conditionally with the proper reported results.

Refs #116758
CC @serhiy-storchaka and @furkanonder

Linked PRs

gh-117364: Add doctest.SkipTest to skip all or some doctest examples #122935

There is a problem with applying this to non-test functions. We do not want to replace docstrings of library functions if doctests are skipped.

Also, what is the solution for conditional skipping of module-level doctests?

There is a problem with applying this to non-test functions. We do not want to replace docstrings of library functions if doctests are skipped.

Hm, indeed. This might be a bit more complex: we can add # doctest: +SKIP to all >>> lines dynamically.

Does not seem too hard, but it will be harder for sure.

Also, what is the solution for conditional skipping of module-level doctests?

This can be adjusted to include doctest.skip_if(condition, module=True).

Or instead of patching the string, we can go with .__doctest_skip__ attribute for a function.

This was my idea too. I thought about making it more general and also setting other options for the whole test, but I don't see a use case for other options for now.

Alternative ideas:

Introduce a special exception doctest.SkipTest, so that raising it will skip all following examples.
Introduce a syntax for conditional skipping, e.g. +SKIP if varname. It is enough to support only simple variable names, not complex expressions. It would be nice to also add syntax to apply options to all subsequent examples instead of just one example.

They are more complex changes of doctest machinery.

pythongh-117364: Add `doctest.SkipTest` to skip all or some doctest e…

Navigation Menu

Repository navigation

Consider adding `doctest.skip_if` decorator #117364

Feature or enhancement

Linked PRs

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

	if int.__doc__: # simple check for --without-doc-strings, skip if lacking
	def non_Python_modules(): r"""

	Finding Doctests in Modules Not Written in Python

	if not hasattr(sys, 'gettrace') or not sys.gettrace():
	def test_pdb_set_trace():
	"""Using pdb.set_trace from a doctest.

	You can use pdb.set_trace from a doctest. To do so, you must

	if supports_unicode:
	def test_unicode(): """
	Check doctest with a non-ascii filename:

	>>> doc = '''
	... >>> raise Exception('clé')

Consider adding doctest.skip_if decorator #117364

Description

Feature or enhancement

Linked PRs

Activity

serhiy-storchaka commented on Mar 29, 2024

sobolevn commented on Mar 29, 2024

sobolevn commented on Mar 31, 2024

serhiy-storchaka commented on Mar 31, 2024

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions

Consider adding `doctest.skip_if` decorator #117364