Repository navigation
Optimize pathlib.PurePath.__fspath__() #102783
Description
Activity
On Windows, an exception should be made to normalize extended paths in
__fspath__(). If an extended path isn't normalized, the open will fail. One needs to replace slashes with backslashes; collapse repeated slashes; and resolve "." and ".." components.Scripts have to be prepared to handle extended paths coming from
os.path.realpath(),__file__, and attributes of thesysmodule such asexecutable,prefix,argv, andpath. Extended paths can also come from extension modules that use API functions such asGetModuleFileNameW()(if an EXE/DLL was loaded using an extended path),GetFinalPathNameByHandleW(), and any of thePathCch*()functions that implements the flagPATHCCH_ALLOW_LONG_PATHS.For path construction, there's an open issue that proposes to modify
ntpath.join()to fully normalize any drive path or UNC path vianormpath(). For paths without a drive, it's proposed to replace slashes with backslashes and collapse repeated slashes. For example, this change would support getting a useful result from something likentpath.join(filename, "../spam//eggs.py")givenfilenameis an extended path such as "\\?\C:\foo\bar.py".On Windows, an exception should be made to normalize extended paths in
__fspath__(). If an extended path isn't normalized, the open will fail. One needs to replace slashes with backslashes; collapse repeated slashes; and resolve "." and ".." components.I don't think pathlib's normalization should change the meaning of paths. It sounds like we should adjust the normalization routine to preserve forward slashes, repeated slashes, and "." components, for these kinds of path. ".." components are already be preserved I think.
I don't think pathlib's normalization should change the meaning of paths. It sounds like we should adjust the normalization routine to preserve forward slashes, repeated slashes, and "." components, for these kinds of path. ".." components are already be preserved I think.
In almost all cases, extended paths are used solely to access a long path. The need to get a literal path is rare, especially in regard to slashes and "." and ".." component names. Microsoft's filesystems will fail an open that has forward slashes or repeated backslashes. NTFS will fail an open that has "." or ".." component names. FAT filesystems allow creating "." and ".." component names as a file's long name, with some other random short name, but it's very dysfunctional.
People assume that
open(filename / "../spam//eggs.txt")will just work. But iffilenameis an extended path -- one that's injected completely outside of the control of the script -- then this open will fail if the path isn't normalized. IMO, no purity in regards to literal paths is worth this headache.Reacted by Barney GaleOS APIs are perfectly happy with unnormalized paths
What happens if a path is mounted on e.g. a FUSE filesystem or a Samba share? Does the OS unnormalize before handing the path to the filesystem?
Resolving as "can't do", because #65238 is resolved as "won't fix"
Feature or enhancement
Return an unnormalized path from
pathlib.PurePath.__fspath__()Pitch
Code like
open(Path('./README.txt'))orPath('/home//barney').iterdir()shouldn't require us to normalize the path in pathlib (e.g. remove.segments, doubled slashes, etc), as OS APIs are perfectly happy with unnormalized paths.We can improve the performance of most
Pathmethods, and the effective performance of passing aPathobject to any API that acceptsos.PathLike, by skipping normalization in__fspath__().Prerequisites
Pathlib must not normalize paths on construction:
Pathlib's normalization must not change the meaning of paths:
Previous discussion
Linked PRs
pathlib.PurePath.__fspath__()by returning raw path #112397