Repository navigation
Description
i)
Run python ./Pyshp/tests/run_benchmarks.py
Reader tests:
Running benchmarks 1 times:
-----------------------------------
parser | exec time
-----------------------+-----------
/opt/hostedtoolcache/Python/3.15.2/x64/lib/python3.15/site-packages/shapefile.py:2987: PossiblyCorruptFileHeader: Declared file size in /home/runner/work/IronPyShp/IronPyShp/Pyshp/tests/shapefiles/blockgroups.dbf header: 236774 smaller than actual file size: 236775 (both in bytes)
warnings.warn(msg, category=PossiblyCorruptFileHeader)
Read Blockgroups | Running...���������� 0.0286 s
Read Edit | Running...���������� 0.012 s
Read Merge | Running...���������� 0.223 s
/opt/hostedtoolcache/Python/3.15.2/x64/lib/python3.15/site-packages/shapefile.py:2987: PossiblyCorruptFileHeader: Declared file size in /home/runner/work/IronPyShp/IronPyShp/PyShp_test_shapefile/ne_10m_admin_1_states_provinces.dbf header: 15161513 smaller than actual file size: 15161514 (both in bytes)
warnings.warn(msg, category=PossiblyCorruptFileHeader)
Read States_35MB | Running...���������� 0.793 s
/opt/hostedtoolcache/Python/3.15.2/x64/lib/python3.15/site-packages/shapefile.py:2987: PossiblyCorruptFileHeader: Declared file size in /home/runner/work/IronPyShp/IronPyShp/PyShp_test_shapefile/ne_110m_admin_0_tiny_countries.dbf header: 139635 smaller than actual file size: 139636 (both in bytes)
warnings.warn(msg, category=PossiblyCorruptFileHeader)
Read Tiny Countries | Running...���������� 0.00745 s
/opt/hostedtoolcache/Python/3.15.2/x64/lib/python3.15/site-packages/shapefile.py:2987: PossiblyCorruptFileHeader: Declared file size in /tmp/tmpmlnacmr9 header: 157486 smaller than actual file size: 157487 (both in bytes)
warnings.warn(msg, category=PossiblyCorruptFileHeader)
Read GIS_OSM_zip_10MB | Running...���������� 0.181 s
https://github.com/JamesParrott/IronPyShp/actions/runs/43751614365/job/132926656877
ii) Level 5 DOS .dbf files should end in 0x1a, but .dbf files of many shapefiles do not.
Added to run_benchmarks,py:
for file_path in SHAPEFILES.values():
file_path.read_bytes()
dbf_file = blockgroups_file.with_suffix(".dbf")
b = dbf_file.read_bytes()
last_byte = b[-1:]
print(f"{last_byte == b"\x1a"}, {last_byte=!r}")>python run_benchmarks.py
True, last_byte=b'\x1a'
True, last_byte=b'\x1a'
True, last_byte=b'\x1a'
True, last_byte=b'\x1a'
True, last_byte=b'\x1a'
True, last_byte=b'\x1a'
The end of the file is marked by a single byte, with value 0x1A.
https://en.wikipedia.org/wiki/.dbf#File_format_of_Level_5_DOS_dBASE
Metadata
Metadata
Assignees
Labels
No labels
Activity
[-]i) .dbf file length check is off by 1. ii) Level 5 DOS .dbf files should end in 0x1a, but .dbf files of many shapefiles do not.[/-][+]i) .dbf file length check is off by 1. ~~ii) Level 5 DOS .dbf files should end in 0x1a, but .dbf files of many shapefiles do not.~~[/+][-]i) .dbf file length check is off by 1. ~~ii) Level 5 DOS .dbf files should end in 0x1a, but .dbf files of many shapefiles do not.~~[/-][+]i) .dbf file length check is off by 1. [/+]