Repository navigation
Conversation
| } | ||
|
|
||
| if (!empty($failMessages)) { | ||
| if (getenv('TYPO3_TESTING_EXPORT_DATASETS')) { |
There was a problem hiding this comment.
Not sure about that ... at least we should use a naming reflecting the case DATASETS are not always exported:
| if (getenv('TYPO3_TESTING_EXPORT_DATASETS')) { | |
| if ((bool)getenv('TYPO3_TESTING_EXPORT_DATASETS_ON_FAILED_ASSERTION')) { |
| if ($value === null) { | ||
| return '\\NULL'; | ||
| } | ||
| $stringValue = (string)$value; |
There was a problem hiding this comment.
Not sure about the implementation here, all values are converted to strings and some generic formating.
Not sure if we should bind that to the real database schema inforamation for table/column ... , having text/string/varchar field values always enquoted and other data (int, float/double, ) in the respected form.
Also thinking about native json fields for supported dbms (not using fallback TEXT/CBOLB for the others etc ) ....
Needs some thinking about this ...
| return '""'; | ||
| } | ||
| if (preg_match('/[,"\n\r\\\\]/', $stringValue)) { | ||
| return '"' . str_replace('"', '""', $stringValue) . '"'; |
There was a problem hiding this comment.
This only quotes text values if they contain , " or some line breaks. If not, they are not quoted ...
GIVEN...: Some Text
EXPECTED: "Some Text"
ACTUAL..: Some Text
ce2c366 to
9baf42c
Compare
Introduce exportCSVDataSet() to write current database state
to CSV files matching the existing fixture format. This completes
the import → assert → export workflow for functional tests.
Usage example:
$this->exportCSVDataSet(
$this->instancePath . '/export.csv',
['pages' => ['uid', 'pid', 'title'], 'tt_content']
);
Tables can specify explicit fields or export all columns.
Additionally, assertCSVDataSet() now auto-exports the actual
database state to a _actual.csv file when assertions fail and
the TYPO3_TESTING_EXPORT_DATASETS environment variable is set.
|
|
CybotTM
left a comment
There was a problem hiding this comment.
Requesting changes: exported files do not always read back correctly through DataSet::read(), and the unit tests would not notice a broken exportCSVDataSet().
I applied the PR to current main (480a74d) and ran a functional probe in TYPO3 core main (8ef5361) on SQLite, MariaDB and PostgreSQL, with identical results:
- Works: exporting the tables of
DataScenarios/Regular/DataSet/ImportDefault.csvandFixtures/be_users_admin.csv, thenassertCSVDataSet()on the export; truncating and re-importing the export, then asserting the original fixtures; native date/datetime/time columns including NULL; the_actual.csvwritten on a failed assertion. - Fails: blob columns and values with a backslash before a quote (inline comments below).
Requested changes:
- Quote blob/binary values. They fall into the
defaultbranch and are written unquoted; a serializedbe_users.ucis cut at its first comma on read back and the following columns shift. - Make writer and reader agree on escaping.
formatCsvValue()doubles", butDataSet::read()callsfgetcsv(..., '"', '\\'). A JSON value{"a":"b,c","q":"x\"y"}comes back as{"a":"b,c","q":"x\"y""}", and a value ending in a backslash (C:\path\) swallows the next row. - Test
exportCSVDataSet()itself.exportedCsvCanBeReadBackByDataSetReadrebuilds the export loop instead of calling the method: with the header line inexportCSVDataSet()broken, all 14 new tests stay green. A functional test that imports a fixture, exports it and reads or asserts the export would cover it, including blob, json and backslash values. - Never overwrite the asserted fixture, and do not report an export that did not happen (inline comments).
- Update the PR description: it names
TYPO3_TESTING_EXPORT_DATASETS, the code readsTYPO3_TESTING_EXPORT_DATASETS_ON_FAILED_ASSERTION.
Smaller points
- Only
1enables the export;true,yesandonare read as off because of(bool)((int)getenv(...)). - On PHP 8.5 (PHPUnit 13) each of the 14 new tests raises the notice "No expectations were configured for the mock object" from
getAccessibleMock(); main has none. - The
Build/phpunit/UnitTests.xmlchange is unrelated to the feature. - The comment in
Resources/Core/Build/FunctionalTests.xmlsays "Comment following line out" where uncommenting is meant, and contains "Do not use provide it here". - The CI runs on this PR are from 2026-02-10, 34 commits behind main. On current main, PHP 8.2 gates are green; on PHP 8.5 the PHPStan error in
Testbase.phpand thecacheResultdeprecation also occur on main without this PR.
| $columnDoctrineType instanceof TimeType, | ||
| $columnDoctrineType instanceof TimeImmutableType => '"' . str_replace('"', '""', (string)$value) . '"', | ||
| $columnDoctrineType instanceof BooleanType => (string)(int)($value), | ||
| default => (string)$value, |
There was a problem hiding this comment.
Blob and binary columns land here and are written unquoted. Exporting be_users.uc with a serialized array containing , and " and reading it back with DataSet::read() returns the value cut at the first comma, and the remaining columns shift (SQLite, MariaDB, PostgreSQL). Other core blob columns: be_users.mfa, fe_users.uc, sys_registry.entry_value. Quoting everything except the numeric and boolean types would avoid listing each string-like type.
| $columnDoctrineType instanceof DateTimeImmutableType, | ||
| $columnDoctrineType instanceof DateType, | ||
| $columnDoctrineType instanceof TimeType, | ||
| $columnDoctrineType instanceof TimeImmutableType => '"' . str_replace('"', '""', (string)$value) . '"', |
There was a problem hiding this comment.
Doubling " does not match the reader: DataSet::read() uses \ as escape character (fgetcsv($fileHandle, 0, ',', '"', '\\')). A json value {"a":"b,c","q":"x\"y"} is read back as {"a":"b,c","q":"x\"y""}", and a string ending in a backslash merges the next row into it. Either the reader needs escape '' or the writer has to escape for the reader.
| foreach ($dataSet->getTableNames() as $tableName) { | ||
| $exportTables[$tableName] = $dataSet->getFields($tableName) ?? []; | ||
| } | ||
| $exportPath = preg_replace('/\.csv$/', '_actual.csv', $fileName); |
There was a problem hiding this comment.
If the fixture name does not end in lowercase .csv (e.g. Result.CSV), the pattern does not match, $exportPath equals $fileName and the export overwrites the fixture being asserted. Reproduced with wrong.CSV. Appending the suffix unconditionally or refusing when both paths are equal would prevent it.
| } | ||
| } | ||
|
|
||
| file_put_contents($path, $output); |
There was a problem hiding this comment.
The return value is not checked. In a read-only directory this raises a PHP warning and the failure message still says "Actual database state exported to: .../wrong_actual.csv" although no file exists.
| ], | ||
| ]; | ||
|
|
||
| // Build CSV output the same way exportCSVDataSet does |
There was a problem hiding this comment.
This rebuilds the export instead of calling exportCSVDataSet(), so the method body is not covered: changing its field header line to implode(';', $fields) leaves all 14 tests green. The copy has also drifted already (str_repeat(',', ...) in the table line, which the implementation dropped).
Introduce exportCSVDataSet() to write current database state to CSV files matching the existing fixture format. This completes the import → assert → export workflow for functional tests.
Usage example:
Tables can specify explicit fields or export all columns.
Additionally, assertCSVDataSet() now auto-exports the actual database state to a _actual.csv file when assertions fail and the TYPO3_TESTING_EXPORT_DATASETS environment variable is set.