Skip to content

Fixes: DB import library - #328

Open
ElminsterAU wants to merge 1 commit into
remobjects:masterfrom
ElminsterAU:fixes/db
Open

ElminsterAU wants to merge 1 commit into
remobjects:masterfrom
ElminsterAU:fixes/db

Conversation

@ElminsterAU

Copy link
Copy Markdown

DB import library: 49 changes, each described in the commit messages.

This is one of a series of pull requests carrying the fixes NexusDB made to Pascal Script, split by area so that each can be reviewed on its own. The fixes are listed one per line in each commit's message.

Applies to master on its own.

Tested. With the whole series applied, every unit of Source compiles, and a script run through TPSScript (strings, Int64, reals, calls into host functions, exceptions, a class from the Classes library, for / Break) gives the expected results, on Delphi 7, 2005, 2006, 2007, 2009, 2010, XE, XE2 - XE8 and 10 Seattle - 13 Florence (Win32, and Win64 from XE2), and on Free Pascal 3.2.2 (i386) and 3.3.1 (x86_64), with and without PS_USESSUPPORT; compilers before XE4 need the last pull request of the series. Master as it stands does not compile on XE4, XE5 Win64, 10 Seattle, 10.1 Berlin, 10.3 Rio Win64, Free Pascal 3.2.2 or 3.3.1, nor on anything before XE4.

Beyond that smoke test, the fixes were developed and are covered in NexusDB, whose suite runs about 6,400 checks on 37 Delphi compiler / platform combinations and on Free Pascal. NexusDB defines PS_USECLASSICINVOKE, so there every Delphi calls native routines through x86.inc / x64.inc; the InvokeCall.inc path (the default from Delphi 10 Seattle) is exercised here by the smoke test only.

These fixes come from NexusDB, which ships Pascal Script as its script engine.  Each was found and fixed
there; most were measured against what Delphi itself does with the same code, and most have a regression
test in NexusDB's own suite, which runs on Delphi 7 through 13 Florence (Win32 and Win64) and on Free Pascal
3.2.2 (i386) and 3.3.1 (x86_64).  NexusDB builds the engine with PS_USECLASSICINVOKE and PS_USESSUPPORT; the
fixes are given here in Pascal Script's own unit names and configuration.

uPSC_DB.pas:
- TParams declared no constructor, so a script's TParams.Create reached TObject.Create and the
  parameter list was never initialized (an access violation in Count or CreateParam, measured as for
  TCollection). Bound to Delphi's parameterless TParams.Create (Delphi 7 and 13 Florence declare it)
- TParamType is the enumeration (ptUnknown, ptInput, ptOutput, ptInputOutput, ptResult) in Delphi 7,
  13 Florence (Data.DB) and Free Pascal, and the script declares it a Byte with these constants as
  bit masks 1, 2, 4, 8, 16: ptInput reached Delphi as ptOutput, ptOutput as ptResult, ptInputOutput
  and ptResult as no member at all (TParam.ParamType's setter and TParams.CreateParam store it
  unchecked). The constants take the ordinals
- TDataSet.FilterOptions is declared with TFilterOptions, which nothing declared, so the property
  was dropped. The members are Delphi 7's and 13 Florence's (Data.DB), one byte as Delphi's set;
  both run-time arms already bind the property
- An enumeration declared to scripts with fewer members than Delphi's, so a script could not name
  the later ones and read them back as ordinals its type does not have. Measured with GetEnumName on
  D7, D2006, D2007, D2009, D2010, XE, XE2, XE8, Tokyo, Rio, Sydney, Alexandria, Athens (Win32) and
  13 Florence (Win64): TFieldType 38 / 42 (2006) / 49 (2009) / 52 (2010) / 53 (13 Florence) members,
  TPenStyle 7 / 9 (2006; the 2006 Graphics unit holds psUserStyle and 2005's does not),
  TListBoxStyle 5 on every version. Free Pascal keeps upstream's declarations (its own differ). And
  TFieldChars (Delphi's set of AnsiChar, 32 bytes on every version measured, as a script set of
  Char) was declared nowhere, so TField.ValidChars was dropped although its helpers exist.
  TField.FieldKind stays undeclared: the library declares fkData.. FkAggregate as LongInt bit
  constants for TFieldKinds, which an enumeration TFieldKind would redeclare with other values
- [Free Pascal] Fcl-db 3.2.2 db.pas 567 / 597: the script's hierarchy as fcl-db's; a script passing
  a TSmallintField / TWordField / TAutoIncField where a TIntegerField is taken is refused at compile
  time, as native Free Pascal code is (pascalscript 08:59)
- [Free Pascal] (db.pas 652)
- [Free Pascal] (db.pas 643)
- [Free Pascal] (db.pas 634)
- [Free Pascal] Unicode mode: Classes, SysUtils and fcl-db are compiled with string = AnsiString,
  while a script `string` is a UnicodeString; this member is bound by address, so the engine would
  hand the RTL a UnicodeString where it reads an AnsiString. Declared AnsiString on Free Pascal, the
  engine converts first. An FPC arm: Delphi-invisible
- [Free Pascal] The script declaration of this Create carried the same gate as its constructor
  routine; without it the script's Create reached TObject.Create and the run-time binding was never
  asked. Delphi selects the same text
- [Free Pascal] Fcl-db's TDefCollection.Find / IndexOf take an AnsiString; the script declared
  string, a UnicodeString under DELPHIUNICODE, and the binding by address passed it unconverted.
  Delphi selects the same text
- [Free Pascal] Fcl-db's TFieldDef.FieldNo is read-only, so the script declares it read-only there.
  An FPC arm
- [Free Pascal] The script's TFieldType, TDataSetState and provider flags follow fcl-db's own lists,
  so every ordinal matches the native one. FPC arms only: Delphi selects the same text
- [Free Pascal] The compile side declares no TFlatList, TFieldDefList, TFieldList or
  TDataSet.FieldDefList / FieldList, as the run time registers none (fcl-db has none). FPC arms
  only: Delphi selects the same text
- [Free Pascal] The script declared fcl-db members the run time never bound. What fcl-db does not
  declare is no longer declared to scripts; what it declares from 3.2.4 (PS_FCLDB_NO_OBJECT_FIELD
  below it) is declared and bound from there. Delphi selects the same text
- TFieldAttributes was declared as a Byte with its members as numeric constants, where Delphi has a
  set of the enumeration TFieldAttribute; in a host that registers the SysUtils library first,
  faReadOnly named the file attribute (1), and `fd.Attributes := faReadOnly` set faHiddenCol.
  Declared as Delphi has them, that assignment is refused there, as is [faReadOnly], and the set
  forms of the other members -- [faRequired] and the like, refused before -- compile
- TGuidField derives from TStringField, and TFieldDefList's default property is FieldDefs, as Delphi
  declares them

uPSR_DB.pas:
- The run-time half of the TParams.Create declaration in uPSC_DB
- These members are bound by the address of a virtual method, so a script passes over a descendant's
  override -- a data set's MoveBy (override from Delphi 2009) skipped its OnFilterRecord check and
  busy tracking when a script called it. RegisterVirtualMethod is no fix: several are virtual only
  in later Delphi versions, and a slot lookup that fails calls slot 0. The constructor routines are
  Delphi only, and FPC keeps the binding by address. The reason first given here -- that under FPC
  the run time hands a constructor routine another Self -- is refuted: the stubs load the class
  first (x86.inc / x64.inc, the IsConstructor arm), measured by the fpc area
- The compiler declares TParams.AssignValues and the run time's registration of it is commented out,
  so a script calling it compiled and did not load (Cannot Import ASSIGNVALUES). The method is
  static and has this signature on Delphi 7 and 13 Florence alike.
- From Delphi 2009 TBlobField.Value is bytes (TBytes), and upstream's helpers Move Length(T) BYTES
  of the script's UnicodeString -- writing 'abcd' stored 61 00 62 00, and reading packed the bytes
  two to a character and left the rest unset. Bound on Delphi to routines reading and writing
  AsString, Delphi's own rule for an untyped blob (the ANSI code page; a wide memo UTF-16), on every
  version alike. Free Pascal keeps upstream's helpers (FPC_PATCHES routes it to their pre-2009 arm)
- TParam.AsBlob is declared to scripts as TBlobData = string, and from Delphi 2009 Delphi's
  TBlobData is bytes (TBytes; TArray<Byte> in 13 Florence, Data.DB), so upstream's helpers handed a
  dynamic array of bytes to the script's UnicodeString and back. Bound on Delphi 2009 and later to
  routines converting between the script's string and the bytes through the ANSI code page, as
  TBlobField.Value (above) does; Delphi 7 - 2007 (TBlobData a string) and Free Pascal keep
  upstream's helpers
- From Delphi 2009 TLookupList is abstract (Add, Clear, ValueOfKey) and TDefaultLookupList
  implements it, so a script creating a TLookupList got an object whose every list call raised
  EAbstractError. Created as TDefaultLookupList when the script names TLookupList itself, as
  Delphi's TField creates its list; the methods are already bound by TDefaultLookupList's slots
- An event property bound with RegisterPropertyHelper -- a script could assign it and stopped
  reading it (Could not call proc; the run time hands an event helper a TMethod, a plain helper the
  script's value)
- [Free Pascal] Fcl-db's TBlobField.Value is a string (AnsiString in its RTL), so Free Pascal takes
  the pre-2009 arm, Self.Value := T, a text conversion. (The 2009+ arm copies Length(T) BYTES of a
  UnicodeString -- half the text; pascalscript replaces it on Delphi,.) The floor pass closes the
  {$IF} with {$IFEND}
- [Free Pascal] As the previous entry, read side: T := Self.Value
- [Free Pascal] Fcl-db has no TDefaultLookupList; its TLookupList declares Add / Clear / ValueOfKey
  itself, as Delphi before 2009 did (the floor pass closes the {$IF} with {$IFEND})
- [Free Pascal] Fcl-db 3.2.2 db.pas 1685 declares TDataSet.Delete virtual (Delphi 13 Data.DB:
  static), so a by-address binding skipped a descendant's override; the class-helper form
- [Free Pascal] Fcl-db 3.2.2 db.pas 1078 declares TIndexDefs.Update virtual; the class-helper form
- [Free Pascal] (the first entry's TFieldDefsCreate_P)
- [Free Pascal] Fcl-db 3.2.2 db.pas 1056 declares TIndexDef.Expression with a setter, so the
  script's write took effect nowhere on FPC; on Delphi the statement is the one it already compiled
- [Free Pascal] (GO by pascalscript 08:59) -- fcl-db 3.2.2 db.pas 567 / 597 / 634 / 643 / 652:
  TIntegerField, TSmallintField, TWordField and TAutoIncField are siblings under TLongintField,
  which carries Value / MinValue / MaxValue; the run-side class the compile side now declares
- [Free Pascal] , TLongintField registered before its descendants
- [Free Pascal] A constructor routine in Delphi's shape receives the class first on Free Pascal too
  (the stubs' IsConstructor arm), so these Delphi-only routines serve it; without them the script's
  Create reached FPC's Create(ItemClass) with none. Delphi selects the same text
- [Free Pascal] Fcl-db's TIndexDefs.Create is virtual; bound by address it ran TIndexDefs' body for
  a descendant class and passed over the descendant's override. The constructor routine calls
  through the class reference. Delphi selects the same text
- [Free Pascal] Seven classes the compile side declares and fcl-db has (TNamedItem, TDefCollection,
  TWideStringField, TBCDField, TFMTBCDField, TVariantField, TGuidField) had no runtime class, so a
  script reaching one got erCouldNotCallProc with no message. Delphi selects the same text
- [Free Pascal] Fcl-db leaves TIndexDefs.FindIndexForFields unimplemented and its GetIndexForFields
  picks a different index than Delphi's; both bind routines with Delphi's rule. Delphi selects the
  same text
- [Free Pascal] Fcl-db has TField.ValidChars and the compile side declares it, so the run time binds
  it too; AutoGenerateValue and ParentField stay excluded. Directives only, between the same lines:
  Delphi selects the same text
- [Free Pascal] Fcl-db has these members (public, in 3.2.2), so the run time binds them too.
  Directives only between the same lines, plus FPC arms: Delphi selects the same text
- [Free Pascal] Fcl-db overloads TFieldDefs.Add with a Word size, so the script's one form is bound
  through a routine that refuses a size outside the Word range rather than truncate it. An FPC arm
- [Free Pascal] Fcl-db's TBlobData is TBytes in either mode while the script's AsBlob is a string,
  so Free Pascal takes the string-converting helpers Delphi 2009+ uses. Directive conditions only:
  Delphi selects the same text
- [Free Pascal] A script method bound to fcl-db's OnGetText / OnSetText gets its text in the
  script's string type where the event's own parameter (read from its RTTI) is an AnsiString. FPC
  arms only: Delphi selects the same text
- [Free Pascal] The script declared fcl-db members the run time never bound. What fcl-db does not
  declare is no longer declared to scripts; what it declares from 3.2.4 (PS_FCLDB_NO_OBJECT_FIELD
  below it) is declared and bound from there. Delphi selects the same text
- [Free Pascal] Fcl-db 3.2.2 db.pas 214 TFieldDefs.Create(ADataSet: TDataSet), 3.2.4 and trunk
  TPersistent
- [Free Pascal] An adapter native code may hold a copy of is never freed ('s D1 rule); the field
  owns every adapter (TPSFieldTextAdapter.Create(aField)), so none outlives it
- [Free Pascal] , the routine's comment
- [Free Pascal] , the class's comment
- [Free Pascal] The class library bound TDataSet's FindFirst & co. Statically where fcl-db declares
  them virtual
- A menu search value is as wide as Delphi's, and EMenuError, TMenuActionLink and TDataSource are
  registered at run time as they are declared
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant