Repository navigation
Fixes: Classes, DateUtils, std and ComObj import libraries - #330
Open
ElminsterAU wants to merge 6 commits into
Open
ElminsterAU wants to merge 6 commits into
ElminsterAU wants to merge 6 commits into
Conversation
These fixes come from NexusDB, which ships Pascal Script as its script engine. Each was found and fixed there; most were measured against what Delphi itself does with the same code, and most have a regression test in NexusDB's own suite, which runs on Delphi 7 through 13 Florence (Win32 and Win64) and on Free Pascal 3.2.2 (i386) and 3.3.1 (x86_64). NexusDB builds the engine with PS_USECLASSICINVOKE and PS_USESSUPPORT; the fixes are given here in Pascal Script's own unit names and configuration. PascalScript.inc: - Delphi 2005 and 2006 stopped with an internal error on a class derived from one a run-time library unit it uses has a helper for; their libraries take the procedure form, as Delphi 7 PascalScriptFPC.inc: - [Free Pascal] The script declared fcl-db members the run time never bound. What fcl-db does not declare is no longer declared to scripts; what it declares from 3.2.4 (PS_FCLDB_NO_OBJECT_FIELD below it) is declared and bound from there. Delphi selects the same text
These fixes come from NexusDB, which ships Pascal Script as its script engine. Each was found and fixed there; most were measured against what Delphi itself does with the same code, and most have a regression test in NexusDB's own suite, which runs on Delphi 7 through 13 Florence (Win32 and Win64) and on Free Pascal 3.2.2 (i386) and 3.3.1 (x86_64). NexusDB builds the engine with PS_USECLASSICINVOKE and PS_USESSUPPORT; the fixes are given here in Pascal Script's own unit names and configuration. uPSUtils.pas: - [older Delphi] NativeUInt only where a 64-bit target exists (XE2 on); before that IPointer stays Cardinal as it always was -- Delphi 2009 dies with internal error C12079 on NativeUInt here - For the GUI libraries' text members (TEXTBUF_REASON) and their characters (CHARPROP_REASON, KEYPRESS_REASON): a script's PChar buffer takes ANSI bytes, a script's Char is an ANSI character - A list's growth counted the new capacity before allocating it, and its periodic trim could raise after an item was added or removed, so an allocation failure left a capacity past the block, or a script stack slot counted but never initialized - The search of the units a unit uses walked every route through units that share the units they use, twice as long with each level of sharing - A comment the text ends inside was reported on the row the text ends on, at a negative column - '(*)' read as a closed comment where Delphi opens one - With EnableComments or EnableWhitespaces, the compiler's parser reported a comment or white space spanning rows at its last row and a negative column - Delphi XE3 - XE5 for Win64 leak the exception object of an except handler left by a jump; each such handler is left after its end - A name several modules declare is the nearest scope's; one module's own external routine, constant and type are checked against each other - AddBlock reads its source before the old block goes; the comments of three opcodes say what is read - PSTextToAnsiBuffer finds the beginning that fits by halves, not by converting once per character
These fixes come from NexusDB, which ships Pascal Script as its script engine. Each was found and fixed
there; most were measured against what Delphi itself does with the same code, and most have a regression
test in NexusDB's own suite, which runs on Delphi 7 through 13 Florence (Win32 and Win64) and on Free Pascal
3.2.2 (i386) and 3.3.1 (x86_64). NexusDB builds the engine with PS_USECLASSICINVOKE and PS_USESSUPPORT; the
fixes are given here in Pascal Script's own unit names and configuration.
uPSCompiler.pas:
- The field does not keep pointing at the freed list. No later use of it was found -- Compile
creates a new list and the destructor does not read it -- so this is hygiene, not the fix of a
known fault
- [older Delphi] Extended -> Double as a typecast is E2089 before Tokyo on Win32; the assignment
converts
- The varUInt64 constant for scripts, from Delphi 2009, where the RTL has it
- AddDelphiFunction ran its duplicate check even with AllowDuplicateRegister on (the default), and
FindProc marks the EARLIER registration of the name as used -- so a script compiled against the
FIRST declaration of a twice-declared routine while the runtime calls the LAST registration
(access violation or wrong value where they differ). Tested first, the flag skips the lookup and a
twice-declared name resolves to its last declaration again, as it did before 2014 upstream. (The
sibling checks in AddInterface and AddClass test the flag second too, but their lookups mark
nothing as used.)
- Upstream skips the temporary of a Char / WideChar cast when the source is a Byte / Word and writes
the source straight into the destination, so "a := Chr(w)" with w: Word and a: AnsiString stops
with Type Mismatch from Delphi 2009, and a Word reaching a Char takes another conversion than an
Integer. The previous release's condition compared the cast type with itself and so always took
the temporary; this states that intent
- A constant expression (a const declaration, an array bound, an attribute argument) took the LAST
registered constant of a name while an ordinary expression takes the FIRST
(TPSPascalCompiler.ProcessSub's constant loop runs 0 to Count - 1), so where two libraries a host
registers both declare one name the two read different values -- in the server and SQL hosts
`const C = faReadOnly` gave the DB field-attribute mask 2 where `faReadOnly` gave SysUtils' file
attribute 1. It also lacked the ordinary lookup's unit-visibility test (IsInLocalUnitList), so it
could take the constant of a script unit the unit being compiled does not use, which with the
first-registered order meant a sibling unit's enumeration member over the unit's own. Both lookups
now take the first VISIBLE constant of the name. Besides the host libraries' duplicates, script
enumeration members can repeat a name (ReadType checks them against types only, AddConstant
against nothing while AllowDuplicateRegister is True), and those follow the same rule; a const
block cannot repeat one (DoConstBlock refuses it)
- The compiler declares IUnknown and IInterface as two separate script interfaces of the IUnknown
GUID (DefineStandardTypes), and TPSInterface.IsCompatibleWith follows the source's ancestry by
object identity, so an interface descending from IInterface was refused by a variable or parameter
declared IUnknown, and one descending from IUnknown (IDispatch, IReadWriteSync) by one declared
IInterface -- in Delphi every interface is assignment-compatible with both.A target that is
IUnknown in all but name (IsUnknownInterfaceType, below: narrowed the test from the GUID alone,
which also took an interface declared with IUnknown's GUID and methods of its own) now takes any
interface; nothing that compiled before means anything else
- A var or out parameter takes a variable of its own type only, by object identity, so a variable
declared IUnknown was refused by a var parameter declared IInterface or an interface declared as
either (and each of those by the others), which in Delphi are one type. Found the same for the
other names the engine declares one Delphi type under: string and UnicodeString or NativeString
(AnsiString and tbtString, and string before Delphi 2009), NativeInt and the integer type of its
size, and dynamic arrays such as TBytes and an alias of it. IsSameScriptType (below) matches all
of them; distinct enumerations, sets, records, classes and interfaces with methods still take a
variable of their own type only
- The tests the IUnknown entries (IsCompatibleWith, IsVarInCompatible, CheckCompatProc) and the
alias entries (IsVarInCompatible, CheckCompatProc, ValidateParameters) share. Matched any two
plain types of one base type; measured that as too wide: AnyString (a parameter type whose
argument keeps its own type) matched AnsiString, so a Char reached a native AnsiString parameter
through a procedural variable; an open array parameter's type matched a dynamic array, whose call
omits the high bound; and TDateTime, a type of its own in Delphi, matched Double. Now only the
names the engine declares one Delphi type under match (ScriptAliasGroup), plus WideString and
UnicodeString before Delphi 2009, where tbtUnicodeString is WideString. Dynamic arrays still match
by element type, which is wider than Delphi's identity and of one representation. The WideString /
UnicodeString alias is keyed on eDefines.inc's WIDESTRING_EQUAL_UNICODESTRING, the property itself
(on Delphi exactly "not UNICODE"; on Free Pascal set with FPC_WIDESTRING_EQUAL_UNICODESTRING), not
on UNICODE, which also aliased them on Free Pascal Windows, where tbtUnicodeString is a
UnicodeString and tbtWideString a COM WideString
- A var or out array parameter compared the element types by identity, so an array of NativeString
was refused where an array of string is declared, which IsSameScriptType makes one type
- A routine fits a procedural type only when each parameter and the result are of the identical
type, so a routine taking an IUnknown did not fit a procedural type taking an IInterface, nor one
taking a UnicodeString a procedural type taking a string, and the like
- See the previous entry
- [Free Pascal] System has UTF8Decode (returning UnicodeString), not UTF8ToWideString
- [Free Pascal] Currency op real computed in the real type, as Delphi computes it (Free Pascal
rounds the real operand to Currency first where Currency is an Int64). FPC arms only: no Delphi
target defines FPC_CURRENCY_IS_INT64
- [Free Pascal] An error of a kind past the enumeration faulted on Free Pascal
- [Free Pascal] A hint of a kind past the enumeration faulted on Free Pascal
- [Free Pascal] A warning of a kind past the enumeration faulted on Free Pascal
- [Free Pascal] A code byte, and the routine's code before it, went through UTF-16 under a UTF-8
page
- [Free Pascal] An exported type byte (bt + 128) and the output before it went through UTF-16 under
a UTF-8 page, so the bytecode would not load
- [Free Pascal] The master routine's calls went through UTF-16 under a UTF-8 page
- [Free Pascal] The master routine's return went through UTF-16 under a UTF-8 page
- [Free Pascal] A block's closing pop took the routine's code through UTF-16 under a UTF-8 page, so
a constant of 200 in it came back as EF BF BD
- [Free Pascal] A routine number above 127 in the debug record went through UTF-16 under a UTF-8
page
- [Free Pascal] An interface method's offset above 127 went through UTF-16 under a UTF-8 page
- [Free Pascal] A local for the label record built below
- [Free Pascal] Under a UTF-8 page the record's first bytes were re-encoded, so every goto was
refused
- [Free Pascal] The join re-encoded the routine number already in the debug record
- [Free Pascal] As above
- [Free Pascal] The debug output joined records of differing pages
- A native routine's var open array parameter (`var a: array of Integer`) refused every script array
with "Type mismatch" -- ValidateParameters compares the element types, and the call it writes then
compared the whole types by identity, which a dynamic array never has with the parameter's own
open array type; the code that passes the array (the !OPENARRAY branch below) was never reached.
The previous release the same (read); 's array rule let it through by accident and 's exclusion of
'!' names took it away again. A static array stays refused (passing it would pass a copy)
- The call passed a var open array parameter through a pointer to a copy of the argument of the open
array's type (the type tells the call layer to pass the data and high bound) and freed that copy
-- popping it off the stack -- before the call, so the pointer named the slot the next parameter
reused and the call raised an access violation, on every convention, Win32 and Win64. The copy now
goes on the stack ahead of the result and the parameters and is released after them
- A native routine's open array of a class (`const aTables: array of TMyTable`, `Items: array of
TMenuItem`) was declared to scripts as an open array of TObject, so a script array of any objects
compiled and the routine then used them as the declared class. The element keeps its class now, as
a record or an enumeration element keeps its type (the next two lines). No shipped script passes
such an array (searched)
- Assigned refused a procedural or event variable and a dynamic array ("Type mismatch"), which
Delphi takes; the run time already answered for an array, and answers for a procedural value with
the uPSRuntime entry of this round
- "and" binds tighter than "or", so the UnicodeString alternative stood alone and a UnicodeString
was compatible with ANY destination -- an Integer, an object, an array element of any kind --
which compiled and then stopped the script with "Type Mismatch". The second line keeps the one
destination that worked: a Char takes a UnicodeString's first character at run time, and from
Delphi 2009 a script's `c := s` relies on it
- An array constructor's elements were written into the array without a check against its element
type -- the constructor itself is typed as an array of Variant, which matches anything -- so
`[TObject.Create]` passed for an open array of a class (kept the class) and the routine used the
object as that class
- A set constructor element of no type (`[nil]`) was dereferenced: an access violation inside
Compile
- A cast written into a destination of another type was compiled as an assignment of its operand to
that destination, so the destination's type decided the value: `i := Byte(x)` with x = 300 gave
300 (Delphi 44), `Result := Byte(a)` and `RI(Byte(x))` the same, `i6 := Integer(c)` with c =
$FFFFFFFF 4294967295 (Delphi -1), `v := Integer(w)` of an empty Variant stayed empty (Delphi 0),
`d := Single(x)` kept the Double. Made the character casts go through a temporary of their own
type; every cast whose destination is of another kind now does
- Completing : an array constructor element for an array of a class may be a number or a Variant by
IsCompatibleType, and the object pointer was then that number -- `SumValues([v])` with v a Variant
holding 0, or an Integer on Win32 / an Int64 on Win64, raised an access violation in the native
routine; an element for an array of an interface was not checked at all, so an IUnknown went where
an IFoo was declared. An object stays allowed there (refused it): SetVariantValue queries it for
the interface, and a nil object gives a nil interface
- An index of no type (`a[nil]`, `s[nil]`) was dereferenced: an access violation inside Compile
- `s[1] := nil` dereferenced a value of no type: an access violation inside Compile
- `Pos(nil, 'x')` dereferenced a value of no type: an access violation inside Compile
- `Length(nil)`, `Low(nil)`, `High(nil)`, `Include(s, nil)` dereferenced a parameter type of no
type: an access violation inside Compile
- `High(e)` of a variable of a two-member enumeration gave 255 (Delphi 1) -- the compiler writes an
enumeration as an integer of its size, and High_ answers by that integer's range. Low needs no
such entry: an enumeration starts at 0. Keyed on the built-in (an untyped parameter), and ByteBool
/ WordBool / LongBool left to the run time as before, whose HighValue is a size marker (answered
2147483647 for a LongBool)
- The constant of the previous entry
- Comparisons of constants still read the right operand in the left one's width and sign -- `Byte(0)
< Integer(-1)` and `Cardinal($FFFFFFFF) = Integer(-1)` folded to True -- and an integer compared
with a Single was first rounded to Single
- Ord was typed as an unsigned UInt64, so `Ord(a) - Ord(b)` was never negative, and since a cast
keeps its own type, `d := Ord(-1)` gave a huge number in a Double (Delphi 7 stopped with "Math
error")
- A routine returning the type of its string argument (Copy, Trim and the case routines) typed its
result as a Char when given a Char, so `Copy(c, 2, 1)` was a one-character value where Delphi
gives an empty string
- A comparison of an AnsiString or Char constant with a wide one read the wide one through the ANSI
code page -- `'A' = #$0100` folded to True and `'A' < #$0100` to False, where variables compare as
UnicodeStrings since. And two Currency constants went through Extended, a Double on Win64:
`Currency(900000000000000) + Currency(0.0001)` lost the 0.0001 there, where variables keep it and
Delphi does
- The sum of a Char or AnsiString constant and a wide one was an AnsiString, the wide one read
through the ANSI code page -- `'A' + #$0100` gave 'AA' on a Western European system, where
variables give U+0100
- A 32-bit constant divided by a 64-bit one read the divisor in 32 bits -- `Integer(10) div
Int64(4294967297)` folded to 10 (divisor 1), where variables give 0, and a divisor whose low half
is 0 was refused as a division by zero
- `mod` as `div` in the previous entry
- A negated unsigned constant kept its width -- `-Byte(1)` folded to 255 and `-Cardinal(1)` to
4294967295, where a negated variable passed to an Integer or Int64 gives -1 and Delphi gives -1
- A constant declaration, as the previous entry -- `const K1 = $FFFFFFFF; K2 = -K1;` gave K2 = 1 (a
script constant declaration refuses casts, so `-Byte(1)` cannot be written there); reason
corrected in
- `not` of a Boolean constant inverted its lowest bit -- `not ByteBool(2)` folded to True (Delphi
False), and `not WordBool(256)` kept the high byte -- where the run time tests the whole value
- A constant declaration, as the previous entry
- A constant cast to an enumeration kept 32 bits -- `Ord(TEn(256))` of a two-member enumeration
folded to 256, where the same value through a variable gives 0 and Delphi gives 0
- An integer not written into a destination of another type was computed in that destination -- `i
:= not b` with a Byte 0 gave -1 (Delphi 255), `RL(not c)` with a Cardinal 0 gave -1 (Delphi
4294967295), while the constant `not Byte(0)` folded to 255; entry is the same shape
- The built-in High's registration, which the High constant of keys on
- Records the built-in High, the previous entry
- Completing change to 's High entry: a host routine named High declared with one untyped parameter,
as the built-in is, was taken for the built-in and never called
- A WideChar argument of Copy, the padding routines and the engine's own Trim and case routines went
through their AnsiString arm, which DefProc picks by the argument's type -- `Copy(wc, 1, 1)` of
U+0100 gave 'A' on a Western European system; and a PChar argument was refused, where Delphi takes
it
- Locals of the next entry
- A call of such a routine stayed typed AnyString, an AnsiString, until its code was written, so as
an operand its result went through an AnsiString temporary -- `Copy(us, 1, 1) = us` with us U+0100
was False and `Copy(Copy(us, 1, 1), 1, 1)` gave 'A'
- An index expression of a 64-bit type was narrowed to a Cardinal before the bounds check -- `a[i6 +
0] := 7` with i6 = 4294967296 wrote a[0]
- StringOfChar and Replicate declared their character as the script's one-byte Char, so
`StringOfChar(wc, 2)` of U+0100 gave 'AA' on a Western European system
- A negated unsigned variable kept its type, so as an operand it was computed in a temporary of that
type -- `(-b) < 0` with a Byte 1 was False (255), `(-c) < 0` with a Cardinal 1 False -- while
`(-Byte(1)) < 0` folds True since
- A constant index was read in 32 bits -- `ar[Int64(4294967296)] := 7` wrote ar[0]
- A static array's index went through a 32-bit subtraction of its start -- `sa[i6] := 7` with i6 =
4294967296 wrote sa[0]
- The constant index of an earlier entry
- A Byte, ShortInt, Word or SmallInt constant div by a 64-bit one read the divisor in 32 bits --
`Byte(10) div Int64(4294967297)` folded to 10
- A Byte, ShortInt, Word or SmallInt constant mod by a 64-bit one read the divisor in 32 bits --
`Byte(10) div Int64(4294967297)` folded to 10
- A negated ShortInt or SmallInt constant kept its width -- `-ShortInt(-128)` folded to -128 --
where a negated variable into a wider destination gives 128, as Delphi does
- A Currency constant with an integer one went through GetReal, a Double on Win64 --
`Currency(400000000000000) + Currency(0.0001) + 1` lost the 0.0001 there, and the same constant
compared with 400000000000000 was not greater, while the run time computes both with the integer
in its own type
- Round and Trunc were declared LongInt, so `Round(5000000000.4)` gave 705032704; Delphi's return an
Int64
- High of a UInt64 variable was typed Int64, so `High(u) > 0` was False (the run time's High_ stores
the UInt64 bound)
- With a WideChar parameter a script Char arrived by its ordinal, so `StringOfChar(c, 2)` of the
byte $80 gave U+0080 where the Char converted to a string is U+20AC (on Delphi 7 it went back
through the code page as "?")
- The end value's temporary of the next entry
- A for loop compared its counter with the end expression on every iteration, so `for i := 1 to n`
with n changed in the body ran to the new n and `for i := 1 to F` called F once per iteration and
once more, where Delphi computes the end once, before setting the counter
- The end value's temporary freed after the loop's own (they are freed last first)
- Break and Continue inside try / finally, try / except or an except handler ran the finally and
then went on at the end of the try statement, in the loop -- `for i := 1 to 3 do try if i = 2 then
Break;... Finally... End` ran its third iteration
- Break and Continue popped one value per with statement, and a case statement turned their jump
into one popping two, so `with r1, r2 do` (two values) and a case within a case (four) left values
on the stack, and the procedure's variables read wrong after the loop (a local set to 7 read 0;
measured on 13 Florence Win64 and Delphi 7)
- A with no longer counts itself for Break and Continue (the previous entry)
- A case no longer rewrites the jumps of a Break or Continue in it (two entries above)
- The field the Break and Continue entries above read
- Each loop records its stack depth
- The field holds a stack depth now, not a count of with statements
- A constant div of a dividend and a divisor of different signedness read the divisor in the
dividend's sign -- `Byte(10) mod Integer(-3)` folded to 10 (Delphi 1), `Integer(10) div
Cardinal(4294967295)` to -10 (Delphi 0); measured on 13 Florence Win64 and Delphi 7
- A constant mod of a dividend and a divisor of different signedness read the divisor in the
dividend's sign -- `Byte(10) mod Integer(-3)` folded to 10 (Delphi 1), `Integer(10) div
Cardinal(4294967295)` to -10 (Delphi 0); measured on 13 Florence Win64 and Delphi 7
- The field of the next entries
- The saved depth of the next entry
- The outermost try statement within a loop records its depth
- The compiler half of the run-time position 4 entry above
- The depth of the next entries
- As the next entry
- A bound reading a field of a function's record result leaves its value on the stack, and freeing
it after the temporary was allocated popped the temporary -- `for i := First.N to n` stopped with
"Out Of Stack Range"
- The registration of Low, which the next entries key on
- Low of a UInt64 variable was typed Int64, so `(Low(u) - u) > 0` with u 1 was False where Delphi's
UInt64 subtraction wraps
- Two ByteBool, WordBool or LongBool values were combined and compared as integers, so the WordBools
256 and 1 gave False for `and` and `=` and True for `xor`, where Delphi gives True, True and False
- The variables of the next entry
- A ByteBool, WordBool or LongBool value went into a Boolean -- an if, while or until condition, an
assignment, a Boolean parameter -- by its low byte, so `if wb` with the WordBool 256 took the else
branch and the LongBool -1 became the Boolean 255
- The temporaries of the next entries
- 's depth is no longer needed (next entry)
- A for loop computed its end before its start, where Delphi computes the start first; and a bound
reading a field of a function's record result left a value on the stack that was freed before the
loop's own temporaries, so `for i := 1 to F.N` stopped with "Out Of Stack Range" and `for i := F.N
to n` followed a changed n
- The bounds stay alive to the loop's end and are freed last first (the previous entry)
- A while condition reading a field of a function's record result (`while F.N < 2`) materialized the
field before the loop's start, so the loop compared a saved value on every pass
- The condition was freed inside the loop (two entries above)
- `repeat... Until F.Done` (a field of a function's record result) went with Continue past the code
computing the field (Out Of Stack Range), with Break popped values the iteration never pushed, and
kept the field on the stack on every pass back
- The selector of the next entries
- `case F.N of` (a field of a function's record result) freed the selector while the case's
temporary lay above its values, so the compiler lost the temporary and stopped with an access
violation
- The registrations of Include and Exclude, which the next entries key on
- The set type of the next entry
- The run time's check by set size let a Char into a set of Byte and into a set of an enumeration of
256 members, setting the member of the Char's ordinal
- A non-constant start was computed after the end was parsed, and parsing an end reading a field of
a function's record result calls the function, so `for i := StartAt to EndAt.N` called EndAt first
- The start is computed above now
- The end's values lie above the start's temporary now (the previous entries)
- The test of the next entry
- It left every constant operand to the integer operation, so `wb and WordBool(1)` with the WordBool
256 was False and `wb xor WordBool(1)` True
- A cast `Boolean(wb)` is typed Boolean, so the assignment's truth test did not see it, and the cast
copied the WordBool 256 by size, False
- Into a destination of another Boolean type it read the value's truth before preparing the
destination, so `a[IndexAt] := lb` with IndexAt changing lb stored the old truth
- `case wb of wb2` (a label the engine accepts) lent ProcessCase's wrapper to the truth test, whose
cleanup freed it under the case's comparison, which freed it again
- `F('x').N` with F taking an Integer recorded a type mismatch while emitting the call and compiled
anyway, reading the field of a result the call never filled
- `Exit(F())` compiled as an Exit and a parenthesized call after it, so the routine returned without
calling F (Delphi 2009 and later return F's value)
- An open-array constructor copied each element by size, so `[wb]` for an `array of Boolean` with
the WordBool 256 gave the Boolean 0
- `s := 'b' + IntToStr(StrToInt(5))` (an operation whose type is not its destination's, with an
operand that fails to compile) compiled without an error and assigned `'b'` alone -- this area's
probe during
- The comment of the next entry
- It left every operation with a False constant to the integer operation, so `wb or WordBool(0)`
with the WordBool 256 was the WordBool 256 where Delphi's or gives a Boolean (Ord 1)
- `with F('x') do` (F taking an Integer) recorded a type mismatch, compiled anyway and emitted a
cm_sp without its source, so the script ran and did nothing
- The test of the next entries
- `WordBool(256) and WordBool(1)` folded to False (Delphi: True)
- It went through a temporary only when the destination's base type differed, and every Boolean type
is btEnum, so `w2 := Boolean(n + 0)` with n 256 gave the WordBool 256 (True) where Delphi and the
previous release give False
- This exit freed the start's temporary before the end's values, which lie above it (a rejected
script; the frees are counted, so no accepted script changes)
- An open-array constructor copied each element into the array without the assignment's
compatibility check, so `[256]` for an `array of Boolean` gave False and `[e1]` entered an `array
of Integer` (the previous release did the same)
- StringOfChar and Replicate take a character, as in Delphi; their AnyString parameter took a string
too (its first character repeated, an empty one gave #0), which the previous release refused
- A set constructor element of an open array of a set type was built as an array, and its assignment
to the element stopped with a type mismatch (`Take([[e1]])`; both releases)
- The minus or not of a Variant written into a variable of another type converted the Variant first,
so `s := -v` into a string stopped with Type Mismatch, where Delphi negates the Variant and
converts the result
- A unit's text without a unit header took the kind of the file that used it, and from the program
its block was added to the program's block, after whose return the program's own block never ran
- An explicit `uses System` did not find the System entry (mixed case against the parser's upper
case) and ran the host's imports again
- A unit that failed to compile left its parser behind, an exception from the unit finder or the
unit left the unit's name as the module of every later message, of later compilations too, and an
exception out of a unit's compilation left its nesting counted, so the next compilation took the
nested branch
- LOCAL FIX (with the entry above): what a unit's compilation changes, saved to give back
- A failing System import left the module 'System' for every later compilation's messages
- A unit's parser reported no lexical error, which ended its text instead
- An unsatisfied forward declaration in a unit was reported at its place in the unit but named the
main script
- A warning made while a unit compiled named no module, where an error names the unit
- An exception out of a compilation left the compiler nested, so every later compilation on it
failed
- LOCAL FIX (with the entries around it): the release of what a compilation made moves to
ReleaseCompileState, so that Compile can run it when an exception ends a compilation
- The release of what a compilation made, run when it ends normally, on an error or by an exception,
and completed when the host's OnBeforeCleanup raises; it also drops the Break and Continue lists
of a loop an exception left, which the next compilation took for a loop around a Break outside any
- The System import's exception handler cleaned up again after its failure exit had, and released
the lists twice
- An integer literal no 64-bit integer holds compiled as 0
- A real literal past the range of its type raised EOverflow out of the compiler where the
floating-point overflow exception is unmasked
- A character literal past $FFFF was truncated, where Delphi makes the surrogate pair or refuses it
- ReadConstant used ReadString's nil -- an access violation out of Compile for a refused string
constant
- A real constant past a type's range stopped Compile with an exception, or was read wrapped
- -- comments made true
- -- a comment made true
- [Free Pascal] _FpuMaskAll / _FpuRestore get Free Pascal arms, the Delphi bodies under {$ASMMODE
INTEL}. FPC arms only: Delphi selects the same text
- A constant past a Single's, Double's or Currency's range put into one without a cast was converted
when the script ran
- -- a constant element of an array constructor converted as an assignment is
- Constant folding ran under the host's floating-point setting -- a real past its range refused the
compile, a Currency past its range was a wrong value
- A real literal read on Win64 left the RTL's DefaultMXCSR with every exception masked
- A folded comparison with a NaN answered as the compiler that built the engine compares one
- -- a Currency constant could hold the minimum, which Delphi refuses, its negation was unchecked,
and an integer compared with a Currency as reals
- -- a Currency constant rounded onto the minimum was taken, and the negation of the least Integer
or Int64 constant wrapped
- An operation of two integer constants past the range of the left one's type wrapped, where Delphi
refuses the constant
- On Win64, where Extended is a Double, a real constant put into a Currency was stored inexactly:
the conversion multiplied a Double by 10000, which past about 900000000000 loses the
ten-thousandths (461168601842738.5 gave.4832), and Florence's Val reads some literals of more than
fifteen digits a unit in the last place off (922337203685477.5 as.625, so it was refused) and
cannot hold more than about sixteen. Delphi converts such a literal exactly on Win64
(P133CurDelphi.dpr). Now a real literal keeps its text -- through a negation and into a declared
constant -- and is read exactly, rounded half to even at the fourth decimal, wherever it is put
into a Currency: assigned, passed, cast or an array constructor's element (Delphi's Win32 answer
on every case measured; Florence's Win64 compiler goes through a Double for a literal of more than
four decimals or with an exponent). A literal whose exact value is past the range is refused
though it rounds back into it (922337203685477.58071), as Delphi's Win32 compilers refuse it. Any
other real constant is converted exactly from its Double, as Win32 does. A literal within 64
characters of the longest string is refused: its counts would pass the range of an Integer ().
- A for statement took a call as its counter -- through a procedural variable of a function type
(for f(7) := 1 to 2) or to a method returning an ordinal (for sl.Add('x') := 1 to 2): the counter
was a temporary holding the result, and the routine was called again at every check, so each check
compared a fresh result and the loop could run for ever (the 4.75.20 engine too). Delphi refuses
it (E1019: a for counter must be a simple local variable). Now refused with Variable Expected,
after the counter's type checks, so every other counter keeps its message (a procedure's call:
Type mismatch).
- A constant of a record or class type -- one a host registers (AddConstantN('C', 'TObject')); a
script cannot declare one -- as a with subject passed the record-or-class check as data, which
WriteOutRec cannot point at, and the statement failed WITHOUT A MESSAGE: the compile failed and
the host had nothing to report. The 4.75.20 engine compiled it and the script ended at the with,
Execute reporting success. Now refused with Variable Expected. (Such a constant cannot be used
anywhere -- an assignment from it stops the script with Invalid Type, before and after: a pass of
its own.)
- A call through a procedural variable of a type whose result is AnyString (type TF = function(S:
AnyString; I: LongInt): AnyString) stayed typed AnyString, an AnsiString, so as an operand its
result went through an AnsiString temporary -- f(us, 0) = us with us U+0100 was False where the
direct call gives True. ReadProcParameters' retyping is now one routine both roads call. 's
character arm in _ProcessFunction stays: a property getter's call is built by CheckClass /
CheckClassArrayProperty, typed by the getter's declaration, so a host property typed AnyString
read with a character index still reaches it.
- A property getter typed AnyString (RegisterProperty('Item', 'AnyString AnyString', iptR)) read
with a PChar index got a PChar result slot, while its index went as a string -- the native
helper's string landed in a slot the run time reads up to its first #0 and never releases as a
string. 's arm in _ProcessFunction maps a Char and a WideChar; a getter reaches only that arm,
since CheckClass / CheckClassArrayProperty type its call by the declaration and do not call
SetAnyStringResultType. The arm now maps a PChar as SetAnyStringResultType does
- An expression that failed to compile into a destination whose index is computed (a[Id(0)] :=
-Id('bad')) left the destination's replacement unfreed -- DoUnCalc, DoBinCalc and WriteCalculation
prepared it (PreWriteOutRec) and their failing exits returned without releasing it
(AfterWriteOutRec): four blocks per refused compile, fifteen where a comparison's prepared operand
was left in its node (the operand it replaced never freed), five where a nil operand's replacement
was not unwrapped. Now DoUnCalc releases the destination it prepared on every way out, DoBinCalc's
comparison undoes each preparation that succeeded and unwraps its nil operands whatever the
outcome, and the other failing exits release what they prepared
- A with statement released its subjects' preparations at its end -- only the LAST subject's, since
each subject overwrote the variable holding it, and none when the body did not compile or a later
subject failed. Two subjects whose address is computed (with a[i].S, b[i].U do) left the first
one's temporary on the stack -- the second pass of a loop around it stopped with Null Pointer
Exception -- and each compilation left the first's replacement allocated; a refused body left them
all. Now every subject's preparation is kept in a list and released at the statement's end and on
every failing exit. The release stays AFTER the body: a function result's preparation is the
storage holding the result the body reads (releasing it once the address was written read popped
storage
- `x as T` copied the object into its destination and checked the class there, so a cast that failed
raised with the object of the wrong class already stored -- `l := o as TStringList` with `o` a
TObject left `l` holding it, the value it had before lost. Delphi checks first and leaves the
destination as it was. Now the object is copied into a temporary, checked there, and only then
copied into the destination. The 4.75.20 engine compiled `as` as an unchecked cast
- An `external` declaration the host's handler refused was reported twice -- the handler's error,
then an error of the compiler's own with no text ("[Error] (2:1): Invalid External [Error]
(2:1):"). The compiler now adds its own error only when the handler reported none
- An `external` declaration in a host that sets no OnExternalProc -- such as a TPSScript with
neither TPSDllPlugin nor a handler of its own -- was refused with "Semicolon (';') expected" right
after its semicolon. It is now refused with "External routines are not supported"
- A message a unit finder made itself while the compiler asked for a unit named that unit, at the
place in the module that uses it
- A message a host made in its OnUses after a unit with a uses clause compiled named that unit, at
the place in the module that uses it; and a hint a host made with no module while a unit compiled
was the main script's
- With the preprocessor on, AdjustMessages placed a message it had placed already a second time,
reading its place in its file as a place in the preprocessed text; a message is now placed once
whoever asks
- A message AdjustMessage could not place yet -- its text not preprocessed yet -- was marked placed
and never placed afterwards
- The negation of a UInt64 constant other than 0 wrapped where Delphi refuses it
- An attribute value of a constant whose type nothing else used made the compiled script fail to
load
- A host constant referring to a type nothing else used was written with an invalid type number
- A static array bound outside the range of a Longint was cut to 32 bits, and the length was
computed in a Longint
- An unknown name in a constant expression or a set type was reported without the name
- An array of an unknown element type was refused a second time, with an error naming nothing
- AddTypeS leaked its parser when it refused a name declared already
- Delphi XE3 - XE5 for Win64 leak the exception object of an except handler left by a jump; each
such handler is left after its end
- An array constructor is refused for a static array, and one nested in an array of arrays is built
as the element type
- A name repeated in a parameter group, a record field list or a script's enumeration is refused, at
the name
- A name repeated in a procedural type's or a host declaration's parameter list is refused
- A name several modules declare is the nearest scope's; one module's own external routine, constant
and type are checked against each other
- A name several modules declare is the nearest scope's whatever its kind, for a routine a host
registers per unit, @name and a constant expression too
- A routine the compiler calls for itself is found by a lookup no script routine can answer;
AddDelphiFunction outside a compilation refuses
- Succ, Pred, Assigned, Break and Continue are the script's where it declares the name
- A routine's number is taken before the host hears of its use; `@Name` and a constant expression
give way to a routine's parameter; a for counter is no with subject's member
- A forward routine's implementation names its parameters as its declaration does
- A Variant's method gets an empty result; an array element that is a call of a routine with no
result is refused
- Two values of one enumeration type, a Boolean among them, compare with <, >, <= and >= by their
ordinals, as in Delphi
- A routine of a module the one being compiled reaches only through another unit, or not at all,
comes after a host's routine of its name, as a constant of such a module does; it is still found
where nothing else has the name
- `+`, `-` and `*` of two set constants wrote into the text the left constant shares, changing that
constant for every later use
- TPSConstant.SetSet wrote into the text a constant declared from it shares, so a host setting a set
constant while a script compiles changed that constant too
- AddClass completes a class declared ahead, as AddClassN does, and the class it returns knows its
native class, so RegisterPublishedProperties declares its properties
- IsValidResource answers for the routine and the data together, and TPSAttributes.FindAttribute
takes a name in any case
- With AllowDuplicateRegister False, AddClassN refuses a class AddClass registered with no ancestor,
as AddClass does
These fixes come from NexusDB, which ships Pascal Script as its script engine. Each was found and fixed
there; most were measured against what Delphi itself does with the same code, and most have a regression
test in NexusDB's own suite, which runs on Delphi 7 through 13 Florence (Win32 and Win64) and on Free Pascal
3.2.2 (i386) and 3.3.1 (x86_64). NexusDB builds the engine with PS_USECLASSICINVOKE and PS_USESSUPPORT; the
fixes are given here in Pascal Script's own unit names and configuration.
uPSRuntime.pas:
- [older Delphi] Low() of a string exists from XE3 (E2198 on 2009.. XE2)
- The compiler declares StrGet2 (StrGet with S by value) and the runtime never registered it, so a
script calling it did not load. StrGet's handler reads a value parameter as well as a var one
(NewTPSVariantIFC dereferences only a btPointer)
- [older Delphi] The RTL has UIntToStr(UInt64) from 2009 and StrToUInt64 / StrToUInt64Def from XE6;
older compilers get these, built on the helper PSTryStrToUInt64
- [older Delphi] On XE2.. XE5 Win64 a script routine called as a method received its floating point
parameters as 0; with the registers stored directly the row passes on all 37 cells. That the cause
is how those assemblers encode movq r64, xmm is a HYPOTHESIS -- the emitted bytes were not read
- [older Delphi] As above, for the floating point result
- [older Delphi] Unqualified, so that it binds to the function above where the RTL has none
- [older Delphi] Str of a UInt64 needs an RTL helper Delphi 2005 does not have (E2158 @str0uint64)
- [C++Builder] The four aliases move below their class bodies (next two patches)
- [C++Builder] XE3 / XE4 header emission
- A script reading an event property that holds a handler Delphi assigned gets a procedure-pointer
value with number 0 and the handler stored beside it (the read paths store it, and Cm_cv calls
it), but every classic call stub (x86.inc, x64.inc) and the published-property write rebuilt the
method from the number alone, so writing the value back cleared the event. Upstream's own
InvokeCall.inc uses the stored method; this helper does the same for the classic call paths
- As above: the published-property write
- A cast of a nil interface to another interface type stopped the script with "Cannot cast an
interface", where Delphi's cast gives nil (System._IntfCast; measured on Delphi 7 and 13 Florence,
Win32 and Win64)
- As the previous entry: a nil object cast to an interface type
- Assigning a nil object to an interface variable stopped the script with "Interface not supported",
where Delphi assigns nil; the cast of already did
- A published property of Delphi type WideChar declared to scripts as Char -- TEdit.PasswordChar
from Delphi 2009 -- was read and written by the published-property fallback as its low byte
(U+0430 read as '0'). Converted through the ANSI code page both ways, as every script character; a
property of Delphi type AnsiChar is unchanged. And: LOCAL FIX of an upstream defect: the
published-property fallback stored an enumerated value outside the property's Delphi type --
TBorderStyle is bsNone..bsSingle in Delphi (Delphi 7 Forms.pas, 13 Florence Vcl.Forms.pas) and
declared to scripts as TFormBorderStyle (six values), so TEdit.BorderStyle := bsSizeable stored 2
and CreateParams then indexed BorderStyles[FBorderStyle], a two-entry array (Vcl.StdCtrls.pas).
The compiler has no subrange type, so the script type stays; the value is refused (Out Of Range)
where it lies outside the RTTI range. Measured D7, D2009, 13 Florence Win32 / Win64: TBorderStyle
0..1, Boolean 0..1, ByteBool / WordBool / LongBool the whole Integer range, so no Boolean value is
refused
- The published-property fallback moves a script set of any size through a four-byte Longint, so a
script set wider than four bytes overran it (and on reading wrote past it into the script's
variable); a published Delphi set is at most four bytes, so such a script set does not match the
property -- refused (Out Of Range). No shipped declaration has the shape
- [Free Pascal] No Dec on a Variant (upstream does not compile on Free Pascal here either)
- [Free Pascal] No Inc on a Variant
- FREE PASCAL (an upstream compile defect on every platform that defines empty_methods_handler): the
interface declared ResultAsRegister while its body is compiled only with a real
MyAllMethodsHandler. The define block moves here from the implementation (next entry) so the
declaration can test it. Delphi-invisible on Win32 / Win64, where the block defines nothing
- [Free Pascal] AlwaysAsVariable sent an 8-byte record BY REFERENCE while x64.inc's FPC arm and Free
Pascal's ABI pass it by value, so the callee read the pointer as the record; and it sent a static
array of 1, 2 or 4 bytes BY VALUE, where x64.inc's FPC arm passes every static array by reference.
Delphi-invisible
- [Free Pascal] The define block moved to the interface (previous entry)
- [Free Pascal] A stated gap: on Free Pascal x86_64 upstream's MyAllMethodsHandler is empty, so a
script method bound to a native event did nothing and returned whatever the register held. Binding
one is refused with a clear error instead; porting the handler is the parity follow-up
- [Free Pascal] The message of the refusal above
- [Free Pascal] An Int64 result of a script method called from native code was lost (the FPC stub
returned EDX = its return address); Free Pascal's i386 register convention is Delphi's, so it
takes the Delphi stub and ResPtr. Delphi-invisible
- [Free Pascal] The comment described a constructor called with Self = 0 in EAX and the class in
EDX; Free Pascal 3 passes the class first and 1 second, as Delphi does. A comment inside {$IFDEF
FPC}: Delphi-invisible
- FPC 3.2.2 has no StringOfChar for a wide character, so the string is built from it here (the floor
ruling's house substitute; Delphi-invisible: FPC_FULLVERSION is undefined there)
- [Free Pascal] The record rule's comment and AlwaysAsVariable follow CONSTREC64 (every record by
reference; no method handler is compiled there). FPC arms only: Delphi selects the same text
- [Free Pascal] Currency op real computed in the real type, as Delphi computes it (Free Pascal
rounds the real operand to Currency first where Currency is an Int64). FPC arms only: no Delphi
target defines FPC_CURRENCY_IS_INT64
- [Free Pascal] BtUnicodeString shares the btWideString arm only where tbtWideString and
tbtUnicodeString are one type (WIDESTRING_EQUAL_UNICODESTRING); Delphi selects the same text
- [Free Pascal] A script method called from native code read a value static array of 1-4 bytes from
a register (Free Pascal passes it on the stack), returned one of 1-4 bytes in a register (Free
Pascal takes every static array through the hidden result pointer) and took a record result of 1,
2 or 4 bytes through that pointer (Free Pascal Win32 returns it in EAX). Non-Delphi arms only:
Delphi-invisible
- [Free Pascal] A script's TIFException(30) passed to ExceptionToString faulted on Free Pascal where
Delphi answers 'Unknown error'
- A dynamic array assigned to itself through two var parameters naming one variable (`procedure
Assign(var d, s: TInts); begin d := s; end;` called as `Assign(a, a)`) came out empty -- the array
branch releases the destination before it reads the source. Measured on 13 Florence Win64 / Win32
and Delphi 7
- SetLength to an array's own length left it shared, so after `b := a; SetLength(a, Length(a)); a[0]
:= 5` b[0] was 5 where Delphi leaves it; assigning a Variant array to an array of its length went
through the same routine
- A PChar assigned from a PChar took the text of a string made from it, which is freed when the
assignment ends, so it pointed at freed memory (the copy read back empty: measured on 13 Florence
Win64 / Win32 and Delphi 7)
- A Variant assigned to an object variable wrote four bytes of the pointer, so on Win64 an
Unassigned Variant left an object variable non-nil
- Before Delphi 2009 a script UnicodeString is a WideString (tbtUnicodeString), yet an array of
WideString assigned to an array of UnicodeString, or back, stopped the script with "Type Mismatch"
-- the element base types differ by number only. The var-parameter half is 's IsSameScriptType.
Keyed on WIDESTRING_EQUAL_UNICODESTRING, as IsSameScriptType
- Assigned of a procedural value, the run-time half of the compiler entry of this round
- Two array types whose elements are one record type were refused ("Type Mismatch") as not of one
base kind, so a dynamic array of records passed to a native routine's open array of that record --
const or, since, var -- stopped the script before the call
- An array of const element without data -- an unset element of a script array passed as one -- is
passed as a nil pointer, and the cleanup after the call read its (nil) type: an access violation
- SetLength to a length whose size overflows a Longint (1073741825 Integers) allocated the
overflowed size and initialised the whole length past it. The bound is the engine's (its sizes are
Longints), not Delphi's: on Win64 Delphi computes the size as a NativeInt
- SetLength of an array to a negative length emptied the array, where Delphi raises a range error
and leaves it
- A static array type whose size passes a Longint (array[0..8191] of an array of 65536 Int64s)
wrapped its size, to 0 there, and a dynamic array of it allocated nothing and initialised the
whole type; such a script is now refused when it loads ("Invalid Type")
- A Variant holding one character of text, assigned to a Byte, a Word or a Cardinal, gave the
character's code -- `v := '7'; b := v` gave 55 (Delphi 7) -- and longer text stopped the script,
while an Integer converted the number
- A comparison of two integers read the right operand in the left one's width and sign -- Byte 0 <
Integer -1 was True, Cardinal $FFFFFFFF = Integer -1 True, Integer -1 < Cardinal $FFFFFFFF False,
Cardinal 1 > Int64 -1 False -- and a Byte, Word, Cardinal or Int64 compared with a real stopped
the script with "Type Mismatch". Used by the next entry. A Currency compared with a real read the
real as a Currency (0 = 0.00004 was True), and an AnsiString or Char compared with a wide string
read the wide one through the code page ('A' = U+0100 was True with the AnsiString on the left,
False the other way round)
- The comparison of integers by value, the previous entry
- Include of a member past the set's size (`Include(s, TEn(8))` for a set of a two-member
enumeration) set a bit in the memory after the set -- the next field of a record
- Exclude, as Include in the previous entry
- The size check of a record type, used by the next entry
- Completing : a record whose field sizes sum past a Longint (four 1 GB static arrays) wrapped its
size, to 0 there, and a static array of it then passed 's check; such a script is refused when it
loads ("Invalid Type")
- A variable whose size is near 2 GB (a static array of 536870911 Integers, which 's check allows)
or several large ones carried the stack's Longint length past High(Longint), and the value was
initialised past the memory allocated
- The size with its type pointer would not fit Push's Longint, the previous entry
- The 64-bit index of the next entry
- An array index variable of a 64-bit type stopped the script with "Type Mismatch" -- `a[i6]` with
i6 an Int64 holding 0; the compiler accepts any integer type as an index
- Pos picked its arm by the substring's type, so `Pos('A', us)` with us = U+0100 + 'A' searched an
AnsiString copy of us and gave 1 (Delphi 2; measured on 13 Florence Win64 and Delphi 7)
- The character of the next entry
- WStrSet's character parameter is AnyString, so a WideChar argument now arrives as a UnicodeString,
which Stack.GetInt refused -- `WStrSet(wc, 1, ws)` stopped with "Type Mismatch"
- The UnicodeString arm of the previous entry
- `o.Free` with o nil stopped the script with "Null Pointer Exception", where Delphi's TObject.Free
does nothing for nil -- a `finally` freeing an object whose creation failed raised again
- A Byte, ShortInt, Word or SmallInt variable div by a 64-bit one read the divisor in 32 bits -- `b
div i6` with b 10 and i6 = 4294967297 gave 10
- A Byte, ShortInt, Word or SmallInt variable mod by a 64-bit one read the divisor in 32 bits -- `b
div i6` with b 10 and i6 = 4294967297 gave 10
- The divisor type of the previous entries
- The run-time half of the previous Round / Trunc entry
- The run-time half of the compiler's Break / Continue entry
- `Include(s, c)` / `Exclude(s, c)` of a Char into a set of Char stopped the script with "Could not
call proc"
- A Byte, ShortInt, Word or SmallInt div by a divisor of the other signedness read the divisor in
the dividend's sign -- `b mod i` with b 10 and i -3 gave 10 (Delphi 1)
- A Cardinal div by a signed divisor read the divisor as a Cardinal -- `c mod i` with c 10 and i -3
gave 10 (Delphi, in Int64: 1)
- An Integer div by a Cardinal read the divisor as an Integer -- `i div c` with i 10 and c
4294967295 gave -10 (Delphi, in Int64: 0)
- A Byte, ShortInt, Word or SmallInt mod by a divisor of the other signedness read the divisor in
the dividend's sign -- `b mod i` with b 10 and i -3 gave 10 (Delphi 1)
- A Cardinal mod by a signed divisor read the divisor as a Cardinal -- `c mod i` with c 10 and i -3
gave 10 (Delphi, in Int64: 1)
- An Integer mod by a Cardinal read the divisor as an Integer -- `i div c` with i 10 and c
4294967295 gave -10 (Delphi, in Int64: 0)
- It entered the finally with the values a case or with inside the try body had pushed still on the
stack, so the finally's own temporaries were read from the wrong slots -- `try case it of 1:
Break; end; finally RI(42) end` printed 0, and with a with the script stopped with "Type Mismatch"
- A Char was accepted into any set, so `Include(es, ch)` with ch remobjects#1 set the second member of an
enumeration's set
- The string index of the next entries
- A string index of a 64-bit type was narrowed to its low 32 bits before the bounds check, so `s[i]`
with i 4294967297 read and wrote s[1]
- A native routine that called a script routine back could grow the script's stack, which moved its
data and freed the old block, so the routine's result and var parameters were written to freed
memory and lost (`li := sl.Add('x')` with a script OnChange gave -1; the previous release lost it
too). Rounds 67 / 68 moved changed slots back after the call; found that inexact (a native routine
that read its var argument after the callback read the old copy; a native write equal to the bytes
saved at the move was lost) and replaced it: the executor's stack grows by chunks and its data
never moves, so a native routine and the script it calls back read and write one copy.: the chunk
lists are arrays made long enough first, so an allocation failure cannot leave a chunk owned twice
or by nobody
- After a script callback raised, RunProc popped its parameters a second time, removing the calling
routine's values (a six-parameter callback raising inside a one-argument native call stopped the
script with Out Of Stack Range instead of reaching its except)
- A pointer variable is rebased when its target lay in the old data, which ends at the old capacity;
the bound was the length the push had already grown, so a target in memory past the old block (a
dynamic array's element) was moved too
- The inlined push onto the temporaries compared the item count with the data's byte capacity
(TPSStack.FCapacity hides TPSList's), so the item list was never grown there and the branch, if
taken, grew the byte capacity and reallocated the items with 4-byte pointers
- A push moved the stack's position, and on a chunked stack switched to the next chunk, before
adding the value's slot, so a failing allocation left the position past the last value, in another
chunk
- Shrinking an array finalized the removed elements without emptying them, so when the smaller block
could not be had a shared nested array kept a pointer whose count was already released
- A push takes its slot first, holding nil, and a stack without chunks that grew rebased that slot
and read a type through the result
- The exception a finally runs for is held by its frame, not executor-wide -- the exception-state
pass
- LOCAL FIX (with the entries around it): dropping a frame's exception, and the nearest frame whose
except handler runs -- the exception-state pass
- LOCAL FIX (with the entries around it): the end of a finally an exception ran -- the
exception-state pass
- LOCAL FIX (with the entries around it): HoldIn -- the exception-state pass
- LOCAL FIX (with the entries around it): an exception escaping a finally replaces the one it ran
for -- the exception-state pass
- LOCAL FIX (with the entries around it): a finally entered for an exception holds it; an except
handler keeps where its exception was raised -- the exception-state pass
- LOCAL FIX (with the entries around it): the end of a first finally an exception ran -- the
exception-state pass
- LOCAL FIX (with the entries around it): the end of a second finally an exception ran -- the
exception-state pass
- LOCAL FIX (with the entries around it): Break / Continue out of a finally an exception runs ends
that exception (it resurfaced later) -- the exception-state pass
- LOCAL FIX (with the entries around it): Exit out of a finally an exception runs ends that
exception (it resurfaced in the caller, uncaught) -- the exception-state pass
- RaiseLastException looked at the top frame only, so under a nested try in the handler it did
nothing, and it passed the handler's state marker as the position -- the exception-state pass
- The Last* functions read the top frame only, so inside a nested try in a handler ExceptionParam
and the rest were empty, and ExceptionPos gave the handler's state marker -- the exception-state
pass
- Whether a callee raised
- LOCAL FIX (with the entries around it): ExceptionProc counts the exceptions it raises
- LOCAL FIX (with the entries around it): RunScript's count before a call
- A callee's error that entered a finally was reported again as erCouldNotCallProc, which passed
that finally by
- LOCAL FIX (with the entry above), the call through a procedural variable
- LOCAL FIX (with the entries around it): RunProc's count before a call
- LOCAL FIX (with the entries above), RunProc's call of an external routine
- An exception escaping an except into its second finally left the frame marked as a running handler
- LOCAL FIX (with the entries around it): Exit into a second finally retires the except before it
- A try body ending normally went on to the second finally with the except before it still armed, so
an exception raised in that finally was caught by it
- LOCAL FIX (with the entries around it): the end of a first finally going on to the second retires
the except
- LOCAL FIX (with the entries around it): Break / Continue into a second finally retires the except
before it
- An exception a script callback raised and handled made its native caller's False return pass
unreported
- A native event handler's saved interpreter state
- A native event handler saves the interpreter's position, stack base and raise count
- The 64-bit native event handler puts back the interpreter's position and stack base after an
exception
- The 32-bit native event handler puts back the interpreter's position and stack base after an
exception
- The 64-bit native event handler hands the script's exception to the native code
- The 32-bit native event handler hands the script's exception to the native code
- An exception caught by the except of the statement whose finally an Exit / Break / Continue ran
left the jump pending
- The end of an except frees the exception it handled before the second finally
- LOCAL FIX (with the entries around it): Exit from an except into its second finally ends the
handled exception
- LOCAL FIX (with the entries around it): Break / Continue from an except into its second finally
ends the handled exception
- SetLength to 0 of a unique array empties the variable before finalizing the elements, as Delphi
does
- LOCAL FIX (with the entry above): the length 0 is handled before the element loop now
- A popped value is finalized before its storage is given back
- LOCAL FIX (with the entry below): the local the array assignment takes its source into -- before
- An assignment over a dynamic array takes the source before it releases the destination, as Delphi
does -- before
- Clear releases the script's values before what a script event needs -- before
- LOCAL FIX (with the entry above): the values are released earlier now -- before
- LOCAL FIX (with the entries above): the frame list is emptied with its frames now -- before
- LOCAL FIX (with the entry below): the local the array release keeps its block in -- before
- The last reference to a dynamic array empties the variable before finalizing the elements, as
Delphi does -- before
- LOCAL FIX (with the entry above): the block is freed from the local -- before
- A released dynamic array is emptied before its count is touched, shared or not, as Delphi does
- Clear releases the stack before the globals
- A Stop inside a nested run stops the script -- before
- A Stop made by a release at a return stands
- A stopped run removes the frames of its try statements
- No instruction runs after a line hook stopped the script
- Clear leaves no current proc behind
- A run stopped in a finally an exception runs ends with the exception -- before
- An exception after a Stop leaves the Stop standing -- before
- An owned pointer value leaves its slot before it is destroyed
- A shrink stops when a release event resized the array
- An interface slot is written before its old reference is released
- A shrink shortens the array before it finalizes the removed elements
- An assignment releases the old contents after the copy
- A dynamic array assigned from a static one is built apart first
- An array of ___Pointer is copied element by element
- An array converted from a Variant is built apart first
- A Variant destination is stored before its old value is released
- A comment: why the Variant array is taken by value
- A call through a nil procedural variable re-ran the main block
- An integer compared with a Currency on its right went through a real -- a Double on Win64, which
cannot hold a Currency's ten-thousandths past about 900000000000, so `i < c` with i
922337203685477 and c 922337203685477.0001 was False and `i = c` True there, while `c > i` (the
Currency on the left) compared exactly and Delphi is exact on every platform (P128CmpDelphi.dpr).
The previous release stopped such a script with Type Mismatch (P128CmpVar.dpr on the tag). Now the
integer is compared with the Currency's raw value exactly; an integer past the range of Currency,
or a UInt64 past High(Int64), is greater or less than every Currency by its sign.
- A Currency compared with an integer on its right converted the integer to a Currency
(PSGetCurrency), which past the range of Currency wraps on Win64, gives the least Currency under a
masked invalid operation on Win32 and raises it (Math error) on Delphi 7, so `c < i` with c the
largest Currency and i 922337203685478 was False while `i > c` (an earlier fix) was True. The
previous release did the same (P129Cmp.dpr on the tag). Now both orders take the integer's exact
order against the Currency's raw value, an integer past the range of Currency greater or less than
every Currency by its sign -- Delphi's answer on Win32; Win64 Delphi wraps on both sides
(P129CmpDelphi.dpr).
- TPSExec.GetProcAsMethodN finds a host routine by its name (GetProc looks at the external
procedures too), but the method it made faulted when native code called it: the method handler
read the routine's declaration as a script routine's ExportDecl, which a host routine's record
does not have -- its declaration in the script holds the modes of its parameters, not their types,
so its arguments cannot be read. The handler now refuses a host routine with an exception; a
script still calls such a method itself, through a procedural variable
- A host can declare an event property to scripts as an integer -- upstream's published-property
fallback and its event property helper both take one, a script routine's number standing for the
method. Reading such a property when it held a method that is no script routine's (nil, or a
Delphi handler) wrote the method beside the number, as for a procedural value, past the four bytes
of the integer: the next variable's type was overwritten (Type Mismatch, or an access violation
escaping the run). Writing one through the fallback made a method of any number, and native code
calling it read past the routine list (an access violation); through the event property helper the
number reached the helper as its method, so even 0 ran a garbage address. Now a read writes beside
the number only for a procedural value; the fallback refuses a number that names no script
routine, as CM_CV does a procedural value's; the helper refuses a number
- The event property helper read an indexed event property with its indices but wrote one without
them -- the loop that passes them was commented out, and the value was taken from the slot below
Self, which for an indexed property holds the last index: the write was refused (Null Pointer
Exception, or Type Mismatch for a Cardinal index) and the write helper never ran. The write now
takes the value from below the indices and passes the indices after it, as the property helpers do
- (a) a var array of const handed to a native routine wrote back every element kind it copies but a
PChar -- the script's element kept its old pointer where Delphi's array holds the routine's; (b)
the event property helper stored the event it read in the finally of the read helper's call, so a
read helper that raised -- or a call that failed -- set the destination to 0 or nil, where Delphi
leaves a destination as it was when the getter raises. Now the PChar arm writes the pointer back,
and the read stores the event only when the call returned
- A Variant array assigned to a dynamic array whose element did not convert was reported twice -- by
the element's SetVariantValue and again, as Type Mismatch, by the one that called
CreateArrayFromVariant. A try took the first report, so the second found that try's handler used
up and went on to an outer handler or stopped the script: the try around the assignment never
caught it, a try... Finally skipped its finally, and an unhandled error read Type Mismatch for the
conversion's own message. Now CreateArrayFromVariant reports every failure itself and its caller
reports none
- A host routine that ran a script routine through ExecuteFunction, RunProcP or RunProcPVar while
the script ran, and handled what the routine raised, broke the running script: inside a try...
Except the routine's exception ran the except -- and the rest of the script -- inside the call,
the host then saw the call succeed, and the script stopped with Invalid Type; inside a try...
Finally it ran the finally inside the call, and the run ended with the handled exception; outside
one the exception stayed pending, and the end of the next try statement, or of the run, raised it
again. Such a call now runs behind a frame of its own and hands the exception to the native code
that made it, as the method handlers do
- The comments at Cm_CG and Cm_CNG named a WordBool or LongBool as reaching their 16- and 32-bit
arms, which this copy's DoBinCalc no longer lets happen -- comment only
- A host routine that ran a script routine through ExecuteFunction, RunProcP or RunProcPVar while
the script ran, where the routine stopped the script while its exception was pending -- a finally
that exception ran called Stop -- and the host handled the exception, still saw the stopped script
end failed with it: the exception was handed to the host only if the script still ran after the
call. It is now handed over whenever a script was running when the call began, as the method
handlers do
- The comments at Cm_CG and Cm_CNG said every jump the compiler writes there is on a Boolean; a
short-circuit or / and whose right operand is a Variant keeps a WordBool or LongBool left
operand's type -- comment only
- A method handler exchanged an event with its native caller as the script's three-slot procedural
value instead of a method
- Two var event arguments naming one caller variable were two values
- Open array of const arguments of a native call naming one script array were separate copies -- a
var argument's write was lost or not seen through the other
- The operand reader finalized a refused managed constant's uninitialized place, and refused an
operand of no kind without an error
- LOCAL FIX: a constant of a type the operand reader has no case for is refused before its place in
the temporaries is made
- Cm_PG / Cm_P2G popped a stack with nothing or a return address on top, and RunScript answered True
for a main routine that is an imported one
- RunProc of an imported routine that reported an error put back the status and the routine the
report had moved, and answered True when the routine did
- LOCAL FIX: RunProc left the parameters it had pushed on the stack when an imported routine raised
or a parameter list held nil
- The debugger's text of a set, of a character past remobjects#255, of a Cardinal or UInt64 property and of a
string property
- LOCAL FIX: the debugger's text lost a character from remobjects#128 the code page cannot hold, and answered
a path past a property that is no object with that property
- Delphi 10 Seattle Win64 called OnException through an unwritten stack slot
- [Free Pascal] PropertyToString (pascalscript ): a Boolean property is tkBool there, which no arm
named, so the debugger wrote [Unknown] where Delphi writes True / False
- [Free Pascal] PropertyToString (): a UInt64 property is tkQWord there, where Delphi's tkInt64 arm
writes it unsigned
- [Free Pascal] PropertyToString (): an AnsiString property is tkAString there, where Delphi names
it tkLString
- Delphi XE3 - XE5 for Win64 leak the exception object of an except handler left by a jump; each
such handler is left after its end
- A value parameter a native caller passes by reference is the script routine's own copy, not the
caller's variable
- A script routine native code calls as a method no longer runs when the copy of one of its
parameters fails; the call is refused
- Before Delphi 10.3 Rio, a Win64 record or static array of 5, 6 or 7 bytes goes by reference
between native code and a script, as Delphi passes it
- A compile under a running script, a reload freeing filled methods, the routine insertion and its
declaration, the catalogue, a method record not found again
- A script routine of a real result that native code calls through a method pointer puts the result
on the x87 stack only when it returns normally, after the cleanup
- A script routine of a real result that native code calls through a method pointer puts the result
on the x87 stack after every release of the call, and loads it only when the call returns
- A routine the compiler calls for itself is found by a lookup no script routine can answer;
AddDelphiFunction outside a compilation refuses
- A Win64 import thunk's displacement is signed
- A Variant's method gets an empty result; an array element that is a call of a routine with no
result is refused
- A number, a character or a text compared with a Variant compares as two Variants, as Delphi does
- An array of ___Pointer element holding no value reached a Variant through a nil type or a nil
place, an access violation for any script assigning such an array to a Variant, comparing one with
a Variant, and for a host taking one from ExecuteFunction; it reads as Null, the routine's answer
for a value of no type
- PSGetArrayField took a static array of ___Pointer for a dynamic one, so the debugger's text of one
read its first element's pointer as the elements -- an access violation
- _IDispatchInvokeInto took the object by value, so every late-bound call added a reference to it
for the call's length; IDispatchInvoke holds it
- A comparison whose right operand is a host's pointer to a pointer (SetPointerToData with a pointer
type) skipped the comparison of a value with a Variant (an earlier fix) and of a text with a wide
one, which tested the pointer's own type, so the typed arms read the value in the left operand's
type; NumericCompare, WideCompare and the Variant rule now read it where the pointer points, as
the arms do
- CreateHeapVariant read the size of a nil type, so a loaded script that lacks a type a helper
builds a value of -- bytes a host altered, or a host retyping the loaded PChar type -- stopped the
event property helper's read with an access violation, though it and the other helpers test the
value for nil; a nil type now gives no value
- COMMENT of an earlier fix corrected: a left Variant converts a nil cell to Null, so not every arm
refuses one
- CreateHeapVariant's nil for a type the loaded script lacks reached two callers that went on --
ClassCallProcPropertyHelperName called the host's accessor without its name argument when the
script held no AnsiString type, and RunProcP / RunProcPVar answered Null without running the
routine when its result named no loaded type; the first now refuses, the second raises Invalid
parameter, as for a parameter of no loaded type
- A value assigned to a Variant goes through a carrier of the pointer type, and where the loaded
script holds no such type the conversion takes it as nothing, so the Variant silently became Null;
such an assignment now stops with Type Mismatch
- A run resumed at the top -- after a host's Pause or an unhandled error -- ended, when the script
reached its end, with the status it was resumed from, paused, so its values stayed on the stack
and the next run went on past the script's end (Out Of Range); it now ends loaded, as a run begun
from the start does
- The executor's Pause inside a run RunProc began -- a routine a host runs by ExecuteFunction, or
native code calls back -- ended that run where it stood: the host got control back with whatever
the routine had done so far and the rest of it was lost; such a run now goes on, and only a run
Execute began takes the executor's pause (the debugger's wait, where an idle call is installed, is
no such pause and still holds a nested run at its line)
- A script routine handed to native code as a method and called after its script ended raises "A
script routine cannot be called after its script has ended" -- it read the method record from
freed memory
- Clear takes each resource out of its list before releasing it -- a host looking a resource up
during a release read the entries already disposed, and a resource it added was never released
- IsValidResource answers for the routine and the data together, and TPSAttributes.FindAttribute
takes a name in any case
- AddBlock reads its source before the old block goes; the comments of three opcodes say what is
read
- A class importer made with CreateAndRegister and AutoFree lives as long as its executor, where the
first LoadData freed it
- RunProcP of a routine whose result no Variant can hold answers Null, where it answered whatever
its result arrived holding
- An assignment of a static array, a dynamic array or a record whose element fails to copy is
refused, where it went on with the destination partly emptied
- A stored native method called through a procedural variable gets its result, its exception reaches
the script and a refused call is reported
- A stored native method that raises leaves nothing behind, where each such call leaked its
parameter list
- Delphi 2010 and XE have no dotted unit names, so System.Rtti was not found there; they use Rtti
These fixes come from NexusDB, which ships Pascal Script as its script engine. Each was found and fixed
there; most were measured against what Delphi itself does with the same code, and most have a regression
test in NexusDB's own suite, which runs on Delphi 7 through 13 Florence (Win32 and Win64) and on Free Pascal
3.2.2 (i386) and 3.3.1 (x86_64). NexusDB builds the engine with PS_USECLASSICINVOKE and PS_USESSUPPORT; the
fixes are given here in Pascal Script's own unit names and configuration.
x86.inc:
- As ProcPtrToMethod in uPSRuntime: a procedure-pointer argument
- As above (the FPC arm, kept alike)
- A cdecl method (a host's RegisterDelphiMethod or an interface method registered cdCdecl) got Self
after its parameters -- the branch put Self first in the stack image and then put every parameter
ahead of it, where the stdcall and safecall branches put Self ahead of the parameters -- so the
method read its first parameter as Self: an access violation on Delphi 7 and 13 Florence Win32
- An open array parameter is two stack values, the data pointer and the high bound; the stack image
put the high bound below the pointer, which is right for the left-to-right conventions (register
beyond the registers, pascal) and reversed for cdecl, stdcall and safecall, so such a routine read
the high bound as its data pointer: an access violation on Delphi 7 and 13 Florence Win32. Both
the value and the var branch
- RealCall_Cdecl skipped storing EDX -- an Int64 result's high half -- when the call had nothing on
the stack, so a cdecl function of no parameters returning an Int64 gave a high half of whatever
the caller held: measured on Delphi 7 and 13 Florence Win32
- [Free Pascal] A 3-byte script set passed by value read 4 bytes, so the byte after it reached the
native 4-byte set. Copied with zero extension, as the Delphi arm does. An {$IFDEF FPC} arm inside
the {$ELSE} of DELPHI_and_MSWINDOWS: Delphi selects the same text
- [Free Pascal] A cdecl or stdcall method takes Self ahead of its hidden result pointer, where
Delphi takes the result pointer first; and a pascal method takes Self above its parameters, where
Delphi takes it below them. FPC arms only: Delphi selects the same text
- [Free Pascal] A cdecl open array goes as its data alone, with no high bound, on both targets. FPC
arms only: Delphi selects the same text
- [Free Pascal] A 1- or 2-byte ordinal or a small set passed by reference goes as a zero-extended
copy, written back after the call, because Free Pascal's RTL keeps such types in 4 bytes. FPC arms
only: Delphi selects the same text
- [Free Pascal] The same script value passed twice by reference shares one narrowed copy, as it
shares one address on Delphi. FPC arms only: Delphi selects the same text
- [Free Pascal] A record result is returned in EAX at 1, 2 or 4 bytes, by a cdecl routine in EAX:EDX
up to 8 bytes, else through the hidden pointer; a static array always through it. FPC arms only:
Delphi selects the same text
- [Free Pascal] A pascal routine returns every record and static array through the hidden pointer,
and a set of 1, 2 or 4 bytes in EAX, a larger one through the hidden pointer, in every convention.
FPC arms only: Delphi selects the same text
- [Free Pascal] A record or static array of 1-4 bytes is passed by value on the stack, never in a
register; under cdecl a record by value at every size and a static array by reference. FPC arms
only: Delphi selects the same text
- [Free Pascal] A dynamic array passed by value or const is its data pointer, in a register when one
is free, as under Delphi. FPC arms only: Delphi selects the same text
- [Free Pascal] A method pointer goes by reference in a copy of its own (by value under cdecl), and
under stdcall a record goes by value at every size, as under cdecl. FPC arms only: Delphi selects
the same text
- [Free Pascal] On i386 a by-value byte above 127 reached a native routine converted under a UTF-8
run-time page: 200 arrived as 195
- [Free Pascal] On i386 a cdecl host routine taking a set of more than 4 bytes by value read garbage
- [Free Pascal] A cdecl array of const reached an external C-varargs routine as its address
- A native routine that called a script routine back could grow the script's stack, which moved its
data and freed the old block, so the routine's result and var parameters were written to freed
memory and lost (`li := sl.Add('x')` with a script OnChange gave -1; the previous release lost it
too). Rounds 67 / 68 moved changed slots back after the call; found that inexact (a native routine
that read its var argument after the callback read the old copy; a native write equal to the bytes
saved at the move was lost) and replaced it: the executor's stack grows by chunks and its data
never moves, so a native routine and the script it calls back read and write one copy.: the chunk
lists are arrays made long enough first, so an allocation failure cannot leave a chunk owned twice
or by nobody
- A native routine with a result the call layer does not return was called on Win64, and under
safecall, without its result pointer
- Open array of const arguments of a native call naming one script array were separate copies -- a
var argument's write was lost or not seen through the other
x64.inc:
- As ProcPtrToMethod in uPSRuntime: a procedure-pointer argument
- As above (the FPC arm, kept alike)
- [Free Pascal] "GPF on exception"): the thunk Free Pascal takes had no unwind description, so an
exception raised by the called routine could not unwind through it and ended the process.
Describing the rbp frame lets the unwinder pass it; the pushes after the prologue are covered by
the frame pointer. This arm is taken only where DELPHI is not defined: Delphi-invisible
- [Free Pascal] A TMethod argument went by value on the stack where the Win64 ABI passes it by
reference (event property writes AV'd); takes the Delphi arm. Delphi-invisible
- [Free Pascal] A cdecl open array goes as its data alone, with no high bound, on both targets. FPC
arms only: Delphi selects the same text
- [Free Pascal] A 1- or 2-byte ordinal or a small set passed by reference goes as a zero-extended
copy, written back after the call, because Free Pascal's RTL keeps such types in 4 bytes. FPC arms
only: Delphi selects the same text
- [Free Pascal] The same script value passed twice by reference shares one narrowed copy, as it
shares one address on Delphi. FPC arms only: Delphi selects the same text
- [Free Pascal] A record result goes through the hidden pointer unless it is 1, 2, 4 or 8 bytes with
no managed field, a set unless it is 1..4 bytes, as Free Pascal returns them. FPC arms only:
Delphi selects the same text
- [Free Pascal] A dynamic array passed by value or const is its data pointer, in a register when one
is free, as under Delphi. FPC arms only: Delphi selects the same text
- [Free Pascal] A record parameter is passed by reference, as Free Pascal passes a const record at
every size; a value record of 1, 2, 4 or 8 bytes, which Free Pascal passes by value, is the stated
gap. FPC arm only: Delphi selects the same text
- [Free Pascal] The record rule's comment and AlwaysAsVariable follow CONSTREC64 (every record by
reference; no method handler is compiled there). FPC arms only: Delphi selects the same text
- [Free Pascal] A cdecl array of const reached an external C-varargs routine as its address
- A native routine that called a script routine back could grow the script's stack, which moved its
data and freed the old block, so the routine's result and var parameters were written to freed
memory and lost (`li := sl.Add('x')` with a script OnChange gave -1; the previous release lost it
too). Rounds 67 / 68 moved changed slots back after the call; found that inexact (a native routine
that read its var argument after the callback read the old copy; a native write equal to the bytes
saved at the move was lost) and replaced it: the executor's stack grows by chunks and its data
never moves, so a native routine and the script it calls back read and write one copy.: the chunk
lists are arrays made long enough first, so an allocation failure cannot leave a chunk owned twice
or by nobody
- A native routine with a result the call layer does not return was called on Win64, and under
safecall, without its result pointer
- X64call kept its saved registers past its own frame, in its caller's locals
- Open array of const arguments of a native call naming one script array were separate copies -- a
var argument's write was lost or not seen through the other
- Before Delphi 10.3 Rio, a Win64 record or static array of 5, 6 or 7 bytes goes by reference
between native code and a script, as Delphi passes it
- A Win64 set of 33 to 64 members passed by value reaches a native routine as Delphi passes it, by
value before 10.4 Sydney where it was given an address under cdecl, stdcall and safecall, and as
an 8-byte copy where it was given the script's shorter value
arm.inc:
- A native routine that called a script routine back could grow the script's stack, which moved its
data and freed the old block, so the routine's result and var parameters were written to freed
memory and lost (`li := sl.Add('x')` with a script OnChange gave -1; the previous release lost it
too). Rounds 67 / 68 moved changed slots back after the call; found that inexact (a native routine
that read its var argument after the callback read the old copy; a native write equal to the bytes
saved at the move was lost) and replaced it: the executor's stack grows by chunks and its data
never moves, so a native routine and the script it calls back read and write one copy.: the chunk
lists are arrays made long enough first, so an allocation failure cannot leave a chunk owned twice
or by nobody
arm64.inc:
- A native routine that called a script routine back could grow the script's stack, which moved its
data and freed the old block, so the routine's result and var parameters were written to freed
memory and lost (`li := sl.Add('x')` with a script OnChange gave -1; the previous release lost it
too). Rounds 67 / 68 moved changed slots back after the call; found that inexact (a native routine
that read its var argument after the callback read the old copy; a native write equal to the bytes
saved at the move was lost) and replaced it: the executor's stack grows by chunks and its data
never moves, so a native routine and the script it calls back read and write one copy.: the chunk
lists are arrays made long enough first, so an allocation failure cannot leave a chunk owned twice
or by nobody
- Open array of const arguments of a native call naming one script array were separate copies -- a
var argument's write was lost or not seen through the other
powerpc.inc:
- A native routine that called a script routine back could grow the script's stack, which moved its
data and freed the old block, so the routine's result and var parameters were written to freed
memory and lost (`li := sl.Add('x')` with a script OnChange gave -1; the previous release lost it
too). Rounds 67 / 68 moved changed slots back after the call; found that inexact (a native routine
that read its var argument after the callback read the old copy; a native write equal to the bytes
saved at the move was lost) and replaced it: the executor's stack grows by chunks and its data
never moves, so a native routine and the script it calls back read and write one copy.: the chunk
lists are arrays made long enough first, so an allocation failure cannot leave a chunk owned twice
or by nobody
InvokeCall.inc:
- A native routine that called a script routine back could grow the script's stack, which moved its
data and freed the old block, so the routine's result and var parameters were written to freed
memory and lost (`li := sl.Add('x')` with a script OnChange gave -1; the previous release lost it
too). Rounds 67 / 68 moved changed slots back after the call; found that inexact (a native routine
that read its var argument after the callback read the old copy; a native write equal to the bytes
saved at the move was lost) and replaced it: the executor's stack grows by chunks and its data
never moves, so a native routine and the script it calls back read and write one copy.: the chunk
lists are arrays made long enough first, so an allocation failure cannot leave a chunk owned twice
or by nobody
- A Single or Double result was taken through a Single(Extended) or Double(Extended) typecast, which
Delphi 10 Seattle and 10.1 Berlin refuse (E2089: the Single one for Win64, the Double one for
Win32), so the unit did not compile there; the assignment converts
These fixes come from NexusDB, which ships Pascal Script as its script engine. Each was found and fixed
there; most were measured against what Delphi itself does with the same code, and most have a regression
test in NexusDB's own suite, which runs on Delphi 7 through 13 Florence (Win32 and Win64) and on Free Pascal
3.2.2 (i386) and 3.3.1 (x86_64). NexusDB builds the engine with PS_USECLASSICINVOKE and PS_USESSUPPORT; the
fixes are given here in Pascal Script's own unit names and configuration.
uPSC_classes.pas:
- From 10.4 Sydney upstream declares CopyFrom with BufferSize, which the script must then pass -- a
script calling CopyFrom(Source, Count), as the previous release declared it on every compiler, no
longer compiled (Invalid number of parameters, measured DFlorence_64). The run time binds it
through a routine (uPSR_classes) that leaves BufferSize to its Delphi default
- Seek is bound by address through TMemoryStream, whose first Seek is the Int64 one from XE2
(TCustomMemoryStream declares no other: System.Classes.hpp of XE2.. Florence), not from 10.2
Tokyo; declared with LongInt parameters there, a script calling Seek failed on every Win32 cell
XE2.. 10.1 Berlin -- an access violation, on XE3 an illegal instruction -- and on Win64 a negative
offset reached it without its sign, so a later Read raised an access violation. The previous
release declared the LongInt form on every compiler, so the same binding failed there from XE2 on:
measured against it on 10.4 Sydney and 13 Florence, Win32 an access violation at the first Seek,
Win64 the Read after a negative offset reading nothing
- TParser.Error takes the message as a string on every Delphi version (Delphi 7 and 13 Florence
read); declared with an Integer, the number reached Delphi as a string and raised an access
violation. Bound by address: on Florence it is overloaded, and @TParser.Error is the first
declaration, the string one
- TParser.SourcePos is NativeInt from 10.2 Tokyo (the shipped System.Classes.hpp of Tokyo..
Florence; Athens and Florence source) and a 32-bit integer before it (Delphi 7 Longint; XE2..
Berlin int, in their System.Classes.hpp) and in FPC (Longint); declared LongInt, a position past 2
GB on Win64 from Tokyo was cut to 32 bits. The script's NativeInt has the width of the target
- TParser.TokenInt is an Int64 (Delphi 7, 12 Athens and 13 Florence read); declared LongInt, a
script read 123456789012 as -1097262572
- With no Create declared, a script's TCollection.Create (or TOwnedCollection's, TDefCollection's,
TParams') compiled against TObject.Create, whose import names TObject, so the run time allocated
the collection and ran TObject.Create on it: the item list was never created, and Count or Add
raised an access violation. No script type holds a class reference, so the real constructor cannot
be declared. Delphi only here, as the other constructor routines
- FmCreate is $FF00 from Delphi 2010 (Classes.hpp of 2010, XE, Berlin, Tokyo read; $FFFF in 2007 and
2009), so a script's fmCreate or fmShareDenyNone was $FFFF, which 13 Florence's TFileStream.Create
takes for an exclusive create. Free Pascal (no DELPHI2010UP) keeps $FFFF, as before
- TStream.Position and Size are Int64 in Delphi (7 and 13 Florence read) and were declared to
scripts and helped as LongInt, so a position or size beyond 2 GB was truncated
- [Free Pascal] Unicode mode: Classes, SysUtils and fcl-db are compiled with string = AnsiString,
while a script `string` is a UnicodeString; this member is bound by address, so the engine would
hand the RTL a UnicodeString where it reads an AnsiString. Declared AnsiString on Free Pascal, the
engine converts first. An FPC arm: Delphi-invisible
- [Free Pascal] The script declared the LongInt Seek (no DELPHI16UP) while the run time binds
TCustomMemoryStream.Seek(const Int64; TSeekOrigin) (VclStreamSeek read garbage positions). The
{$IF} is DELPHI16UP's value on every Delphi
- [Free Pascal] The script declaration of this Create carried the same gate as its constructor
routine; without it the script's Create reached TObject.Create and the run-time binding was never
asked. Delphi selects the same text
- [Free Pascal] TStrings / TStringList Create were declared and bound only under DELPHI2005UP, which
Free Pascal never has; the script's Create reached TObject.Create and TStrings.Create's
initialisation never ran (StringListValues). Delphi selects the same text
- [Free Pascal] The script constants take Free Pascal's own values ($400 / $1FF / $FF00). FPC arms
or an FPC term in the condition: Delphi selects the same text
- [Free Pascal] The constructor the run side now binds
- [Free Pascal] Classesh.inc declares THelpEvent's Data Longint on every target
- [Free Pascal] Classesh.inc's TGetStrProc takes the RTL's string, an AnsiString in the Unicode mode
- The Classes library registered TParser without streams too, where its constructor, which takes a
stream, could not be declared and TObject's took its place -- a parser made by it was never set up
and faulted at its first token; TParser now comes with the streams
- TBits.Bits, Delphi's default property, was not the script's, so B[0] was refused; it is now the
default
- TParser is declared where the stream class is -- registered by the Classes library's streams or by
the host itself -- not by the Streams flag alone
uPSR_classes.pas:
- These members are bound by the address of a virtual method, so a script passes over a descendant's
override -- a data set's MoveBy (override from Delphi 2009) skipped its OnFilterRecord check and
busy tracking when a script called it. RegisterVirtualMethod is no fix: several are virtual only
in later Delphi versions, and a slot lookup that fails calls slot 0
- The run-time half of the CopyFrom declaration in uPSC_classes: bound by address, the Sydney+
CopyFrom would take its BufferSize from whatever the caller's register held
- Measured on DFlorence_64, a script's TParser.Create raised an access violation in the constructor
(Win64 passes the method pointer by reference); on Win32 it comes from the stack and the callee's
RET 8 unbalances it. Probes\ProbeParserOnError.dpr lists the cells where SOME Create(Stream, nil)
compiles -- not that it is the first overload, which a binding by address takes; that was read for
Athens and Florence only. The routine supplies the default on every version. Delphi only, as the
other constructor routines
- Upstream's routine for this (STRINGSTREAMFIX) takes the class and the constructor flag as two
leading parameters of a class-helper method, whose hidden Self shifts every parameter by one -- a
script's TStringStream.Create('object x end') gave an empty stream on DFlorence_64 and killed the
process on DFlorence_32. STRINGSTREAMFIX is no longer defined, so upstream's routines compile out.
The previous release declared no TStringStream
- The run-time half of the TCollection.Create declaration in uPSC_classes
- A script Char is one byte (AnsiChar) on every compiler, Delphi's Char two from Delphi 2009, and a
read helper writes through a pointer to the SCRIPT's value -- a script reading the property into a
record field had the field after it overwritten. The helper takes an AnsiChar. And both directions
convert through the ANSI code page (PSWideCharToAnsiChar / PSAnsiCharToWideChar) from Delphi 2009:
a cast took the character's low byte (U+20AC read as $AC, $80 written as U+0080). The write
helpers take the script's character zero-extended, as the call layer passes it
- The read helper wrote a Delphi Char through a pointer to the script's one-byte Char (as for
TStrings above). It takes an AnsiChar and the token's LOW BYTE: TParser's token is the byte it
read, not decoded through a code page (13 Florence System.Classes, TParser.NextToken: "Result :=
Char(FBuffer[P])", for a byte CharType classes ctOther), and NextToken / CheckToken, bound by
address, pass it to a script as that byte. Converted it through the ANSI code page, which read
byte $80 of an ANSI stream as ? On code page 1252
- A script PChar is a PAnsiChar, and from Delphi 2009 TStrings.SetText takes and GetText returns a
PWideChar, so Delphi read the script's ANSI bytes as UTF-16 (past their terminator) or the script
read Delphi's UTF-16 as ANSI (to the first character's high zero byte). Bound on Delphi 2009 and
later to routines converting through the ANSI code page. GetText calls the virtual GetText (it
copied Text, passing over a descendant's override
- A script PChar is a PAnsiChar, and from Delphi 2009 TResourceStream.Create and CreateFromID take
the resource type as a PWideChar, so Delphi read the script's ANSI bytes as UTF-16 (past their
terminator) or the script read Delphi's UTF-16 as ANSI (to the first character's high zero byte).
Bound on Delphi 2009 and later to routines converting through the ANSI code page
- From Delphi 10.2 Tokyo ReadBuffer and WriteBuffer take their count as a NativeInt, and the
script's LongInt reached it for Win64 without its sign (x64.inc passes a btS32 zero-extended), so
a negative count arrived as about 4 billion. The routines take a LongInt and pass it on, widened
with its sign where the count is wider
- TStream.Position and Size are Int64 in Delphi (7 and 13 Florence read) and were declared to
scripts and helped as LongInt, so a position or size beyond 2 GB was truncated
- [Free Pascal] TStringStream.Create bound by address took Create(const ABytes: TBytes), FPC's first
overload (VclStringStream / VclParser*: "Invalid pointer operation" at Free). An FPC arm
- [Free Pascal] A constructor routine in Delphi's shape receives the class first on Free Pascal too
(the stubs' IsConstructor arm), so these Delphi-only routines serve it; without them the script's
Create reached FPC's Create(ItemClass) with none. Delphi selects the same text
- [Free Pascal] TStrings / TStringList Create were declared and bound only under DELPHI2005UP, which
Free Pascal never has; the script's Create reached TObject.Create and TStrings.Create's
initialisation never ran (StringListValues). Delphi selects the same text
- [Free Pascal] Classesh.inc declares Equals(Obj: TObject) override first, Equals(TheStrings:
TStrings) second, so @TStrings.Equals bound the first by address
- [Free Pascal] Both forms
- [Free Pascal] Classesh.inc declares TBits.Create(TheSize: longint = 0) virtual (trunk aSize:
SizeInt); a script's Create ran TObject's, so a descendant's constructor never ran
- [Free Pascal] The class-helper form
- [Free Pascal] The plain form
- The Classes library registered TParser without streams too, where its constructor, which takes a
stream, could not be declared and TObject's took its place -- a parser made by it was never set up
and faulted at its first token; TParser now comes with the streams
- TParser is declared where the stream class is -- registered by the Classes library's streams or by
the host itself -- not by the Streams flag alone
uPSC_dateutils.pas:
- [Free Pascal] Unicode mode: Classes, SysUtils and fcl-db are compiled with string = AnsiString,
while a script `string` is a UnicodeString; this member is bound by address, so the engine would
hand the RTL a UnicodeString where it reads an AnsiString. Declared AnsiString on Free Pascal, the
engine converts first. An FPC arm: Delphi-invisible
uPSR_dateutils.pas:
- [Free Pascal] StrToDate / FormatDateTime bound by address took FPC's first overloads --
StrToDate(ShortString), and FormatDateTime with its defaulted Options left to whatever the
register held. An FPC arm
- A TDateTime before 30 December 1899 is negative with its time of day still added (noon of 29
December 1899 is -1.5), so subtracting the epoch put such a date a day out:
UnixToDateTime(-2209204800) gave noon of the 30th. Delphi 7's DateUtils has the same formulas; 13
Florence converts through a time stamp. Rounding to the nearest second is kept, as before
uPSC_std.pas:
- [Free Pascal] Unicode mode: Classes, SysUtils and fcl-db are compiled with string = AnsiString,
while a script `string` is a UnicodeString; this member is bound by address, so the engine would
hand the RTL a UnicodeString where it reads an AnsiString. Declared AnsiString on Free Pascal, the
engine converts first. An FPC arm: Delphi-invisible
uPSR_comobj.pas:
- [Free Pascal] SysUtils' and ComObj's text parameters are the RTL's string; bound directly, a valid
GUID or ProgID arrived as UTF-16 bytes in the Unicode mode
- [Free Pascal] )
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Classes, DateUtils, std and ComObj import libraries: 46 changes, each described in the commit messages.
This is one of a series of pull requests carrying the fixes NexusDB made to Pascal Script, split by area so that each can be reviewed on its own. The fixes are listed one per line in each commit's message.
Depends on #325: this branch holds their commits as well; the last commit is this pull request's own.
Tested. With the whole series applied, every unit of
Sourcecompiles, and a script run throughTPSScript(strings, Int64, reals, calls into host functions, exceptions, a class from the Classes library,for/Break) gives the expected results, on Delphi 7, 2005, 2006, 2007, 2009, 2010, XE, XE2 - XE8 and 10 Seattle - 13 Florence (Win32, and Win64 from XE2), and on Free Pascal 3.2.2 (i386) and 3.3.1 (x86_64), with and withoutPS_USESSUPPORT; compilers before XE4 need the last pull request of the series. Master as it stands does not compile on XE4, XE5 Win64, 10 Seattle, 10.1 Berlin, 10.3 Rio Win64, Free Pascal 3.2.2 or 3.3.1, nor on anything before XE4.Beyond that smoke test, the fixes were developed and are covered in NexusDB, whose suite runs about 6,400 checks on 37 Delphi compiler / platform combinations and on Free Pascal. NexusDB defines
PS_USECLASSICINVOKE, so there every Delphi calls native routines throughx86.inc/x64.inc; theInvokeCall.incpath (the default from Delphi 10 Seattle) is exercised here by the smoke test only.