Skip to content

New features and fixes for 2.6.0 - #135

Merged
qiaoyuang merged 27 commits into
release/2.6.0from
feature/new-features-2.6.0
Nov 8, 2026
Merged

qiaoyuang merged 27 commits into
release/2.6.0from
feature/new-features-2.6.0

Conversation

@qiaoyuang

@qiaoyuang qiaoyuang commented Nov 8, 2026 •

Copy link
Copy Markdown
Collaborator

The new features of 2.6.0, and the bugs found while making them, from feature/new-features-2.6.0, which started from release/2.6.0 after the first bug fixes and contains all of it. Every item has its entry in CHANGELOG.md, and the guides are updated; new DSL API is experimental, behind @ExperimentalDSLDatabaseAPI.

New features

  • F7: joins start from a relation, as (PersonTable INNER_JOIN BookTable ON (...)) SELECT X<R>(), without FROM, with every join operator. (f05b95c)
  • F66: table version tokens, Database#versioned { }, awaitChange and onChange, which tell a reader when the tables it read change through the same Database. (113cd45)
  • F23: INSERT_OR_ABORT, INSERT_OR_FAIL and INSERT_OR_ROLLBACK. (c432843)
  • F27: transaction(mode) { }, which begins a transaction DEFERRED, IMMEDIATE or EXCLUSIVE. (5f7f480)
  • F26: INDEXED_BY and NOT_INDEXED. (15175d0)
  • F29: EXPLAIN_QUERY_PLAN(select), read as a tree of QueryPlan steps. (36a5269)
  • F30: the PRAGMAs that check the database, read its schema and what the SQLite has, checkpoint the WAL, read the size and header of the file, and set how statements run. (8e755ea, ae7a9ec)
  • F19: @CollateRTrim, and COLLATE on the columns of an index. (63eb2eb)
  • F25: ORDER_BY, LIMIT and OFFSET after DELETE and UPDATE, written as the subquery of rowids that SQLite turns them into, so that they run on every platform, Android's SQLite lacking SQLITE_ENABLE_UPDATE_DELETE_LIMIT. (5ae53ce)
  • F13: a new module, com.ctrip.kotlin:sqllin-dsl-paging, on every platform of sqllin-dsl: Database#pagingSource { } returns androidx.paging's PagingSource of a SELECT, built on F66, as Room's LimitOffsetPagingSource is. CI builds and publishes it. (8dd6317)
  • F12: columns of the types SQLite doesn't store, converted by the serializer that kotlinx.serialization writes the property with: Uuid, Instant and Duration, value classes, types whose classes declare their serializers, and any type given one with @Serializable(with = ...) or @file:UseSerializers, as a List stored as JSON. The column has the type the serializer stores, and its element, ClauseConverted<T>, converts the values of its conditions and of SET {} with it. (e617d73)

Fixes

  • B9: the processor's Kotlin classes were compiled for the JDK that ran the build, which an older JDK can't load. (8a328d5)
  • B10: a clause appended to a statement held in a variable replaced the last statement of the scope, rather than the one it completes. (01a6171)
  • B11: an inner transaction { } ended the outer transaction, since 2.0.0 at least. (59b86cd)
  • B12: on Android 13+, PRAGMA_FOREIGN_KEYS(flag) and the other PRAGMAs that set how a connection runs statements didn't apply to statements whose SQL had run before. (12fbbbf)
  • B13: on Kotlin/Native and Android 8.1 and lower, journalMode hasn't taken effect since 1.2.0. (517491e)
  • B14: an enum with a serializer of its own, as one storing its name, or any property given a serializer, was written by it in INSERT but compared and set as its type is by default, so that conditions didn't match and a row SET {} changed couldn't be read. It is now a column converted by its serializer. (e617d73)
  • B15: the object generated for an entity annotated with an opt-in marker, as @ExperimentalUuidApi, didn't opt in to it, so it didn't compile. (21da315)
  • B16: a SELECT whose results weren't read kept its cursor and statement open, so that its tables couldn't be dropped, and on Kotlin/Native the connection stayed open after close(), holding the database file, which Windows can't delete: the first CI run of this PR failed 78 of the 116 Windows tests, which all opened the database that testTableVersions left. A SELECT now reads its rows when its scope runs it. (691685c)

Breaking changes

  • Removed the joins written as clauses of SELECT, which 2.5.0 announced would be removed (F7), and the UNION {} and UNION_ALL {} blocks, which 2.5.0 deprecated (0d8efdc).
  • UPDATE ... WHERE returns a WhereUpdateDeleteStatement rather than the String of its SQL (F25); DELETE returns a statement rather than Unit, which only breaks binary compatibility.
  • A property with a serializer that isn't its type's default one gets a ClauseConverted element rather than a ClauseEnum, ClauseString or ClauseNumber (B14), whose operators are experimental. Tables created before need no migration.

Other

  • AGP 9.5.0, kotlinx.serialization 1.12.0 and sqlite-jdbc 3.54.1.0 (087341f), then AGP 9.5.1 and Gradle 9.9.0 (22f8e62).
  • DatabaseScope no longer has @Volatile fields, which suggested it could be shared between threads; its documentation says it isn't thread-safe. (63628a4)
  • CI uploads the test reports of sqllin-dsl-test on macOS, Windows and Linux, rather than those of sqllin-dsl, which has no tests, so that a failed run's report is kept. (de0df18)
  • The guide says that a schema PRAGMA may miss what another connection created (1698c80). The ROADMAP has a Shelved section, with incremental_vacuum (F68, 340797b) and data_version (F69, f454e0a).

Testing

  • Locally, sqllin-dsl-test passes jvmTest (117 tests) and testAndroidHostTest on Robolectric API 26 and 37 (234 tests), and sqllin-driver its JVM (13) and Android (27) tests. Each fix but B9, a build setting checked by the version of the class files, has a test that fails without it.
  • For F12, the 56 table and view objects that the processor generated before for the existing test entities are byte for byte the same.
  • The native tests run only on CI, as the development machine is an Intel Mac: they are compiled and linked locally for macosArm64, iosSimulatorArm64 and watchosArm32, and compiled for linuxX64 and mingwX64. The first CI run of this PR passed on macOS, Linux and Android, and failed on Windows for B16.

🤖 Generated with Claude Code

qiaoyuang and others added 27 commits October 29, 2026 09:31
… relation (F7)

2.5.0 replaced the joins written as clauses of SELECT, such as
table SELECT INNER_JOIN<R>(other) USING column, with joins of relations,
which check the type their rows are read into against the joined
relations, can join any number of relations, each with its own
condition, and select result columns. It announced that the old ones
would be removed in the next version, which this commit does: the
functions CROSS_JOIN, JOIN, INNER_JOIN, NATURAL_JOIN, NATURAL_INNER_JOIN,
LEFT_OUTER_JOIN and NATURAL_LEFT_OUTER_JOIN that took relations as
arguments, the SELECT and SELECT_DISTINCT that took what they returned,
and the types BaseJoinClause, JoinClause, NaturalJoinClause,
JoinStatementWithoutCondition and JoinSelectStatement, with the ON,
USING, WHERE, GROUP_BY, ORDER_BY and LIMIT that took them. It is a
breaking change: such a join becomes
(table INNER_JOIN other USING column) SELECT X<R>().

With them gone, a join can start with its first relation, as in
PersonTable INNER_JOIN BookTable ON (...), without FROM: every join
operator is now an infix function of a relation too, which starts the
join FROM(relation) would. It couldn't be before, as such a function
took the place of the old CROSS_JOIN(CategoryTable) in the lambda of a
table, whose implicit receiver is a relation. FROM still starts a join.

The test of the old joins now covers the joins that start from a
relation, of every kind each platform has; the other tests and the
sample use them too. The sample only keeps the joins that its Student
type fits, as the new ones check it. The guide to advanced queries
starts joins from their first relation, and describes how to migrate.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2.5.0 deprecated the UNION {} and UNION_ALL {} blocks, and the
beginUnion, createUnionSelectStatement and endUnion functions they
used, for bugs that wouldn't be fixed: a block read its rows into its
first SELECT's result type while returning the table's row type, so a
union of projections, result columns or joins failed with a
ClassCastException when its results were used, and a nested block was
flattened into the SQL, which SQLite evaluates from left to right, so
that it changed its meaning. The infix UNION, UNION_ALL, INTERSECT and
EXCEPT replaced them. This removes them, with UnionSelectStatementGroup,
the container they collected their SELECTs in, so that a SELECT is now
always added to the scope, transaction or trigger body it is written
in. It is a breaking change: UNION { a; b } becomes (a) UNION (b).

The test of the block is removed, as the infix operators have their
own, and the guide to advanced queries describes the migration in place
of the deprecated block.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An observed query gives results, while a reader that reads on its own
and only has to learn when the tables it read change, such as a
PagingSource that invalidates itself, needs a signal. And a version
read apart from the read can't be sound, as a table only has one once
it is tracked, which the read itself starts.

Database#versioned { } runs a scope of SELECTs, and returns its result
with a TableVersions token of the versions of the tables they read.
Like observe, it tracks the tables, captures their versions and runs
the SELECTs under the executive lock, in that order, so that no change
after the read is missed. A scope that writes is rejected before any of
its statements runs, as its token couldn't tell its own changes from
others'. Tokens combine with +, which takes the earlier version of a
table that both read, so that the combined token is stale when either
is, as for a query built from the results of another; tokens of
another Database are rejected.

awaitChange suspends until a table of the token changes. onChange
registers a one-shot callback, and registers before it checks: when
the token is already stale, it removes the registration and returns
null, so that the caller acts on the change itself, as a PagingSource
returns LoadResult.Invalid(). The registrations are a list replaced
atomically, so that a registration is either fired by a writer or
removed by onChange, never both. Callbacks run on the thread that
wrote, after the writing scope, blocking or suspended, has released the
lock, so that they may use the database.

The tracker's weak registrations for F13 are left to F13.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
SQLlin had INSERT OR IGNORE and INSERT OR REPLACE. This adds the other
resolutions a statement can give: INSERT_OR_ABORT, INSERT_OR_FAIL and
INSERT_OR_ROLLBACK, which take objects, expressions, DEFAULT_VALUES and
a SELECT, as INSERT does, and UPDATE_OR_ABORT, UPDATE_OR_FAIL,
UPDATE_OR_IGNORE, UPDATE_OR_REPLACE and UPDATE_OR_ROLLBACK, written as
UPDATE. A statement's resolution takes the place of the conflict clause
of a constraint, which onConflict declares since F17: INSERT_OR_ABORT
fails where a UNIQUE ON CONFLICT IGNORE column would have skipped the
row, which is the use of a resolution that is otherwise the default.

Like INSERT_OR_IGNORE and INSERT_OR_REPLACE, the new INSERTs write the
primary key even when the database assigns it, as a conflict on it
couldn't be seen otherwise, so the three now share one builder,
Insert.insertOr, and Update.update takes the keyword it starts with.

The guide to modifying the database describes the resolutions in a new
section, including that in transaction { }, where a failing statement
rolls the transaction back, ABORT, FAIL and ROLLBACK only differ outside
one. The test checks that each failure is the constraint's, rather than
one of the SQL, and runs every form once successfully.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…VE (F27)

Since B5, every transaction begins IMMEDIATE, taking the write lock at
once. This lets a transaction begin in another TransactionMode:
EXCLUSIVE, which also keeps other connections from reading in DELETE
journal mode, and is IMMEDIATE in WAL mode; and DEFERRED, which takes
no lock until the transaction reads, nor the write lock until it
writes, but makes a transaction that reads before it writes fail at
once while another connection writes, without waiting for the busy
timeout.

In sqllin-driver, DatabaseConnection#beginTransaction(mode) begins one,
and withTransaction takes a mode. The new beginTransaction has a default
implementation, which only begins an IMMEDIATE transaction, so that a
DatabaseConnection implemented outside SQLlin still compiles; the JVM
and Kotlin/Native drivers run BEGIN with the mode, or a deferred BEGIN
on a read-only connection, as they did. Android's framework can't begin
a deferred transaction that writes, as it turns a BEGIN statement into
an EXCLUSIVE transaction of its own, so there DEFERRED begins an
IMMEDIATE one, which only takes the write lock earlier, while EXCLUSIVE
is its beginTransaction.

In the DSL, transaction(mode) { } begins one; a transaction in another
one joins it and keeps its mode. The guide to modifying the database
describes the modes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
INDEXED_BY makes a statement look its rows up through an index, and
NOT_INDEXED through none, rather than through the one SQLite's query
planner chooses: (PersonTable INDEXED_BY "idx_person_name") SELECT
WHERE(...) and PersonTable.NOT_INDEXED DELETE WHERE(...).

They return an IndexedTable, a Relation whose FROM is the table with the
clause, and whose name and columns are the table's, so that a SELECT
reads it as it reads the table, joins included, and an observed query
or a versioned read tracks the table. It isn't a Table, so INSERT,
which SQL doesn't allow it in, doesn't take it, which is checked at
compile time; UPDATE, the UPDATE_OR_ ones and DELETE take it through
overloads, which give the table with the clause to their builders as
the target they write.

The test shows the clauses are in the SQL of each statement by giving
an index that doesn't exist, which fails with "no such index". The
guide to modifying the database describes them with indexes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…to 3.54.1.0

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
sqllin-processor only gave its Java classes a JVM target, 21, through
the java block, which left its Kotlin classes, all of its code, to the
default: the JDK that runs the build. Built with JDK 25, they were Java
25 class files, major version 69, which a KSP run on an older JDK can't
load, failing with UnsupportedClassVersionError; the Kotlin Gradle
plugin warned of the inconsistent targets. CI, on JDK 21, built them
for 21.

The processor now uses the JVM toolchain of the other modules, JDK 21,
which compiles its Java and Kotlin classes for it, whichever JDK runs
the build: the jar's classes are major version 65.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
EXPLAIN_QUERY_PLAN(select) tells how SQLite would run a SELECT without
running it: which tables it scans, which indexes it searches, and where
it sorts with a temporary B-tree. Any SELECT can be planned, joins,
subqueries and compound ones included. It is taken out of the
statements the scope runs, as a subquery is, and planned with its
parameters, as it would run.

The QueryPlanStatement it returns is a statement of its own, rather
than a SELECT, so that it can't become a subquery or part of a
compound, which SQL doesn't allow, and isn't counted among the tables a
versioned read covers; it changes nothing. It reads the plan when it
runs and closes its cursor, as the plan is small. getResults() returns
a QueryPlan: its steps, each with its detail and its children, the
details of all of them, and a toString() that writes them as a tree, as
the sqlite3 shell does.

The plan is read by the columns SQLite returns, rather than by its
version, which no platform tells reliably: since SQLite 3.24.0 they are
id, parent, notused and detail, and a step's parent is the id of the
step it belongs to; before, as on Android below API 30, they tell no
such relation, so the steps are listed without children. The details
are SQLite's own text, which differs between versions, so they are
left as they are, and the guide to advanced queries says to check a
plan by the names of tables and indexes and by phrases such as USING
INDEX, rather than by whole details.

The helper that takes a SELECT out of the scope's statements is renamed
takeOut(), as it no longer serves subqueries only, and gets back its
documentation, which a later section had separated from it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A clause appended to a statement, as ORDER_BY, LIMIT, OFFSET, GROUP_BY,
HAVING, WHERE after result columns, and UPDATE's WHERE, replaced the
last statement of its container, whatever it was, as long as it was a
SELECT or an UPDATE without WHERE. When another statement had been built
since, as when a statement is held in a variable and completed later,
that statement was dropped and never ran, while the incomplete one still
did: an UPDATE held as `val update = table UPDATE SET(...)`, followed by
a SELECT and then completed with `update WHERE (...)`, updated every row
of the table. UPDATE's WHERE also added its parameters to the list of
the UPDATE without it, which then failed with too many parameters when
its SET had some.

The clauses now replace the statement they complete through
replaceStatement, which finds it wherever it is, as CREATE INDEX's WHERE
already did, and changeLastStatement is removed. A clause appended to a
statement that a clause was already appended to, or that became part of
another one, as a subquery, a compound SELECT or EXPLAIN QUERY PLAN,
fails with an IllegalStateException that says so, while the scope is
built, so that none of its statements runs, rather than replacing
another statement. UPDATE's WHERE merges the parameters into a new list.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A transaction in another one, which the documentation of DatabaseScope
says can be nested, ended the outer one when it ended, since 2.0.0 at
least: transaction { } called endTransaction() in its finally block
whether its beginTransaction() had begun a transaction or joined one,
and endTransaction() cleared the transaction being built. The
statements after the inner transaction were then no longer in the
transaction, so they ran on their own, and when one of them failed,
those before it weren't rolled back; a second transaction in the outer
one began a transaction of its own, in its own mode. The F27 test put
the inner transaction last, so it didn't show.

beginTransaction() and endTransaction() now count how deeply the
transactions are nested, and only the outermost endTransaction() ends
the transaction. The fix is in these two functions rather than in the
inline transaction { } and transaction(mode) { }, which call them, so
that it also applies to code compiled against an earlier version, whose
inlined bodies still call them as before, and to nested calls of the
two functions themselves. beginTransaction() still returns true only for
the outermost transaction. endTransaction() gets documentation that
says what it does.

The test rolls back an outer transaction whose statement after an inner
one fails, with two inner ones in a row, with modes, after an inner one
whose block threw, and with nested calls of beginTransaction() and
endTransaction(), and commits the statements of the inner ones with the
outer one; it fails without the fix.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
transactionStatementsGroup and transactionDepth were @volatile, which
suggested that a scope could be shared between threads. It can't: the
statements are collected in unsynchronized lists, and beginning a
transaction checks the depth, creates a group and adds it in several
steps, which neither @volatile nor an atomic would make safe. A scope is
used by the code that runs its block; a suspending block may resume on
another thread, but a coroutine's steps run one after another, its
dispatcher making the writes before a suspension visible after it, so
the fields need no @volatile, as the other fields of the scope have
none.

The documentation of DatabaseScope now says that a scope isn't
thread-safe, and that its functions mustn't be called from threads or
coroutines that run at the same time, as coroutines launched on a
dispatcher of several threads in the block of suspendedScope would.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…t 1)

The first part of F30: PRAGMA_INTEGRITY_CHECK(maxErrors) and
PRAGMA_QUICK_CHECK(maxErrors), which return the problems, or an empty
list rather than SQLite's "ok"; PRAGMA_FOREIGN_KEY_CHECK(), of all tables
or of one; PRAGMA_TABLE_INFO(table), of a table, a view or a name,
PRAGMA_INDEX_LIST(table), PRAGMA_INDEX_XINFO(index) and
PRAGMA_FOREIGN_KEY_LIST(table); PRAGMA_WAL_CHECKPOINT(mode), which a
transaction rejects, as SQLite fails it there with "database table is
locked"; PRAGMA_PAGE_COUNT(), PRAGMA_FREELIST_COUNT() and
PRAGMA_PAGE_SIZE(); and PRAGMA_DEFER_FOREIGN_KEYS(flag), which only a
transaction takes, as SQLite turns it off when one ends.

A PragmaStatement runs a PRAGMA as a query when it returns rows, and
with execSQL when it returns none, as Android's framework doesn't run a
statement that returns rows with execSQL, and sqlite-jdbc doesn't run one
that returns none as a query. It reads the rows when it runs and closes
its cursor; getResults() returns them once the scope has run, as
ColumnInfo, IndexInfo, IndexColumn, ForeignKeyInfo, ForeignKeyViolation,
a CheckpointResult or a value, in com.ctrip.sqllin.dsl.sql.pragma. A
PRAGMA changes no table, so it can be in a versioned read. Names are
written as string literals, and columns are looked up by name once there
is a row, as an older SQLite names no columns for a PRAGMA that returns
no rows.

PRAGMA_RECURSIVE_TRIGGERS and reading PRAGMA_FOREIGN_KEYS(), which the
first part was to have, are left out: on Android 13 and higher, a
database in WAL mode has several connections, and the framework steps a
PRAGMA that sqlite3_stmt_readonly() reports as read-only on another
connection than the one that writes, which only prepares it, so the
statements cached there keep running as they were compiled, which a test
on Robolectric API 37 showed for both; they need the PRAGMA to run on
every connection, which PRAGMA_FOREIGN_KEYS(flag) needs too.

The test reads auto_vacuum before checking freelist_count after a
delete, as Android's SQLite creates a database with auto_vacuum FULL,
which leaves no unused pages. The guide to modifying the database
describes the PRAGMAs in a new section. The ROADMAP leaves
case_sensitive_like out of F30, as SQLite deprecates it, and adds F67,
the PRAGMAs that set how each connection runs statements, as parameters
of the configurations.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…nnection on Android (B12)

On Android 13 and higher, PRAGMA foreign_keys = ON, and OFF, and PRAGMA
recursive_triggers, run with execSQL, as the DSL's PRAGMA_FOREIGN_KEYS
does, didn't apply to the statements whose SQL had run before: they kept
checking foreign keys or not, and firing triggers recursively or not.
There, a database in WAL mode has several connections, the one that
writes and those that read outside a transaction. The framework prepares
such a PRAGMA on the connection that writes, but steps it on one that
reads, as sqlite3_stmt_readonly() reports it doesn't write. SQLite
changes the setting when it prepares the PRAGMA, but only expires the
statements the connection has compiled when it steps it, so the
statements the connection that writes had cached kept running as they
were compiled, while those it compiled afterwards had the new setting.
On Robolectric API 37, an INSERT whose SQL had run before
PRAGMA_FOREIGN_KEYS(true) inserted a row that violated a foreign key.

The Android driver's execSQL now runs these two PRAGMAs, outside a
transaction, on every connection of the database, those opened later
included, with execPerConnectionSQL, as the busy timeout is set, so that
each connection steps them. Inside a transaction, which runs on the
connection that writes, it runs them there, where SQLite ignores a
change of foreign_keys, as on the other platforms, and applies
recursive_triggers to the statements after it. Below Android 13, a
database that SQLlin opens has a single connection, and nothing changes.
No API changes: the driver recognizes the two PRAGMAs in the SQL.

The test fails without the fix on API 37, and passes on API 26 and the
other platforms either way.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The second part of F30, with the two the first part left out, which B12
made work on every connection:

- PRAGMA_FOREIGN_KEYS(), which reads what PRAGMA_FOREIGN_KEYS(flag) sets;
  PRAGMA_RECURSIVE_TRIGGERS(flag), PRAGMA_LEGACY_ALTER_TABLE(flag) and
  PRAGMA_IGNORE_CHECK_CONSTRAINTS(flag), which the Android driver now
  runs on every connection too, as it does foreign_keys;
- PRAGMA_INTEGRITY_CHECK(table) and PRAGMA_QUICK_CHECK(table), which need
  SQLite 3.33.0, below which they check the whole database;
- PRAGMA_TABLE_XINFO(table), with ColumnInfo.kind for the hidden and
  generated columns, which needs SQLite 3.26.0, and PRAGMA_TABLE_LIST(),
  which needs 3.37.0;
- PRAGMA_COMPILE_OPTIONS(), which Android's SQLite is built without,
  PRAGMA_COLLATION_LIST(), PRAGMA_DATABASE_LIST(), and
  PRAGMA_FUNCTION_LIST(), PRAGMA_MODULE_LIST() and PRAGMA_PRAGMA_LIST(),
  which need SQLite 3.30.0;
- PRAGMA_APPLICATION_ID() and PRAGMA_APPLICATION_ID(id), and
  PRAGMA_AUTO_VACUUM() and PRAGMA_AUTO_VACUUM(mode).

SQLite ignores a PRAGMA it lacks rather than failing, so those that only
newer versions or some builds have check that the rows SQLite returns
have their first column, and throw an IllegalStateException that tells
what they need rather than return nothing, without asking SQLite for its
version; they are marked with @PlatformDependentSQLiteAPI.

SQLite takes auto_vacuum and page_size from what the connection last read
of the header of the file, which it reads again when a transaction
begins, so they are read in a transaction of their own outside one: on
Robolectric API 37, auto_vacuum read outside a transaction, on a
connection that reads, still told FULL after INCREMENTAL and VACUUM.

Left out: incremental_vacuum, which returns a row without columns for
each page it removes, so that sqlite-jdbc runs one step of it and can't
query it, and Android's framework below 13 rejects it in execSQL;
journal_size_limit, wal_autocheckpoint, max_page_count, secure_delete and
cell_size_check, which each connection has, and some of which Android's
framework sets on each connection it opens, so they go to F67.

Android's SQLite turns legacy_alter_table on by default, which the test
and the documentation take into account, and its own test reads the
auto_vacuum and the build options of the SQLite it runs on. The guide to
modifying the database describes them; the ROADMAP moves F30 to
Supported.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
On Kotlin/Native and on Android 8.1 and lower, journalMode didn't take
effect, since 1.2.0: a database stayed in the journal mode it was in,
DELETE for a new one on Kotlin/Native and that of the framework on
Android, rather than WAL, the default. updateJournalMode ran
`PRAGMA journal_mode=WAL` as a query whose cursor it never read, and a
query there only runs as its cursor is read: NativeCursor steps the
statement in next(), and Android's framework fills its window when the
cursor is first moved. On Robolectric API 26, the mode stayed `memory`,
and became `wal` once the row was read. The JVM sets the mode through
SQLiteConfig, and Android 9 and higher through OpenParams.

The row is now read. A connection that is actually read-only, as one
that isReadOnly opens where the file can't be written, keeps the mode
the file is in, as SQLite writes WAL mode to the header of the file:
switching a file in DELETE mode to WAL fails there with "attempt to
write a readonly database".

testJournalMode checks the mode after opening with the default, with
DELETE, and with the default again, and fails on API 26 without the fix.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
PRAGMA incremental_vacuum removes a page each time it returns a row,
which has no columns, so it has to run as a query whose rows are all
read. F30 left it out as "no driver steps through it", which was wrong:
measured, Android's framework runs it fully as a query, on API 26 and
37, though execSQL removes one page and then throws, and Kotlin/Native
would, as it steps each row of a query. Only sqlite-jdbc can't: it
doesn't run a statement that returns no columns as a query, failing
with "Query does not return results", and runs it as a statement, as
DB.exec does too, only as far as its first row, removing one page.

The KDoc of PRAGMA_AUTO_VACUUM(mode), the guide to modifying the
database, in English and Chinese, the CHANGELOG entry of F30 and the
ROADMAP line of F30 now say so.

The JVM would have to run `PRAGMA incremental_vacuum(1)` once for each
page, in a transaction, and few databases are in INCREMENTAL mode, which
Android's SQLite doesn't create them in, so it goes to a new part of the
ROADMAP, Shelved, of the items that aren't planned until a need for them
comes up, as F68.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…n created

A connection reads the schema once and keeps it, and when the PRAGMAs
that read the schema of a table or an index by its name don't find it,
SQLite doesn't always check that schema is still current: index_list,
index_info, index_xinfo and foreign_key_list never do, up to its trunk,
and table_info and table_xinfo don't before 3.33.0, Android 13 and
lower. So a table or index that another connection, such as one in
another process, has just created looks as if it didn't exist, until
this connection runs a statement that reads or writes a table, which
does check it. Measured on SQLite 3.43.2 with two connections, and on
Robolectric API 33.

A connection's own DDL updates its schema at once, and on Android 13
and higher, where a Database reads through several connections, the
framework closes those that read after each DDL that runs through
execSQL, so a single Database, which SQLlin's apps almost always are,
doesn't meet it. It isn't worked around: that would cost an extra query,
and a transaction on Android, for every such PRAGMA. The guide to
modifying the database, in English and Chinese, now says so where it
says a table or index that doesn't exist has none.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
F46 already added the COLLATE operator, with BINARY, NOCASE and RTRIM,
for conditions and ORDER BY, so two parts of F19 were left:

- @CollateRTrim, experimental, for COLLATE RTRIM in a column's
  definition, which compares the column's values ignoring the spaces at
  their ends, as its UNIQUE constraint and indexes do, while storing them
  as they are written. The processor reports a property that has both it
  and @CollateNoCase, as a column has one collation, and one that isn't a
  String or a Char, and rejects it on views and FTS tables, as it does
  the other constraints. A column needs no @CollateBinary, as BINARY is
  its default.
- COLLATE on the columns of an index. CREATE_INDEX already wrote the
  operator's expression without the table's name, and SQLite reads
  `(name COLLATE NOCASE)` in an index as the column with that collation,
  as index_xinfo tells, rather than as an expression, so it only needed
  saying and testing, with an order too, as
  `(name COLLATE NOCASE) to DESC`.

testCollations checks the column's SQL, a lookup that ignores the
trailing spaces of a stored value, which keeps them, the UNIQUE
constraint that rejects the value without them, the collations and
orders of two indexes through PRAGMA_INDEX_XINFO, one of them BINARY on a
NOCASE column, and that a query comparing by NOCASE uses the NOCASE
index, through EXPLAIN_QUERY_PLAN.

The guides to getting started, to modifying the database and to SQL
functions, in English and Chinese, the KDoc of CREATE_INDEX and of
COLLATE, the CHANGELOG and the ROADMAP, where F19 moves to Supported,
say so.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`PersonTable DELETE WHERE(PersonTable.age LT 18) ORDER_BY (PersonTable.age
to ASC) LIMIT 10`, and the same after DELETE X, UPDATE ... WHERE and an
UPDATE without WHERE, with OFFSET after LIMIT, experimental.

SQLite only takes ORDER BY and LIMIT in a DELETE or UPDATE when it is
built with SQLITE_ENABLE_UPDATE_DELETE_LIMIT. sqlite-jdbc and Apple's
SQLite are; Android's isn't, as AOSP's Android.bp shows, nor the
libraries SQLlin links on Linux and Windows, where the statement fails
with `near "ORDER": syntax error`, as it does on Python's SQLite 3.53.4.
When it is, sqlite3LimitWhere() turns them into a subquery of the rowids
of the rows they choose, so SQLlin writes that subquery itself, which
every SQLite runs:

    DELETE FROM person WHERE rowid IN (SELECT rowid FROM person
        WHERE person.age<? ORDER BY person.age ASC LIMIT 10)

EXPLAIN QUERY PLAN of the two is the same, for DELETE and UPDATE, and
with INDEXED BY, which SQLite keeps for the subquery while the statement
finds the rows by their rowids: SQLlin puts INDEXED BY and NOT INDEXED in
the subquery only, as with both the statement scans the index. Unlike
SQLite's syntax, the subquery is allowed in the body of a trigger. A
table with a column named rowid, which rowid then means, reads its rowid
as _rowid_ or oid, which FTS tables take too.

ORDER BY without LIMIT, which SQLite rejects, fails the scope with an
IllegalArgumentException before any of its statements runs: an
UPDATE or DELETE with ORDER BY is an OrderByUpdateDeleteStatement, whose
checkComplete() throws. checkComplete() moves to SingleStatement, which
SelectStatement overrides, so that the scope, a transaction and
CREATE_TRIGGER check every statement alike.

DELETE X and DELETE WHERE(...), of a Table and of an IndexedTable, now
return a WhereUpdateDeleteStatement, which ORDER_BY and LIMIT follow,
rather than Unit, which only needs code compiled against an earlier
version to be compiled again; UPDATE ... WHERE returns one too rather
than the String of its SQL, which is a Breaking change. Their parts are
kept in UpdateDeleteParts, from which each clause builds the next
statement. ROADMAP F15 now says that WITHOUT ROWID tables will have to
choose the rows by their primary key.

testUpdateDeleteOrderByLimit covers DELETE with WHERE and without,
UPDATE with WHERE and without, with the parameters of SET before those
of WHERE, OFFSET with a negative LIMIT, LIMIT without ORDER BY,
INDEXED BY, a column named rowid, a transaction, a trigger that keeps the
latest rows, and ORDER BY without LIMIT in a scope, in a transaction and
in a trigger, after which nothing ran.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A new module, com.ctrip.kotlin:sqllin-dsl-paging, on every platform of
sqllin-dsl, which androidx.paging's paging-common 3.5.3, its latest
release, has too. `database.pagingSource { PersonTable SELECT
ORDER_BY(PersonTable.id to ASC) }`, experimental, returns a
PagingSource<Int, T> that loads the rows a page at a time by their
offsets, as Room's LimitOffsetPagingSource does:

- a page is `SELECT * FROM (select) LIMIT ... OFFSET ...`, so the SELECT
  can have a WHERE, a join or compound operators; the first page also
  counts the rows, `SELECT count(*) FROM (select)`, so that each page
  tells how many rows are before and after it, and a refresh past the
  last row loads the last rows;
- the keys are offsets at the borders between pages: a prepend loads at
  most as many rows as there are before its key, so that two pages never
  share a row wherever a refresh starts, which needs no grid of the page
  size, as the ROADMAP said; a page after the first row can always
  prepend, as after a RemoteMediator invalidates the source;
- getRefreshKey starts half the initial load before the row last read,
  adding the first page's itemsBefore without placeholders, whose
  anchorPosition doesn't count the rows before it.

It is built on the table version tokens of F66: it registers with
onChange from its first page, through a callback that only holds the
source weakly, so that a source its Pager drops can be collected, and
closes the registration when it is invalidated otherwise. A page read
after a change since the first is LoadResult.Invalid, and every Invalid
is returned through invalidate(), so that the source is invalid once it
returns one, also for a TestPager, which doesn't invalidate it as a
Pager does; a source invalidated before it loads reads nothing. Pages
load in Dispatchers.IO by default.

The module can only reach sqllin-dsl's public API, so sqllin-dsl marks
what it loads through with the new @InternalSQLlinAPI, an opt-in of
ERROR level for SQLlin's own modules: Database.loadPage, LoadedPage and
TableVersions.isStale, over the internal DatabaseScope.page() and
RowCountStatement, which reads its count under the lock it runs under.

The module's build script, which a template made, now follows
sqllin-dsl's: its package, targets without iosX64, the SDK levels of the
version catalog and Maven Central publishing; the template's lint
plugin, kotlin-stdlib, kotlin-test, test runner and junit entries, its
.gitignore, manifest and example tests are gone. CI builds it on each
host, and publish_all.sh publishes it.

CommonPagingTest, in sqllin-dsl-test, loads it through TestPager and a
Pager: the keys and counts of refreshes, appends and prepends, at the
start, the middle, the end and past it; the refresh key with
placeholders and without; invalidation by a change, not by one of
another table nor before the first page, before loading, between the
two reads of a refresh, and before a later page; WHERE, an outer join
and a compound SELECT; and a Pager that reloads around the position read
after a delete. Each of four fixes that are undone fails it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A property of a type SQLite doesn't store can be a column when its
serializer encodes it as one value: the serializer kotlinx.serialization
writes the property with is the converter, so that no annotation of
SQLlin's is needed. That covers the built-in serializers of Uuid,
Instant and Duration, value classes, types whose classes declare their
serializers, as kotlinx-datetime's do, and any type, a List stored as
JSON included, given a serializer with @serializable(with = ...) on the
property, on the use of its type or in a type alias, or by
@file:UseSerializers.

- sqllin-processor finds that serializer as the plugin does, and
  generates a ClauseConverted element that holds it, with a SET {}
  setter that stores what it encodes; a type no column can hold, a
  collection or a class serialized as a structure, or a type serialized
  contextually, without a SerializersModule to find its serializer in,
  is an error whose message says how to give it one.
- The column's type is the one the serializer stores, which the table's
  createSQL, then a getter, computes with Table#columnType, and which
  ALTER_ADD_COLUMN now computes too, through the kind of the descriptor;
  a serializer that encodes a structure fails CREATE, as does a
  collation of a column that doesn't store text. The tables of every
  other entity are generated as before, byte for byte.
- ClauseConverted, experimental, converts the values of EQ, NEQ, LT,
  LTE, GT, GTE, IN, of values and of a subquery, BETWEEN, IS and ISNOT
  with the serializer, and checks that it is the property's, so that a
  serializer the processor took wrongly fails rather than compares other
  values than those stored. Comparisons and ORDER BY compare the stored
  values, which the guide warns the ISO-8601 strings of Instant aren't
  in the order of.

B14: an enum whose class declares a serializer, as one storing its
name, or any property given a serializer of its own, was written by it
in INSERT but compared and set as its type is by default, as an enum's
ordinal, in a column of INT, so that conditions didn't match and a row
SET {} changed couldn't be read. Such a property is now converted by
its serializer, which changes the type of its element: a breaking
change. An enum with its default serializer, or serialized
contextually, is still stored as its ordinal.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An entity annotated with an opt-in marker, as @ExperimentalUuidApi on
its class, requires every use of the class to opt in, which the object
generated for it, a Table, a View or an FtsTable of it, didn't, so it
didn't compile, with a converted column or without. The object now opts
in to the markers that the class, its properties and its file are
annotated with, the annotations that are themselves annotated with
@RequiresOptIn, besides those that their @optins give, which a table
with converted columns needs for their types. The objects of the other
entities are generated as before.

Found by the review of F12.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
It could let the table version tokens of F66, and the PagingSource of
F13, see the commits of other connections and processes, which they
don't, but it doesn't tell which tables changed, and each connection
counts on its own, which Android's pool of connections needs a design
for. It was left out of F30 without being written down.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A SELECT kept its cursor, and its statement, open until its results
were read, which they may never be, as the SELECTs of versioned { }
whose token alone is wanted. Meanwhile the statement kept reading its
tables, which couldn't be dropped, failing with "database table is
locked", and on Kotlin/Native it kept the connection open after
close(), as sqlite3_close_v2 waits for the statements to end, holding
the database file. Windows can't delete an open file, so on CI every
test after testTableVersions opened its database, with its rows and
without the tables of its own configuration, and 78 of 116 failed;
Linux and macOS delete it anyway.

The SELECT now reads and decodes its rows when it runs, under the lock
of its scope and in its transaction, and closes the cursor, so that its
statement ends with the scope; getResults() returns the rows, and an
error decoding them is thrown by the scope. testUnreadSelect drops the
table of an unread SELECT, which failed with SQLITE_LOCKED on the JVM,
and deletes the closed database, which failed on Windows.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The macOS, Windows and Linux jobs uploaded sqllin-dsl/build/reports on
failure, but the DSL's tests run in sqllin-dsl-test, so that the report
of a failed run, as that of the Windows tests of #135, wasn't uploaded.
The Android job already uploaded the right one.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@qiaoyuang
qiaoyuang merged commit f484f64 into release/2.6.0 Nov 8, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant