Skip to content

[Query] Release UDF query scope when IoTDBLocal rejects a non-query statement - #19240

Merged
JackieTien97 merged 1 commit into
masterfrom
ty/iotdblocal-rejected-query-cleanup
Nov 12, 2026
Merged

JackieTien97 merged 1 commit into
masterfrom
ty/iotdblocal-rejected-query-cleanup

Conversation

@JackieTien97

Copy link
Copy Markdown
Contributor

Description

Follow-up to #19107. When IoTDBLocal.query() rejects a statement that is not a query, the QueryExecution built for that statement never finalizes its UDF query scope. The class loaders pinned for the UDFs used by the statement therefore stay open after CREATE FUNCTION deprecates them.

Scenario

A table scalar UDF localq calls IoTDBLocal.query() in beforeStart with a write statement that uses the external UDF lazy_f:

local.query("INSERT INTO db.t2(time, device_id, v) SELECT time, device_id, lazy_f(v) FROM db.t");

The statement is rejected as expected with Only query is supported for IoTDBLocal query interface. After SELECT localq(v) FROM one runs 5 times, a CREATE FUNCTION deprecates the class loader that defined lazy_f:

Class loader that defined lazy_f
Before activeQueriesCount=5, deprecated, never closed
After activeQueriesCount=0, deprecated, closed

Read-only internal queries such as SELECT lazy_f(v) FROM db.t already left activeQueriesCount=0, whether the UDF closes the result set or leaves it to the framework. They are unchanged.

Cause

Coordinator.execution() constructs the QueryExecution first. Its analysis runs inside the execution's UDF query scope, where reflect() pins the class loader that defined each external UDF. Only then does InternalQueryExecutor.validateReadOnlyQuery() throw SemanticException for a statement that is not a query. At that point the execution is not in queryExecutionMap and has not been started, so cleanupQueryExecution() cannot find it and nothing calls stopAndCleanup(). The scope is never finalized.

Fix

Coordinator.execution() now calls stopAndCleanup() on the rejected execution and rethrows the same exception, so the message and status code returned to the caller do not change.

  • For a QueryExecution that was never started, stopAndCleanup() releases what the analysis acquired and finalizes the UDF query scope. ConfigExecution.stopAndCleanup() does nothing.
  • Nothing else between construction and queryExecutionMap.put() / start() can throw. Construction failures are already handled by analyzeWithCleanupOnFailure(), and start() finalizes the scope itself when it fails.

Validation

  • The new CoordinatorTest#testRejectedReadOnlyInternalQueryClosesUDFQueryScope calls Coordinator.execution(..., readOnlyInternalQuery = true, ...) with a real QueryExecution whose analysis marks the statement as WRITE. It checks that:

    • the same SemanticException is thrown;
    • the execution is not registered;
    • its UDF query scope is closed.

    On master (54bff3d) it fails at the scope check, and with the fix it passes.

  • With the fix on master, these 7 classes pass (52 tests) with Spotless and Checkstyle enabled: CoordinatorTest, QueryExecutionRetryTest, QueryOwnershipTest, LoadTsFileAnalyzerDiagnosticsTest, LoadTsFileDataTypeConverterSchemaTest, CteMaterializerTest and CteSubqueryTest.

  • The scenario above ran on a 1C1D cluster built from 449a2b6, the head merged as Protect UDF classloader lifetime with query scopes #19107, once without and once with the fix. With the fix on that head, IoTDBLocalIT (TableSimpleIT) passed: 22 tests passed, including testRejectMutatingStatementsViaIoTDBLocal, and 1 was skipped because it requires cluster execution. The query execution, IoTDBLocal and UDF class loader code is unchanged on master since that head.


This PR has:

  • been self-reviewed.
  • added comments explaining the "why" and the intent of the code wherever would not be obvious
    for an unfamiliar reader.
  • added unit tests or modified existing tests to cover new code paths, ensuring the threshold
    for code coverage.
  • been tested in a test IoTDB cluster.

Key changed/added classes (or packages if there are too many classes) in this PR
  • Coordinator
  • CoordinatorTest (new)

🤖 Generated with Claude Code

IoTDBLocal.query() runs its statement through Coordinator.execution() with
readOnlyInternalQuery set. The QueryExecution is constructed and analyzed
first, then InternalQueryExecutor.validateReadOnlyQuery() rejects a statement
that is not a query. At that point the execution has not been registered in
queryExecutionMap and has not been started, so nothing ever called
stopAndCleanup() on it. The UDF query scope opened for its analysis was never
finalized, and the class loaders pinned for the UDFs used by the statement
stayed open after CREATE FUNCTION deprecated them.

Clean up the rejected execution before rethrowing the same exception, so the
error returned to the caller does not change. Add a regression test that
drives Coordinator.execution() with a statement whose analysis marks it as a
write.
@sonarqubecloud

Copy link
Copy Markdown

Quality Gate Failed Quality Gate failed

Failed conditions
C Reliability Rating on New Code (required ≥ A)

See analysis details on SonarQube Cloud

Catch issues before they fail your Quality Gate with our IDE extension SonarQube for IDE

@codecov

codecov Bot commented Nov 12, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 60.00000% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 48.94%. Comparing base (54bff3d) to head (64f04e6).
⚠️ Report is 1 commits behind head on master.

Files with missing lines Patch % Lines
.../apache/iotdb/db/queryengine/plan/Coordinator.java 60.00% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff            @@
##             master   #19240   +/-   ##
=========================================
  Coverage     48.94%   48.94%           
  Complexity      689      689           
=========================================
  Files          5589     5589           
  Lines        413857   413861    +4     
  Branches      54594    54594           
=========================================
+ Hits         202544   202581   +37     
+ Misses       211313   211280   -33     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@JackieTien97
JackieTien97 merged commit 364ff1f into master Nov 12, 2026
41 of 42 checks passed
@JackieTien97
JackieTien97 deleted the ty/iotdblocal-rejected-query-cleanup branch November 12, 2026 10:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant