Skip to content

Conversation

@haridsv
Copy link
Contributor

@haridsv haridsv commented Feb 10, 2026

Summary

  • Adds new configuration property phoenix.mutate.preserveOnLimitExceeded that changes mutation limit behavior, which can be overridden at the connection level via connection properties.
  • When enabled, limit checks occur BEFORE state modification, preserving existing buffered mutations
  • Introduces MutationLimitReachedException (for executeUpdate) and MutationLimitBatchException (for executeBatch) to signal recoverable limit conditions
  • Allows applications to commit buffered mutations and continue processing instead of losing state, which facilitates "dynamic sizing" for the batch.

Test plan

  • Added integration tests for row count and byte size limits with preserve mode
  • Added tests for executeBatch with autoCommit on/off
  • Added tests for edge cases: row merge, conflicting rows (ON DUPLICATE KEY), constructor limit exceeded
  • Added test for legacy behavior (preserveOnLimitExceeded=false)
  • Verified estimatedSize tracking for conflicting rows

JIRA: PHOENIX-7759

@haridsv
Copy link
Contributor Author

haridsv commented Feb 10, 2026

@sanjeet006py Good eye on catching the gaps! Yes, both would be backwards incompatible but I didn't make it conditional because both are pretty corner case scenarios that are very unlikely to be of practical use cases. In fact, it can be considered a good idea if some remote use case is slipping through larger mutations than intended, as they would get caught and will make the developers to make adjustments, after all these can be considered as bug fixes. However, I am open to make them conditional if anyone has a solid counter argument.

@haridsv
Copy link
Contributor Author

haridsv commented Feb 11, 2026

The spotless errors are outside the changed files in the PR and they are coming even though I already ran spotless.

@haridsv
Copy link
Contributor Author

haridsv commented Feb 11, 2026

Most of checkstyle errors are also in existing lines. Only the below are in the changed lines:

./phoenix-core-client/src/main/java/org/apache/phoenix/execute/MutationState.java:2469:            colValuesSizeDiff -= (entry.getKey().getEstimatedSize() + oldValue.length);�:31: Unnecessary parentheses around assignment right-hand side. [UnnecessaryParentheses]
./phoenix-core-client/src/main/java/org/apache/phoenix/query/QueryServices.java:100:  public static final String PRESERVE_MUTATIONS_ON_LIMIT_EXCEEDED_ATTRIB =�:17: Redundant 'final' modifier. [RedundantModifier]

The second one is following the existing pattern so I will leave it as is for consistency. I will fix the first one.

@haridsv
Copy link
Contributor Author

haridsv commented Feb 11, 2026

There us ibe junit test failure:

[org.apache.phoenix.end2end.LongViewIndexDisabledBaseRowKeyMatcherIT.testViewsWithExtendedPK](https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-2371/1/testReport/org.apache.phoenix.end2end/LongViewIndexDisabledBaseRowKeyMatcherIT/testViewsWithExtendedPK/)
 Error Details
isMultiTenant(false), extendPK(true), partition(640), tenant(3), rowId(6120), pkInfo(ID1,ID2,ID3), testInfo(DECIMAL,DECIMAL,INTEGER), sortInfo(DESC,DESC,DESC)
 Stack Trace
java.lang.AssertionError: isMultiTenant(false), extendPK(true), partition(640), tenant(3), rowId(6120), pkInfo(ID1,ID2,ID3), testInfo(DECIMAL,DECIMAL,INTEGER), sortInfo(DESC,DESC,DESC)
	at org.junit.Assert.fail(Assert.java:89)
	at org.apache.phoenix.end2end.BaseRowKeyMatcherTestIT.createViewHierarchy(BaseRowKeyMatcherTestIT.java:983)
	at org.apache.phoenix.end2end.BaseRowKeyMatcherTestIT.createViewHierarchy(BaseRowKeyMatcherTestIT.java:917)
	at org.apache.phoenix.end2end.BaseRowKeyMatcherTestIT.testViewsWithExtendedPK(BaseRowKeyMatcherTestIT.java:795)
...
 Standard Error
...
2026-02-09T22:47:18,744 ERROR [main] end2end.BaseRowKeyMatcherTestIT(981): 1
java.lang.ArrayIndexOutOfBoundsException: 1
	at org.apache.phoenix.schema.types.PDataType.getDecimalPrecisionAndScale(PDataType.java:767)
	at org.apache.phoenix.schema.types.PDecimal.isSizeCompatible(PDecimal.java:335)
	at org.apache.phoenix.schema.PTableImpl.newKey(PTableImpl.java:1251)
	at org.apache.phoenix.compile.UpsertCompiler.setValues(UpsertCompiler.java:181)
	at org.apache.phoenix.compile.UpsertCompiler.access$500(UpsertCompiler.java:125)
	at org.apache.phoenix.compile.UpsertCompiler$UpsertValuesMutationPlan.execute(UpsertCompiler.java:1393)
	at org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:665)
	at org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:612)
	at org.apache.phoenix.call.CallRunner.run(CallRunner.java:52)
	at org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:612)
	at org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:593)
	at org.apache.phoenix.jdbc.PhoenixPreparedStatement.execute(PhoenixPreparedStatement.java:182)
	at org.apache.phoenix.end2end.BaseRowKeyMatcherTestIT.upsertTenantViewRows(BaseRowKeyMatcherTestIT.java:428)
	at org.apache.phoenix.end2end.BaseRowKeyMatcherTestIT.createViewHierarchy(BaseRowKeyMatcherTestIT.java:972)
	at org.apache.phoenix.end2end.BaseRowKeyMatcherTestIT.createViewHierarchy(BaseRowKeyMatcherTestIT.java:917)
	at org.apache.phoenix.end2end.BaseRowKeyMatcherTestIT.testViewsWithExtendedPK(BaseRowKeyMatcherTestIT.java:795)
...

This can't be related to my change and must be a flapper.

Copy link
Contributor

@sanjeet006py sanjeet006py left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, Thanks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants