-
Notifications
You must be signed in to change notification settings - Fork 22
CASSANALYTICS-116 Fix ByteBuffer flip() in StreamBuffer.copyBytes() causing data corruption #165
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
bbotella
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good to me
a58005f to
57732bb
Compare
|
VertxStreamBuffer.copyBytes() was calling flip() after each write, resetting buffer position to 0. When multiple chunks were written to the same buffer, subsequent chunks overwrote previous data instead of appending, causing Stream API failures. Example:
Solution:
tests passed without flip() because:
However, flip() remains essential for correctness in partial-fill scenarios (e.g., last buffer in file, or when less data is available than buffer capacity). |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We should avoid the changes in the analytics-sidecar-client and the other sidecar-copied subprojects at the best. Those files are supposed to be in sync with Sidecar.
I would revert the changes in this file and the test file. The VertxStreamBuffer and its test in another package.
| * Copies bytes from this {@link StreamBuffer} into the {@link ByteBuffer destination}. | ||
| * <p> | ||
| * This method writes {@code length} bytes starting at the destination buffer's current position | ||
| * and advances the position by {@code length}. The caller is responsible for calling | ||
| * {@link ByteBuffer#flip()} on the destination buffer before reading from it. | ||
| * |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for adding the docs. Let's actually add the docs in org.apache.cassandra.spark.utils.streaming.StreamBuffer instead, not this file.
| @@ -1,5 +1,6 @@ | |||
| 0.3.0 | |||
| ----- | |||
| * Fix ByteBuffer flip() in StreamBuffer.copyBytes() causing data corruption (CASSANALYTICS-116) | |||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I believe the description is no longer accurate. Please update to reflect the actual fix.
8aedd87 to
589ad78
Compare
…ausing data corruption Patch by Jyothsna Konisa; Reviewed by Yifan Cai and Bernardo Botella for CASSANALYTICS-116
589ad78 to
9f31559
Compare
No description provided.