Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Integrate UploadHandler in BufferedWriteHandler struct #2670

Open
wants to merge 4 commits into
base: uploadHandler
Choose a base branch
from

Conversation

ashmeenkaur
Copy link
Collaborator

Description

This PR integrates the UploadHandler into the BufferedWriteHandler to enable efficient and synchronized uploads of filled blocks to GCS.

Key changes

  • Trigger block uploads: When a block in the BufferedWriteHandler is complete, it's automatically sent to the UploadHandler for asynchronous uploading.
  • Implement Flush method: A Flush method is added to BufferedWriteHandler. This method ensures current block is sent for upload and then calls the Finalize method on the UploadHandler to finalize the GCS object.

Link to the issue in case of a bug fix.

NA

Testing details

  1. Manual - Manually verified the happy path upload of a file with buffered writes enabled.
  2. Unit tests - updated
  3. Integration tests - NA

@ashmeenkaur ashmeenkaur force-pushed the buffered_write_handler_changes branch 2 times, most recently from a251009 to 84c159d Compare November 11, 2024 06:05
@ashmeenkaur ashmeenkaur changed the base branch from master to uploadHandler November 11, 2024 06:05
@ashmeenkaur ashmeenkaur marked this pull request as ready for review November 11, 2024 06:08
@ashmeenkaur ashmeenkaur requested a review from a team as a code owner November 11, 2024 06:08
Copy link

codecov bot commented Nov 11, 2024

Codecov Report

Attention: Patch coverage is 80.00000% with 3 lines in your changes missing coverage. Please review.

Project coverage is 77.58%. Comparing base (039429f) to head (fa29ea7).

Files with missing lines Patch % Lines
internal/bufferedwrites/buffered_write_handler.go 78.57% 2 Missing and 1 partial ⚠️
Additional details and impacted files
@@                Coverage Diff                @@
##           uploadHandler    #2670      +/-   ##
=================================================
+ Coverage          77.55%   77.58%   +0.02%     
=================================================
  Files                112      112              
  Lines              15797    15803       +6     
=================================================
+ Hits               12252    12261       +9     
+ Misses              3022     3019       -3     
  Partials             523      523              
Flag Coverage Δ
unittests 77.58% <80.00%> (+0.02%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@ashmeenkaur ashmeenkaur force-pushed the uploadHandler branch 2 times, most recently from e5945ac to 039429f Compare November 13, 2024 06:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants