Implement multipart upload(AST-24859) #1343
Open
+613
−9
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
cx-sumit-morchhale
requested review from
cx-anjali-deore,
cx-anurag-dalke and
cx-umesh-waghode
as code owners
cx-sumit-morchhale
changed the title
cx-sumit-morchhale
changed the title
|
Great job! No new security vulnerabilities introduced in this pull request |
cx-sumit-morchhale
force-pushed
the
feature/AST-24859
branch
from
06bb69c to
5b95d6d
Compare
cx-sumit-morchhale
force-pushed
the
feature/AST-24859
branch
from
5b95d6d to
a8c7e42
Compare
cx-anurag-dalke
requested changes
Nov 17, 2025
| flagResponse, _ := featureFlagsWrapper.GetSpecificFlag(wrappers.IncreaseFileUploadLimit) | ||
| if flagResponse.Status && fileInfo.Size() > MaxSizeBytes { | ||
| // File size >5GB, proceed with multipart upload | ||
| logger.PrintIfVerbose("File size >5GB and INCREASE_FILE_UPLOAD_LIMIT flag is enabled,hence uploading file in multiple parts...") |
Collaborator
There was a problem hiding this comment.
update message & removed Flag name from log message
Collaborator
There was a problem hiding this comment.
Uploading source code in multiple parts.
| preSignedURL, zipFilePathErr = uploadsWrapper.UploadFileInMultipart(zipFilePath, featureFlagsWrapper) | ||
| } else { | ||
| // File size is within <=5GB, proceed with upload in single part | ||
| logger.PrintIfVerbose("File size is within the limit and uploading file in a single part...") |
| // calculate file size and compare with 5GB limit | ||
| fileInfo, err := os.Stat(sourcesFile) | ||
| if err != nil { | ||
| return nil, errors.Wrapf(err, "Failed to stat - %s", sourcesFile) |
Collaborator
There was a problem hiding this comment.
remove error condition
| // Generate presigned URL | ||
| presignedURL, err := getPresignedURLForMultipartUploading(startMultipartUploadResponse, partNumber) | ||
| if err != nil { | ||
| return nil, fmt.Errorf("failed to get presigned URL for part%d - %s", partNumber, err.Error()) |
Collaborator
There was a problem hiding this comment.
Failed - change f to F
| // Upload part | ||
| etag, err := uploadPart(presignedURL, partPath, featureFlagsWrapper) | ||
| if err != nil { | ||
| return nil, fmt.Errorf("failed to upload part%d - %s", partNumber, err.Error()) |
| path := viper.GetString(commonParams.CompleteMultipartUploadPathEnv) | ||
| jsonBytes, err := json.Marshal(completeMultipartUpload) | ||
| if err != nil { | ||
| return nil, errors.Errorf("Failed to marshal complete multipart upload request body - %s", err.Error()) |
Collaborator
There was a problem hiding this comment.
change the error message.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
By submitting this pull request, you agree to the terms within the Checkmarx Code of Conduct. Please review the contributing guidelines for guidance on creating high-quality pull requests.
Description
Implemented code changes for multi part upload when source folder size exceeds the 5GB and introduce 'multipart_file_size' environment variable in checkmarx.yml file where customer can give the file part size according to internet banwidth.
Type of Change
Related Issues
Link any related issues or tickets.
Checklist
Screenshots (if applicable)
Add screenshots to help explain your changes.
Additional Notes
Add any other relevant information.