Quantcast
Channel: Active questions tagged rest - Stack Overflow
Viewing all articles
Browse latest Browse all 3619

SpringBoot 3 Upload large file efficiently and performantly

$
0
0

I am using spring boot 3, java 21 for a rest api.

One of the API will handle file upload. Files can be pretty big, 2~3GB. Reading the spring documentation, I read that the MultipartFile request will always either save the data to memory or save it as a temporary file. Either way does not work. If possible I want a way to directly stream the MultipartFile data as it is being uploaded. I have read some SO posts, some of them is using Apache commons fileupload stream. Those posts are pretty old.

I will share two rest controller methods.

@PostMapping("/{fileId}/content")@PreAuthorize("hasAuthority('SCOPE_write:files')")public ResponseEntity<String> postFileFileId(@PathVariable Long fileId,        @RequestParam("file") MultipartFile multipartFile) {    Optional<FileResourceEntity> fileResourceEntityOptional = fileRepository.findByFileIdAndIsTrashFalse(fileId);    if (!fileResourceEntityOptional.isPresent()) {        return ResponseEntity.notFound().build();    }    try {        FileResourceEntity fileResourceEntity = fileResourceEntityOptional.get();        createFoldersAndSaveUploadFile(fileResourceEntity, multipartFile);        fileRepository.save(fileResourceEntity);        return ResponseEntity.noContent().build();    } catch (Exception ex) {        return ResponseEntity.badRequest().body(UPLOAD_FAILED_MESSAGE);    }}@PostMapping(consumes = { "multipart/form-data" })@PreAuthorize("hasAnyAuthority('SCOPE_write:files')")public ResponseEntity<String> postFile(        @RequestPart("filedetail") FileResourceDetail fileResource,        @RequestPart("content") MultipartFile multipartFile,        UriComponentsBuilder ubc) {    FileResourceEntity savedFile = new FileResourceEntity();    try {        FileResourceEntity fileData = fileResourceDTOService.convertToEntity(fileResource);        savedFile = fileRepository.save(fileData);        if (!multipartFile.isEmpty()) {            createFoldersAndSaveUploadFile(savedFile, multipartFile);        }        URI locationOfNewFile = ubc                .path("/api/v4/files/{fileId}")                .buildAndExpand(savedFile.getFileId())                .toUri();        return ResponseEntity.created(locationOfNewFile).build();    } catch (Exception ex) {        fileRepository.deleteById(savedFile.getFileId());        return ResponseEntity.badRequest()                .body(UPLOAD_FAILED_MESSAGE);    }}private void createFoldersAndSaveUploadFile(FileResourceEntity fileResourceEntity, MultipartFile multipartFile)        throws IOException {    String createDate = fileResourceEntity.getCreatedAt().format(DATE_FORMATTER);    createFolders(Path.of(rootDataFolder, createDate));    Files.copy(multipartFile.getInputStream(), getFilePath(fileResourceEntity));}

We are running the application as pod in Kubernetes. Files are saved to a PV, which is actually mounted FSX. Right now, after client finishes upload, server takes a long time to save the multipart file to actual file server, then returns response.

Is there any way to directly save/stream uploads to file using Spring Boot?


Viewing all articles
Browse latest Browse all 3619

Trending Articles



<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>