how can i use this to upload every 10 files at the same time

[khadija-mouaddah]

how can i use this to upload every 10 files at the same time

ThreadPoolExecutor executor = new ThreadPoolExecutor(50, 50,
    10, TimeUnit.SECONDS,
    new LinkedBlockingQueue<>(10_000),
    new ThreadFactoryBuilder()
      .threadNamePrefix("sdk-async-response").build());

// Allow idle core threads to time out
executor.allowCoreThreadTimeOut(true);

S3AsyncClient clientThread = S3AsyncClient.builder()
  .asyncConfiguration(
    b -> b.advancedOption(SdkAdvancedAsyncClientOption
      .FUTURE_COMPLETION_EXECUTOR,
      executor
    )
  )
  .build();

i have this code

public String uploadFolder2(Path filePath, String filepath_cloud) {

	PutObjectRequest objectRequest = PutObjectRequest.builder()
			.bucket(bucketName)
			.key(filepath_cloud)
			.build();

	// Put the object into the bucket
	CompletableFuture<PutObjectResponse> future = s3AsyncClient.putObject(objectRequest,
			AsyncRequestBody.fromFile(filePath));
	future.whenComplete((resp, err) -> {
		try {
			if (resp != null) {
				System.out.println("Object uploaded. Details: " + resp);
			} else {
				err.printStackTrace();
			}
		} finally {

			// s3Client.close();
		}
	});

	future.join();

	return "Your file " + filePath + " was successfully uploaded ASync";
}

```

public void savecontent() throws IOException {

	Path source = Paths.get("C:\\var\\kh\\ff");
	try (Stream<Path> paths = Files.walk(source)) {
		paths.filter(Files::isRegularFile).forEach(x -> uploadFolder2(x, source.relativize(x).toString().replace("\\","/")));
	}
   
}

[debora-ito]

Hi @khadija-mouaddah this channel is for questions regarding the AWS SDK for Java specifically, we don't really give general implementation suggestions or debug code.

Your sample code is using the S3 async client, which makes asynchronous calls, so you're on the right path to make concurrent uploads. Also take a look at TransferManager v2, it's a high-level library that makes high performance operations to S3.
https://aws.amazon.com/blogs/developer/introducing-amazon-s3-transfer-manager-in-the-aws-sdk-for-java-2-x/

[zoewangg]

Hi @khadija-mouaddah, you can try with S3TransferManager#uploadDirectory. Note that S3TransferManager is still in Developer Preview at the moment

https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/transfer/s3/S3TransferManager.html#uploadDirectory-software.amazon.awssdk.transfer.s3.UploadDirectoryRequest-

[github-actions[bot]]

Hello! Reopening this discussion to make it searchable.