Uploads via S3AsyncClient drop all data when put object is invoked with AsyncRequestBody.fromPublisher

[yankeenjg]

Hi,

I'm running into an issue where it appears that any data uploaded by the S3AsyncClient via PutObject is dropped if and only if the PutObject requests 1/ does not contain a content length in the top-level request 2/ the async request body does not contain a content length. As an example, any request submitted with AsyncRequestBody.fromPublisher will reflect (2).

Below is a snippet of test that reproduces the issue -- unfortunately I'm unable to post the full code. Note that I've run the same test 1/ against a real S3 endpoint 2/ with AsyncRequestBody.fromByteArray and both of those tests work. I'm using version 3.11.0.

Here are a few additional things I've tried, but which did not seem to fix the issue:

1. I tried running this via Docker
2. I tried running this with Tomcat using the pull request from AWS CRT-based S3 client does not work with Jetty 12.0.13+ #2136 as I falsely assumed the issue reported there might be related.
3. I tried using the netty client instead of the crt client

Please let me know if there's an issue in the code below or if you're able to look into this.

Thanks!

S3MockApplication.start(new HashMap<>(Map.of(
    S3MockApplication.PROP_HTTP_PORT,  9090,
    S3MockApplication.PROP_HTTPS_PORT, 9191,
    S3MockApplication.PROP_ROOT_DIRECTORY, root().toString(),
    S3MockApplication.PROP_SECURE_CONNECTION, true,
    S3MockApplication.PROP_SILENT, false)));

S3AsyncClient client = S3AsyncClient.builder()
    .credentialsProvider(StaticCredentialsProvider.create(AwsBasicCredentials.create("foo", "bar")))
    .endpointOverride(URI.create("https://localhost:9191"))
    .forcePathStyle(true)
    .httpClient(AwsCrtAsyncHttpClient.builder()
        .connectionTimeout(Duration.ofSeconds(5))
        .maxConcurrency(100)
        .buildWithDefaults(AttributeMap.builder()
            .put(TRUST_ALL_CERTIFICATES, true)
            .build()))
    .region(US_EAST_1)
    .build();

client.createBucket(b -> b.bucket("bucket")).get();

String key = UUID.randomUUID().toString();

PutObjectRequest put = PutObjectRequest.builder()
    .bucket("my-bucket")
    .key(key)
    .build();

String content = "This is a test";

SimplePublisher<ByteBuffer> publisher = new SimplePublisher<>(); // this class is defined by the aws sdk
CompletableFuture<?> future = client.putObject(put, fromPublisher(publisher));
publisher.send(ByteBuffer.wrap(content.getBytes(UTF_8))).get();
publisher.complete().get();
future.get();

GetObjectRequest get = GetObjectRequest.builder()
  .bucket("bucket")
  .key(key)
  .build();

byte[] results = client.getObject(get, AsyncResponseTransformer.toBytes()).get().asByteArray();

System.out.println(new String(results, UTF_8)); // this prints out an empty string

assertEquals(content, new String(results, UTF_8)); // this fails

[yankeenjg]

It looks like I had forgotten to enable the CRT-client's native multipart upload functionality when I ran the test against S3 mock but had it enabled when I ran the test against real S3. There's still a difference here in that with S3Mock the upload seems to silently drop data (as if content length was zero) instead of throwing an exception as real S3 does. The update to Tomcat appears to have fixed the other issues I was having so I'll close this one and leave a comment on the issue that already exists for that.