S3 HeadObject throws S3Exception: null instead of NoSuchKeyException when no object exists. #1941

dhegberg · 2020-07-06T18:56:09Z

Expected Behavior

Documentation states HeadObject will throw NoSuchKeyException if the object does not exist.

Current Behavior

HeadObject throws S3Exception: null instead.

Steps to Reproduce

For a key that does not exist.

        final HeadObjectRequest request = HeadObjectRequest.builder()
                .bucket(bucket)
                .key(objectKey)
                .build();
        final HeadObjectResponse response = s3.headObject(request)

Context

Documentation states HeadObject will throw NoSuchKeyException (https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/services/s3/S3Client.html#headObject-software.amazon.awssdk.services.s3.model.HeadObjectRequest-) but a S3Exception: null is thrown instead.

This is stated to be fixed in #544

Your Environment

AWS Java SDK version used: 2.13.25

The text was updated successfully, but these errors were encountered:

debora-ito · 2020-07-07T16:56:46Z

@dhegberg I was not able to reproduce, I get
Exception in thread "main" software.amazon.awssdk.services.s3.model.NoSuchKeyException: The specified key does not exist. (Service: S3, Status Code: 404, Request ID: xxx, Extended Request ID: null)

Can you provide the full stacktrace with the error you're seeing?
And a code snippet showing how your s3 client is being constructed?

dhegberg · 2020-07-07T17:17:33Z

The s3 client is being created as part of a dagger module:

@provides
@singleton
static S3Client getS3Client() {
return S3Client.builder().build();
}

Stack Trace (This is being run in a Java8 lambda):
`
software.amazon.awssdk.services.s3.model.S3Exception

Cause

{
"errorMessage": "null (Service: S3, Status Code: 403, Request ID: null, Extended Request ID: D08D8ysH68NGgukI780pV3NZKFu06OQOIkP7H9q56MlnVlXgXaFgxPk1QSbienVjrjT9iPJmIvE=)",
"errorType": "software.amazon.awssdk.services.s3.model.S3Exception",
"stackTrace": [
"software.amazon.awssdk.protocols.xml.internal.unmarshall.AwsXmlPredicatedResponseHandler.handleErrorResponse(AwsXmlPredicatedResponseHandler.java:156)",
"software.amazon.awssdk.protocols.xml.internal.unmarshall.AwsXmlPredicatedResponseHandler.handleResponse(AwsXmlPredicatedResponseHandler.java:106)",
"software.amazon.awssdk.protocols.xml.internal.unmarshall.AwsXmlPredicatedResponseHandler.handle(AwsXmlPredicatedResponseHandler.java:84)",
"software.amazon.awssdk.protocols.xml.internal.unmarshall.AwsXmlPredicatedResponseHandler.handle(AwsXmlPredicatedResponseHandler.java:42)",
"software.amazon.awssdk.awscore.client.handler.AwsSyncClientHandler$Crc32ValidationResponseHandler.handle(AwsSyncClientHandler.java:94)",
"software.amazon.awssdk.core.internal.handler.BaseClientHandler.lambda$successTransformationResponseHandler$4(BaseClientHandler.java:214)",
"software.amazon.awssdk.core.internal.http.pipeline.stages.HandleResponseStage.execute(HandleResponseStage.java:40)",
"software.amazon.awssdk.core.internal.http.pipeline.stages.HandleResponseStage.execute(HandleResponseStage.java:30)",
"software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)",
"software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptTimeoutTrackingStage.execute(ApiCallAttemptTimeoutTrackingStage.java:73)",
"software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptTimeoutTrackingStage.execute(ApiCallAttemptTimeoutTrackingStage.java:42)",
"software.amazon.awssdk.core.internal.http.pipeline.stages.TimeoutExceptionHandlingStage.execute(TimeoutExceptionHandlingStage.java:77)",
"software.amazon.awssdk.core.internal.http.pipeline.stages.TimeoutExceptionHandlingStage.execute(TimeoutExceptionHandlingStage.java:39)",
"software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage.execute(RetryableStage.java:64)",
"software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage.execute(RetryableStage.java:34)",
"software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)",
"software.amazon.awssdk.core.internal.http.StreamManagingStage.execute(StreamManagingStage.java:56)",
"software.amazon.awssdk.core.internal.http.StreamManagingStage.execute(StreamManagingStage.java:36)",
"software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.executeWithTimer(ApiCallTimeoutTrackingStage.java:80)",
"software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.execute(ApiCallTimeoutTrackingStage.java:60)",
"software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.execute(ApiCallTimeoutTrackingStage.java:42)",
"software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)",
"software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)",
"software.amazon.awssdk.core.internal.http.pipeline.stages.ExecutionFailureExceptionReportingStage.execute(ExecutionFailureExceptionReportingStage.java:37)",
"software.amazon.awssdk.core.internal.http.pipeline.stages.ExecutionFailureExceptionReportingStage.execute(ExecutionFailureExceptionReportingStage.java:26)",
"software.amazon.awssdk.core.internal.http.AmazonSyncHttpClient$RequestExecutionBuilderImpl.execute(AmazonSyncHttpClient.java:189)",
"software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.invoke(BaseSyncClientHandler.java:121)",
"software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.doExecute(BaseSyncClientHandler.java:147)",
"software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.execute(BaseSyncClientHandler.java:101)",
"software.amazon.awssdk.core.client.handler.SdkSyncClientHandler.execute(SdkSyncClientHandler.java:45)",
"software.amazon.awssdk.awscore.client.handler.AwsSyncClientHandler.execute(AwsSyncClientHandler.java:55)",
"software.amazon.awssdk.services.s3.DefaultS3Client.headObject(DefaultS3Client.java:4080)"
`

dhegberg · 2020-07-07T18:12:48Z

This seems to be a problem with the execution environment rather than the client.

Local runs were successful, the issue only occurred in a Lambda.

jeffzoch · 2020-08-06T01:06:27Z

I actually have this exact issue. I run in EKS and am seeing this happen regularly.

software.amazon.awssdk.services.s3.model.S3Exception: null (Service: S3, Status Code: 400, Request ID: null, Extended Request ID: rwXHjbMhpEbHQ68JyLAtGmI7wbQszGNgPekt9kr76ALmN61JkuB3RnwcTRkINBAjvwk5KfCFzEE=)
	at software.amazon.awssdk.protocols.xml.internal.unmarshall.AwsXmlPredicatedResponseHandler.handleErrorResponse(AwsXmlPredicatedResponseHandler.java:156)
	at software.amazon.awssdk.protocols.xml.internal.unmarshall.AwsXmlPredicatedResponseHandler.handleResponse(AwsXmlPredicatedResponseHandler.java:106)
	at software.amazon.awssdk.protocols.xml.internal.unmarshall.AwsXmlPredicatedResponseHandler.handle(AwsXmlPredicatedResponseHandler.java:84)
	at software.amazon.awssdk.protocols.xml.internal.unmarshall.AwsXmlPredicatedResponseHandler.handle(AwsXmlPredicatedResponseHandler.java:42)
	at software.amazon.awssdk.awscore.client.handler.AwsSyncClientHandler$Crc32ValidationResponseHandler.handle(AwsSyncClientHandler.java:94)
	at software.amazon.awssdk.core.internal.handler.BaseClientHandler.lambda$successTransformationResponseHandler$5(BaseClientHandler.java:229)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.HandleResponseStage.execute(HandleResponseStage.java:40)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.HandleResponseStage.execute(HandleResponseStage.java:30)
	at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptTimeoutTrackingStage.execute(ApiCallAttemptTimeoutTrackingStage.java:73)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptTimeoutTrackingStage.execute(ApiCallAttemptTimeoutTrackingStage.java:42)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.TimeoutExceptionHandlingStage.execute(TimeoutExceptionHandlingStage.java:77)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.TimeoutExceptionHandlingStage.execute(TimeoutExceptionHandlingStage.java:39)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptMetricCollectionStage.execute(ApiCallAttemptMetricCollectionStage.java:50)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptMetricCollectionStage.execute(ApiCallAttemptMetricCollectionStage.java:36)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage.execute(RetryableStage.java:64)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage.execute(RetryableStage.java:34)
	at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
	at software.amazon.awssdk.core.internal.http.StreamManagingStage.execute(StreamManagingStage.java:56)
	at software.amazon.awssdk.core.internal.http.StreamManagingStage.execute(StreamManagingStage.java:36)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.executeWithTimer(ApiCallTimeoutTrackingStage.java:80)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.execute(ApiCallTimeoutTrackingStage.java:60)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.execute(ApiCallTimeoutTrackingStage.java:42)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallMetricCollectionStage.execute(ApiCallMetricCollectionStage.java:48)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallMetricCollectionStage.execute(ApiCallMetricCollectionStage.java:31)
	at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
	at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.ExecutionFailureExceptionReportingStage.execute(ExecutionFailureExceptionReportingStage.java:37)
	at software.amazon.awssdk.core.internal.http.pipeline.stages.ExecutionFailureExceptionReportingStage.execute(ExecutionFailureExceptionReportingStage.java:26)
	at software.amazon.awssdk.core.internal.http.AmazonSyncHttpClient$RequestExecutionBuilderImpl.execute(AmazonSyncHttpClient.java:193)
	at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.invoke(BaseSyncClientHandler.java:128)
	at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.doExecute(BaseSyncClientHandler.java:154)
	at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.lambda$execute$1(BaseSyncClientHandler.java:107)
	at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.measureApiCallSuccess(BaseSyncClientHandler.java:162)
	at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.execute(BaseSyncClientHandler.java:91)
	at software.amazon.awssdk.core.client.handler.SdkSyncClientHandler.execute(SdkSyncClientHandler.java:45)
	at software.amazon.awssdk.awscore.client.handler.AwsSyncClientHandler.execute(AwsSyncClientHandler.java:55)
	at software.amazon.awssdk.services.s3.DefaultS3Client.headObject(DefaultS3Client.java:4590)

The client is built

S3Client.builder().apply {
            credentialsProvider { credentials.resolveCredentials() }
            region(Region.of(config.region))
            overrideConfiguration { override ->
                override.apiCallTimeout(Duration.ofSeconds(5))
                override.apiCallAttemptTimeout(Duration.ofSeconds(15))
            }
            serviceConfiguration { s3Config ->
                s3Config.apply {
                    pathStyleAccessEnabled(false)
                    checksumValidationEnabled(true)
                    chunkedEncodingEnabled(true)
                }.build()
            }
        }

jeffzoch · 2020-08-06T01:16:12Z

I should clarify that this request fails very frequently (think thousands of times per week) but also succeeds frequently. There are random 8 hour periods where we dont see one of them and then other 8 hour periods where we see 500 an hour.

magJ · 2021-01-28T08:53:17Z

For what it's worth, in my case at least, S3 seems to respond with these errors when specifying a non-existent versionId.
So seems like a bug on the service side, rather than the client implementation.

Dharmesh2015 · 2021-04-01T02:24:13Z

My application create files on S3 if its not exist , which is dependent s3.headObject call , suddenly function started throwing error with null, any suggested solution ? we are running code through lambda

makarthikeyan1 · 2021-12-03T23:44:33Z

happened to me when i used "us-east-1" instead of "us-east-2" where the bucket was located.

…f2be0747a Pull request: release <- staging/9fd7d256-c1fd-492d-ad6f-9a5f2be0747a

fredex42 · 2022-04-14T11:50:10Z

In case anyone else finds it useful, I got this same error when I was updating a lambda to work with versioned objects. In my case, I needed to add the s3:GetObjectVersion permission to the lambda's role in order to fix.

selalerercapitolis · 2022-09-28T12:17:13Z

And I got it when I was using S3Proxy in a docker (out testing environment), was putting a file in S3 and the file key started with a slash /

campidelli · 2023-06-14T00:13:14Z

If it helps, I was having this issue with my Springboot v3 application when I tried to call headBucket using the async client. I think it was an authentication issue that was fixed after I added the following to my pom.xml:

        <dependency>
            <groupId>software.amazon.awssdk</groupId>
            <artifactId>sso</artifactId>
        </dependency>
        <dependency>
            <groupId>software.amazon.awssdk</groupId>
            <artifactId>ssooidc</artifactId>
        </dependency>

reed53 · 2024-12-24T16:38:20Z

Was running into this, my fix was to add the ListBucket permission for the bucket I was trying to read from.

dhegberg added bug This issue is a bug. needs-triage This issue or PR still needs to be triaged. labels Jul 6, 2020

debora-ito added response-requested Waiting on additional info and feedback. Will move to "closing-soon" in 10 days. and removed needs-triage This issue or PR still needs to be triaged. labels Jul 7, 2020

github-actions bot removed the response-requested Waiting on additional info and feedback. Will move to "closing-soon" in 10 days. label Jul 7, 2020

dhegberg closed this as completed Jul 7, 2020

aws-sdk-java-automation added a commit that referenced this issue Feb 21, 2022

Merge pull request #1941 from aws/staging/9fd7d256-c1fd-492d-ad6f-9a5…

57f73f2

…f2be0747a Pull request: release <- staging/9fd7d256-c1fd-492d-ad6f-9a5f2be0747a

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

S3 HeadObject throws S3Exception: null instead of NoSuchKeyException when no object exists. #1941

S3 HeadObject throws S3Exception: null instead of NoSuchKeyException when no object exists. #1941

dhegberg commented Jul 6, 2020

debora-ito commented Jul 7, 2020

dhegberg commented Jul 7, 2020

dhegberg commented Jul 7, 2020

jeffzoch commented Aug 6, 2020 •

edited

Loading

jeffzoch commented Aug 6, 2020

magJ commented Jan 28, 2021

Dharmesh2015 commented Apr 1, 2021

makarthikeyan1 commented Dec 3, 2021

fredex42 commented Apr 14, 2022

selalerercapitolis commented Sep 28, 2022

campidelli commented Jun 14, 2023

reed53 commented Dec 24, 2024

S3 HeadObject throws S3Exception: null instead of NoSuchKeyException when no object exists. #1941

S3 HeadObject throws S3Exception: null instead of NoSuchKeyException when no object exists. #1941

Comments

dhegberg commented Jul 6, 2020

Expected Behavior

Current Behavior

Steps to Reproduce

Context

Your Environment

debora-ito commented Jul 7, 2020

dhegberg commented Jul 7, 2020

dhegberg commented Jul 7, 2020

jeffzoch commented Aug 6, 2020 • edited Loading

jeffzoch commented Aug 6, 2020

magJ commented Jan 28, 2021

Dharmesh2015 commented Apr 1, 2021

makarthikeyan1 commented Dec 3, 2021

fredex42 commented Apr 14, 2022

selalerercapitolis commented Sep 28, 2022

campidelli commented Jun 14, 2023

reed53 commented Dec 24, 2024

jeffzoch commented Aug 6, 2020 •

edited

Loading