Getting SignatureDoesNotMatch on PutObject

[poopoothegorilla]

I am receiving

bad response: SignatureDoesNotMatch: The request signature we calculated does not match the signature you provided. Check your key and signing method. status code 403

when I run the following code... (to minimize code size i excluded some error checks)

func (conn *Connection) newS3Connection() (*s3.S3, error) {
	creds := credentials.NewStaticCredentials(conn.cfg.S3ID, conn.cfg.S3Key, "")
	if _, err := creds.Get(); err != nil {
		return nil, err
	}

	cfg := aws.NewConfig().WithRegion("us-west-2").WithCredentials(creds)
	return s3.New(session.New(), cfg), nil
}

	filename := `test.psv.gz`
	file, _ := ioutil.TempFile("", filename)
        defer  os.Remove(file.Name())

	w := gzip.NewWriter(file)
	defer w.Close()

	w.Write("hello")

	fileInfo, _ := file.Stat()

	fileLen := fileInfo.Size()
	fileBytes, _ := ioutil.ReadAll(file)

	fileType := http.DetectContentType(fileBytes)
	path := "/dumps/" + filename
	params := &s3.PutObjectInput{
		Bucket:        aws.String(conn.cfg.S3Bucket),
		Key:           aws.String(path),
		Body:          bytes.NewReader(fileBytes),
		ContentLength: aws.Int64(fileLen),
		ContentType:   aws.String(fileType),
	}
        svc, _ := conn.newS3Connection()
	resp, _ := svc.PutObject(params) // Where the error is returned

What am I missing here? Both the Key and ID work on other calls... it seems just with this file example. The input does not explicitly set the metadata so it doesn't look like it is related to this past issue #642.

[xibz]

Hello @poopoothegorilla, thank you for reaching out to us. By other calls, are you saying this function works with other objects?

Can you enable logging and post back the data? Also what Go and SDK version are you using?

cfg := aws.NewConfig().WithRegion("us-west-2").WithCredentials(creds).WithLogLevel(aws.LogDebug)

[poopoothegorilla]

Thank you for taking the time @xibz. Yep this function has worked with other objects.

Go: go1.8 darwin/amd64
SDK: Release v1.7.1

I was not quite sure what was or wasnt sensitive information here so I took out some sections. The code above uses the filename "test.psv.gz" but this code i ran with "team_path_items.psv.gz" it is the same code however.

2017/03/01 14:36:53 DEBUG: Request s3/PutObject Details:
---[ REQUEST POST-SIGN ]-----------------------------
PUT /dumps/team_path_items.psv.gz HTTP/1.1
Host: [---]
User-Agent: aws-sdk-go/1.7.1 (go1.8; darwin; amd64)
Content-Length: 17770
Authorization: AWS4-HMAC-SHA256 Credential=[---]/us-west-2/s3/aws4_request, SignedHeaders=content-length;content-type;host;x-amz-content-sha256;x-amz-date, Signature=[---]
Content-Type: text/plain; charset=utf-8
X-Amz-Content-Sha256: [---]
X-Amz-Date: 20170301T193653Z
Accept-Encoding: gzip

---

2017/03/01 14:36:54 DEBUG: Response s3/PutObject Details:
---[ RESPONSE ]--------------------------------------
HTTP/1.1 403 Forbidden
Transfer-Encoding: chunked
Content-Type: application/xml
Date: Wed, 01 Mar 2017 19:36:53 GMT
Server: AmazonS3
X-Amz-Id-2: [---]
X-Amz-Request-Id: [---]

---

[xibz]

@poopoothegorilla , the logs look correct as far as I can tell. Can you post a request that succeeds?

[poopoothegorilla]

2017/03/01 19:01:11 DEBUG: Request s3/PutObject Details:
---[ REQUEST POST-SIGN ]-----------------------------
PUT /dumps/team_path_items.psv.gz HTTP/1.1
Host: [---]
User-Agent: aws-sdk-go/1.7.1 (go1.8; darwin; amd64)
Content-Length: 101321
Authorization: AWS4-HMAC-SHA256 Credential= [---]/20170302/us-west-2/s3/aws4_request, SignedHeaders=content-length;content-type;host;x-amz-content-sha256;x-amz-date, Signature= [---]
Content-Type: text/plain; charset=utf-8
X-Amz-Content-Sha256: [---]
X-Amz-Date: [---]
Accept-Encoding: gzip

---

2017/03/01 19:01:13 DEBUG: Response s3/PutObject Details:
---[ RESPONSE ]--------------------------------------
HTTP/1.1 200 OK
Content-Length: 0
Date: Thu, 02 Mar 2017 00:01:14 GMT
Etag: [---]
Server: AmazonS3
X-Amz-Id-2: [---]
X-Amz-Request-Id: [---]

---

[jasdel]

@poopoothegorilla And additional level of debuging output might help here. The following will log the body of the HTTP Request/Response.

cfg := aws.NewConfig().
    WithRegion("us-west2").
    WithCredentials(creds).
    WithLogLevel(aws.LogDebugWithHTTPBody | aws.LogDebugWithSigning)

This will give you a log of everything the SDK is using to sign the request with.

I notice the ContentLength is being manually set. Is it possible that the value for that field is different than the actual length of the body being sent? Its possible that you're running into a file flush issue. Where the length of the file when stat'ed does not match the length of the content written by gzip. What about trying something like:

	w := gzip.NewWriter(file)
	w.Write("hello")
	// Move Close from a defer to inline so everything will be written before trying
	// to stat the file.
	w.Close()

	fileInfo, _ := file.Stat()
	fileLen := fileInfo.Size()
	fileBytes, _ := ioutil.ReadAll(file)

[poopoothegorilla]

@jasdel your suggestion of removing the ContentLength field worked! I did not know that it was optional. Thanks alot both of you guys @xibz @jasdel ... I really appreciate it 😄

[poopoothegorilla]

I see it now...
http://docs.aws.amazon.com/sdk-for-go/api/service/s3/#PutObjectInput

Size of the body in bytes. This parameter is useful when the size of the
body cannot be determined automatically.
ContentLength *int64 location:"header" locationName:"Content-Length" type:"long"

[bryevo]

I ran into the same issue. I was using an IAM user with an old access key that didn't have S3FullAcess in the group permissions. Once I added the correct permissions and created a new access key it worked! Hopefully that fixes it!