Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bad types after smithy upgrade #4928

Closed
3 tasks done
xconverge opened this issue Jul 5, 2023 · 26 comments
Closed
3 tasks done

Bad types after smithy upgrade #4928

xconverge opened this issue Jul 5, 2023 · 26 comments
Assignees
Labels
bug This issue is a bug. p2 This is a standard priority issue

Comments

@xconverge
Copy link

xconverge commented Jul 5, 2023

Checkboxes for prior research

Describe the bug

Here are 2 examples of some issues I am experiencing with types with 3.363.0

I scanned through https://github.com/aws/aws-sdk-js-v3/blob/main/CHANGELOG.md and can't find anything that makes these seem like intentional changes, sorry if this is all a bit vague but maybe someone with more information can ask more pointed questions/add something that helps.

3.362.0 is all good, 3.363.0 has all sorts of issues. The only change that seems relevant to me is #4873

Issues I see with @aws-sdk/s3-request-presigner:

Argument of type 'S3' is not assignable to parameter of type 'Client<any, ServiceInputTypes, MetadataBearer, any>'.
  The types of 'middlewareStack.add' are incompatible between these types.
    Type '{ (middleware: InitializeMiddleware<ServiceInputTypes, ServiceOutputTypes>, options?: (InitializeHandlerOptions & AbsoluteLocation) | undefined): void; (middleware: SerializeMiddleware<...>, options: SerializeHandlerOptions & AbsoluteLocation): void; (middleware: BuildMiddleware<...>, options: BuildHandlerOptions & ...' is not assignable to type '{ (middleware: InitializeMiddleware<ServiceInputTypes, MetadataBearer>, options?: (InitializeHandlerOptions & AbsoluteLocation) | undefined): void; (middleware: SerializeMiddleware<...>, options: SerializeHandlerOptions & AbsoluteLocation): void; (middleware: BuildMiddleware<...>, options: BuildHandlerOptions & Abso...'.
      Types of parameters 'middleware' and 'middleware' are incompatible.
        Types of parameters 'next' and 'next' are incompatible.
          Type 'InitializeHandler<ServiceInputTypes, ServiceOutputTypes>' is not assignable to type 'InitializeHandler<ServiceInputTypes, MetadataBearer>'.
            Type 'ServiceOutputTypes' is not assignable to type 'MetadataBearer'.ts(2345)

image

Issues I see with @aws-sdk/client-s3:

Property 'Body' does not exist on type 'GetObjectCommandOutput'.ts(2339)

image

SDK version number

@aws-sdk/[email protected]

Which JavaScript Runtime is this issue in?

Node.js

Details of the browser/Node.js/ReactNative version

node v18.16.0

Reproduction Steps

Code snippets in screenshots are one example

Observed Behavior

Lots of typescript errors and weirdness with 3.363.0

Expected Behavior

Types work correctly like in 3.362.0

Possible Solution

No response

Additional Information/Context

No response

@xconverge xconverge added bug This issue is a bug. needs-triage This issue or PR still needs to be triaged. labels Jul 5, 2023
@xconverge
Copy link
Author

I also just noticed this was reported while I was reporting my issue, and it looks related IMO #4927

@RanVaknin RanVaknin self-assigned this Jul 5, 2023
@RanVaknin
Copy link
Contributor

RanVaknin commented Jul 5, 2023

Hi @xconverge ,

Thanks for opening this issue.

The code snippets you attached as images are incomplete and hard to assess since they are in image form.

I wrote some repro code loosly based on the information you have provided:

import { S3Client, GetObjectCommand, HeadObjectCommand, GetObjectCommandOutput } from "@aws-sdk/client-s3";

const main = async () => {
  const client = new S3Client({ region: "us-east-1" });
  const params = { Bucket: "testbucket", Key: "hey.txt" };
  const headCommand = new HeadObjectCommand(params);

  return client.send(headCommand).then(async () => {
    const getCommand = new GetObjectCommand(params);
    const data: GetObjectCommandOutput = await client.send(getCommand);

    if (data.Body) {
      console.log("Body exists on data");
      return;
    }
    console.log("Body does not exist on data");
  });
};

main()
  .catch(console.error);

This compiles and runs successfully:

$ npm list
[email protected] /Users/rvaknin/test_folder/4928
├── @aws-sdk/[email protected]
└── @types/[email protected]
$ tsc src/sample.ts 
$ node src/sample.js
Body exists on data

This is odd, because I was not able to reproduce #4927 either. To give you an idea of how I'm reproducing this, with each reproduction I'm creating a new project from scratch, so Im wondering maybe your code has some dependencies that aren't resolving correctly.
Can you deleting your node_modules and package-lock.json and try running npm install again to see if it solves the problem?

Otherwise, if you can upload a minimal github repo that can consistently reproduce this behavior, it would be extremely helpful.

Thanks again,
Ran~

@RanVaknin RanVaknin added response-requested Waiting on additional info and feedback. Will move to \"closing-soon\" in 7 days. p2 This is a standard priority issue and removed needs-triage This issue or PR still needs to be triaged. labels Jul 5, 2023
@xconverge
Copy link
Author

xconverge commented Jul 6, 2023

@RanVaknin I did not use client.send, I used the alternative API, so perhaps that makes a difference and is why your repro code doesnt repro it. Sorry for the images, I knew it was a half-baked bug report but was hoping there was something obvious.

If instead of what you wrote you were to do something like this, what do you see?

import { S3Client, GetObjectCommand, HeadObjectCommand, GetObjectCommandOutput } from "@aws-sdk/client-s3";

const main = async () => {
  const client = new S3Client({ region: "us-east-1" });
  const params = { Bucket: "testbucket", Key: "hey.txt" };

  return client.send(headCommand).then(async () => {
    //const getCommand = new GetObjectCommand(params);
    const data: GetObjectCommandOutput = await client.getObject(params);

    if (data.Body) {
      console.log("Body exists on data");
      return;
    }
    console.log("Body does not exist on data");
  });
};

main()
  .catch(console.error);

I will try to get to an actual small reproducible example tomorrow, but I experienced this over a handful of repositories and all of it looks similar (albeit slight differences between which client/lib I was using)

@Joel-PeakMetrics
Copy link

Joel-PeakMetrics commented Jul 6, 2023

I'm seeing the same problem with deno (strict TS checks). Deno version 1.34.2

import { GetObjectCommand, PutObjectCommand, S3Client } from "npm:@aws-sdk/[email protected]";
const s3Client = new S3Client({
    "region":"us-east-1",
    "endpoint":"http://localhost:4566",
    "forcePathStyle": true
})

const command = new GetObjectCommand({
    Bucket: "local-raw-data",
    Key: "*",
});
const objectsFound = await s3Client.send(command);
$ deno check s3test.ts
Check file:///.../s3test.ts
TS2345 [ERROR]: Argument of type 'GetObjectCommand' is not assignable to parameter of type 'Command<ServiceInputTypes, GetObjectCommandInput, ServiceOutputTypes, GetObjectCommandOutput, SmithyResolvedConfiguration<...>>'.
  Types of property 'resolveMiddleware' are incompatible.
    Type '(clientStack: MiddlewareStack<ServiceInputTypes, ServiceOutputTypes>, configuration: S3ClientResolvedConfig, options?: HttpHandlerOptions | undefined) => Handler<...>' is not assignable to type '(stack: MiddlewareStack<ServiceInputTypes, ServiceOutputTypes>, configuration: SmithyResolvedConfiguration<HttpHandlerOptions>, options: any) => Handler<...>'.
      Types of parameters 'clientStack' and 'stack' are incompatible.
        Type 'import("file:///.../node_modules/.deno/@[email protected]/node_modules/@smithy/types/dist-types/middleware.d.ts").MiddlewareStack<import("file:///.../node_modules/.deno/@[email protected]/node_mod...' is not assignable to type 'import("file:///.../node_modules/.deno/@[email protected]/node_modules/@smithy/types/dist-types/middleware.d.ts").MiddlewareStack<import("file:///.../node_modules/.deno/@[email protected]/node_modu...'.
          Types of property 'concat' are incompatible.
            Type '<InputType extends import("file:///.../node_modules/.deno/@[email protected]/node_modules/@aws-sdk/client-s3/dist-types/S3Client.d.ts").ServiceInputTypes, OutputType extends import("file:///.../node_module...' is not assignable to type '<InputType extends import("file:///.../node_modules/.deno/@[email protected]/node_modules/@aws-sdk/client-s3/dist-types/S3Client.d.ts").ServiceInputTypes, OutputType extends import("file:///.../node_module...'. Two different types with this name exist, but they are unrelated.
              Types of parameters 'from' and 'from' are incompatible.
                Type 'MiddlewareStack<InputType, OutputType>' is not assignable to type 'MiddlewareStack<InputType, ServiceOutputTypes>'.
                  Types of property 'addRelativeTo' are incompatible.
                    Type '(middleware: MiddlewareType<InputType, OutputType>, options: RelativeMiddlewareOptions) => void' is not assignable to type '(middleware: MiddlewareType<InputType, ServiceOutputTypes>, options: RelativeMiddlewareOptions) => void'.
                      Types of parameters 'middleware' and 'middleware' are incompatible.
                        Type 'MiddlewareType<InputType, ServiceOutputTypes>' is not assignable to type 'MiddlewareType<InputType, OutputType>'.
                          Type 'InitializeMiddleware<InputType, ServiceOutputTypes>' is not assignable to type 'MiddlewareType<InputType, OutputType>'.
                            Type 'InitializeMiddleware<InputType, ServiceOutputTypes>' is not assignable to type 'InitializeMiddleware<InputType, OutputType>'.
                              Call signature return types 'InitializeHandler<InputType, ServiceOutputTypes>' and 'InitializeHandler<InputType, OutputType>' are incompatible.
                                Type 'Promise<InitializeHandlerOutput<ServiceOutputTypes>>' is not assignable to type 'Promise<InitializeHandlerOutput<OutputType>>'.
                                  Type 'InitializeHandlerOutput<ServiceOutputTypes>' is not assignable to type 'InitializeHandlerOutput<OutputType>'.
                                    Types of property 'output' are incompatible.
                                      Type 'ServiceOutputTypes' is not assignable to type 'OutputType'.
                                        'ServiceOutputTypes' is assignable to the constraint of type 'OutputType', but 'OutputType' could be instantiated with a different subtype of constraint 'ServiceOutputTypes'.
                                          Type 'AbortMultipartUploadCommandOutput' is not assignable to type 'OutputType'.
                                            'AbortMultipartUploadCommandOutput' is assignable to the constraint of type 'OutputType', but 'OutputType' could be instantiated with a different subtype of constraint 'ServiceOutputTypes'.
const objectsFound = await s3Client.send(command);
                                         ~~~~~~~
    at file:///.../s3test.ts:14:42

Found 2 errors.

@RanVaknin
Copy link
Contributor

Hi @xconverge,

I noticed some inconsistencies in your code. It seems like you were trying to use S3.headObject() rather than s3Client.send(new HeadObjectCommand({})) but I think I understand what you were trying to demonstrate:

import { S3, GetObjectCommandOutput } from "@aws-sdk/client-s3";

const main = async () => {
  const client = new S3({ region: "us-east-1" });
  const params = { Bucket: "testbucket", Key: "hey.txt" };

  return client.headObject(params).then(async () => {
    const data: GetObjectCommandOutput = await client.getObject(params);

    if (data.Body) {
      console.log("Body exists on data");
      return;
    }
    console.log("Body does not exist on data");
  });
};

main()
  .catch(console.error);

Yet, this modification doesn't seem to alter the outcome. The body is still present, which aligns with the type definitions.

We are aware that some users are encountering this issue, but it's been challenging for our team to replicate. If you could provide a minimal repository demonstrating this issue, it would be greatly appreciated.

Thanks again,
Ran~

@xconverge
Copy link
Author

Deleting my lock file and reinstalling seems to resolve the issues...

@github-actions github-actions bot removed the response-requested Waiting on additional info and feedback. Will move to \"closing-soon\" in 7 days. label Jul 7, 2023
@richgarner-nsw-gov
Copy link

richgarner-nsw-gov commented Jul 7, 2023

I think this is the same as I raised here

Especially the comment

@frontendphil
Copy link

I see a very similar error. The s3Client method returns new S3Client and fills in the AWS region. Key and Bucket are both strings.

We're running on TS 5.0.4 right now if that helps.

CleanShot 2023-07-07 at 10 24 12

@Naddiseo
Copy link

Naddiseo commented Jul 7, 2023

Deleting my lock file and reinstalling seems to resolve the issues...

This also seems to work for me, which seems to indicate a peer, or deep, dependency mismatch?

I dug a little bit deeper into the yarn.lock file and experimented.
Here's my starting package.json wrt @aws-sdk:

{
   "dependencies": {
    "@aws-sdk/client-s3": "^3.362.0",
    "@aws-sdk/client-sns": "^3.362.0",
    "@aws-sdk/s3-request-presigner": "^3.362.0",
   "@aws-sdk/client-secrets-manager": "^3.362.0",
   }
}

and after upgrade

{
   "dependencies": {
   "@aws-sdk/client-s3": "^3.367.0",
    "@aws-sdk/client-sns": "^3.363.0",
    "@aws-sdk/s3-request-presigner": "^3.367.0",
    "@aws-sdk/client-secrets-manager": "^3.363.0",
   }
}

I looked at my yarn.lock file at the "@aws-sdk/*" and "@smithy*" related packages to compare, the output is in a gist, It has 4 files:

  1. The yarn.lock before the upgrade (works)
  2. yarn.lock after upgrading via yarn upgrade-interactive (does not work)
  3. yarn.lock after removing all @aws-sdk and @smithy entries, then upgrading (DOES WORK)
  4. The diff between files 2 and 3

here's the diff for quick reference:

726c726
< "@aws-sdk/types@npm:3.357.0":
---
> "@aws-sdk/types@npm:3.357.0, @aws-sdk/types@npm:^3.222.0":
735,743d734
< "@aws-sdk/types@npm:^3.222.0":
<   version: 3.257.0
<   resolution: "@aws-sdk/types@npm:3.257.0"
<   dependencies:
<     tslib: ^2.3.1
<   checksum: eab2154f7d27d0bec29df41374481652525e1658ad987c64ed02cbe7b7388c563e6a8d317c8fc564ac6fbd97c98be700a29352c9acebc3903cf3305431bf0a86
<   languageName: node
<   linkType: hard
< 
796,797c787,788
<   version: 3.55.0
<   resolution: "@aws-sdk/util-locate-window@npm:3.55.0"
---
>   version: 3.310.0
>   resolution: "@aws-sdk/util-locate-window@npm:3.310.0"
799,800c790,791
<     tslib: ^2.3.1
<   checksum: 5bc79779a17ae4dd5cbcc221df6e85c976a149c0868745ae58075a3c51506aeeb087dfde4b3e5179004461243b11f3a458de720ee92866c6a1b65182326259ae
---
>     tslib: ^2.5.0
>   checksum: d552ce5f0f836ecb13d7920ae650552c56706f26a5e8abf894ba471e18775a3791869bda95269153735bac9d211efc3ba78ea01c34428c3fed4318ac693a08bc
834,835c825,826
<   version: 3.109.0
<   resolution: "@aws-sdk/util-utf8-browser@npm:3.109.0"
---
>   version: 3.259.0
>   resolution: "@aws-sdk/util-utf8-browser@npm:3.259.0"
838c829
<   checksum: 8311763b04261dab5995ec67abf31795f41e9c4b1ad635ed305735e14c7e3bc48e9ae349a06aab485390358a6a58e97190144ea51190983cec4ae665887b219b
---
>   checksum: b6a1e580da1c9b62c749814182a7649a748ca4253edb4063aa521df97d25b76eae3359eb1680b86f71aac668e05cc05c514379bca39ebf4ba998ae4348412da8
1089,1099c1080
< "@smithy/protocol-http@npm:^1.0.1":
<   version: 1.0.1
<   resolution: "@smithy/protocol-http@npm:1.0.1"
<   dependencies:
<     "@smithy/types": ^1.0.0
<     tslib: ^2.5.0
<   checksum: ba9ac4880fed48eeea0813663c94c765fe5b900f2fdac4f5de6524306bbc6645829f48bc175d202076b83acaccf008ed77f4b5546a4c180315f253e22fe6c89f
<   languageName: node
<   linkType: hard
< 
< "@smithy/protocol-http@npm:^1.1.0, @smithy/protocol-http@npm:^1.1.1":
---
> "@smithy/protocol-http@npm:^1.0.1, @smithy/protocol-http@npm:^1.1.0, @smithy/protocol-http@npm:^1.1.1":
1175,1184c1156
< "@smithy/types@npm:^1.0.0":
<   version: 1.0.0
<   resolution: "@smithy/types@npm:1.0.0"
<   dependencies:
<     tslib: ^2.5.0
<   checksum: ec05163564af050088f3c21cb047640ca842bea645c2a73624475b486d5df8ad9c494bf683a498f4b467b84fab2817cc199893dfb5cee30dce1e0172ab38db00
<   languageName: node
<   linkType: hard
< 
< "@smithy/types@npm:^1.1.0, @smithy/types@npm:^1.1.1":
---
> "@smithy/types@npm:^1.0.0, @smithy/types@npm:^1.1.0, @smithy/types@npm:^1.1.1":

From my reading of it, without removing the yarn.lock first, there seems to be more versions of @smithy/types and @aws-sdk/types hanging around in the working version

@benj-dobs
Copy link

I also had this issue and was only able to resolve it by deleting yarn.lock and regenerating it from scratch. I tried at first just deleting the top-level @aws-sdk/* and @smithy/* entries, but this was not successful.

My type errors indicate that there are two incompatible versions of @smithy/types installed, one at the root of node_modules and one inside @smithy/signature-v4. Here's an example of a type error when using client-cognito-identity-provider

Argument of type 'VerifyUserAttributeCommand' is not assignable to parameter of type 'Command<ServiceInputTypes, VerifyUserAttributeCommandInput, ServiceOutputTypes, VerifyUserAttributeCommandOutput, SmithyResolvedConfiguration<...>>'.
  Types of property 'resolveMiddleware' are incompatible.
    Type '(clientStack: MiddlewareStack<ServiceInputTypes, ServiceOutputTypes>, configuration: CognitoIdentityProviderClientResolvedConfig, options?: HttpHandlerOptions | undefined) => Handler<...>' is not assignable to type '(stack: MiddlewareStack<ServiceInputTypes, ServiceOutputTypes>, configuration: SmithyResolvedConfiguration<HttpHandlerOptions>, options: any) => Handler<...>'.
      Types of parameters 'clientStack' and 'stack' are incompatible.
        Type 'import("<PROJECT_ROOT>/node_modules/@smithy/signature-v4/node_modules/@smithy/types/dist-types/middleware").MiddlewareStack<import("<PROJECT_ROOT>/node_modules/@aws-sdk/client-cognito-identity-provider/dist-types/CognitoIdentityProviderC...' is not assignable to type 'import("<PROJECT_ROOT>/node_modules/@smithy/types/dist-types/middleware").MiddlewareStack<import("<PROJECT_ROOT>/node_modules/@aws-sdk/client-cognito-identity-provider/dist-types/CognitoIdentityProviderClient").ServiceInputTypes, import(...'.

@xconverge
Copy link
Author

What is the latest on this @RanVaknin ?

It got quite a few thumbs-ups, so I was obviously not alone, I have resolved it for myself by deleting the lock file, but I imagine you want to close the issue?

@switz
Copy link

switz commented Jul 24, 2023

Seeing this error – deleting lockfiles shouldn't be a solution (nor does it fix it for me).

@RanVaknin
Copy link
Contributor

Hi to all the folks still running into this issue.
Can you please update your SDK dependencies and see if the issue is still present?

Thanks,
Ran~

@RanVaknin RanVaknin added the response-requested Waiting on additional info and feedback. Will move to \"closing-soon\" in 7 days. label Jul 28, 2023
@github-actions
Copy link

github-actions bot commented Aug 5, 2023

This issue has not received a response in 1 week. If you still think there is a problem, please leave a comment to avoid the issue from automatically closing.

@github-actions github-actions bot added the closing-soon This issue will automatically close in 4 days unless further comments are made. label Aug 5, 2023
@Joel-PeakMetrics
Copy link

Hello @RanVaknin, I upgraded to AWS SDK v3.379.1, but I am still seeing the type check issue.

@github-actions github-actions bot removed closing-soon This issue will automatically close in 4 days unless further comments are made. response-requested Waiting on additional info and feedback. Will move to \"closing-soon\" in 7 days. labels Aug 7, 2023
@xconverge
Copy link
Author

I just had to delete and recreate my lockfile again to update

@frankleng
Copy link

frankleng commented Sep 1, 2023

issue went away after forcing @smithy/types : 2.2.2
s3 was using 2.0.2 according to lock file

@oscarhermoso
Copy link

Ditto - I also resolved this issue by installing @smithy/[email protected] instead of relying on nested dependendencies in the lock file.

@shellscape
Copy link

Just ran across this with a Property 'TopicArn' does not exist on type 'ServiceOutputTypes' using CreateTopicCommand from @aws-sdk/client-sns. My repo is a monorepo with pnpm, and another project in the monorepo is using an older version of aws-sdk (the context is valid in this case). Installing @smithy/types@latest directly in the affected project solved the errors as well.

@Gkleinereva
Copy link

I ran into this issue on @aws-sdk/[email protected]. I'm using pnpm - re-generating the lock file and reinstalling/rebuilding node_modules fixed the issue for me

@anna-kol
Copy link

anna-kol commented Dec 5, 2023

it happened to me in the newest version 3.465.0

it thinks it should be "new S3Client([{ region: 'us-east1' }])", which throws region not found error
instead of "new S3Client({ region: 'us-east1' })" which is correct but the typescript doesn't accept

i tried to delete node_modules and the lock file like suggested, and also install the latest smithy/types 2.6.0 instead of relying on the dependency and it didn't help

reverting to "@aws-sdk/client-s3": "3.13.1" helped.

@espetro
Copy link

espetro commented Apr 3, 2024

The issue with getSignedUrl still happens! I'm using the following versions:

{
  "@aws-sdk/client-s3": "3.540.0",
  "@aws-sdk/s3-request-presigner": "3.540.0",
}

@letharion
Copy link

For what it's worth I ended up here after googling for a similar error. rm yarn.lock && yarn resolved the issue for me.
The git diff after that is gigantic, >16000 LOC, so I won't post that here, but the parts I'm assuming are relevant to this are a large number of minor upgrades in the form of:

-    "@aws-sdk/client-s3": "^3.554.0",
+    "@aws-sdk/client-s3": "^3.621.0",

and

-    "@smithy/middleware-content-length": "npm:^3.0.4"
-    "@smithy/middleware-endpoint": "npm:^3.0.5"
-    "@smithy/middleware-retry": "npm:^3.0.10"
+    "@smithy/middleware-content-length": "npm:^3.0.5"
+    "@smithy/middleware-endpoint": "npm:^3.1.0"
+    "@smithy/middleware-retry": "npm:^3.0.13"

@RanVaknin
Copy link
Contributor

Hi everyone on the thread. It seems like there is a mix of people that solved this by deleting the lock file and npm installing the latest versions of the SDK. For others it was to install a newer version of smithy types directly, for others its a different issue altogether.

Because this was not reproducible, and people are chiming in with different reasoning, we are not able to make actionable suggestions. I ask that if you are still affected to do the following:

  1. Delete the lock file and re-install your dependencies - preferably use the latest version
  2. If the issue persists please open a new issue, and provide a small github repository that we can clone and test locally ourselves to better help you.

Thanks,
Ran~

@shellscape
Copy link

It's fairly irresponsible to recommend deleting a lock file to correct a single dependency issue. they're not supposed to be superfluous like that.

Copy link

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs and link to relevant comments in this thread.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Aug 18, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug This issue is a bug. p2 This is a standard priority issue
Projects
None yet
Development

No branches or pull requests