[bug] @azure-tools/typespec-python `keyError: OperationGroups` #2975

l0lawrence · 2024-12-17T23:22:22Z

With the recent typespec versioning updates to the rest-api-specs repo it looks like this error is back again for EventGrid System Events (a models only package): cmdout [automation_generate.sh] 2024-12-17 21:40:57 [ERROR] KeyError: 'operationGroups'

as seen here: https://github.com/Azure/azure-rest-api-specs/pull/31739/checks?check_run_id=34561185899

Was there a design change? I believe we ran into this before and resolved it.

The text was updated successfully, but these errors were encountered:

l0lawrence · 2024-12-17T23:25:08Z

Ah looks like this still might be open? #2630. But see here it passed.

swathipil · 2024-12-18T03:45:28Z

I'm also seeing it after bumping from 0.35.1 to 0.37.1. Stack trace:

Error: Command failed: /home/swathip/repos/azure-sdk-tools/packages/python-packages/apiview-stub-generator/TempTypeSpecFiles/apiview-treestyle-parser-schema/node_modules/@typespec/http-client-python/venv/bin/python /home/swathip/repos/azure-sdk-tools/packages/python-packages/apiview-stub-generator/TempTypeSpecFiles/apiview-treestyle-parser-schema/node_modules/@typespec/http-client-python/eng/scripts/setup/run_tsp.py --output-folder=/home/swathip/repos/azure-sdk-tools/packages/python-packages/apiview-stub-generator --cadl-file=/tmp/cadl-codegen/python-yaml-path1cce935f-08a8-4ee4-82a0-c60f918da792.yaml --package-version=0.3.13 --generate-packaging-files=false --flavor=azure --package-dir=apiview-stub-generator --package-name=treestyle-parser --generate-test=false --generate-sample=false --models-mode=dpg --emit-cross-language-definition-file=true --from-typespec=true
Traceback (most recent call last):
  File "/home/swathip/repos/azure-sdk-tools/packages/python-packages/apiview-stub-generator/TempTypeSpecFiles/apiview-treestyle-parser-schema/node_modules/@typespec/http-client-python/eng/scripts/setup/run_tsp.py", line 41, in <module>
    codegen.CodeGenerator(output_folder=args.output_folder, cadl_file=args.cadl_file, **unknown_args).process()
  File "/home/swathip/repos/azure-sdk-tools/packages/python-packages/apiview-stub-generator/TempTypeSpecFiles/apiview-treestyle-parser-schema/node_modules/@typespec/http-client-python/generator/pygen/codegen/__init__.py", line 336, in process
    self.sort_exceptions(yaml_data)
  File "/home/swathip/repos/azure-sdk-tools/packages/python-packages/apiview-stub-generator/TempTypeSpecFiles/apiview-treestyle-parser-schema/node_modules/@typespec/http-client-python/generator/pygen/codegen/__init__.py", line 247, in sort_exceptions
    for group in client["operationGroups"]:
KeyError: 'operationGroups'

    at genericNodeError (node:internal/errors:983:15)
    at wrappedFn (node:internal/errors:537:14)
    at checkExecSyncError (node:child_process:882:11)
    at execSync (node:child_process:954:15)
    at $onEmit (file:///home/swathip/repos/azure-sdk-tools/packages/python-packages/apiview-stub-generator/TempTypeSpecFiles/apiview-treestyle-parser-schema/node_modules/@typespec/http-client-python/dist/emitter/emitter.js:110:9)
    at async Object.$onEmit [as emitFunction] (file:///home/swathip/repos/azure-sdk-tools/packages/python-packages/apiview-stub-generator/TempTypeSpecFiles/apiview-treestyle-parser-schema/node_modules/@azure-tools/typespec-python/dist/src/emitter.js:3:5)
    at async runEmitter (file:///home/swathip/repos/azure-sdk-tools/packages/python-packages/apiview-stub-generator/TempTypeSpecFiles/apiview-treestyle-parser-schema/node_modules/@typespec/compiler/dist/src/core/program.js:366:13)
    at async compile (file:///home/swathip/repos/azure-sdk-tools/packages/python-packages/apiview-stub-generator/TempTypeSpecFiles/apiview-treestyle-parser-schema/node_modules/@typespec/compiler/dist/src/core/program.js:123:9)
    at async compileTsp (file:///home/swathip/.nvm/versions/node/v23.4.0/lib/node_modules/@azure-tools/typespec-client-generator-cli/dist/typespec.js:97:21)
    at async generateCommand (file:///home/swathip/.nvm/versions/node/v23.4.0/lib/node_modules/@azure-tools/typespec-client-generator-cli/dist/commands.js:248:35)

--------------------------------------------------
Library Version                0.37.1
TypeSpec Compiler Version      0.63.0
--------------------------------------------------
    at runEmitter (file:///home/swathip/repos/azure-sdk-tools/packages/python-packages/apiview-stub-generator/TempTypeSpecFiles/apiview-treestyle-parser-schema/node_modules/@typespec/compiler/dist/src/core/program.js:369:19)
    at async compile (file:///home/swathip/repos/azure-sdk-tools/packages/python-packages/apiview-stub-generator/TempTypeSpecFiles/apiview-treestyle-parser-schema/node_modules/@typespec/compiler/dist/src/core/program.js:123:9)
    at async compileTsp (file:///home/swathip/.nvm/versions/node/v23.4.0/lib/node_modules/@azure-tools/typespec-client-generator-cli/dist/typespec.js:97:21)
    at async generateCommand (file:///home/swathip/.nvm/versions/node/v23.4.0/lib/node_modules/@azure-tools/typespec-client-generator-cli/dist/commands.js:248:35)
    at async updateCommand (file:///home/swathip/.nvm/versions/node/v23.4.0/lib/node_modules/@azure-tools/typespec-client-generator-cli/dist/commands.js:307:5)
    at async Object.handler (file:///home/swathip/.nvm/versions/node/v23.4.0/lib/node_modules/@azure-tools/typespec-client-generator-cli/dist/index.js:151:5) {
  info: {
    kind: 'emitter',
    metadata: {
      type: 'module',
      name: '@azure-tools/typespec-python',
      homepage: 'https://github.com/Azure/autorest.python',
      bugs: [Object],
      version: '0.37.1'
    },
    error: Error: Command failed: /home/swathip/repos/azure-sdk-tools/packages/python-packages/apiview-stub-generator/TempTypeSpecFiles/apiview-treestyle-parser-schema/node_modules/@typespec/http-client-python/venv/bin/python /home/swathip/repos/azure-sdk-tools/packages/python-packages/apiview-stub-generator/TempTypeSpecFiles/apiview-treestyle-parser-schema/node_modules/@typespec/http-client-python/eng/scripts/setup/run_tsp.py --output-folder=/home/swathip/repos/azure-sdk-tools/packages/python-packages/apiview-stub-generator --cadl-file=/tmp/cadl-codegen/python-yaml-path1cce935f-08a8-4ee4-82a0-c60f918da792.yaml --package-version=0.3.13 --generate-packaging-files=false --flavor=azure --package-dir=apiview-stub-generator --package-name=treestyle-parser --generate-test=false --generate-sample=false --models-mode=dpg --emit-cross-language-definition-file=true --from-typespec=true
    Traceback (most recent call last):
      File "/home/swathip/repos/azure-sdk-tools/packages/python-packages/apiview-stub-generator/TempTypeSpecFiles/apiview-treestyle-parser-schema/node_modules/@typespec/http-client-python/eng/scripts/setup/run_tsp.py", line 41, in <module>
        codegen.CodeGenerator(output_folder=args.output_folder, cadl_file=args.cadl_file, **unknown_args).process()
      File "/home/swathip/repos/azure-sdk-tools/packages/python-packages/apiview-stub-generator/TempTypeSpecFiles/apiview-treestyle-parser-schema/node_modules/@typespec/http-client-python/generator/pygen/codegen/__init__.py", line 336, in process
        self.sort_exceptions(yaml_data)
      File "/home/swathip/repos/azure-sdk-tools/packages/python-packages/apiview-stub-generator/TempTypeSpecFiles/apiview-treestyle-parser-schema/node_modules/@typespec/http-client-python/generator/pygen/codegen/__init__.py", line 247, in sort_exceptions
        for group in client["operationGroups"]:
    KeyError: 'operationGroups'

msyyc · 2024-12-18T08:47:24Z

Thanks for your feedback and let me take an investigation.

msyyc · 2024-12-23T02:33:05Z

Update: I have found the root cause and will fix it ASAP.

msyyc · 2025-01-03T06:00:26Z

@swathipil @l0lawrence The issue is fixed in latest emitter 0.37.2.

iscai-msft · 2025-01-06T16:20:58Z

Closing, please reopen if it's still an issue. Thanks @msyyc for your help!

swathipil · 2025-01-07T01:07:42Z

Hi @msyyc - I'm still seeing this issue after updating to the latest typespec-python version. Here's the trace:

Using output directory '/home/swathip/repos/azure-sdk-tools/packages/python-packages/apiview-stub-generator'

888                                      888 d8b                   888    
888                                      888 Y8P                   888    
888                                      888                       888    
888888 .d8888b  88888b.          .d8888b 888 888  .d88b.  88888b.  888888 
888    88K      888 "88b        d88P"    888 888 d8P  Y8b 888 "88b 888    
888    "Y8888b. 888  888 888888 888      888 888 88888888 888  888 888    
Y88b.       X88 888 d88P        Y88b.    888 888 Y8b.     888  888 Y88b.  
 "Y888  88888P' 88888P"          "Y8888P 888 888  "Y8888  888  888  "Y888 
                888                                                       
                888                                                       
                888                                                       

0.14.3
Found emitter package @azure-tools/typespec-python@latest
Installing dependencies from npm...

added 97 packages, and audited 98 packages in 21s

14 packages are looking for funding
  run `npm fund` for details

found 0 vulnerabilities
Resolved path: /home/swathip/repos/azure-sdk-tools/packages/python-packages/apiview-stub-generator/TempTypeSpecFiles/apiview-treestyle-parser-schema/node_modules/@typespec/compiler
Compiling tsp using @azure-tools/typespec-python...
Traceback (most recent call last):
  File "/home/swathip/repos/azure-sdk-tools/packages/python-packages/apiview-stub-generator/TempTypeSpecFiles/apiview-treestyle-parser-schema/node_modules/@typespec/http-client-python/eng/scripts/setup/run_tsp.py", line 41, in <module>
    codegen.CodeGenerator(output_folder=args.output_folder, cadl_file=args.cadl_file, **unknown_args).process()
  File "/home/swathip/repos/azure-sdk-tools/packages/python-packages/apiview-stub-generator/TempTypeSpecFiles/apiview-treestyle-parser-schema/node_modules/@typespec/http-client-python/generator/pygen/codegen/__init__.py", line 336, in process
    self.sort_exceptions(yaml_data)
  File "/home/swathip/repos/azure-sdk-tools/packages/python-packages/apiview-stub-generator/TempTypeSpecFiles/apiview-treestyle-parser-schema/node_modules/@typespec/http-client-python/generator/pygen/codegen/__init__.py", line 247, in sort_exceptions
    for group in client["operationGroups"]:
KeyError: 'operationGroups'
update

Sync and generate from a TypeSpec project

Options:
  -d, --debug            Enable debug logging                          [boolean]
  -o, --output-dir       Specify an alternate output directory for the generated
                         files.                          [string] [default: "."]
  -y, --no-prompt        Skip any interactive prompts.                 [boolean]
      --help             Show help                                     [boolean]
      --repo             Repository where the project is defined        [string]
      --commit           Commit hash to be used                         [string]
      --tsp-config       Path to tspconfig.yaml                         [string]
      --local-spec-repo  Path to local spec repo                        [string]
      --emitter-options  The options to pass to the emitter             [string]
      --save-inputs      Don't clean up the temp directory after generation
                                                                       [boolean]
  -v, --version          Show version number                           [boolean]

ExternalError: Emitter "@azure-tools/typespec-python" crashed! This is a bug.
Please file an issue at https://github.com/Azure/autorest.python/issues

Error: Command failed: /home/swathip/repos/azure-sdk-tools/packages/python-packages/apiview-stub-generator/TempTypeSpecFiles/apiview-treestyle-parser-schema/node_modules/@typespec/http-client-python/venv/bin/python /home/swathip/repos/azure-sdk-tools/packages/python-packages/apiview-stub-generator/TempTypeSpecFiles/apiview-treestyle-parser-schema/node_modules/@typespec/http-client-python/eng/scripts/setup/run_tsp.py --output-folder=/home/swathip/repos/azure-sdk-tools/packages/python-packages/apiview-stub-generator --cadl-file=/tmp/cadl-codegen/python-yaml-path3e7a3279-3ef7-4a45-9b82-59d8fad79e04.yaml --package-version=0.3.13 --generate-packaging-files=false --flavor=azure --package-dir=apiview-stub-generator --package-name=treestyle-parser --generate-test=false --generate-sample=false --models-mode=dpg --emit-cross-language-definition-file=true --from-typespec=true
Traceback (most recent call last):
  File "/home/swathip/repos/azure-sdk-tools/packages/python-packages/apiview-stub-generator/TempTypeSpecFiles/apiview-treestyle-parser-schema/node_modules/@typespec/http-client-python/eng/scripts/setup/run_tsp.py", line 41, in <module>
    codegen.CodeGenerator(output_folder=args.output_folder, cadl_file=args.cadl_file, **unknown_args).process()
  File "/home/swathip/repos/azure-sdk-tools/packages/python-packages/apiview-stub-generator/TempTypeSpecFiles/apiview-treestyle-parser-schema/node_modules/@typespec/http-client-python/generator/pygen/codegen/__init__.py", line 336, in process
    self.sort_exceptions(yaml_data)
  File "/home/swathip/repos/azure-sdk-tools/packages/python-packages/apiview-stub-generator/TempTypeSpecFiles/apiview-treestyle-parser-schema/node_modules/@typespec/http-client-python/generator/pygen/codegen/__init__.py", line 247, in sort_exceptions
    for group in client["operationGroups"]:
KeyError: 'operationGroups'

    at genericNodeError (node:internal/errors:983:15)
    at wrappedFn (node:internal/errors:537:14)
    at checkExecSyncError (node:child_process:882:11)
    at execSync (node:child_process:954:15)
    at $onEmit (file:///home/swathip/repos/azure-sdk-tools/packages/python-packages/apiview-stub-generator/TempTypeSpecFiles/apiview-treestyle-parser-schema/node_modules/@typespec/http-client-python/dist/emitter/emitter.js:110:9)
    at async Object.$onEmit [as emitFunction] (file:///home/swathip/repos/azure-sdk-tools/packages/python-packages/apiview-stub-generator/TempTypeSpecFiles/apiview-treestyle-parser-schema/node_modules/@azure-tools/typespec-python/dist/src/emitter.js:3:5)
    at async runEmitter (file:///home/swathip/repos/azure-sdk-tools/packages/python-packages/apiview-stub-generator/TempTypeSpecFiles/apiview-treestyle-parser-schema/node_modules/@typespec/compiler/dist/src/core/program.js:366:13)
    at async compile (file:///home/swathip/repos/azure-sdk-tools/packages/python-packages/apiview-stub-generator/TempTypeSpecFiles/apiview-treestyle-parser-schema/node_modules/@typespec/compiler/dist/src/core/program.js:123:9)
    at async compileTsp (file:///home/swathip/.nvm/versions/node/v23.5.0/lib/node_modules/@azure-tools/typespec-client-generator-cli/dist/typespec.js:97:21)
    at async generateCommand (file:///home/swathip/.nvm/versions/node/v23.5.0/lib/node_modules/@azure-tools/typespec-client-generator-cli/dist/commands.js:248:35)

--------------------------------------------------
Library Version                0.37.2
TypeSpec Compiler Version      0.63.0
--------------------------------------------------
    at runEmitter (file:///home/swathip/repos/azure-sdk-tools/packages/python-packages/apiview-stub-generator/TempTypeSpecFiles/apiview-treestyle-parser-schema/node_modules/@typespec/compiler/dist/src/core/program.js:369:19)
    at async compile (file:///home/swathip/repos/azure-sdk-tools/packages/python-packages/apiview-stub-generator/TempTypeSpecFiles/apiview-treestyle-parser-schema/node_modules/@typespec/compiler/dist/src/core/program.js:123:9)
    at async compileTsp (file:///home/swathip/.nvm/versions/node/v23.5.0/lib/node_modules/@azure-tools/typespec-client-generator-cli/dist/typespec.js:97:21)
    at async generateCommand (file:///home/swathip/.nvm/versions/node/v23.5.0/lib/node_modules/@azure-tools/typespec-client-generator-cli/dist/commands.js:248:35)
    at async updateCommand (file:///home/swathip/.nvm/versions/node/v23.5.0/lib/node_modules/@azure-tools/typespec-client-generator-cli/dist/commands.js:307:5)
    at async Object.handler (file:///home/swathip/.nvm/versions/node/v23.5.0/lib/node_modules/@azure-tools/typespec-client-generator-cli/dist/index.js:151:5) {
  info: {
    kind: 'emitter',
    metadata: {
      type: 'module',
      name: '@azure-tools/typespec-python',
      homepage: 'https://github.com/Azure/autorest.python',
      bugs: [Object],
      version: '0.37.2'
    },
    error: Error: Command failed: /home/swathip/repos/azure-sdk-tools/packages/python-packages/apiview-stub-generator/TempTypeSpecFiles/apiview-treestyle-parser-schema/node_modules/@typespec/http-client-python/venv/bin/python /home/swathip/repos/azure-sdk-tools/packages/python-packages/apiview-stub-generator/TempTypeSpecFiles/apiview-treestyle-parser-schema/node_modules/@typespec/http-client-python/eng/scripts/setup/run_tsp.py --output-folder=/home/swathip/repos/azure-sdk-tools/packages/python-packages/apiview-stub-generator --cadl-file=/tmp/cadl-codegen/python-yaml-path3e7a3279-3ef7-4a45-9b82-59d8fad79e04.yaml --package-version=0.3.13 --generate-packaging-files=false --flavor=azure --package-dir=apiview-stub-generator --package-name=treestyle-parser --generate-test=false --generate-sample=false --models-mode=dpg --emit-cross-language-definition-file=true --from-typespec=true
    Traceback (most recent call last):
      File "/home/swathip/repos/azure-sdk-tools/packages/python-packages/apiview-stub-generator/TempTypeSpecFiles/apiview-treestyle-parser-schema/node_modules/@typespec/http-client-python/eng/scripts/setup/run_tsp.py", line 41, in <module>
        codegen.CodeGenerator(output_folder=args.output_folder, cadl_file=args.cadl_file, **unknown_args).process()
      File "/home/swathip/repos/azure-sdk-tools/packages/python-packages/apiview-stub-generator/TempTypeSpecFiles/apiview-treestyle-parser-schema/node_modules/@typespec/http-client-python/generator/pygen/codegen/__init__.py", line 336, in process
        self.sort_exceptions(yaml_data)
      File "/home/swathip/repos/azure-sdk-tools/packages/python-packages/apiview-stub-generator/TempTypeSpecFiles/apiview-treestyle-parser-schema/node_modules/@typespec/http-client-python/generator/pygen/codegen/__init__.py", line 247, in sort_exceptions
        for group in client["operationGroups"]:
    KeyError: 'operationGroups'
    
        at genericNodeError (node:internal/errors:983:15)
        at wrappedFn (node:internal/errors:537:14)
        at checkExecSyncError (node:child_process:882:11)
        at execSync (node:child_process:954:15)
        at $onEmit (file:///home/swathip/repos/azure-sdk-tools/packages/python-packages/apiview-stub-generator/TempTypeSpecFiles/apiview-treestyle-parser-schema/node_modules/@typespec/http-client-python/dist/emitter/emitter.js:110:9)
        at async Object.$onEmit [as emitFunction] (file:///home/swathip/repos/azure-sdk-tools/packages/python-packages/apiview-stub-generator/TempTypeSpecFiles/apiview-treestyle-parser-schema/node_modules/@azure-tools/typespec-python/dist/src/emitter.js:3:5)
        at async runEmitter (file:///home/swathip/repos/azure-sdk-tools/packages/python-packages/apiview-stub-generator/TempTypeSpecFiles/apiview-treestyle-parser-schema/node_modules/@typespec/compiler/dist/src/core/program.js:366:13)
        at async compile (file:///home/swathip/repos/azure-sdk-tools/packages/python-packages/apiview-stub-generator/TempTypeSpecFiles/apiview-treestyle-parser-schema/node_modules/@typespec/compiler/dist/src/core/program.js:123:9)
        at async compileTsp (file:///home/swathip/.nvm/versions/node/v23.5.0/lib/node_modules/@azure-tools/typespec-client-generator-cli/dist/typespec.js:97:21)
        at async generateCommand (file:///home/swathip/.nvm/versions/node/v23.5.0/lib/node_modules/@azure-tools/typespec-client-generator-cli/dist/commands.js:248:35) {
      status: 1,
      signal: null,
      output: [Array],
      pid: 31159,
      stdout: <Buffer >,
      stderr: <Buffer 54 72 61 63 65 62 61 63 6b 20 28 6d 6f 73 74 20 72 65 63 65 6e 74 20 63 61 6c 6c 20 6c 61 73 74 29 3a 0a 20 20 46 69 6c 65 20 22 2f 68 6f 6d 65 2f 73 ... 935 more bytes>
    }
  }
}

Not sure if I'm missing any other package updates? Any help would be appreciated - thanks!

msyyc · 2025-01-07T03:25:29Z

After fix PR microsoft/typespec#5506 merged, we will release a new version.

msyyc · 2025-01-07T08:18:21Z

@swathipil / @l0lawrence typespec-python 0.37.3 is released and it shall fix this issue.

swathipil · 2025-01-07T17:02:59Z

thanks @msyyc!

msyyc self-assigned this Dec 18, 2024

iscai-msft closed this as completed Jan 6, 2025

swathipil reopened this Jan 7, 2025

tadelesh mentioned this issue Jan 7, 2025

[http-client-python] do not do exception sort if there is no operation groups microsoft/typespec#5506

Merged

msyyc closed this as completed in microsoft/typespec#5506 Jan 7, 2025

msyyc reopened this Jan 7, 2025

msyyc mentioned this issue Jan 7, 2025

release for typespec-python 0.37.3 #2993

Merged

msyyc closed this as completed in #2993 Jan 7, 2025

lirenhe mentioned this issue Jan 7, 2025

Python CodeGen should provide clear message for self-service trouble shooting microsoft/typespec#5509

Open

3 tasks

msyyc reopened this Jan 7, 2025

v-hongli1 closed this as completed Jan 7, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[bug] @azure-tools/typespec-python `keyError: OperationGroups` #2975

[bug] @azure-tools/typespec-python `keyError: OperationGroups` #2975

l0lawrence commented Dec 17, 2024 •

edited

Loading

l0lawrence commented Dec 17, 2024 •

edited

Loading

swathipil commented Dec 18, 2024

msyyc commented Dec 18, 2024

msyyc commented Dec 23, 2024

msyyc commented Jan 3, 2025

iscai-msft commented Jan 6, 2025

swathipil commented Jan 7, 2025

msyyc commented Jan 7, 2025

msyyc commented Jan 7, 2025

swathipil commented Jan 7, 2025

[bug] @azure-tools/typespec-python keyError: OperationGroups #2975

[bug] @azure-tools/typespec-python keyError: OperationGroups #2975

Comments

l0lawrence commented Dec 17, 2024 • edited Loading

l0lawrence commented Dec 17, 2024 • edited Loading

swathipil commented Dec 18, 2024

msyyc commented Dec 18, 2024

msyyc commented Dec 23, 2024

msyyc commented Jan 3, 2025

iscai-msft commented Jan 6, 2025

swathipil commented Jan 7, 2025

msyyc commented Jan 7, 2025

msyyc commented Jan 7, 2025

swathipil commented Jan 7, 2025

[bug] @azure-tools/typespec-python `keyError: OperationGroups` #2975

[bug] @azure-tools/typespec-python `keyError: OperationGroups` #2975

l0lawrence commented Dec 17, 2024 •

edited

Loading

l0lawrence commented Dec 17, 2024 •

edited

Loading