Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Feature Req][Azure.Extensions.AspNetCore.Configuration.Secrets] Ability to lazily load secrets from Azure KV #18410

Closed
jananiva opened this issue Feb 4, 2021 · 3 comments
Closed

[Feature Req][Azure.Extensions.AspNetCore.Configuration.Secrets] Ability to lazily load secrets from Azure KV #18410

jananiva opened this issue Feb 4, 2021 · 3 comments
Assignees
@heaths @pakrym
Labels
Client This issue points to a problem in the data-plane of the library. customer-reported Issues that are reported by GitHub users external to the Azure organization. Extensions ASP.NET Core extensions feature-request This issue requires a new behavior in the product in order be resolved. needs-team-attention Workflow: This issue needs attention from Azure service team or SDK team
Milestone
Backlog

Comments

@jananiva
Copy link

jananiva commented Feb 4, 2021 •
edited
Loading

Our code uses Azure.Extensions.AspNetCore.Configuration.Secrets package to load secrets from KV into configuration instance during app startup. Since there is no option but to load all secrets at once from KV, when multiple app instances are trying to connect to KV during startup, KV throttles us.

And during high loads, when we scale up the number of VMs/instances, all of these instances try to come up at the same time and try to read it from KV and KV keeps throttling us.

Service Fabric cluster doesn't take to startup failures pretty well and it keeps aggressively retrying to startup which further worsens the load on KV!

The app does not need 'ALL' the secrets in the keyvault during start-up, it only needs a subset. So, please add the ability to lazily load secrets on demand from KV.
@ghost ghost added needs-triage Workflow: This is a new issue that needs to be triaged to the appropriate team. customer-reported Issues that are reported by GitHub users external to the Azure organization. question The issue doesn't require a change to the product in order to be resolved. Most issues start as that labels Feb 4, 2021
@jananiva jananiva changed the title Ability to lazily load values from Azure KV Ability to lazily load secrets from Azure KV Feb 4, 2021
@jananiva jananiva changed the title Ability to lazily load secrets from Azure KV [Azure.Extensions.AspNetCore.Configuration.Secrets] Ability to lazily load secrets from Azure KV Feb 4, 2021
@jananiva jananiva changed the title [Azure.Extensions.AspNetCore.Configuration.Secrets] Ability to lazily load secrets from Azure KV [Feature Req][Azure.Extensions.AspNetCore.Configuration.Secrets] Ability to lazily load secrets from Azure KV Feb 4, 2021
@Mohit-Chakraborty Mohit-Chakraborty added Client This issue points to a problem in the data-plane of the library. feature-request This issue requires a new behavior in the product in order be resolved. KeyVault labels Feb 4, 2021
@ghost ghost removed the needs-triage Workflow: This is a new issue that needs to be triaged to the appropriate team. label Feb 4, 2021
@Mohit-Chakraborty
Copy link
Contributor

Thank you for your feedback. Tagging and routing to the team best able to assist.

@Mohit-Chakraborty Mohit-Chakraborty added the needs-team-triage Workflow: This issue needs the team to triage. label Feb 4, 2021
@jsquire jsquire added needs-team-attention Workflow: This issue needs attention from Azure service team or SDK team and removed feature-request This issue requires a new behavior in the product in order be resolved. needs-team-triage Workflow: This issue needs the team to triage. labels Feb 4, 2021
@heaths
Copy link
Member

heaths commented Feb 4, 2021

Actually, this is an ASP.NET extensions ask with @pakrym owns but I can take a look as well, so adding him as co-assignee.

@heaths heaths added Extensions ASP.NET Core extensions feature-request This issue requires a new behavior in the product in order be resolved. and removed KeyVault question The issue doesn't require a change to the product in order to be resolved. Most issues start as that labels Feb 4, 2021
@pakrym
Copy link
Contributor

pakrym commented Feb 4, 2021

Unfortunately, the ASP.NET Core Options system doesn't support lazy loading. All configuration sources are enumerated eagerly when on ConfigurationBuilder.Build()

https://github.com/dotnet/runtime/blob/8241168b2d85e8e7ff25fa8984d91fed8da7922a/src/libraries/Microsoft.Extensions.Configuration/src/ConfigurationRoot.cs#L35

You can use the https://github.com/Azure/azure-sdk-for-net/blob/master/sdk/extensions/Azure.Extensions.AspNetCore.Configuration.Secrets/src/KeyVaultSecretManager.cs to only load a subset of secrets by overriding the Load method.

@jsquire jsquire added this to the Backlog milestone Feb 8, 2021
@pakrym pakrym closed this as completed Feb 17, 2021
openapi-sdkautomation bot pushed a commit to AzureSDKAutomation/azure-sdk-for-net that referenced this issue Mar 30, 2022
@dosegal @NoaGoren @ushasan
CodeGen from PR 18410 in Azure/azure-rest-api-specs
2da2d6d 
dev-Sentinel-2022-04-01-preview (Azure#18410)

* Adds base for updating Microsoft.SecurityInsights from version preview/2022-01-01-preview to version 2022-04-01-preview

* Updates readme

* Updates API version in new specs and examples

* Fix ThreatInteliignece accordig to latest preview for alignment (Azure#18231)

* Add UebaEntityProviders to EntityAnalyticsSettings (Azure#18196)

* Add UebaEntityProviders to EntityAnalyticsSettings

* remove ueba

* Added additionalProperties to itemsKeyValue to resolve autorest issues (Azure#18170)

* added additionalProperties to itemsKeyValue to resolve autorest issues

* Update Watchlists.json

* ran prettier-fix

* Remove readonly property from ownerType (Azure#18450)

Co-authored-by: dosegal <[email protected]>
Co-authored-by: NoaGoren <[email protected]>
Co-authored-by: ushasan <[email protected]>
@github-actions github-actions bot locked and limited conversation to collaborators Mar 28, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@heaths heaths

@pakrym pakrym

Labels
Client This issue points to a problem in the data-plane of the library. customer-reported Issues that are reported by GitHub users external to the Azure organization. Extensions ASP.NET Core extensions feature-request This issue requires a new behavior in the product in order be resolved. needs-team-attention Workflow: This issue needs attention from Azure service team or SDK team
Projects
None yet
Milestone
Backlog
Development

No branches or pull requests
5 participants
@jsquire @heaths @pakrym @Mohit-Chakraborty @jananiva