Using Kratos for Service Accounts for Service-to-Service Auth

[taylow]

Hello, I would love to open a discussion about the use of Ory Kratos for service accounts, for use in service-to-service authentication/authorisation where a user would normally be present.

Setup

Currently, we are using middleware for our IAM solution; one that makes a request to Kratos’ session/whoami endpoint for a user session based on the ory_kratos_session cookie sent from the frontend - this session is then injected into the request flow’s context - and another that performs various Keto Check requests using the injected user session. Once a request makes it past the middleware, we run the request in the context of that user.

Problem

Our problem arose when we began exposing the functionality of our services as user-definable HTTP APIs, where a user is no longer involved in the process. For reference, we are building a no-code tool that allows organisations to build functions-as-a-service.

Our endpoint service is responsible for taking the requests to these user-defined endpoints and executing them. In order to do this, we must contact a few services to prepare the request, and finally make a request to the execution service to execute the request. As these requests are made without the context of a user, we hit the (purposely erected) wall of our middleware.

Solutions

A good few hours of discussion later and we came up with a few possible solutions.

1. Setup a separate pool of internal and external services - the internal pool would be configured to not contain Kratos/Keto middleware.
   1. We would use Hashicorp Consul to configure the two pools and would protect the internal pool using their rules engine to prevent services from having free rein.
   2. This is a fairly complicated solution, as it ties us to another technology, requires potentially complicated rules, and adds many more doors for problems.
2. Ory Hydra - Users would allow 3rd parties to make requests on their behalf, essentially running the user-defined APIs in the context of the user that published them.
   1. We have an organisation layer that would make this fairly awkward, as the APIs that are exposed are owned by an organisation/teams and not a single person.
   2. If a user leaves an organisation, their Keto relation tuples are deleted, rendering the APIs unusable.
      1. We could mitigate this by transferring the ownership/publishing account to the one next in line when a user leaves/is removed.
   3. Our understanding is that Hydra would allow 3rd parties to act on behalf of a user, but this isn’t really what we want/need, as we ideally don’t want a single user to have that responsibility.
      1. That said, we can still see Hydra being useful in our final solution.
3. Second instance of Kratos for service users - this would not be publicly accessible and would be used purely for Service Users.
   1. Service Users will be generated either automatically (per user-defined API) or via some screen that allows a user to tie 1-to-many APIs to a service user.
   2. The username and password for these service users will be used as API key + secret key auth and given to users on a one time basis.
      1. We wouldn’t store any passwords, only expose them once to the user on creation.
      2. API keys/usernames can be hashed for extra security of the API key.
      3. Offers the ability to add OAuth using Hydra in the future (request made on behalf of organisation via service user instead of a real user member of the organisation).
      4. Allows a service user to be deleted by team members with the right privileges, revoking the current API key/secret key pair.
      5. We can apply Keto relations to these service users so that they can use our Keto middleware and have permissions to access an organisation’s resources.
      6. We wouldn’t have to modify our middleware much. We would simply modify our configs to map a specific cookie to a specific instance of Kratos
         (e.g. ory_session_cookie: localhost:4466 and ory_service_cookie: localhost:4499) and have the service that needs to make these requests use the ory_service_cookie in its requests to other services.

Diagrams

I pulled together some diagrams for our current request flow and our proposed “Kratos 2” flow (as we’re calling it internally).

Kratos 1 - User Accounts

Kratos 2 - Service Accounts

Issues/Concerns

At the moment, it appears that Kratos doesn’t clear any of its flow or session data once expired. As a service account could potentially be logged in many times in a short period (maybe even concurrently), these tables begin to fill with a lot of redundant data. I believe there have been PRs/issues open for these redundant entries, and I believe a clean up tool is due to be released in the future.

Without testing, we are unsure if Kratos is even suitable for this level of traffic with constant login flows being generated, and whether concurrent/multiple sessions are even possible.

We would also like to have the TTL of these sessions be around the same time as our execution timeout, so very short. Again, without testing we’re unsure of the minimum expiry time of a session.

Conclusion

We are essentially looking to build something similar to AWS’ IAM page, where you can create users that represent either a person (regular users) or a service (service users). In our setup, this would not be a single account that the service logs into, but instead one that the “API logs into” using the credentials provided in the HTTP request. We want to enable our customers to generate and manage API tokens which can be authorise to use our backend on an organisation’s behalf.

We think we’re trying to tackling something here that others may find useful. Any input is massively appreciated!

[aeneasr]

Thank you for this detailed explanation. I think that you can go with the two-variant version of Ory Kratos, or potentially also introduce a second identity schema for machine users and only create them via the Admin API.

Another alternative is to use the OAuth2 Client Credentials grant which is basically what a service account could do. It doesn't require user interaction and works with a variety of authentication mechanisms, including mTLS at some point. This might come in handy when we're talking Istio!

You might also be able to leverage RFC7523 ( https://www.ory.sh/docs/hydra/guides/oauth2-grant-type-jwt-bearer ) to generate tokens on behalf of someone without user interaction.

[taylow]

Thank you very much for the input @aeneasr!

A second identity schema! 💡 This managed to sneak past us while coming up with solutions, even after recently migrating to 0.9.0 and changed our configs to schemas and default_schema_id 🤦‍♀️ I'm assuming identity schema-based flow control isn't a feature at the moment? (i.e. something like identity.schemas.service_user_schema.selfservice.flows.registration.enabled: false). We also wouldn't want anyone to login to our regular system using these accounts. This could be enforced at the application/middleware layer, but schema-based flow origin whitelisting would be a much stronger solution (i.e. something like identity.schemas.service_user_schema.selfservice.flows.login.whitelisted_urls: - url: kratos.internal).

We have looked further into Client Credentials since and believe this definitely could work. I personally need to brush up on the concept and understand what entities are provided once a successful request is made, and how they can fit into our current middleware (i.e. Keto checks for specific resources are currently tied to identity IDs) or if we would need another piece of middleware. I wouldn't be able to make that decision until I do some more research.

As for RFC7523, I think our problem lies in having an organisation layer, where we ideally don't want a single (real) user to be responsible for access to the FaaS APIs hosted by the organisation. Our fear is that a user may leave an organisation, rendering the JWT as unusable; breaking all clients in the process. Again, this may be something we either need to do more research on, or need to mitigate. We discussed implementing a process that swaps a user's responsibilities/permissions to the organisation owner when they leave/are removed, but again we aren't sure if this is possible with JWTs without testing.

Thanks again, we really appreciate you taking the time to input on this!

[till]

I think the issue tying it to an actual user is my biggest concern as well. As in, person leaves, everything breaks. Or the account is kept around, people share a password. That's what I see users do.

The other is, that if options for the keys granular, then users usually get "lazy" and deal with a "I can do everything" key attached to a user that can literally do everything because they don't want to go through this again and again. Rotation also doesn't happen then (speaking from personal experience). :D So this has to be taken into account but may be another concern (not Kratos).

I like the idea of using a second schema and creating (machine) accounts through the admin API best currently. Though I am not sure how login flows for these accounts would work. As in, I have a machine user that is supposed to authenticate. I think that the browser based flow would not be it, but I asked this on Slack before: is there a way to supply a schema somewhere in the flows?

[taylow]

We found an example of where this is handled in another SaaS; Miro. If you delete a user on Miro that owns some boards within a Team, it offers the ability to straight up wipe the data or transfer the ownership to yourself (presumably restricted to owner/admin).

There is definitely risks in giving an account access to everything! Our Keto middleware that performs the necessary checksare unique to each endpoint, so we essentially configure this through code atm. A simplified config file with some sort of permissions->endpoint mapping could be achieved, but it was a little out of scope for us at the minute! The Ory Summit YouTube video I linked below by Ashley Manraj over at Pvotal have achieved this using Protobuf Options in each service method. They define the required permissions to perform the role, and presumably this is used within some Keto check.

The one thing that steered us away from using a second Kratos schema was the chattiness when signing in/validating a user. This may be incorrect, as we didn't end up going down this route, but I believe you need to first request a login flow, then use the flow ID from the response to perform a login request. At this point, you either receive the identity/session, or you much do another request to the session/whoami endpoint to fetch the session! If there is a better way to programatically login, please let me know! :D

[till]

Maybe we hit a bug (with Ory Cloud), but as far as I thought, once a schema is changed, we can no longer use it in the flows.

I have 2 users:

• user A, first schema
• user B, second schema (default)

I can no longer login with user A.

But I'll investigate and try again.

[till]

Maybe we hit a bug (with Ory Cloud), but as far as I thought, once a schema is changed, we can no longer use it in the flows.

I have 2 users:

• user A, first schema
• user B, second schema (default)

I can no longer login with user A.

But I'll investigate and try again.

This is resolved, may have been related to the SDK woes.

[taylow]

I just replied to someone on Slack asking about api keys, but thought it would be nice to have here for searchability

Copied from Slack

A few solutions include:

• Separate identity schemas for service users
• Separate instances of kratos for service users, using the username and passwords of the identities as basic auth credentials, or writing middleware that splits an api key on a given delimiter which is then used as the username/password in a login flow
• Having your reverse proxy configured to forward auth requests (Caddy forward_auth, nginx subrequest, etc.) to both Kratos and an API key service, or potentially as an authoriser in Oathkeeper
  Use Ory Hydra’s client credentials
• Create a custom service to handle api keys separately to Kratos

There is also a blog post that touches on how you can use Kong’s api key plugin paired with Oathkeeper and Kratos to add api keys to your auth stack https://www.ory.sh/zero-trust-api-security-ory-tutorial/

A similar configuration to the reverse proxy stuff can be seen in this Ory Summit video too! https://www.youtube.com/watch?v=A_IH_1NW7cM&t=245s
This isn’t trivial though, we have gone through much research to come up with a solution that woks for us!

Also worth noting that we use gRPC/gRPC-Web, which I believe isn’t supported by Oathkeeper, so this made things a little more complicated for us!

[vinckr]

This may be relevant for this topic:
The Ory Kratos and Ory Hydra integration just landed with Hydra 2.0,
see the release notes for more details!