Add support for standard TOTP to auth backend (MFA)

[radeksimko]

This would allow using TOTP along with existing auth.

All these apps use this standard:
http://en.wikipedia.org/wiki/Google_Authenticator
https://www.authy.com/
http://www.windowsphone.com/en-us/store/app/authenticator/e7994dbc-2336-4950-91ba-ca22d653759b
https://guide.duosecurity.com/third-party-accounts

[ketzacoatl]

++

[jimjh]

Is this enhancement being considered for an upcoming release, or is it waiting for someone to pick it up?

If it's the latter, I would be happy to look into it.

[jefferai]

It's currently the latter, although I would suggest holding off for the moment. The reason is that we are working on redesigning the frameworks that provide most of the support logic for all backends in order to make them more standardized and support more features. I'm hoping that one of the standard things we can do at that point is enable multi-factor authentication support for any backend.

[timothykimball]

+1 - would love this feature

[antonlva]

+1

[skyzyx]

👍

[elconas]

Would this feature cover this user story ??

I would also like to use Vault as a "MFA" device for machine 2 machine communication, protected by MFA. 
I would like to store MFA Token Secrets (mainly TOTP) on VAULT and I would like to use the VAULT API to retrieve a TOTP OTP. This means VAULT would act like a "multi tenant Google Authenticator".
This can be used in cloud automation scenarios that enforce MFA access to API's. The user could use all authentication methods (e.g. Puppet CA) to authenticate against the Vault MFA Service.

  User -> vault:auth_vault() -> vault:get_mfa_otp('mfa_name') -> aws:get_session_token(IAMAccount) -> aws:assume_role(targetAccount)

This would ease the usage of AWS MFA enforcement for all m2m communications (e.g. Jenkins).

[jefferai]

Hi @elconas

This feature doesn't, however, that does sound like a really neat user story. Feel free to post a new issue with a request!

[CreatixEA]

Hi @jefferai

I am really interested in standard MFA. There are multiple of lib that provide these feature in all languages (PyOTP, ROTP). I suggest reusing (or write your own based on) the Ruby One Time Password :

• Already implemented backend secret library
• Make all other opensource MFA usable (you only support DUO which is not open sourced and still commercial, probably they support you)

Is it in your plan ? Have any idea of time table ?

Thank you really for the job you've made,
Sincerely,

CreatixEA

[jefferai]

@CreatixEA extensive MFA support is on the roadmap but when we were designing it we realized that in order to do it properly there are other things that need to be added first that it depends on. So it's a ways out still but very much on our mind!

[keymon]

+1

[mayakacz]

Hi all, I work for Google and we're trying to get a better idea of the demand for this FR (for Google Authenticator). If this is something you're still interested in, please thumbs up the issue. Thanks!
@emilymye

[sethvargo]

This is available as of Vault 0.7.1: https://github.com/hashicorp/vault/blob/master/CHANGELOG.md#071-may-5th-2017

https://www.vaultproject.io/docs/secrets/totp/index.html

[CreatixEA]

Hi @sethvargo

Thank you for this new feature, but I think there is a misunderstanding : the feature request was to implement a TOTP as AUTH BACKEND (not secret backend) : to be able to generate an access token after having identified ourself by TOTP (like DUO but using opensourced TOTP : a lot of lib already implements this features)

Is it possible to update this issue ?

Thanks a lot in advance,
Sincerely

[sethvargo]

Ah my misunderstanding. Reopened. Very sorry about that.

[tristanmorgan]

Does this mean the Open Source Vault won't have TOTP? #3109

[jefferai]

What's going into Enterprise is a completely new mfa system written from scratch. We do not currently plan to add this to oss, although it could happen in the future.

[tilgovi]

@jefferai I am not sure that answers @tristanmorgan question. Is the system going into Enterprise related to using TOTP in auth backends, like this issue? Or is it separate (e.g., secret backend, something else entirely related to Enterprise, etc)

[jefferai]

The system going into Enterprise will enable MFA everywhere in Vault.

[CreatixEA]

As I understand the commited file : the TOTP as Auth Backend only exists in the Entreprise version (https://www.vaultproject.io/api/system/mfa.html) since those Endpoints are only available to Entreprise version.

I can't understand that this basic feature which is the basic way to protect everything (like your Gmail account) is not OSS. Vault without MFA is useless since there is a rootToken ...

Maybe Google could propose a pull to this or a plugin... @emilymye ? @mayakacz ?

[jefferai]

Hi all,

In my previous responses I didn't disclose much information because the release of 0.8 is imminent and I was hoping to avoid detailed discussion based on documentation that isn't yet finalized, but I don't want to leave people hanging so I'll add more detail at this point.

I can confirm that the new MFA system is only coming to Vault Enterprise, at least for now. To understand the reason for this it helps to have an understanding of why the current MFA system exists, the challenges it presents, and why bringing it to the open source side is not something we can currently support.

First, though, I want to address the Enterprise vs. Open Source issue head-on. Every HashiCorp tool to this point has started out as open source and every such tool is actively developed by full-time HashiCorp engineers on both the open source and enterprise sides. However, meeting the needs of both our community and our enterprise customers requires a significant investment in our products and our development/engineering capabilities...the more users Vault has, the more disparate requests and requirements we see. To address this, the Vault team has ramped up from one full-time engineer to two, then three, then six. Keeping engineers employed is costly, so HashiCorp needs revenue, and out of the two options generally available for companies built around open source (charge for support, or move to open core), HashiCorp chose to move to an open core model. This is where Vault Enterprise comes in.

Figuring out the right OSS/Enterprise feature set for not just Vault but the other enterprise HashiCorp products is complicated (and as you can imagine, an easy way to have emotions run high internally, since in an ideal world all features would be open source). Without going into the meat of a lot of internal discussions, I'll simply say that we generally try to organize our thinking around who has need for a particular feature (the persona) and which edition of Vault is targeted for that persona, and we put together some overall guidelines. Two of the guidelines relevant for Vault Enterprise are Governance and Organization. You can see this in the description of Vault Enterprise Premier from the Vault product page:

For teams to organize and collaborate on secrets, identity, and policies across many Vault clusters to comply with business policy.

"Business policy" includes both a business' own internal security policy as well any industry or governmental compliance requirements they must meet; and this is the key here with respect to this particular feature, and where a bit of history comes into play.

When MFA was first added, it was proposed by and implemented by someone at a specific company (I won't name it, but it's easy to find in the Git history) because they had specific industry regulatory requirements they needed to meet; in other words, governance needs. We accepted the implementation in order to help them meet these needs. However, Vault was just past 0.2 and we didn't have a good feel for whether features being added outside of Vault's original scope were necessarily good ideas (in terms of scope or implementation), and problems with the design immediately became apparent; some of the more major issues are:

• It requires specific integration (at a code level) into every auth backend
• It only works for auth backends; moreover, it only works for login paths for auth backends
• It cannot be configured centrally or via ACLs so it must be configured for each mount of each supporting auth backend

Additionally, relevant to this specific issue, even if we could centrally configure MFA to use it outside of login points, there was no good way to tie a TOTP key to a specific person, making the workflow extremely onerous. I actually went down the path of an "MFA everywhere" capability a while back and realized that it was really not feasible at any scale with TOTP beyond a set of hard-coded keys.

The answer to this ends up being something coming in 0.8 we refer to as Identity, and it's no coincidence they're coming at the same time. Identity allows us to correlate information about a person across their tokens, which means something like storing a TOTP key to be used for an MFA factor can work across logins. It's the only reasonable way we have found to approach this problem (within the context of Vault, that is, but which is actually a very complicated context when the various storage possibilities and their various limitations are taken into account).

The new MFA system built around Identity can do the things that are completely impossible with the old system: it doesn't require integration specific to each backend, it's not limited to auth backends, it's able to be configured centrally and activated via ACLs.

However, the motivation behind this feature is still exactly the same as before: it's for governance and compliance requirements for our enterprise customers; the underlying technology that finally made this happen (Identity) is there to enable features around organizational requirements; and the significant amount of engineering effort that went into planning and executing on these features were directed specifically towards these organizational and governance needs. In a world in which Vault Enterprise exists, the focus of these features on organization and governance place Identity and the new MFA system squarely on the Enterprise side. I am sure that it's not a satisfying answer viewed through an individual's lens...you see it as a fundamental need, and that's understandable. In your shoes I'd probably feel the same way, and I really am sorry that you're disappointed, but it's not something that is going to change at this point in time.

I do want to point out that this doesn't mean it will never come to the open source side; Terraform state locking used to be an enterprise-only feature, but we eventually were able to make it available to everyone. As I said before, in an ideal world we'd love for all features would be open source; open source is in our DNA and most of the engineers and management team at HashiCorp have strong histories with open source software and communities. But a revenue stream allows us to keep engineers employed to keep making Vault better, faster, including the open source side; and speaking personally as the manager of the Vault team, I want to keep my team employed, as they're all fantastic engineers and good people.

A couple of other points to the previous comment:

• Root tokens should generally not be used; they can only be generated in specific ways and we recommend revoking them as soon as possible and multiple eyes on them at any given time
• Google is working on some integration between GCP/Vault but that is the extent of their involvement

I'm going to close this issue at this point as we have no current plans to work on the legacy MFA system.

[tilgovi]

Beautiful response. Thank you, @jefferai.

[evq]

@jefferai First off, thanks for the in-depth response. Having more background on the existence of the existing implementation is always helpful. The fact that Duo is the only existing option did seem like a business decision. Also, thank you to the amazing work that you and the team have done so far!

With that said, I disagree very strongly with the idea that MFA is a "compliance feature" and frankly even hinting at such is embarrassing. With the attention to detail paid everywhere else in vault it is totally bizarre that the weakest link in the chain seems to be login. Options for end user authentication boil down to a single static secret. Sure, the open source version of vault is being targeted as "Centralized secrets management for individuals"... but I wouldn't trust a system that relies on my ability to keep a single static secret safe. Neither should any anyone else! MFA or hardware token support is a must in this day and age.

I may be an individual using vault but I've also been a part of two companies where vault has come up and I've been in a position to make the call on it's use internally. I can't claim to know how vault enterprise sales are made, so maybe upselling from the open source version just isn't a thing you care about. As you pointed out there are a number of issues with the current MFA system, especially once one has a number of different users. To me this is a perfect driver for existing users of the open source version to upgrade to enterprise. But without some form of basic, limited MFA support in the open source version you won't be able to get a number of people, myself included, in the door.

By all means focus internal effort on the new MFA system, but to say that the community isn't allowed to solve the basic problem on their own just seems wrong. The core isn't really that open then, is it?

[mitchellh]

Hey @evq. I'm just hopping in to give my own response. @jefferai may have other things to add and I'm not trying to speak for him. I will repeat him in some cases because of course our experiences overlap. However, I will try to answer the question in your final paragraph, so bear with me.

First, I don't disagree with what you're saying. I can't prove it but I ask you trust me that this issue in particular was the subject of dozens of hours of discussion and the sum total of participant hours that went into those discussions is probably in the high hundreds. It wasn't an easy decision. In fact, our initial decision was to build and place an MFA system into OSS for many reasons you stated. I still believe we'll get there.

However, as the design process for that system began to implement this generically in Vault, we realized that there were some serious shortcomings, and that a better concept of identity was required to make this possible in the ways that many of our users and customers wanted. Through that discussion, we concluded that the use cases we're hearing from folks fit well into our enterprise feature guidelines.

As a conclusion to this, we were able to implement an extremely powerful MFA system. As @jefferai stated, you can add MFA to any endpoint via ACLs, rather than restrict it purely by auth and require per-backend integration.

So you asked: can the community solve the basic problem on their own? Yes, of course. Our software is liberally licensed with an OSS license for that reason. Its a protection for our users and its philosophically important to me.

Unfortunately, its not that simple for us to merge a PR because a merge is a big commitment. In many cases, the initial author of that PR can walk away after they contribute the work (but thank you thank you, we are grateful for the contributions). But as maintainers of the project, we're on the line for as long as that feature exists (which usually ends up being forever for all intents and purposes). For all our paying customers, that includes support for our OSS. That means we have to train engineering staff to test it and keep it stable, we have to train support staff to answer user issues about it, and we have to train sales and sales engineers to know how to sell that feature (the idea of it, not the feature itself which would itself be in OSS in this example).

And in 99% of cases, we're willing and happy to take that burden on. And honestly, we feel obligated as well! It is open source software! We're thankful for our community and contributors and they're all so awesome for helping make the projects we started something great. We of course have room for lots of improvement, but we believe we do our best to accept as much as possible.

The difficulty with merging work that overlaps with an enterprise feature is that it challenges our ability to build a sustainable environment for us to maintain the software we're trying really hard to give away. Its frustrating for you I'm sure, but its also frustrating for us for the same reasons because we would really rather just make it all OSS.

I'm not trying to be greedy, I'm not trying to be evil, I'm just trying to make sure our company will be around for years to come and that we can ship more open source. And @jefferai is right: we try to open source everything. Every new feature we plan is by default open source. But every so often one does cause a flurry of internal discussion for why it should be enterprise.

MFA fell into that infrequent bucket.

@jefferai said that MFA could eventually become OSS. That is true. That is something we're holding onto. I'll add that we are still discussing ways to make basic MFA that perhaps is less flexible (restricted by architecture, not by paywall reasons) available in OSS. I do agree with many of your points @evq and we are motivated to find more solutions to this problem.

[CreatixEA]

Thanks @jefferai and @mitchellh for your complete answers ! That's important to know and understand how you make things and how decisions are taken.

I understand you got a lot of work to do to implement the MFA in the whole application. I did not understood it was so complicated : here is how I saw the problem.

I've a login/password. I enterd it to get a temporary token which I use to get other secret backends
The only thing I hopped was to enter a login/password/TOTP to get the same temporary token.

It's less secured than a TOTP for each backend request, but enough to my view : the weakness of a static login/password couple is reinforced with a TOTP. It's the responsability to each organisation to define the TTL of the temporary token.

Anyway, thank you for the work you're doing. But I stay on my position : I think it's a necessary feature to any product of security.

You can crypt anything you want, if you only need a login/password to get it, that's easy to crack a password... The whole chain must be secured, in my opinion.

Looking forward to evolutions.

Thank's again !

[rfc1036]

Maybe a reasonable middle ground could be to implement an authentication backend which just calls an external program (providing to it username, password and MFA token): this way interested users could implement locally whatever they like and large customers would keep looking for an integrated solution.

[valorl]

So, it's 2021, how's everyone doing?
With MFA an even more of a ubiquitous and standard feature than back in 2017, are there any plans to revisit this decision 🤞 ?

I think there is a considerable user segment that needs Vault with MFA but doesn't need anything else from Enterprise.
We're talking about centralized management of every single secret, you don't need to be a Fortune 500 company to have the need for MFA. It should probably be discouraged to not have MFA.

Without MFA in OSS, and without a more affordable license (e.g. OSS + MFA), this user segment is basically cut off from ever being able to enjoy Vault.
I personally find this pretty sad and disappointing, given how likely you are to run into Vault in connection with cloud native, open source software only to figure out you can't actually use it.

[keymon]

Agree with @valorl .

one solution can use some auth provider with MFA support, and integrate it via OIDC.

[tristanmorgan]

could it really be?... #14025