add redshift backend

[eladapps]

No description provided.

[jefferai]

Hi there,

Thanks for providing this. A few comments up-front:

1. This is adding a JWT backend, not just Redshift.

2. What exactly is different between Redshift and Postgres in terms of what Vault needs? I wonder if rather than a separate backend it might be easy to simply enhance the Postgres backend to work for both Postgres and Redshift.

[eladapps]

JWT removed
Redshift does not include all of postgresql capabilites. adding and removing users (creating new credentials and revoking them in vault) requires different commands. additionally, the uuid created in vault for the username includes dashes which are not allowed in redshift usernames.
Actually at first I did alter the postgres code to work with postgres, but I did not feel comfortable enough to change the original code and decided to compile it as a different backend.
I would be happy to add a tutorial to help people get started with this backend

[jefferai]

I think I still have the same question -- what exactly is the scope of differences? Is it just those two items you mentioned?

In terms of maintainability it would be far, far easier to simply have a boolean configuration option for "redshift mode" -- or perhaps a string option with "postgres" as a default and "redshift" allowed, in order to allow other variants in the future. Then it's easy to key off of those to adjust which commands need to be run.

Don't be too worried about changing the original backend -- that's what code review is for :-) You'll do fine.

[brockoffdev]

@eladapps Jeff and I have been discussing this.

First of all, I actually didn't have any issue utilizing dashes in my usernames or passwords on Redshift...so not sure if you are using an older version? But my first issue actually was that Redshift requires an Uppercase letter in your password, which Vault does not provide (my workaround was simply to put two uppercase letters into the SQL command's password creation (i.e. CREATE USER "{{name}}" WITH PASSWORD "HI-{{password}}")...and just dealing with having to remember to enter those letters when you receive a Password. It's not perfect, but it works.)

So the actual creation of the user and password always worked and got me in!

The real issue is revocation. See my response to Jeff in #760, but I think the simple fix may be to have an additional value for Postgres mount point "roles", that would allow you to write a revocation SQL query.

Thoughts?

[jefferai]

But my first issue actually was that Redshift requires an Uppercase letter in your password

As far as I have seen from docs, Redshift uses IAM accounts. If there is a requirement for an uppercase letter in your password, this is likely coming from the complexity requirements set in IAM for your account, and can be changed.

[eladapps]

Hi Bryant.

Its been a while since I wrote that code.
Let me dig in and see what my issues were and get back to you

On Mon, Nov 9, 2015 at 1:48 AM, Bryant Rockoff [email protected]
wrote:

@eladapps https://github.com/eladapps Jeff and I have been discussing
this.

First of all, I actually didn't have any issue utilizing dashes in my
usernames or passwords on Redshift...so not sure if you are using an older
version? But my issue actually was that Redshift requires an Uppercase
letter in your password, which Vault does not provide (my workaround was
simply to put two uppercase letters into the SQL command's password
creation (i.e. CREATE USER "{{name}}" WITH PASSWORD
"HI-{{password}}")...and just dealing with having to remember to enter
those letters when you receive a Password. It's not perfect, but it works.

So the actual creation of the user and password always worked and got me
in!

The real issue is revocation. See my response to Jeff in #760
#760, but I think the simple
fix may be to have an additional value for Postgres mount point "roles",
that would allow you to write a revocation SQL query.

Thoughts?

—
Reply to this email directly or view it on GitHub
#690 (comment).

[jimjh]

@jefferai Redshift doesn't use IAM accounts for SQL access. IAM is used to control access for creating/modifying/deleting clusters (which are AWS resources), but not for SQL login. Users generally login and run SQL queries against Redshift using the regular PostgreSQL client.

I cannot find a way to change the password policy (for SQL login) in Redshift. It not configurable through IAM.

However, as @brockoffdev mentioned, it's possible to workaround the password requirements. The real issue is revocation and renewal.

$ vault renew myLeaseID 600
Renew error: Error making API request.

URL: PUT https://vault/v1/sys/renew/db/creds/readonly/xxx-xxx-xxx-xxx-xxx
Code: 400. Errors:

* failed to renew entry: pq: syntax error at or near "ROLE"
``

[jefferai]

@jimjh Thanks, good to know. I'm still inclined to think that this is easily implemented as a config option on top of the existing postgresql backend.

Over time (next few Vault releases) I think the plan will try to be to combine the various SQLish backends into one, where each role specifies the type of DB it's connecting to along with the other config params and all of the shared logic can be reused.

[jimjh]

@jefferai thanks for the update

[crazed]

Definitely interested in a Redshift backend. Was thinking about copying the postgres one for my own needs, but not sure if you are still planning to consolidate the SQL pieces into one rather than separate code bases.

[jefferai]

We are planning that!

[cheesefactory]

@jefferai Has there been any more discussion/progress on getting something going for a Redshift backend?

[jefferai]

All that I can say right now is that it's planned. I don't have any hard dates to give.

[endzyme]

ping

[jefferai]

pong

[endzyme]

Just checking in and seeing if there was any forward motion on this. :)

[jefferai]

If the current postgres capabilities in the combined backend do not support redshift (I think they do) adding a plugin variant is the right way to do this at this point, so I'll close this.

[briankassouf]

@endzyme see #2788 for more info about redsift support

[jefferai]

Ah thanks @briankassouf , I knew support existed but forgot just how recent :-)