Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix issue with obtaining and releasing locks #16

Closed
wants to merge 3 commits into from

Conversation

cmawhorter
Copy link

@cmawhorter cmawhorter commented Nov 19, 2020

fixes #15

first... this is the first time i've ever touched go. some more experienced eyes would be much appreciated.

summary of changes:

  • added dynamo for interacting with ddb, which is a PITA in the best of times
  • removed separate LOCK-* ddb records. Lock is now a property of the main Item record. this means there is no longer a need to document TTL because there is no longer a need for it at all
  • switched to the same "active locking" functionality that file storage supports. added idempotent ddb updates to support it
  • decreased timeouts to better service on demand generation. (is LockTimeout too low now though? 1m)
  • added documentation about single region support

there is one breaking change: i renamed "Table" -> "TableName" without good reason. the rename doesn't impact use. the name in caddy config remains "table".

i did have to make some small modifications to tests because some stuff was renamed or removed. otherwise though tests are unchanged and pass. some additional tests would be great but i don't really know how to do that.

@fillup
Copy link
Contributor

fillup commented Nov 29, 2020

Hi @cmawhorter, thanks for the contribution. For your foray into Go you've done a good job. At a broader level though I'm still not clear how there is a problem with locks today. I have tests to confirm that existing but expired locks are removed and new locks are created. Are you able to duplicate or provide steps to prove there is a problem with the existing code? The 3rd party dynamo library you brought in provides some nice syntactic sugar but I'm not sure it's worth bringing a 3rd party library in just for that unless it's fixing something wrong with the core AWS SDK, especially for very simple queries/updates.

I'm not saying no to merging this, I just want to understand the problem better and how this solves it.

Thanks!

@cmawhorter
Copy link
Author

this is the issue:
image

a 48h guarantee on ttl deletion makes it a great option for record lifecycle but notsomuch for locking. additionally that link says the problems will happen based on usage/capacity which means this will start failing at the worst possible time (when you're busy).

why dynamo? using ddb directly is a horrible and error-prone experience and that is doubly so when writing conditional updates, etc. ddb just launched partiql which may fix this, but for now some kind of non-aws lib is pretty much a requirement of working with ddb in any way.

i could've kept the lock a separate record but i merged the two together 1) because i didn't realize how many other non-cert records were also being written and 2) because it'd help people in advanced use cases *

  • consider a ddb stream. if the lock is in the same record it means the lambda processing the stream could do something with that information. also global tables.

i've been using my changes while i'm testing and they seem to work ok so far. i've done minimal on-demand testing yet. the one issue i've come across is there is an ocsp record for a wildcard that -- when it exists -- prevent usage of the wildcard. i'm not sure if this is an issue with my changes or caddy itself. the issue only happens when i'm using multiple caddy servers simultaneously.

@briskt
Copy link
Contributor

briskt commented Oct 12, 2023

@cmawhorter unfortunately, we have left this stale for a very long time. Can you give me an update on this? Are you still using this patch in your fork? I would be happy to merge it if I knew it was still helpful.

@cmawhorter
Copy link
Author

i've been using it for a while now, but i think the two have diverged pretty significantly. feel free to grab whatever is useful and close this

@briskt briskt changed the base branch from master to develop October 18, 2023 13:37
@briskt
Copy link
Contributor

briskt commented Oct 18, 2023

We aren't using this library presently, but I am not sure whether other people are using it. Would you mind if I add a note to the README suggesting your fork? I might just archive this repo.

@fillup
Copy link
Contributor

fillup commented Oct 18, 2023

@briskt just FYI this is listed as a plugin on Caddy's site, so if you archive it or transfer ownership be sure to let the Caddy team know to update their references as well. https://caddyserver.com/docs/modules/

@briskt
Copy link
Contributor

briskt commented Oct 18, 2023

@fillup good to know. Thanks for bringing that to my attention.

@briskt briskt deleted the branch silinternational:develop December 19, 2023 17:21
@briskt briskt closed this Dec 19, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

Possible bug in locking mechanism
3 participants