Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Projections #9

Merged
merged 37 commits into from
Nov 18, 2016
Merged

Projections #9

merged 37 commits into from
Nov 18, 2016

Conversation

prolic
Copy link
Member

@prolic prolic commented Nov 16, 2016

No description provided.

@prolic
Copy link
Member Author

prolic commented Nov 16, 2016

Please carefully check the locking mechanisms!

$statement->execute();

$row = $statement->fetch(PDO::FETCH_OBJ);
//die;

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

// todo : remove

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

*/
protected function acquireLock(): void
{
$statement = $this->connection->prepare("SELECT GET_LOCK('$this->name', 1) as 'lock';");

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no need to surround aliases with quotes

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's required!

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why? can't you do simply as lock?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No.

mysql> SELECT get_lock('ff', 3) as lock;
ERROR 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'lock' at line 1

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh ok, fair enough, the 'lock' keyword is reserved :)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

btw, I see you pass 1 as a second parameter of GET_LOCK, which means your lock will be released after 1 second. This is not what we want, right?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

damn, that's a bug. or maybe we do the same implementation as in psql?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think having the same implementation in place might be the best scenario. Will update the code tomorrow.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok but don't forget to expose the timeout value in the config of the package so the user can configure it. I would actually be in favor of letting the user chose how he really wants to manage his locks by exposing a locking interface. Another solution would be to make your locked column a datetime meaning until when the projection is locked (instead of storing a random int).

//die;
if (false === $row) {
$sql = <<<EOT
INSERT INTO $this->projectionsTable (name, position, state, locked)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so you use GET_LOCK() for MySQL but a locked column for Postgres?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. PostgreSQL has not the same feature.

@prolic
Copy link
Member Author

prolic commented Nov 16, 2016

There is also a pthreads implementation planned for multi stream processing. Problem is different threads cannot share db connection. So how to construct this object? Inject a container and service names? Any ideas?

Copy link
Member

@codeliner codeliner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was not able to review the PR in detail due to limited time today. Implementation looks good. Some issues are already discussed so I only add a micro issue to the list.
Can't say it often enough, really great work.

@@ -151,6 +151,10 @@ public function __construct(
$sql = 'INSERT INTO ' . $tableName . ' (' . implode(', ', $columnNames) . ') VALUES ' . $allPlaces;

$statement = $this->connection->prepare($sql);
/*
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rm

@codeliner
Copy link
Member

There is also a pthreads implementation planned for multi stream processing. Problem is different threads cannot share db connection.

I followed @bendavies suggestion in the chat and took a look at icicle. I played a bit with reactPHP in the past and also read a few things about icicle.
Most of the examples are dealing with non-blocking streams but a google search took me there: https://packagist.org/packages/asyncphp/icicle-database
Except my experiments at home, I did not use non-blocking techniques in php so I cannot say much about it. Maybe we can ask the async php community for help or ideas ... non-blocking I/O works great for node.js and Go so why not also for php, but database connections would need to support it.

@prolic
Copy link
Member Author

prolic commented Nov 17, 2016

  • removed deadlocks
  • added lock until timeout in milliseconds
  • added projection factories
  • added query factories
  • added read model projection factories

$lockUntilString = $now->modify('+' . (string) $this->lockTimeoutMs . ' ms')->format('Y-m-d\TH:i:s.u');

$sql = <<<EOT
UPDATE $this->projectionsTable SET locked_until = ? WHERE name = ? AND (locked_until IS NuLL OR locked_until < ?);

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NULL

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@prolic
Copy link
Member Author

prolic commented Nov 17, 2016

About multi-stream-processing:

This will require pthreads and will probably be added with in a distinct implementation (much stuff differs then). This will be added later and is not required at the moment.

@prolic
Copy link
Member Author

prolic commented Nov 17, 2016

Note: This PR is now also cleaned up, so a DistributedLock interface can be added very easy (but is not yet done in this PR).

$this->connection->beginTransaction();

$listener = $this->eventStore->getActionEventEmitter()->attachListener(
ActionEventEmitterAware::EVENT_APPEND_TO,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wrong event?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

$this->connection->beginTransaction();

$listener = $this->eventStore->getActionEventEmitter()->attachListener(
ActionEventEmitterAware::EVENT_APPEND_TO,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wrong event?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

removed

@prolic prolic merged commit 07840ce into master Nov 18, 2016
@prolic prolic deleted the projections branch November 18, 2016 09:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants