Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Kraken ws orders #349

Merged
merged 50 commits into from
Aug 6, 2022
Merged

Kraken ws orders #349

merged 50 commits into from
Aug 6, 2022

Conversation

goodboy
Copy link
Contributor

@goodboy goodboy commented Jul 5, 2022

WS based order management with fixes for #310, #296 and hopefully most of #290.

Leverage the new 3.10 match: syntax like crazy which turned out very nicely for making the msg loops way more grokable

Includes safeguard code to cancel all persistent orders found on startup until we get existing order loading working the EMS as per #293.


TODOs:


Follow up (mostly pertaining to the wishlist in #345):

@goodboy goodboy added perf efficiency and latency optimization clearing auction and mm tech: EMS, OMS, algo-trading broker-backend `brokerd`/`datad` related backend tech labels Jul 5, 2022
@goodboy goodboy requested review from iamzoltan and guilledk July 5, 2022 02:39
@goodboy goodboy changed the base branch from kraken_ledger_pps to notokeninwswrapper July 5, 2022 02:39
@goodboy
Copy link
Contributor Author

goodboy commented Jul 5, 2022

Order "edits" now fully working 🏄🏼

@goodboy goodboy marked this pull request as ready for review July 5, 2022 15:50
@@ -20,19 +20,19 @@
'''
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't forget this file! 😂

@goodboy goodboy mentioned this pull request Jul 5, 2022
@goodboy goodboy force-pushed the kraken_ws_orders branch 2 times, most recently from 0459c8e to ff38e72 Compare July 5, 2022 21:50
Base automatically changed from notokeninwswrapper to master July 6, 2022 00:57
@goodboy goodboy force-pushed the kraken_ws_orders branch 3 times, most recently from 5100036 to f649869 Compare July 9, 2022 17:14
@goodboy goodboy changed the base branch from master to drop_pydantic July 9, 2022 17:15
@goodboy
Copy link
Contributor Author

goodboy commented Jul 9, 2022

Pretty sure everything is mostly working.

Just need to test some live clears and darks.

Base automatically changed from drop_pydantic to master July 9, 2022 18:15
@goodboy goodboy force-pushed the kraken_ws_orders branch 2 times, most recently from db9b3c8 to 7fe5e4c Compare July 14, 2022 13:28
@goodboy goodboy force-pushed the kraken_ws_orders branch 3 times, most recently from aa9ed79 to 9729066 Compare July 21, 2022 16:11
@goodboy goodboy changed the base branch from master to pptables July 21, 2022 16:11
@goodboy goodboy force-pushed the kraken_ws_orders branch from 9729066 to 2d0f926 Compare July 21, 2022 19:32
Base automatically changed from pptables to master July 21, 2022 21:54
goodboy added 11 commits July 30, 2022 17:33
This ended up driving the rework of the `piker.pp` apis to use context
manager + table style which resulted in a much easier to follow
state/update system B). Also added is a flag to do a manual simulation
of a "fill triggered rt pp msg" which requires the user to delete the
last ledgered trade entry from config files and then allowing that trade
to emit through the `openOrders` sub and update client shortly after
order mode boot; this is how the rt updates were verified to work
without doing even more live orders 😂.

Patch details:
- open both `open_trade_ledger()` and `open_pps()` inside the trade
  dialog startup and conduct a "pp state sync" logic phase where we now
  pull the account balances and incrementally load pp data (in order,
  from `pps.toml`, ledger, api) until we can generate the asset balance
  by reverse incrementing through trade history eventually erroring out
  if we can't reproduce the balance value.
- rework the `trade2pps()` to take in the `PpTable` and generate new
  ems msgs from table updates.
- return the new `dict[str, Transaction]` expected from
  `norm_trade_records()`
- only update pp config and ledger on dialog exit.
If we don't have a pos table built out already (in mem) we can't figure
out the likely dst asset (since there's no pair entry to guide us) that
we should use to search for withdrawal transactions; so move it later.

Further this ports to the new api changes in `piker.pp`` that will land
with #365.
@goodboy goodboy force-pushed the kraken_ws_orders branch from c0617a5 to e6a3e8b Compare July 30, 2022 21:34
goodboy and others added 12 commits July 30, 2022 17:34
Turns out you can pass both thus making mapping an ems `oid` to
a brokerd-side `reqid` much more simple. This allows us to avoid keeping
as much local dialog state but with still the following caveats:

- ok `editOrder` msgs must update the reqid<->txid map
- only pop `reqids2txids` entries inside the `cancelOrderStatus` handler
Why we need so many fields to accomplish passing through a dialog key to
orders is beyond me but this is how they do it with edits..

Allows not having to handle `editOrderStatus` msgs to update the dialog
key table and instead just do it in the `openOrders` sub by checking the
canceled msg for a 'cancel_reason' of 'Order replaced', in which case we
just pop the txid and wait for the new order the kraken backend engine
will submit automatically, which will now have the correct 'userref'
value we passed in via the `newuserref`, and then we add that new `txid`
to our table.
Since we figured out how to pass through ems dialog ids to the
`openOrders` sub we don't really need to do much with status updates
other then error handling. This drops `process_status()` and moves the
error handling logic into a status handler sub-block; we now just
info-log status updates for troubleshooting purposes.
The (partial) fills from this sub are most indicative of clears (also
says support) whereas the msgs in the `ownTrades` sub are only emitted
after the entire order request has completed - there is no size-vlm
remaining.

Further enhancements:
- this also includes proper subscription-syncing inside `subscribe()` with
  a small pre-msg-loop which waits on ack-msgs for each sub and raises any
  errors. This approach should probably be implemented for the data feed
  streams as well.
- configure the `ownTrades` sub to not bother sending historical data on
  startup.
- make the `openOrders` sub include rate limit counters.
- handle the rare case where the ems is trying to cancel an order which
  was just edited and hasn't yet had it's new `txid` registered.
`kraken`: use `userref` field AND `reqid`, utilize `openOrders` sub for most msging
@goodboy goodboy merged commit a9185e7 into master Aug 6, 2022
@goodboy goodboy deleted the kraken_ws_orders branch August 6, 2022 01:01
@goodboy goodboy mentioned this pull request Mar 2, 2023
6 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
broker-backend `brokerd`/`datad` related backend tech clearing auction and mm tech: EMS, OMS, algo-trading perf efficiency and latency optimization
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant