Epoch time-align (indexing) #426
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
goodboy
force-pushed
the
multichartz
branch
3 times, most recently
from
670ba16 to
9589896
Compare
goodboy
force-pushed
the
epoch_index
branch
2 times, most recently
from
47b8ddd to
c61a46a
Compare
We have this method on our `ChartPlotWidget` but it makes more sense to directly associate axis-labels with, well, the label's parent axis XD. We add `._stickies: dict[str, YAxisLabel]` to replace `ChartPlotWidget._ysticks` and pass in the `pg.PlotItem` to each axis instance, stored as `Axis.pi` instead of handing around linked split references (which are way out of scope for a single axis). More work needs to be done to remove dependence on `.chart: ChartPlotWidget` references in the date axis type as per comments.
Main "public" API change is to make `GodWidget.get/set_chart_symbol()` accept and cache-on fqsn tuples to allow handling overlayed chart groups and adjust method names to be plural to match. Wrt `LinkedSplits`, - create all chart widget axes with a `None` plotitem argument and set the `.pi` field after axis creation (since apparently we have another object reference causality dilemma..) - set a monkeyed `PlotItem.chart_widget` for use in axes that still need the widget reference. - drop feed pause/resume for now since it's leaking feed tasks on the `brokerd` side and we probably don't really need it any more, and if we still do it should be done on the feed not the flume. Wrt `ChartPlotItem`, - drop `._add_sticky()` and use the `Axis` method instead and add some overlay + axis sanity checks. - refactor `.draw_ohlc()` to be a lighter wrapper around a call to `.add_plot()`.
Initial support for real-time multi-symbol overlay charts using an
aggregate feed delivered by `Feed.open_multi_stream()`.
The setup steps for constructing the overlayed plot items is still very
very rough and will likely provide incentive for better refactoring high
level "charting APIs". For each fqsn passed into `display_symbol_data()`
we now synchronously,
- create a single call to `LinkedSplits.plot_ohlc_main() -> `ChartPlotWidget`
where we cache the chart in scope and for all other "sibling" fqsns
we,
- make a call to `ChartPlotWidget.overlay_plotitem()` -> `PlotItem`, hide its axes,
make another call with this plotitem input to
`ChartPlotWidget.draw_curve()`, set a sym-specific view box auto-yrange maxmin callback,
register the plotitem in a global `pis: dict[str, list[pgo.PlotItem, pgo.PlotItem]] = {}`
Once all plots have been created we then asynchronously for each symbol,
- maybe create a volume chart and register it in a similar task-global
table: `vlms: dict[str, ChartPlotWidget] = {}`
- start fsp displays for each symbol
Then common entrypoints are entered once for all symbols:
- a single `graphics_update_loop()` loop-task is started wherein
real-time graphics update components for each symbol are created,
* `L1Labels`
* y-axis last clearing price stickies
* `maxmin()` auto-ranger
* `DisplayState` (stored in a table `dss: dict[str, DisplayState] = {}`)
* an `increment_history_view()` task
and a single call to `Feed.open_multi_stream()` is used to create
a symbol-multiplexed quote stream which drives a single loop over all
symbols wherein for each quote the appropriate components are looked
up and passed to `graphics_update_cycle()`.
- a single call to `open_order_mode()` is made with the first symbol
provided as input, though eventually we want to support passing in the
entire list.
Further internal implementation details:
- special tweaks to the `pg.LinearRegionItem` setup wherein the region
is added with a zero opacity and *after* all plotitem overlays to
avoid and issue where overlays weren't being shown within the region
area in the history chart.
- all symbol-specific graphics oriented update calls are adjusted to
pass in the fqsn:
* `update_fsp_chart()`
* `ChartView._set_yrange()`
* ChartPlotWidget.update_graphics_from_flow()`
- avoid a double increment on sample step updates by not calling the
increment on any vlm chart since it seems the vlm-ohlc chart linking
already takes care of this now?
- use global counters for the last epoch time step to avoid incrementing
all views more then once per new time step given underlying shm array
buffers may be on different array-index values from one another.
Before this axes were being stacked from the outside in (for `'right'` and 'bottom'` axes) which is somewhat non-intuitive for an `.append()` operation. As such this change makes a symbol list stack a set of `'right'` axes from left-to-right. Details: - rename `ComposeGridLayout.items` -> `.pitems` - return `(int, list[AxisItem])` pairs from `.insert/append_plotitem()` and the down stream `PlotItemOverlay.add_plotitem()`. - drop `PlotItemOverlay.overlays` and add it back as `@property` around the underlying `.layout.pitems`.
Facepalm, obviously absolute array indexes are not going to necessarily align vs. time over multiple feeds/history. Instead use `np.searchsorted()` on whatever curve has the smallest support and find the appropriate index of intersection in time so that alignment always starts at a sensible reference. Also adds a `debug_print: bool` input arg which can enable all the prints when working on this.
Curve-path colouring and cache mode settings are used (and can thus be factored out of) all child types; this moves them into the parent type's `.__init__()` and adjusts all sub-types match: - the bulk was moved out of the `Curve.__init__()` including all previous commentary around cache settings. - adjust `BarItems` to use a `NoCache` mode and instead use the `last_step_pen: pg.Pen` and `._pen` inside it's `.pain()` instead of defining functionally duplicate vars. - adjust all (transitive) calls to `BarItems` to use the new kwargs names.
use the new `do_overlay_scaling: bool` since we know each feed will have its own updates (cuz multiplexed by feed..) and we can avoid ranging/scaling overlays that will make their own calls. Also, pass in the last datum "brighter" color for ohlc curves as it was originally (and now that we can pass that styling bit through).
A super snappy `numpy.median()` calculator (per input range) which we slap an `lru_cache` on thanks to handy dunder method hacks for such things on mutable types XD
Massively speeds up scaling transform cycles (duh). Also includes a draft for an "overlay transform" type/api; obviously still a WIP 🏄..
In the (incrementally updated) display loop we have range logic that is incrementally updated in real-time by streams, as such we don't really need to update all linked chart's (for any given, currently updated chart) y-ranges on calls of each separate (sub-)chart's `ChartView.interact_graphics_cycle()`. In practise there are plenty of cases where resizing in one chart (say the vlm fsps sub-plot) requires a y-range re-calc but not in the OHLC price chart. Therefore we always avoid doing more resizing then necessary despite it resulting in potentially more method call overhead (which will later be justified by better leveraging incrementally updated `Viz.maxmin()` and `media_from_range()` calcs).
It's way faster since it uses a uniform time arithmetic to narrow the `numpy.searchsorted()` range before actually doing the index search B)
This was a subtle logic error when building the `plots: dict` we weren't adding the "main (ohlc or other source) chart" from the `LinkedSplits` set when interacting with some sub-chart from `.subplots`.. Further this tries out bypassing `numpy.median()` altogether by just using `median = (ymx - ymn) / 2` which should be nearly the same?
In situations where clients are (dynamically) subscribing *while* broadcasts are starting to taking place we need to handle the `set`-modified-during-iteration case. This scenario seems to be more common during races on concurrent startup of multiple symbols. The solution here is to use another set to take note of subscribers which are successfully sent-to and then skipping them on re-try. This also contains an attempt to exception-handle throttled stream overruns caused by higher frequency feeds (like binance) pushing more quotes then can be handled during (UI) client startup.
goodboy
force-pushed
the
multichartz
branch
3 times, most recently
from
320e4c2 to
f1f1515
Compare
|
Replaced with a slew of other sub-PRs namely #451 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
As I'm sure many have hear me complain, been working on epoch (time) aligning overlays to get #420 to where we'd likely want it.
Much more to come but putting this up for those who want to test.
Likely this will get merged with #420 eventually but breaking it out to have a testable and working version of multichartz so as to stay sane.. 🏄🏼