Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Null op_object #70

Open
bitphage opened this issue Nov 1, 2019 · 14 comments
Open

Null op_object #70

bitphage opened this issue Nov 1, 2019 · 14 comments

Comments

@bitphage
Copy link

bitphage commented Nov 1, 2019

When using /get_account_history sometimes entry['operation_history']['op_object'] is null (None), I got this only on ES wrapper v2. Example:

curl -X GET --header 'Accept: application/json' 'https://wrapper.elasticsearch.bitshares.ws/get_single_operation?operation_id=1.11.1000908945'
@oxarbitrage
Copy link
Member

It depends on the configuration of the elasticsearch plugins in the back end.

For example, in this one: https://explorer.bitshares-kibana.info/es/single_operation?operation_id=1.11.1000908945 both things are provided; the string op and the object op_object.

By the way, the single_operation needs cache. Added at 7146310

@bitphage
Copy link
Author

bitphage commented Nov 4, 2019

It looks like a bug because op_object is available for some operations. In my view, it should be available for all ops or none at all. Looks like I need to switch from op_object to op completely.

Here is a log from my app to demonstrate inconsistency:

2019-11-04 14:13:17,081 INFO: Sold 620.29219 RUBLE for 0.00122387 RUDEX.BTC @ 0.0000019730540 RUDEX.BTC/RUBLE (506828.49485648 RUBLE/RUDEX.BTC)
2019-11-04 14:13:17,082 INFO: Sold 642.30435 RUBLE for 0.00127997 RUDEX.BTC @ 0.0000019927780 RUDEX.BTC/RUBLE (501812.03465706 RUBLE/RUDEX.BTC)
2019-11-04 14:13:17,083 INFO: Sold 526.94717 RUBLE for 0.00106059 RUDEX.BTC @ 0.0000020127065 RUDEX.BTC/RUBLE (496843.42677189 RUBLE/RUDEX.BTC)
2019-11-04 14:13:17,083 ERROR: None op_object in 1.11.1000908383
2019-11-04 14:13:17,084 INFO: Sold 0.00072506 RUDEX.BTC for 420.33415 RUBLE @ 579723.26 RUBLE/RUDEX.BTC (0.00000172 RUDEX.BTC/RUBLE)
2019-11-04 14:13:17,085 INFO: Sold 0.00039141 RUDEX.BTC for 229.17859 RUBLE @ 585520.53 RUBLE/RUDEX.BTC (0.00000171 RUDEX.BTC/RUBLE)
2019-11-04 14:13:17,085 ERROR: None op_object in 1.11.1000908945
2019-11-04 14:13:17,085 ERROR: None op_object in 1.11.1000909172
2019-11-04 14:13:17,085 ERROR: None op_object in 1.11.1000909214
2019-11-04 14:13:17,086 ERROR: None op_object in 1.11.1000909742
2019-11-04 14:13:17,086 ERROR: None op_object in 1.11.1000909745
2019-11-04 14:13:17,086 ERROR: None op_object in 1.11.1000909766
2019-11-04 14:13:17,086 ERROR: None op_object in 1.11.1000910175
2019-11-04 14:13:17,087 INFO: Sold 0.00001025 RUDEX.BTC for 6.06159 RUBLE @ 591374.63 RUBLE/RUDEX.BTC (0.00000169 RUDEX.BTC/RUBLE)
2019-11-04 14:13:17,090 INFO: Sold 0.00056306 RUDEX.BTC for 336.30920 RUBLE @ 597288.39 RUBLE/RUDEX.BTC (0.00000167 RUDEX.BTC/RUBLE)
2019-11-04 14:13:17,090 ERROR: None op_object in 1.11.1000922799
2019-11-04 14:13:17,091 ERROR: None op_object in 1.11.1001036046
2019-11-04 14:13:17,091 ERROR: None op_object in 1.11.1001036048
2019-11-04 14:13:17,091 ERROR: None op_object in 1.11.1001036050
2019-11-04 14:13:17,091 ERROR: None op_object in 1.11.1001036052
2019-11-04 14:13:17,092 INFO: Sold 0.00076436 RUDEX.BTC for 452.02023 RUBLE @ 591370.86 RUBLE/RUDEX.BTC (0.00000169 RUDEX.BTC/RUBLE)
2019-11-04 14:13:17,092 ERROR: None op_object in 1.11.1001465303
2019-11-04 14:13:17,092 ERROR: None op_object in 1.11.1001465313
2019-11-04 14:13:17,093 INFO: Sold 0.00005969 RUDEX.BTC for 35.29893 RUBLE @ 591370.92 RUBLE/RUDEX.BTC (0.00000169 RUDEX.BTC/RUBLE)
2019-11-04 14:13:17,097 ERROR: None op_object in 1.11.1001492004
2019-11-04 14:13:17,097 ERROR: None op_object in 1.11.1001492063
2019-11-04 14:13:17,100 INFO: Sold 0.00000582 RUDEX.BTC for 3.47618 RUBLE @ 597281.79 RUBLE/RUDEX.BTC (0.00000167 RUDEX.BTC/RUBLE)
2019-11-04 14:13:17,101 INFO: Sold 0.00025834 RUDEX.BTC for 155.84524 RUBLE @ 603256.33 RUBLE/RUDEX.BTC (0.00000166 RUDEX.BTC/RUBLE)
2019-11-04 14:13:17,102 ERROR: None op_object in 1.11.1001538512
2019-11-04 14:13:17,103 ERROR: None op_object in 1.11.1001800020
2019-11-04 14:13:17,107 ERROR: None op_object in 1.11.1001882410
2019-11-04 14:13:17,107 ERROR: None op_object in 1.11.1001882413
2019-11-04 14:13:17,108 INFO: Sold 0.00050508 RUDEX.BTC for 310.81803 RUBLE @ 615383.76 RUBLE/RUDEX.BTC (0.00000163 RUDEX.BTC/RUBLE)
2019-11-04 14:13:17,109 INFO: Sold 0.00103936 RUDEX.BTC for 639.60527 RUBLE @ 615383.77 RUBLE/RUDEX.BTC (0.00000163 RUDEX.BTC/RUBLE)

@oxarbitrage
Copy link
Member

I see what you mean, but it is something happening specifically in https://wrapper.elasticsearch.bitshares.ws where i don't have access.

In current versions of bitshares-explorer-api and bitshares-core elasticsearch plugin this is not happening: https://explorer.bitshares-kibana.info/es/single_operation?operation_id=1.11.1000908383

I dont know what version or how the backend running the elasticsearch is configured there.

@bitphage
Copy link
Author

bitphage commented Nov 4, 2019

Ok, so it's a instance-specific issue, need to contanct it's admin.

@oxarbitrage
Copy link
Member

@sschiessl-bcp might be able to help.

I agree you can change your program to use op as a string if that is always there. Will make your program a bit slower but should work.

@oxarbitrage
Copy link
Member

By the way, i am trying to add https://flask-limiter.readthedocs.io/en/stable/ into explorer.bitshares-kibana.info to fix server performance issues. I am having too much queries from bots. Should make the service more reliable.

@sschiessl-bcp
Copy link

That might be an issue due to our ES cluster ... it looks like there is one instance that is not feeding the es_object ..

The UI integration always uses fallback: if object not given, parse the op string. I guess that why I haven't noticed that yet ...

@sschiessl-bcp
Copy link

There was indeed one feeding without objects ...

Since 2 minutes ago you should always see the object, but not the string. I will resync the database from scratch again to have that setting consistent everywhere

@sschiessl-bcp
Copy link

Resyncing is running, when it has caught up the state should be consistent in having the object.

@bitphage
Copy link
Author

bitphage commented Dec 3, 2019

I've got the same issue with explorer.bitshares-kibana.info:
https://explorer.bitshares-kibana.info/es/single_operation?operation_id=1.11.1005897922

@sschiessl-bcp
Copy link

Your expectancy is to have op_object field, yes?

@bitphage
Copy link
Author

bitphage commented Dec 3, 2019

@sschiessl-bcp right.

@bitphage
Copy link
Author

I tried to switch to op field but found an opposite situation may exists as well: empty op with non-empty op_object 🤦‍♂️

https://explorer.bitshares-kibana.info/es/single_operation?operation_id=1.11.964985433

bitphage added a commit to bitphage/bitshares-tradehistory-analyzer that referenced this issue Dec 11, 2019
@bitphage
Use 'op' field when 'op_object' is null
96374e2 
See bitshares/bitshares-explorer-api#70 for
details
@sschiessl-bcp
Copy link

I triggered resync again, now all ES feed nodes should provide op_object. It should replace all in the past as well because I started syncing from 0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@bitphage @oxarbitrage @sschiessl-bcp