H-3947: Implement higher precision when convert data types in the graph

[TimDiekmann]

🌟 What is the purpose of this PR?

As a stop-gap solution to do math for any numbers, we're going to

• Use our own JSON type (instead of serde_json::Value)
• Use a decimal library to do conversions in the graph
• Store only f64

🔍 What does this change?

• Introduce a new Value type which replaces serde_json::Value to have more control over it
• Implement a Real type which represents a decimal number
• Use the Real type in the newly added Value

Real behavior:

• Creation from integers is lossless
• Conversion from f32 and f64 fails if it's NaN
• Converting to primitives may round and truncate
• Can be converted to/from FLOAT4 and FLOAT8 in Postgres
• It can be deserialized from number primitives and strings

Value behavior:

• Parses and outputs Number from/as f64
• This means, outside of the graph (API and Postgres) only f64 precision is retained, but internally, a higher precision is used

Pre-Merge Checklist 🚀

🚢 Has this modified a publishable library?

This PR:

• does not modify any publishable blocks or libraries, or modifications do not need publishing

📜 Does this require a change to the docs?

The changes in this PR:

• are internal and do not require a docs change

🕸️ Does this require a change to the Turbo Graph?

The changes in this PR:

• do not affect the execution graph

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 20.87%. Comparing base (9b2708f) to head (4b1b3d9).
Report is 7 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #6303      +/-   ##
==========================================
- Coverage   21.03%   20.87%   -0.16%     
==========================================
  Files         580      580              
  Lines       19832    19792      -40     
  Branches     2929     2929              
==========================================
- Hits         4171     4132      -39     
+ Misses      15609    15608       -1     
  Partials       52       52              

Flag	Coverage Δ
apps.hash-ai-worker-ts	1.34% <ø> (ø)
apps.hash-api	1.14% <ø> (ø)
blockprotocol.type-system	46.42% <ø> (ø)
local.harpc-client	48.69% <ø> (ø)
local.hash-backend-utils	8.81% <ø> (ø)
local.hash-graph-sdk	58.62% <ø> (ø)
local.hash-isomorphic-utils	0.91% <ø> (ø)
local.hash-subgraph	24.54% <ø> (ø)
rust.deer	6.66% <ø> (ø)
rust.error-stack	72.53% <ø> (ø)
rust.sarif	88.81% <ø> (+0.15%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[vilkinsons]

How many number-related data types are we exposing to users as HASH "Data Types" in the ontology/entities UI? (Integer, 64 bit float, 128 bit float, etc.)

[TimDiekmann]

How many number-related data types are we exposing to users as HASH "Data Types" in the ontology/entities UI? (Integer, 64 bit float, 128 bit float, etc.)

Currently, we only expose number in TypeScript, which corresponds to a 64 bit floating point (IEEE 754). For embeddings, we use 32 bit. See H-3988 for more thoughts on this topic.
The frontend may restrict number further depending on constraints such as whole numbers, min, max, etc.

[CiaranMn]

Looks good, thanks @TimDiekmann!

[TimDiekmann]

Thanks! I need a re-approval due to merge conflicts due to changes in rustfmt in the toolchain bump merged earlier today.

[github-actions]

Benchmark results

@rust/hash-graph-benches – Integrations

scaling_read_entity_complete_zero_depth

Function	Value	Mean	Flame graphs
entity_by_id	25 entities	$3.34\mathrm{ms}\pm 9.81\mathrm{\mu }\mathrm{s}\left(3.76\mathrm{\%}\right)$	Flame Graph
entity_by_id	1 entities	$1.99\mathrm{ms}\pm 11.5\mathrm{\mu }\mathrm{s}\left(0.432\mathrm{\%}\right)$	Flame Graph
entity_by_id	5 entities	$2.00\mathrm{ms}\pm 8.72\mathrm{\mu }\mathrm{s}\left(-0.088\mathrm{\%}\right)$	Flame Graph
entity_by_id	10 entities	$2.20\mathrm{ms}\pm 6.62\mathrm{\mu }\mathrm{s}\left(0.246\mathrm{\%}\right)$	Flame Graph
entity_by_id	50 entities	$4.17\mathrm{ms}\pm 23.4\mathrm{\mu }\mathrm{s}\left(-17.113\mathrm{\%}\right)$	Flame Graph

representative_read_entity

Function	Value	Mean	Flame graphs
entity_by_id	entity type ID: https://blockprotocol.org/@alice/types/entity-type/organization/v/1	$16.8\mathrm{ms}\pm 198\mathrm{\mu }\mathrm{s}\left(-26.609\mathrm{\%}\right)$	Flame Graph
entity_by_id	entity type ID: https://blockprotocol.org/@alice/types/entity-type/block/v/1	$17.1\mathrm{ms}\pm 202\mathrm{\mu }\mathrm{s}\left(-24.996\mathrm{\%}\right)$	Flame Graph
entity_by_id	entity type ID: https://blockprotocol.org/@alice/types/entity-type/song/v/1	$16.0\mathrm{ms}\pm 196\mathrm{\mu }\mathrm{s}\left(-0.472\mathrm{\%}\right)$	Flame Graph
entity_by_id	entity type ID: https://blockprotocol.org/@alice/types/entity-type/uk-address/v/1	$17.0\mathrm{ms}\pm 181\mathrm{\mu }\mathrm{s}\left(0.321\mathrm{\%}\right)$	Flame Graph
entity_by_id	entity type ID: https://blockprotocol.org/@alice/types/entity-type/playlist/v/1	$16.1\mathrm{ms}\pm 167\mathrm{\mu }\mathrm{s}\left(-4.151\mathrm{\%}\right)$	Flame Graph
entity_by_id	entity type ID: https://blockprotocol.org/@alice/types/entity-type/page/v/2	$15.3\mathrm{ms}\pm 185\mathrm{\mu }\mathrm{s}\left(-5.990\mathrm{\%}\right)$	Flame Graph
entity_by_id	entity type ID: https://blockprotocol.org/@alice/types/entity-type/book/v/1	$15.5\mathrm{ms}\pm 164\mathrm{\mu }\mathrm{s}\left(-4.156\mathrm{\%}\right)$	Flame Graph
entity_by_id	entity type ID: https://blockprotocol.org/@alice/types/entity-type/person/v/1	$15.8\mathrm{ms}\pm 182\mathrm{\mu }\mathrm{s}\left(-30.383\mathrm{\%}\right)$	Flame Graph
entity_by_id	entity type ID: https://blockprotocol.org/@alice/types/entity-type/building/v/1	$16.1\mathrm{ms}\pm 185\mathrm{\mu }\mathrm{s}\left(6.82\mathrm{\%}\right)$	Flame Graph

representative_read_entity_type

Function	Value	Mean	Flame graphs
get_entity_type_by_id	Account ID: d4e16033-c281-4cde-aa35-9085bf2e7579	$2.11\mathrm{ms}\pm 8.05\mathrm{\mu }\mathrm{s}\left(-0.014\mathrm{\%}\right)$	Flame Graph

representative_read_multiple_entities

Function	Value	Mean	Flame graphs
link_by_source_by_property	depths: DT=0, PT=2, ET=2, E=2	$85.6\mathrm{ms}\pm 381\mathrm{\mu }\mathrm{s}\left(-0.502\mathrm{\%}\right)$	Flame Graph
link_by_source_by_property	depths: DT=0, PT=0, ET=2, E=2	$80.5\mathrm{ms}\pm 311\mathrm{\mu }\mathrm{s}\left(-0.927\mathrm{\%}\right)$	Flame Graph
link_by_source_by_property	depths: DT=0, PT=0, ET=0, E=2	$72.4\mathrm{ms}\pm 426\mathrm{\mu }\mathrm{s}\left(-0.030\mathrm{\%}\right)$	Flame Graph
link_by_source_by_property	depths: DT=2, PT=2, ET=2, E=2	$89.2\mathrm{ms}\pm 549\mathrm{\mu }\mathrm{s}\left(0.349\mathrm{\%}\right)$	Flame Graph
link_by_source_by_property	depths: DT=255, PT=255, ET=255, E=255	$97.5\mathrm{ms}\pm 486\mathrm{\mu }\mathrm{s}\left(-1.190\mathrm{\%}\right)$	Flame Graph
link_by_source_by_property	depths: DT=0, PT=0, ET=0, E=0	$37.0\mathrm{ms}\pm 259\mathrm{\mu }\mathrm{s}\left(-3.754\mathrm{\%}\right)$	Flame Graph
entity_by_property	depths: DT=0, PT=2, ET=2, E=2	$51.9\mathrm{ms}\pm 337\mathrm{\mu }\mathrm{s}\left(-0.046\mathrm{\%}\right)$	Flame Graph
entity_by_property	depths: DT=0, PT=0, ET=2, E=2	$46.8\mathrm{ms}\pm 225\mathrm{\mu }\mathrm{s}\left(-0.535\mathrm{\%}\right)$	Flame Graph
entity_by_property	depths: DT=0, PT=0, ET=0, E=2	$42.2\mathrm{ms}\pm 388\mathrm{\mu }\mathrm{s}\left(1.14\mathrm{\%}\right)$	Flame Graph
entity_by_property	depths: DT=2, PT=2, ET=2, E=2	$55.5\mathrm{ms}\pm 369\mathrm{\mu }\mathrm{s}\left(0.582\mathrm{\%}\right)$	Flame Graph
entity_by_property	depths: DT=255, PT=255, ET=255, E=255	$64.6\mathrm{ms}\pm 267\mathrm{\mu }\mathrm{s}\left(0.267\mathrm{\%}\right)$	Flame Graph
entity_by_property	depths: DT=0, PT=0, ET=0, E=0	$37.1\mathrm{ms}\pm 226\mathrm{\mu }\mathrm{s}\left(-0.264\mathrm{\%}\right)$	Flame Graph

scaling_read_entity_complete_one_depth

Function	Value	Mean	Flame graphs
entity_by_id	25 entities	$174\mathrm{ms}\pm 745\mathrm{\mu }\mathrm{s}\left(-4.078\mathrm{\%}\right)$	Flame Graph
entity_by_id	1 entities	$20.7\mathrm{ms}\pm 65.9\mathrm{\mu }\mathrm{s}\left(0.770\mathrm{\%}\right)$	Flame Graph
entity_by_id	5 entities	$27.4\mathrm{ms}\pm 269\mathrm{\mu }\mathrm{s}\left(2.30\mathrm{\%}\right)$	Flame Graph
entity_by_id	10 entities	$31.4\mathrm{ms}\pm 157\mathrm{\mu }\mathrm{s}\left(-45.531\mathrm{\%}\right)$	Flame Graph
entity_by_id	50 entities	$5.60\mathrm{s}\pm 281\mathrm{ms}\left(-0.049\mathrm{\%}\right)$	Flame Graph

scaling_read_entity_linkless

Function	Value	Mean	Flame graphs
entity_by_id	100 entities	$2.21\mathrm{ms}\pm 8.99\mathrm{\mu }\mathrm{s}\left(1.03\mathrm{\%}\right)$	Flame Graph
entity_by_id	1000 entities	$2.89\mathrm{ms}\pm 11.4\mathrm{\mu }\mathrm{s}\left(-7.374\mathrm{\%}\right)$	Flame Graph
entity_by_id	10000 entities	$9.29\mathrm{ms}\pm 91.1\mathrm{\mu }\mathrm{s}\left(2.87\mathrm{\%}\right)$	Flame Graph
entity_by_id	1 entities	$1.99\mathrm{ms}\pm 6.10\mathrm{\mu }\mathrm{s}\left(0.648\mathrm{\%}\right)$	Flame Graph
entity_by_id	10 entities	$2.00\mathrm{ms}\pm 8.65\mathrm{\mu }\mathrm{s}\left(0.697\mathrm{\%}\right)$	Flame Graph