Skip to content
This repository has been archived by the owner on Feb 21, 2024. It is now read-only.

BSI Range-Encoding: schema support for range frames #616

Closed
travisturner opened this issue Jun 7, 2017 · 0 comments · Fixed by #670
Closed

BSI Range-Encoding: schema support for range frames #616

travisturner opened this issue Jun 7, 2017 · 0 comments · Fixed by #670
Assignees

Comments

@travisturner
Copy link
Member

travisturner commented Jun 7, 2017

We need a way to specify that a frame should contain range-encoded, integer values (as opposed to equality-encoded bitmaps achieved with SetBit()), and a way to define a schema (list of fields) for the range frame.

Modify the create frame HTTP endpoint:

/index/i/frame/f" -d '{"options":{"rangeEnabled": true}, "fields": [{"name": "age", "type": "int", "min": 1, "max": 120}, {"name": "salary", "type": "int", "min": 10000, "max": 1000000}]}}'

The above command should create a frame with a schema containing two fields: age and salary. Because the rangeEncoded option is provided and set to true, this frame is considered a range frame and as such the following options must be forced to:

The schema needs to be in memory, available to all nodes in the cluster as part of the frame struct. It also needs to be stored on disk in data-dir/index/frame/.schema.

One thing to note about this schema is that it effectively needs to maintain the mapping of fields to rows. So in our example above, rows would be assigned like below:

rowID represents
0 age: not-null
1 age: component 1
2 age: component 2
3 age: component 3
4 age: component 4
5 age: component 5
6 age: component 6
7 age: component 7
8 salary: not-null
9 salary: component 1
10 salary: component 2
11 salary: component 3
12 salary: component 4
13 salary: component 5
14 salary: component 6
15 salary: component 7
16 salary: component 8
17 salary: component 9
18 salary: component 10
19 salary: component 11
20 salary: component 12
21 salary: component 13
22 salary: component 14
23 salary: component 15
24 salary: component 16
25 salary: component 17
26 salary: component 18
26 salary: component 19
27 salary: component 20

Additionally, we need to maintain the offset of the range. In our salary example, the offset is 10,000 (the min value). By doing this, we can support ranges that don't necessarily start at 0. In theory, we could also support ranges that contain negative numbers.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants