-
Notifications
You must be signed in to change notification settings - Fork 48
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
load() and DOM APIs #557
load() and DOM APIs #557
Conversation
The tests are running fine, but it looks like there's an issue with |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Overall, this looks pretty good, some questions/comments below.
src/dom/Value.ts
Outdated
/** | ||
* Returns null if the Value is not a Struct, List, or SExpression. | ||
* Otherwise, will attempt to index into the Value with each pathElement in turn | ||
* until: | ||
* - A value is found that cannot use the current PathElement as an index. | ||
* (e.g. The current Value is a List but the current PathElement is a number.) | ||
* - An undefined value is encountered. | ||
* (e.g. The current Value is a Struct and the current PathElement is a string, | ||
* but the current PathElement is not a fieldName that exists in the Struct.) | ||
* - The pathElements are exhausted and the requested Value is discovered. | ||
* | ||
* @param pathElements One or more values to be used to index into the Value. | ||
* @returns null if no value is found at the specified path. Otherwise, returns the discovered Value. | ||
*/ | ||
get(...pathElements: PathElement[]): Value | null; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should we use undefined
here? I ask because of the potential asymmetry with respect to someone using null
as a value on the serialization side.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
-1 for undefined
. If pathElements
resolves to a null value, I'm expecting an instance of Value
to be returned for which isNull()
returns true
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ion null
values loaded from a stream are represented in the DOM as instances of dom.Null
(which implements dom.Value
). If get(...)
returns a Javascript null
, it always means that no value was found at the specified path.
In the upcoming serialization API, if you wrote:
let value: Value = ion.dump(null);
I would expect value
to be an instance of dom.Null
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If pathElements resolves to a null value, I'm expecting an instance of Value to be returned for which isNull() returns true.
Can you elaborate on this? If there isn't an Ion value at the specified path, why would an instance of Value
be returned?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
To clarify the current behavior:
If pathElements resolves to a null value
- If
pathElements
resolves to an Ion null,get
will currently return aValue
for whichisNull()
is true andgetType()
returns the correct Ion type. - If
pathElements
resolves to a JSnull
, it means that no value was found at the specified path.
src/dom/DomValue.ts
Outdated
this._ionType = ionType; | ||
this._annotations = []; | ||
// Setting the 'enumerable' attribute of these properties to `false` prevents them | ||
// from appearing in the iterators returned by Object.keys(), Object.entries(), etc. | ||
// This guarantees that users iterating over the fields of a struct or values in a list | ||
// will see only Values from the source data or that they have created themselves. | ||
Object.defineProperty(this, "_ionType", {enumerable: false}); | ||
Object.defineProperty(this, "_annotations", {enumerable: false}); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Minor: We may want to have _annotations
be spelled _ionAnnotations
just to be consistent.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, good call.
src/dom/DomValue.ts
Outdated
// Class expressions (like this mixin) cannot have private or protected methods. | ||
_setAnnotations(annotations: string[]) { | ||
this._annotations = annotations; | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is the implication that one cannot set annotations or is that handled elsewhere?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The intent was for users not to mutate the annotations that were set during construction (at least for now). Ordinarily you could signal this by flagging the method as protected
or private
, but that mechanism isn't permitted in class
expressions. I've used the _
prefix (as is done elsewhere in ion-js
) to signal that this is meant to be an internal method, but I added the comment to clarify that I couldn't provide stronger guarantees.
src/dom/Value.ts
Outdated
/** | ||
* Returns null if the Value is not a Boolean. | ||
* Otherwise, returns the boolean value of the Boolean. | ||
*/ | ||
booleanValue(): boolean | null; | ||
|
||
/** | ||
* Returns null if the Value is not an Integer, Float, or Decimal. | ||
* Otherwise, returns the JS number representation of the Value. | ||
* In some cases, this method will perform conversions that inherently involve loss of | ||
* precision. See bigIntValue() and decimalValue() for lossless alternatives. | ||
*/ | ||
numberValue(): number | null; | ||
|
||
/** | ||
* Returns null if the Value is not an Integer. | ||
* Otherwise, returns a lossless BigInt representation of the Value. | ||
*/ | ||
bigIntValue(): JSBI | null; | ||
|
||
/** | ||
* Returns null if the Value is not a Decimal. | ||
* Otherwise, returns a lossless Decimal representation of the Value. | ||
*/ | ||
decimalValue(): Decimal | null; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A common theme in this API is that for the cases where the type does not match, null
is returned. What if the type aligns and the value is null
? Should the mis-typed cases throw (similar to Java) or return something like undefined
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
-1 for undefined
(there are times when it is tempting, but still poor imo), but throwing sounds appropriate as it represents a coding error which would ideally be identified by the compiler (but can't be in these cases). Throwing would also align with ion-js's IonReader
contract.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What if the type aligns and the value is null?
I don't believe there is such a case. Can you give me an example?
Note that:
let value: Value = ion.load('"foo"');
assert.equal(value.getType(), IonTypes.STRING);
assert.equal("foo", value.stringValue());
assert.isFalse(value.isNull());
assert.isTrue(value instanceof ion.dom.String);
assert.isFalse(value instanceof ion.dom.Null);
let value: Value = ion.load('null.string');
assert.equal(value.getType(), IonTypes.STRING);
assert.isNull(value.stringValue());
assert.isTrue(value.isNull());
assert.isFalse(value instanceof ion.dom.String);
assert.isTrue(value instanceof ion.dom.Null);
src/dom/Value.ts
Outdated
/** | ||
* Returns null if the Value is not a Struct. | ||
* Otherwise, returns an iterator over the field name/value pairs in the Struct. | ||
*/ | ||
fields(): IterableIterator<[string, Value]> | null; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why return null
over throwing?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
+1 for throwing (as well as for fieldNames()
and values()
)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I went back and forth on this. ion-java
throws.
I opted to return null
because it allows users to cheaply test for unexpectedly missing values.
let greeting = ion.load('hello')!.stringValue();
if (greeting !== null) {
console.log(greeting + ", friend.");
}
If we throw, the only way to avoid the expense of throwing and catching an Error
would be rather tedious type inspection.
let greeting: Value = ion.load('hello')!;
if (value.getType() === IonTypes.STRING && !value.isNull()) { // OR (value instanceof dom.String)
console.log(greeting.stringValue());
}
src/dom/index.ts
Outdated
function createReader(ionData: ReaderBuffer | Reader): Reader { | ||
// If the provided parameter is already a reader, no new work is required. | ||
// However, we cannot simply test `ionData instanceof Reader` because `Reader` | ||
// is an interface. | ||
if (ionData instanceof TextReader || ionData instanceof BinaryReader) { | ||
return ionData as Reader; | ||
} | ||
return makeReader(ionData as ReaderBuffer); | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is it cleaner to test against the concrete types of ReaderBuffer
to avoid the potentially more open concrete type space of Reader
? (e.g. mitigating when we add a DOM Reader
or something in the future).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Interesting observation. I'll switch it over.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I remember the original challenge I encountered here. ReaderBuffer
is comprised of:
string
ReaderOctetBuffer
ArrayBufferLike
ArrayBuffer
ArrayBufferView
- ... (different view sizes)
SharedArrayBuffer
- ...
ArrayLike<number>
Many of these are interfaces or type aliases and do not exist at runtime. There's no comprehensive list of concrete types we can test for with (e.g.) typeof
or instanceof
.
I'm going to leave this as-is for now. I suggest providing a public AbstractReader
class as a common base class for Reader
implementations, even if it offers no concrete functionality. Then we can simply test instanceof AbstractReader
to make this distinction.
src/dom/Null.ts
Outdated
* text encoding | Ion data type | ||
* ====================================== | ||
* null | IonType.NULL (implicit) | ||
* null.null | IonType.NULL (explicit) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What is implicit vs. explicit mean here? null.null
and null
are merely syntactic ways to spell the same thing, this doc seems to imply something more, but I can't think of what could be meant.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
null.null and null are merely syntactic ways to spell the same thing
This is all I meant to say. One encoding spells out the Ion type, one doesn't. I'll remove the words "implicit" and "explicit" from the comment, since they appear to do more harm than good.
src/dom/Struct.ts
Outdated
constructor(fields: IterableIterator<[string, Value]>, annotations: string[] = []) { | ||
super(); | ||
for(let [fieldName, value] of fields) { | ||
Object.defineProperty(this, ''+fieldName, { | ||
configurable: true, | ||
enumerable: true, | ||
value: value, | ||
writable: true | ||
}) | ||
} | ||
this._setAnnotations(annotations); | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this is burying the lede if I am reading this correctly. You do surface field names as properties, yes? If so, this should be documented explicitly in the doc comment.
What happens when they incidentally shadow methods or the internal fields? Does this need to gate against that?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You do surface field names as properties, yes?
Yes.
If so, this should be documented explicitly in the doc comment.
Agreed, I'll add this.
What happens when they incidentally shadow methods or the internal fields? Does this need to gate against that?
My plan (not implemented in this already quite large PR) is for all property accesses on structs to resolve to method names and internal fields first. If you have data that would shadow one of those fields, it can be accessed by .get(fieldName)
instead. The iterators returned by fields()
and fieldNames()
would behave properly, returning the custom property values instead of the methods/fields.
let s = ion.load('{stringValue: "Hello", foo: "bar"}')!; // The `stringValue` field shadows Value#stringValue()
let greeting = s.stringValue; // This resolves to the method handle and is probably not what the user wanted
let greeting = s['stringValue']; // This resolves to the method handle and is probably not what the user wanted
let greeting = s.stringValue(); // `greeting` is now "Hello"
let greeting = s.get('stringValue'); // `greeting is now "Hello"
This means that there are ~16 Struct
field names that can have unexpected behavior, which seemed a better compromise than simply forbidding them.
src/dom/Symbol.ts
Outdated
stringValue(): string { | ||
return this.valueOf() as string; | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should this class override toString
like dom.String
does?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good catch, thanks. It's already storing symbolText: string
, so we might as well avoid the cost of converting from String
to string
via valueOf()
. I'll add that.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good work, Zack. :) Two high-level thoughts:
- please check the first parameter of the dom-class constructors for consistency. should others specify
protected readonly
? looks like we're not using a "_" prefix consistently? the following four stood out, but worth checking all of them:- Clob:
protected readdonly bytes
- Float:
protected readonly value
- String:
protected readonly _text
- Symbol:
protected readonly symbolText
- Clob:
- it's entirely possible i don't understand the beauty of
[...x]
;), but i expectx
to be equivalent; am i missing out on the magic?
src/dom/Value.ts
Outdated
/** | ||
* Returns null if the Value is not a Boolean. | ||
* Otherwise, returns the boolean value of the Boolean. | ||
*/ | ||
booleanValue(): boolean | null; | ||
|
||
/** | ||
* Returns null if the Value is not an Integer, Float, or Decimal. | ||
* Otherwise, returns the JS number representation of the Value. | ||
* In some cases, this method will perform conversions that inherently involve loss of | ||
* precision. See bigIntValue() and decimalValue() for lossless alternatives. | ||
*/ | ||
numberValue(): number | null; | ||
|
||
/** | ||
* Returns null if the Value is not an Integer. | ||
* Otherwise, returns a lossless BigInt representation of the Value. | ||
*/ | ||
bigIntValue(): JSBI | null; | ||
|
||
/** | ||
* Returns null if the Value is not a Decimal. | ||
* Otherwise, returns a lossless Decimal representation of the Value. | ||
*/ | ||
decimalValue(): Decimal | null; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
-1 for undefined
(there are times when it is tempting, but still poor imo), but throwing sounds appropriate as it represents a coding error which would ideally be identified by the compiler (but can't be in these cases). Throwing would also align with ion-js's IonReader
contract.
* Value's [type]Value() functions will throw an Error if the requested operation is not supported by the Value's ionType. * Value's fields(), fieldNames(), and elements() methods return arrays instead of iterators. * ion.load() will only call next on the provided Reader if it is not already positioned over a value. * Cleanup for spacing, typedoc, etc.
UpdatesThe most recent commit includes the following changes:
Blocking issueI've identified a bug in |
Due to an issue[1] in typedoc's mixin support, the documentation generation phase of the release process fails. This commit points the typedoc dependency to a custom fork[2] that contains a fix. We can and should back this change out later when mainline is fixed. [1] TypeStrong/typedoc#1207 [2] https://github.com/zslayton/typedoc/
I've unblocked the build by creating a temporary |
The I've updated the |
src/dom/Null.ts
Outdated
if (Null._operationIsSupported(this.getType(), operation)) { | ||
return null; | ||
} | ||
throw new Error(`'${operation}' is not supported by Ion type ${this.getType().name}`); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
drop single quotes, add ()
? as in: ${operation}() is not supported...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sure.
src/dom/Null.ts
Outdated
// Otherwise, throw an Error indicating that the requested operation is not supported. | ||
private _unsupportedOperationOrNullDerefence(operation: string): never { | ||
if (Null._operationIsSupported(this.getType(), operation)) { | ||
throw new Error(`${operation} called on a null ${this.getType().name}.`); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
add ()
, as in: ${operation}() called...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done.
src/dom/Null.ts
Outdated
if (Null._operationIsSupported(this.getType(), operation)) { | ||
throw new Error(`${operation} called on a null ${this.getType().name}.`); | ||
} | ||
throw new Error(`'${operation}' is not supported by Ion type ${this.getType().name}`); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
drop single quotes around ${operation}
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
lgtm :)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Overall looks great--minor comments below.
super(value); | ||
this._setAnnotations(annotations); | ||
} | ||
|
||
public numberValue(): number { | ||
return this.valueOf() as number; | ||
return +this.valueOf(); | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A comment of intent here would be helpful as to what unary plus is trying to accomplish here (presumably forces some kind of coercion?).
['fields', new Set([IonTypes.STRUCT])], | ||
['fieldNames', new Set([IonTypes.STRUCT])], | ||
['elements', new Set([IonTypes.LIST, IonTypes.SEXP, IonTypes.STRUCT])] | ||
]); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Magic strings always makes me a bit nervous, would it be prudent to just model these symbols as an enum
? Also, do we really need this table to be a map? Wouldn't an enumeration of integers serve this purpose?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm going to see if I can add the supported types as a property directly on the methods' Function
objects. I'll do so in an upcoming PR.
src/dom/Struct.ts
Outdated
* Struct fields can be accessed as properties on this object. For example: | ||
* | ||
* let s: any = ion.load('{foo: 1, bar: 2, baz: qux::3}'); | ||
* assert.equal(6, s.foo + s['bar'] + s.baz); | ||
* | ||
* [1] http://amzn.github.io/ion-docs/docs/spec.html#struct |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If someone defines a field name that collides with a method name, which is masked? We should be clear about that in the docs.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I have a PR brewing that will introduce the new methods-take-precedence behavior. I'll be sure to spell out that behavior in the updated docs.
Issue #, if available: #531
Description of changes:
Currently,
ion-js
only provides a streamingReader
API to access values in an Ion stream. This API is efficient, but has a rather steep learning curve and can often require verbose code. This PR creates a DOM-style API for loading values from an Ion stream into memory, allowing users to interact with them as they would any other JS value/Array/Object.load()
andloadAll()
Two new top-level functions have been introduced:
Javascript (
any
) interfaceEach Ion data type is represented by a new class that extends an existing Javascript type:
dom.List
extendsArray
dom.Integer
extendsNumber
dom.Timestamp
extendsDate
This means that in most cases, you can treat values returned by
load()
the same way you would treat a standard Javascript equivalent.Typescript (
Value
) interfaceThe Javascript API above is concise and convenient, but does not offer any static type safety. Each of the new classes in this PR also implements a new interface called
dom.Value
, which provides methods for indexing into Ion values as well as converting them to JS representations.Recommended reading order
I suggest reading the following files first:
src/dom/Value.ts
: All of the new classes introduced in this PR implement this interface.src/dom/DomValue.ts
: Serves as a common default implementation for theValue
interface.src/dom/index.ts
: Defines theload()
andloadAll()
methods.test/dom/dom.ts
: Demonstrates how to work with each data type afterload()
ing some Ion data.The rest of the files contain class definitions for each Ion type and are pretty straightforward.
Notes
dom
module. There are some name collisions that are easily remedied via package qualifiers or custom import aliases:ion.dom.String
vsglobal.String
ion.dom.Boolean
vsglobal.Boolean
ion.dom.Decimal
vsion.Decimal
ion.dom.Timestamp
vsion.Timestamp
toString()
does not attempt to produce valid Ion. OverridingtoString()
in classes that extendString
(i.e.dom.String
,dom.Symbol
) causes someString
functionality (e.g. thematch
method) to break.Follow-on work
ion.dump(...)
method to perform the inverse ofion.load()
. I held off on this to confirm that the API proposed in this PR is acceptable.TODO
s in this code that will be turned intoion-js
issues after the PR gets merged. Another PR will add the issue links to the comments.dom.Decimal
from anumber
. However, this is not required for use of theload()
API.ion.dump()
method is available, we should design a formal integration withion-tests
.By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.