Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: more generic BlockCodec, remove encoder & decoder props #75

Merged
merged 1 commit into from
Apr 26, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/mikeals-workflow.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ jobs:
runs-on: ubuntu-latest
strategy:
matrix:
node-version: [12.x, 14.x]
node-version: [12.x, 14.x, 16.x]
steps:
- uses: actions/checkout@v2
- name: Use Node.js ${{ matrix.node-version }}
Expand Down
44 changes: 12 additions & 32 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,8 @@
# multiformats

This library defines common interfaces and low level building blocks for various interrelated multiformat technologies (multicodec, multihash, multibase,
and CID). They can be used to implement custom custom base
encoders / decoders / codecs, codec encoders /decoders and multihash hashers that comply to the interface that layers above assume.
This library defines common interfaces and low level building blocks for various interrelated multiformat technologies (multicodec, multihash, multibase, and CID). They can be used to implement custom custom base encoders / decoders / codecs, codec encoders /decoders and multihash hashers that comply to the interface that layers above assume.

Library provides implementations for most basics and many others can be found in linked repositories.
This library provides implementations for most basics and many others can be found in linked repositories.

## Interfaces

Expand Down Expand Up @@ -45,19 +43,15 @@ block = await Block.create({ bytes: block.bytes, cid: block.cid, codec, hasher }

### Multibase Encoders / Decoders / Codecs

CIDs can be serialized to string representation using multibase encoders that
implement [`MultibaseEncoder`](https://github.com/multiformats/js-multiformats/blob/master/src/bases/interface.ts) interface. Library
provides quite a few implementations that can be imported:
CIDs can be serialized to string representation using multibase encoders that implement [`MultibaseEncoder`](https://github.com/multiformats/js-multiformats/blob/master/src/bases/interface.ts) interface. This library provides quite a few implementations that can be imported:

```js
import { base64 } from "multiformats/bases/base64"
cid.toString(base64.encoder)
//> 'mAYAEEiCTojlxqRTl6svwqNJRVM2jCcPBxy+7mRTUfGDzy2gViA'
```

Parsing CID string serialized CIDs requires multibase decoder that implements
[`MultibaseDecoder`](https://github.com/multiformats/js-multiformats/blob/master/src/bases/interface.ts) interface. Library provides a
decoder for every encoder it provides:
Parsing CID string serialized CIDs requires multibase decoder that implements [`MultibaseDecoder`](https://github.com/multiformats/js-multiformats/blob/master/src/bases/interface.ts) interface. This library provides a decoder for every encoder it provides:

```js
CID.parse('mAYAEEiCTojlxqRTl6svwqNJRVM2jCcPBxy+7mRTUfGDzy2gViA', base64.decoder)
Expand Down Expand Up @@ -93,35 +87,24 @@ v0.toV1().toString()

### Multicodec Encoders / Decoders / Codecs

Library defines [`BlockEncoder`, `BlockDecoder` and `BlockCodec` interfaces](https://github.com/multiformats/js-multiformats/blob/master/src/codecs/interface.ts)
and utility function to take care of the boilerplate when implementing them:
This library defines [`BlockEncoder`, `BlockDecoder` and `BlockCodec` interfaces](https://github.com/multiformats/js-multiformats/blob/master/src/codecs/interface.ts). Codec implementations should conform to the `BlockCodec` interface which implements both `BlockEncoder` and `BlockDecoder`.

```js
import { codec } from 'multiformats/codecs/codec'

const json = codec({
/**
* @template T
* @type {BlockCodec<0x0200, T>}
*/
export const { name, code, encode, decode } = {
name: 'json',
// As per multiformats table
// https://github.com/multiformats/multicodec/blob/master/table.csv#L113
code: 0x0200,
encode: json => new TextEncoder().encode(JSON.stringify(json)),
decode: bytes => JSON.parse(new TextDecoder().decode(bytes))
})
```

Just like with multibase, here codecs are duals of `encoder` and `decoder` parts,
but they also implement both interfaces for convenience:

```js
const hello = json.encoder.encode({ hello: 'world' })
json.decode(b1)
//> { hello: 'world' }
}
```

### Multihash Hashers

This library defines [`MultihashHasher` and `MultihashDigest` interfaces](https://github.com/multiformats/js-multiformats/blob/master/src/hashes/interface.ts)
and convinient function for implementing them:
This library defines [`MultihashHasher` and `MultihashDigest` interfaces](https://github.com/multiformats/js-multiformats/blob/master/src/hashes/interface.ts) and convinient function for implementing them:

```js
import * as hasher from 'multiformats/hashes/hasher')
Expand All @@ -141,8 +124,6 @@ CID.create(1, json.code, hash)
//> CID(bagaaierasords4njcts6vs7qvdjfcvgnume4hqohf65zsfguprqphs3icwea)
```



# Implementations

By default, no base encodings (other than base32 & base58btc), hash functions,
Expand Down Expand Up @@ -179,7 +160,6 @@ import the ones you need yourself.
| `dag-pb` | `@ipld/dag-pb` | [ipld/js-dag-pb](https://github.com/ipld/js-dag-pb) |
| `dag-jose` | `dag-jose`| [ceramicnetwork/js-dag-jose](https://github.com/ceramicnetwork/js-dag-jose) |


## TypeScript support

This project is distributed with type definitions for TypeScript.
Expand Down
25 changes: 12 additions & 13 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -7,15 +7,14 @@
"type": "module",
"scripts": {
"build": "npm run build:js && npm run build:types",
"build:js": "npm_config_yes=true ipjs build --tests --main && npm run build:copy",
"build:js": "ipjs build --tests --main && npm run build:copy",
"build:copy": "cp -a tsconfig.json src vendor test dist/ && rm -rf dist/test/ts-use",
"build:types": "npm run build:copy && cd dist && tsc --build",
"build:vendor": "npm run build:vendor:varint && npm run build:vendor:base-x",
"build:vendor:varint": "npx brrp -x varint > vendor/varint.js",
"build:vendor:base-x": "npx brrp -x @multiformats/base-x > vendor/base-x.js",
"publish": "npm_config_yes=true ipjs publish",
"build:vendor:varint": "npm_config_yes=true npx brrp -x varint > vendor/varint.js",
"build:vendor:base-x": "npm_config_yes=true npx brrp -x @multiformats/base-x > vendor/base-x.js",
"publish": "ipjs publish",
"lint": "standard",
"check": "tsc --build --noErrorTruncation",
"test:cjs": "npm run build:js && mocha dist/cjs/node-test/test-*.js && npm run test:cjs:browser",
"test:node": "hundreds mocha test/test-*.js",
"test:cjs:browser": "polendina --page --worker --serviceworker --cleanup dist/cjs/browser-test/test-*.js",
Expand Down Expand Up @@ -75,9 +74,6 @@
"./hashes/identity": {
"import": "./src/hashes/identity.js"
},
"./codecs/codec": {
"import": "./src/codecs/codec.js"
},
"./codecs/json": {
"import": "./src/codecs/json.js"
},
Expand All @@ -86,16 +82,16 @@
}
},
"devDependencies": {
"@types/node": "^14.14.37",
"@typescript-eslint/eslint-plugin": "^4.20.0",
"@typescript-eslint/parser": "^4.20.0",
"c8": "^7.6.0",
"@types/node": "^14.14.41",
"@typescript-eslint/eslint-plugin": "^4.22.0",
"@typescript-eslint/parser": "^4.22.0",
"c8": "^7.7.1",
"hundreds": "0.0.9",
"ipjs": "^5.0.0",
"mocha": "^8.3.2",
"polendina": "^1.1.0",
"standard": "^16.0.3",
"typescript": "^4.2.3"
"typescript": "^4.2.4"
},
"standard": {
"ignore": [
Expand All @@ -120,6 +116,9 @@
"homepage": "https://github.com/multiformats/js-multiformats#readme",
"typesVersions": {
"*": {
"bases/base64": [
"types/bases/base64-import.d.ts"
],
"*": [
"types/*"
],
Expand Down
71 changes: 0 additions & 71 deletions src/codecs/codec.js

This file was deleted.

14 changes: 4 additions & 10 deletions src/codecs/interface.ts
Original file line number Diff line number Diff line change
Expand Up @@ -16,18 +16,12 @@ export interface BlockDecoder<Code extends number, T> {
}

/**
* IPLD codec that is just Encoder + Decoder however it is
* separate those capabilties as sender requires encoder and receiver
* requires decoder.
* An IPLD codec is a combination of both encoder and decoder.
*/
export interface BlockCodec<Code extends number, T> extends BlockEncoder<Code, T>, BlockDecoder<Code, T> {
encoder: BlockEncoder<Code, T>,
decoder: BlockDecoder<Code, T>
}

export interface BlockCodec<Code extends number, T> extends BlockEncoder<Code, T>, BlockDecoder<Code, T> {}

// This just a hack to retain type information abouth the data that
// is incoded `T` Because it's a union `data` field is never going
// This just a hack to retain type information about the data that
// is encoded `T` Because it's a union `data` field is never going
// to be usable anyway.
export type ByteView<T> =
| Uint8Array
Expand Down
14 changes: 11 additions & 3 deletions src/codecs/json.js
Original file line number Diff line number Diff line change
@@ -1,10 +1,18 @@
// @ts-check

import { codec } from './codec.js'
/**
* @template {number} Code
* @template T
* @typedef {import('./interface').BlockCodec<Code, T>} BlockCodec
*/

export const { name, code, decode, encode, decoder, encoder } = codec({
/**
* @template T
* @type {BlockCodec<0x0200, T>}
*/
export const { name, code, encode, decode } = {
name: 'json',
code: 0x0200,
encode: json => new TextEncoder().encode(JSON.stringify(json)),
decode: bytes => JSON.parse(new TextDecoder().decode(bytes))
})
}
27 changes: 21 additions & 6 deletions src/codecs/raw.js
Original file line number Diff line number Diff line change
@@ -1,11 +1,26 @@
// @ts-check

import { coerce } from '../bytes.js'
import { codec } from './codec.js'

export const { name, code, decode, encode, decoder, encoder } = codec({
/**
* @template {number} Code
* @template T
* @typedef {import('./interface').BlockCodec<Code, T>} BlockCodec
*/

/**
* @param {Uint8Array} bytes
* @returns {Uint8Array}
*/
const raw = (bytes) => coerce(bytes)

/**
* @template T
* @type {BlockCodec<0x55, Uint8Array>}
*/
export const { name, code, encode, decode } = {
name: 'raw',
code: 85,
decode: coerce,
encode: coerce
})
code: 0x55,
decode: raw,
encode: raw
}
3 changes: 1 addition & 2 deletions src/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,5 @@ import * as varint from './varint.js'
import * as bytes from './bytes.js'
import * as hasher from './hashes/hasher.js'
import * as digest from './hashes/digest.js'
import * as codec from './codecs/codec.js'

export { CID, hasher, digest, varint, bytes, codec }
export { CID, hasher, digest, varint, bytes }
5 changes: 2 additions & 3 deletions test/test-legacy.js
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,6 @@ import { legacy } from 'multiformats/legacy'
import * as rawCodec from 'multiformats/codecs/raw'
import * as jsonCodec from 'multiformats/codecs/json'
import { sha256, sha512 } from 'multiformats/hashes/sha2'
import { codec } from 'multiformats/codecs/codec'
import { CID } from 'multiformats/cid'

const same = assert.deepStrictEqual
Expand Down Expand Up @@ -36,7 +35,7 @@ describe('multicodec', () => {
raw = legacy(rawCodec, { hashes })
json = legacy(jsonCodec, { hashes })
link = await raw.util.cid(Buffer.from('test'))
custom = legacy(codec({
custom = legacy({
name: 'custom',
code: 6787678,
encode: o => {
Expand All @@ -52,7 +51,7 @@ describe('multicodec', () => {
if (obj.o.link) obj.link = CID.asCID(link)
return obj
}
}), { hashes })
}, { hashes })
})
test('encode/decode raw', () => {
const buff = raw.util.serialize(Buffer.from('test'))
Expand Down
32 changes: 1 addition & 31 deletions test/test-multicodec.js
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ import * as bytes from '../src/bytes.js'
import assert from 'assert'
import * as raw from 'multiformats/codecs/raw'
import * as json from 'multiformats/codecs/json'
import { codec } from 'multiformats/codecs/codec'

const same = assert.deepStrictEqual
const test = it

Expand Down Expand Up @@ -31,37 +31,7 @@ describe('multicodec', () => {
same(json.decode(buff), { hello: 'world' })
})

test('json.encoder', () => {
const { encoder } = json
same(encoder === json.encoder, true, 'getter cached decoder')

const buff = encoder.encode({ hello: 'world' })
same(buff, bytes.fromString(JSON.stringify({ hello: 'world' })))
})

test('json.decoder', () => {
const { decoder } = json
same(decoder === json.decoder, true, 'getter cached encoder')

const buff = json.encode({ hello: 'world' })
same(decoder.decode(buff), { hello: 'world' })
})

test('raw cannot encode string', async () => {
await testThrow(() => raw.encode('asdf'), 'Unknown type, must be binary type')
})

test('add with function', () => {
const blip = codec({
code: 200,
name: 'blip',
encode: (a) => a[1],
decode: (a) => a
})

const two = bytes.fromString('two')
const three = bytes.fromString('three')
same(blip.encode(['one', two, three]), two)
same(blip.decode(three, 200), three)
})
})
Loading