Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Multi wildcard support finalization #40

Merged
merged 11 commits into from
Jan 20, 2022
27 changes: 27 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
name: ci

on: [push, pull_request]

jobs:
test:
runs-on: ubuntu-latest

strategy:
matrix:
node-version: [6.x, 8.x, 10.x, 12.x, 14.x, 16.x]

steps:
- uses: actions/checkout@v2

- name: Use Node.js
uses: actions/setup-node@v1
with:
node-version: ${{ matrix.node-version }}

- name: Install
run: |
npm install

- name: Run tests
run: |
npm run test
3 changes: 2 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
node_modules
.nyc_output
package-lock.json
coverage
coverage
.idea
11 changes: 0 additions & 11 deletions .travis.yml

This file was deleted.

16 changes: 16 additions & 0 deletions benchmark/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,8 @@ const redactIntermediateWildCensorFunction = fastRedact({ paths: ['as.*.c'], cen
const redactCensorFunctionWithPath = fastRedact({ paths: ['at.d.b'], censor: censorFn, serialize: false })
const redactWildCensorFunctionWithPath = fastRedact({ paths: ['au.d.*'], censor: censorFnWithPath, serialize: false })
const redactIntermediateWildCensorFunctionWithPath = fastRedact({ paths: ['av.*.c'], censorFnWithPath, serialize: false })
const redactMultiWild = fastRedact({ paths: ['aw.*.*'] })
const redactMultiWildCensorFunction = fastRedact({ paths: ['ax.*.*'], censor: censorFn, serialize: false })

const getObj = (outerKey) => ({
[outerKey]: {
Expand Down Expand Up @@ -201,6 +203,20 @@ var run = bench([
redactIntermediateWildCensorFunctionWithPath(obj)
}
setImmediate(cb)
},
function benchFastRedactMultiWild (cb) {
const obj = getObj('aw')
for (var i = 0; i < max; i++) {
redactMultiWild(obj)
}
setImmediate(cb)
},
function benchFastRedactMultiWildCensorFunction (cb) {
const obj = getObj('ax')
for (var i = 0; i < max; i++) {
redactMultiWildCensorFunction(obj)
}
setImmediate(cb)
}
], 500)

Expand Down
11 changes: 11 additions & 0 deletions example/multi-wildcard-array-depth.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
'use strict'
const fastRedact = require('..')
const redact = fastRedact({ paths: ['a[*].c.d[*].i'] })
const obj = {
a: [
{ c: { d: [ { i: 'redact me', j: 'not me' } ], e: 'leave me be' } },
{ c: { d: [ { i: 'redact me too', j: 'not me' }, { i: 'redact me too', j: 'not me' } ], f: 'I want to live' } },
{ c: { d: [ { i: 'redact me 3', j: 'not me' } ], g: 'I want to run in a stream' } }
]
}
console.log(redact(obj))
11 changes: 11 additions & 0 deletions example/multi-wildcard-array-end.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
'use strict'
const fastRedact = require('..')
const redact = fastRedact({ paths: ['a[*].c.d[*]'] })
const obj = {
a: [
{ c: { d: ['hide me', '2'], e: 'leave me be' } },
{ c: { d: ['and me'], f: 'I want to live' } },
{ c: { d: ['and also I'], g: 'I want to run in a stream' } }
]
}
console.log(redact(obj))
11 changes: 11 additions & 0 deletions example/multi-wildcard-array.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
'use strict'
const fastRedact = require('..')
const redact = fastRedact({ paths: ['a[*].c[*].d'] })
const obj = {
a: [
{ c: [{ d: 'hide me', e: 'leave me be' }, { d: 'hide me too', e: 'leave me be' }, { d: 'hide me 3', e: 'leave me be' }] },
{ c: [{ d: 'and me', f: 'I want to live' }] },
{ c: [{ d: 'and also I', g: 'I want to run in a stream' }] }
]
}
console.log(redact(obj))
71 changes: 60 additions & 11 deletions lib/modifiers.js
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,20 @@ function nestedRestore (arr) {
const length = arr.length
for (var i = 0; i < length; i++) {
const { key, target, value } = arr[i]
target[key] = value
if (has(target, key)) {
target[key] = value
}
/* istanbul ignore else */
if (typeof target === 'object') {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it possible that target is not an object? It's not covered by tests anyway

Copy link
Contributor Author

@lrecknagel lrecknagel Jan 19, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think its possible, applies for the example below:

const redact = fastRedact({
  paths: ['a[*].x.d[*].i.*.i']
})
const o = {
  a: [
    { x: { d: [ { i: undefined, j: 'NR' } ] } }
  ]
}

Only applies when the given path expects a deeper nested object but is undefined somewhere.
I`am not sure if this should be a case to be considered.

const targetKeys = Object.keys(target)
for (var j = 0; j < targetKeys.length; j++) {
const tKey = targetKeys[j]
const subTarget = target[tKey]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what if subtarget is also an object?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you pls explain your point here - I dont get it right now
subTarget can be an object, and it works imho

For the example, which is also in the tests (https://github.com/lrecknagel/fast-redact/blob/multi-wildcard-support-finalization/test/index.js#L1228)

const redactOut = fastRedact({
  paths: ['a[*].x.d[*].p.*.i'],
})
const obj = {
  a: [
    { x: { d: [ { i: undefined, j: 'NR' } ] }},
  ]
}
redactOut(obj)

we get the following state for subtarget:

Bildschirmfoto 2022-01-19 um 13 48 31

if (has(subTarget, key)) {
subTarget[key] = value
}
}
}
}
}

Expand All @@ -67,7 +80,9 @@ function nestedRedact (store, o, path, ns, censor, isCensorFct, censorFctTakesPa
}

function has (obj, prop) {
return Object.prototype.hasOwnProperty.call(obj, prop)
return obj !== undefined && obj !== null
? ('hasOwn' in Object ? Object.hasOwn(obj, prop) : Object.prototype.hasOwnProperty.call(obj, prop))
: false
}

function specialSet (o, k, path, afterPath, censor, isCensorFct, censorFctTakesPath) {
Expand All @@ -80,23 +95,57 @@ function specialSet (o, k, path, afterPath, censor, isCensorFct, censorFctTakesP
var ov
var oov = null
var exists = true
var wc = null
ov = n = o[k]
if (typeof n !== 'object') return { value: null, parent: null, exists }
while (n != null && ++i < afterPathLen) {
k = afterPath[i]
oov = ov
if (!(k in n)) {
if (k !== '*' && !wc && !(typeof n === 'object' && k in n)) {
exists = false
break
}
ov = n[k]
nv = (i !== lastPathIndex)
? ov
: (isCensorFct
? (censorFctTakesPath ? censor(ov, [...path, originalKey, ...afterPath]) : censor(ov))
: censor)
n[k] = (has(n, k) && nv === ov) || (nv === undefined && censor !== undefined) ? n[k] : nv
n = n[k]
if (k === '*') {
wc = k
if (i !== lastPathIndex) {
continue
}
}
if (wc) {
const wcKeys = Object.keys(n)
for (var j = 0; j < wcKeys.length; j++) {
const wck = wcKeys[j]
const wcov = n[wck]
const kIsWc = k === '*'
if (kIsWc || (typeof wcov === 'object' && k in wcov)) {
if (kIsWc) {
ov = wcov
} else {
ov = wcov[k]
}
nv = (i !== lastPathIndex)
? ov
: (isCensorFct
? (censorFctTakesPath ? censor(ov, [...path, originalKey, ...afterPath]) : censor(ov))
: censor)
if (kIsWc) {
n[wck] = nv
} else {
wcov[k] = (nv === undefined && censor !== undefined) || (has(wcov, k) && nv === ov) ? wcov[k] : nv
}
}
}
wc = null
} else {
ov = n[k]
nv = (i !== lastPathIndex)
? ov
: (isCensorFct
? (censorFctTakesPath ? censor(ov, [...path, originalKey, ...afterPath]) : censor(ov))
: censor)
n[k] = (has(n, k) && nv === ov) || (nv === undefined && censor !== undefined) ? n[k] : nv
n = n[k]
}
if (typeof n !== 'object') break
}
return { value: ov, parent: oov, exists }
Expand Down
1 change: 0 additions & 1 deletion lib/parse.js
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,6 @@ function parse ({ paths }) {
const before = path.slice(0, star)
const beforeStr = before.join('.')
const after = path.slice(star + 1, path.length)
if (after.indexOf('*') > -1) throw Error('fast-redact – Only one wildcard per path is supported')
const nested = after.length > 0
wcLen++
wildcards.push({
Expand Down
1 change: 0 additions & 1 deletion lib/restorer.js
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,6 @@ function restorer ({ secret, wcLen }) {
return function compileRestore () {
if (this.restore) return
const paths = Object.keys(secret)
.filter((path) => secret[path].precensored === false)
const resetters = resetTmpl(secret, paths)
const hasWildcards = wcLen > 0
const state = hasWildcards ? { secret, groupRestore, nestedRestore } : { secret }
Expand Down
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "fast-redact",
"version": "3.0.1",
"version": "3.0.2",
"description": "very fast object redaction",
"main": "index.js",
"scripts": {
Expand Down
Loading