Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: respect all data namespaces for sync auto-stop #769

Merged
merged 1 commit into from
Aug 29, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions src/constants.js
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,11 @@ export const NAMESPACES = /** @type {const} */ ([
/** @type {ReadonlyArray<Namespace>} */
export const PRESYNC_NAMESPACES = ['auth', 'config', 'blobIndex']

/** @type {ReadonlyArray<Namespace>} */
export const DATA_NAMESPACES = NAMESPACES.filter(
(namespace) => !PRESYNC_NAMESPACES.includes(namespace)
)
Comment on lines +16 to +19
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was removed recently in 2b36edd, and I'm adding it back same as it was.


export const NAMESPACE_SCHEMAS = /** @type {const} */ ({
data: ['observation', 'track'],
config: [
Expand Down
12 changes: 10 additions & 2 deletions src/sync/sync-api.js
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,11 @@ import { TypedEmitter } from 'tiny-typed-emitter'
import { SyncState } from './sync-state.js'
import { PeerSyncController } from './peer-sync-controller.js'
import { Logger } from '../logger.js'
import { NAMESPACES, PRESYNC_NAMESPACES } from '../constants.js'
import {
DATA_NAMESPACES,
NAMESPACES,
PRESYNC_NAMESPACES,
} from '../constants.js'
import { ExhaustivenessError, assert, keyToId, noop } from '../utils.js'
import { NO_ROLE_ID } from '../roles.js'
/** @import { CoreOwnership as CoreOwnershipDoc } from '@mapeo/schema' */
Expand Down Expand Up @@ -206,7 +210,11 @@ export class SyncApi extends TypedEmitter {
* | yes | yes | yes | none | off |
*/
#updateState(namespaceSyncState = this[kSyncState].getState()) {
const dataHave = namespaceSyncState.data.localState.have
const dataHave = DATA_NAMESPACES.reduce(
(total, namespace) =>
total + namespaceSyncState[namespace].localState.have,
0
)
const hasReceivedNewData = dataHave !== this.#previousDataHave
if (hasReceivedNewData) {
this.#clearAutostopDataSyncTimeoutIfExists()
Expand Down
49 changes: 37 additions & 12 deletions test-e2e/sync.js
Original file line number Diff line number Diff line change
Expand Up @@ -294,21 +294,34 @@ test('auto-stop', async (t) => {
const clock = FakeTimers.install({ shouldAdvanceTime: true })
t.after(() => clock.uninstall())

const managers = await createManagers(2, t)
const [invitor, ...invitees] = managers
const invitor = createManager('invitor', t)

const fastify = Fastify()
const fastifyController = new FastifyController({ fastify })
t.after(() => fastifyController.stop())
const invitee = createManager('invitee', t, { fastify })

const managers = [invitor, invitee]

await Promise.all([
invitor.setDeviceInfo({ name: 'invitor' }),
invitee.setDeviceInfo({ name: 'invitee' }),
fastifyController.start(),
])
t.after(() => fastifyController.stop())

const disconnect = connectPeers(managers, { discovery: false })
t.after(disconnect)

const projectId = await invitor.createProject({ name: 'mapeo' })
await invite({ invitor, invitees, projectId })
await invite({ invitor, invitees: [invitee], projectId })

const projects = await Promise.all(
managers.map((m) => m.getProject(projectId))
)
const [invitorProject, inviteeProject] = projects

const generatedObservations = generate('observation', { count: 3 })
const generatedObservations = generate('observation', { count: 2 })

invitorProject.$sync.start({ autostopDataSyncAfter: 10_000 })
inviteeProject.$sync.start({ autostopDataSyncAfter: 10_000 })
Expand All @@ -333,13 +346,25 @@ test('auto-stop', async (t) => {

await clock.tickAsync(9000)

const observation2 = await invitorProject.observation.create(
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pseudocode for this diff:

-createObservation()
+createBlob()

 checkThatAutostopTimeoutIsUpdated()

valueOf(generatedObservations[1])
const fixturePath = new URL(
'../tests/fixtures/images/02-digidem-logo.jpg',
import.meta.url
).pathname
const blob = await invitorProject.$blobs.create(
{ original: fixturePath },
blobMetadata({ mimeType: 'image/jpeg' })
)
await waitForSync(projects, 'full')
assert(
await inviteeProject.observation.getByDocId(observation2.docId),
'invitee receives doc'
const blobUrl = await inviteeProject.$blobs.getUrl({
...blob,
variant: 'original',
})
const response = await request(blobUrl, { reset: true })
assert.equal(response.statusCode, 200)
assert.deepEqual(
Buffer.from(await response.body.arrayBuffer()),
await fs.readFile(fixturePath),
'invitee receives blob'
)

await clock.tickAsync(9000)
Expand Down Expand Up @@ -388,12 +413,12 @@ test('auto-stop', async (t) => {
invitorProject.$sync.start()
inviteeProject.$sync.start()

const observation3 = await invitorProject.observation.create(
valueOf(generatedObservations[2])
const observation2 = await invitorProject.observation.create(
valueOf(generatedObservations[1])
)
await waitForSync(projects, 'full')
assert(
await inviteeProject.observation.getByDocId(observation3.docId),
await inviteeProject.observation.getByDocId(observation2.docId),
'invitee receives doc'
)

Expand Down
Loading