Code

Merge wip-v2 into master#178

Code

Merged

mvdan merged 101 commits intomasteripld/go-car:masterfrom wip-v2ipld/go-car:wip-v2Copy head branch name to clipboardJul 16, 2021

+5,404-18

Commits

Commits on Jun 10, 2021

• Define CAR v2 as a separate go module

  Show description for f2ff2ca
  masih

  committed
  f2ff2ca

  Copy full SHA for f2ff2ca

• Add doc.go to avoid CI issues for empty modules

  Show description for 0c1de83
  masih

  committed
  0c1de83

  Copy full SHA for 0c1de83

Commits on Jun 11, 2021

• Define basic CAR v2 constants along with tests

  Show description for b76ee51
  masih

  committed
  b76ee51

  Copy full SHA for b76ee51

• Export constant header size

  Show description for 29e325a
  masih

  committed
  29e325a

  Copy full SHA for 29e325a

• Add CarV1Offset field to CAR v2 header

  Show description for 17e0aa9
  masih

  committed
  17e0aa9

  Copy full SHA for 17e0aa9

• Fix test case name to reflect what the test is

  Show description for 968459c
  masih

  committed
  968459c

  Copy full SHA for 968459c

Commits on Jun 14, 2021

• Implement CAR v2 header construction and marshalling

  Show description for e571973
  masih

  committed
  e571973

  Copy full SHA for e571973

Commits on Jul 16, 2021

• Implement CAR v2 writer that uses format.NodeGetter

  Show description for 875a654
  

  masih

  authored and

  mvdan

  committed
  875a654

  Copy full SHA for 875a654

• Copy carbs indexing mechanism int go-car repo

  Show description for d9c8b76
  

  masih

  authored and

  mvdan

  committed
  d9c8b76

  Copy full SHA for d9c8b76

• Address staticcheck errors

  Show description for d5e22dd
  

  masih

  authored and

  mvdan

  committed
  d5e22dd

  Copy full SHA for d5e22dd

• Implement OffsetReader

  Show description for 7cbf448
  

  masih

  authored and

  mvdan

  committed
  7cbf448

  Copy full SHA for 7cbf448

• Implement CAR v2 reader with API that provides BlockStore

  Show description for 1d3cbb3
  

  masih

  authored and

  mvdan

  committed
  1d3cbb3

  Copy full SHA for 1d3cbb3

• initial implementation of the 'fd' based carbon

  

  willscott

  authored and

  mvdan

  committed
  0f1e5aa

  Copy full SHA for 0f1e5aa

• close write handle on 'finish'

  

  willscott

  authored and

  mvdan

  committed
  7593b75

  Copy full SHA for 7593b75

• add test for expected round-trip behavior

  

  willscott

  authored and

  mvdan

  committed
  3e05a31

  Copy full SHA for 3e05a31

• Address staticcheck issues and remove dedicated carbon go module

  Show description for c7430a1
  

  masih

  authored and

  mvdan

  committed
  c7430a1

  Copy full SHA for c7430a1

• Remove the copied carbs without history

  Show description for 93bd623
  

  masih

  authored and

  mvdan

  committed
  93bd623

  Copy full SHA for 93bd623

• original willscott/carbs history in squashed form

  Show description for efbdc87
  

  willscott

  authored and

  mvdan

  committed
  efbdc87

  Copy full SHA for efbdc87

• Bump github.com/ipfs/go-ipld-cbor from 0.0.4 to 0.0.5

  Show description for c2a3b1a
  

  dependabot[bot]

  authored and

  mvdan

  committed
  c2a3b1a

  Copy full SHA for c2a3b1a

• Bump github.com/ipfs/go-ipfs-blockstore from 1.0.1 to 1.0.2

  Show description for 882fd42
  

  dependabot[bot]

  authored and

  mvdan

  committed
  882fd42

  Copy full SHA for 882fd42

• Bump github.com/ipfs/go-ipfs-blockstore from 1.0.2 to 1.0.3

  Show description for cef5063
  

  dependabot[bot]

  authored and

  mvdan

  committed
  cef5063

  Copy full SHA for cef5063

• Update version of github.com/ipld/go-car

  

  iand

  authored and

  mvdan

  committed
  260b871

  Copy full SHA for 260b871

• Bump github.com/multiformats/go-multihash from 0.0.14 to 0.0.15

  Show description for 44d3842
  

  dependabot[bot]

  authored and

  mvdan

  committed
  44d3842

  Copy full SHA for 44d3842

• Bump github.com/ipfs/go-block-format from 0.0.2 to 0.0.3

  Show description for e387959
  

  dependabot[bot]

  authored and

  mvdan

  committed
  e387959

  Copy full SHA for e387959

• Bump github.com/cheggaaa/pb/v3 from 3.0.5 to 3.0.7

  Show description for 9c64ad7
  

  dependabot[bot]

  authored and

  mvdan

  committed
  9c64ad7

  Copy full SHA for 9c64ad7

• Bump github.com/cheggaaa/pb/v3 from 3.0.7 to 3.0.8

  Show description for 67fcd1a
  

  dependabot[bot]

  authored and

  mvdan

  committed
  67fcd1a

  Copy full SHA for 67fcd1a

• add 'of' for creation of new Carbs

  

  willscott

  authored and

  mvdan

  committed
  4c5f1ae

  Copy full SHA for 4c5f1ae

• allow 'Record' Index to be public to allow for other implementations of Index

  

  willscott

  authored and

  mvdan

  committed
  4e343d7

  Copy full SHA for 4e343d7

• update to go-car v0.2.2

  

  willscott

  authored and

  mvdan

  committed
  0509247

  Copy full SHA for 0509247

• Bump github.com/ipld/go-car from 0.2.2 to 0.3.0

  Show description for 9773052
  

  dependabot[bot]

  authored and

  mvdan

  committed
  9773052

  Copy full SHA for 9773052

• Bump github.com/ipld/go-car from 0.3.0 to 0.3.1

  Show description for 067dae6
  

  dependabot[bot]

  authored and

  mvdan

  committed
  067dae6

  Copy full SHA for 067dae6

• Address staticcheck errors

  Show description for 94190bc
  

  masih

  authored and

  mvdan

  committed
  94190bc

  Copy full SHA for 94190bc

• Remove carbs dedicated go module

  Show description for 9e407d6
  

  masih

  authored and

  mvdan

  committed
  9e407d6

  Copy full SHA for 9e407d6

• Implement read-only random access blockstore

  Show description for d336f84
  

  masih

  authored and

  mvdan

  committed
  d336f84

  Copy full SHA for d336f84

• Run gofumpt -l -w .

  Show description for 8b5f18d
  

  masih

  authored and

  mvdan

  committed
  8b5f18d

  Copy full SHA for 8b5f18d

• Rename Prefix to Pragma for consistency

  Show description for e6a626c
  

  masih

  authored and

  mvdan

  committed
  e6a626c

  Copy full SHA for e6a626c

• Implement CAR introspector API

  Show description for 806cff8
  

  masih

  authored and

  mvdan

  committed
  806cff8

  Copy full SHA for 806cff8

• Replace introspector.go with simple ReadPragma

  Show description for 885b0d0
  

  masih

  authored and

  mvdan

  committed
  885b0d0

  Copy full SHA for 885b0d0

• Add TODO to reduce reader wrapping when working with CAR v1 reader

  Show description for 588d68e
  

  masih

  authored and

  mvdan

  committed
  588d68e

  Copy full SHA for 588d68e

• Use unsigned integer to represent width and count in sorted index

  Show description for aef90df
  

  masih

  authored and

  mvdan

  committed
  aef90df

  Copy full SHA for aef90df

• Move index out of internal package to unblock Ignite team

  Show description for 40097aa
  

  masih

  authored and

  mvdan

  committed
  40097aa

  Copy full SHA for 40097aa

• Implement read-while-writing blockstore

  Show description for 01e55a9
  

  masih

  authored and

  mvdan

  committed
  01e55a9

  Copy full SHA for 01e55a9

• Format code using gofumpt

  Show description for 1e464d9
  

  masih

  authored and

  mvdan

  committed
  1e464d9

  Copy full SHA for 1e464d9

• Fix attach index in ReadOnly blockstore

  Show description for 3ab2658
  

  masih

  authored and

  mvdan

  committed
  3ab2658

  Copy full SHA for 3ab2658

• Revert use of t.TempDir due to access issue in windows

  Show description for 8a3fdb1
  

  masih

  authored and

  mvdan

  committed
  8a3fdb1

  Copy full SHA for 8a3fdb1

• Rename ReadPragma to ReadVersion and drop returning roots

  Show description for 6bd284c
  

  masih

  authored and

  mvdan

  committed
  6bd284c

  Copy full SHA for 6bd284c

• Remove dependency to CAR v1 go module

  Show description for ff99aaf
  

  masih

  authored and

  mvdan

  committed
  ff99aaf

  Copy full SHA for ff99aaf

• copy v1 license into the v2 module

  Show description for 567a4c6
  mvdan

  committed
  567a4c6

  Copy full SHA for 567a4c6

• Remove CAR v1 walk function until it is explicitly asked for

  Show description for 2f36f2e
  

  masih

  authored and

  mvdan

  committed
  2f36f2e

  Copy full SHA for 2f36f2e

• Fix the writer index codec to sorted

  Show description for a6127a0
  

  masih

  authored and

  mvdan

  committed
  a6127a0

  Copy full SHA for a6127a0

• Improve documentation and fix bug in writer

  Show description for c954c99
  

  masih

  authored and

  mvdan

  committed
  c954c99

  Copy full SHA for c954c99

• index: use the IndexSorted multicodec

  Show description for 48d5a7f
  mvdan

  committed
  48d5a7f

  Copy full SHA for 48d5a7f

• add Close methods and NewReaderMmap

  Show description for cacae14
  mvdan

  committed
  cacae14

  Copy full SHA for cacae14

• blockstore: close AllKeysChan channel when done

  Show description for 73d15ae
  mvdan

  committed
  73d15ae

  Copy full SHA for 73d15ae

• blockstore: make unsupported methods panic

  Show description for c25c235
  mvdan

  committed
  c25c235

  Copy full SHA for c25c235

• consistently use CID hashes for uniqueness

  Show description for bd5908e
  mvdan

  committed
  bd5908e

  Copy full SHA for bd5908e

• blockstore: don't error on Has not finding a CID

  Show description for acc013c
  mvdan

  committed
  acc013c

  Copy full SHA for acc013c

• blockstore: make it safe for concurrent use

  Show description for 307cc4c
  mvdan

  committed
  307cc4c

  Copy full SHA for 307cc4c

• add WrapV1File

  Show description for 2611339
  mvdan

  committed
  2611339

  Copy full SHA for 2611339

• blockstore: add option to deduplicate by CID

  Show description for 08f751c
  mvdan

  committed
  08f751c

  Copy full SHA for 08f751c

• replace util.ReadCid with go-cid

  Show description for 695f454
  mvdan

  committed
  695f454

  Copy full SHA for 695f454

• replace internal/io.ReadCid with cid.CidFromReader

  Show description for 5f5aa1c
  mvdan

  committed
  5f5aa1c

  Copy full SHA for 5f5aa1c

• avoid exposing io.SectionReader in the API

  Show description for eaeb24b
  mvdan

  committed
  eaeb24b

  Copy full SHA for eaeb24b

• add carv2.WrapV1 based on io interfaces

  Show description for e54e235
  mvdan

  committed
  e54e235

  Copy full SHA for e54e235

• Replace binary.Put/ReadUvarint with go-varint

  Show description for f598e59
  

  masih

  authored and

  mvdan

  committed
  f598e59

  Copy full SHA for f598e59

• Rename ReadOnlyOf to NewReadOnly and accept both v1 and v2 payload

  Show description for 4e863b2
  

  masih

  authored and

  mvdan

  committed
  4e863b2

  Copy full SHA for 4e863b2

• Accommodate index.Generate taking io.ReadSeeker

  Show description for c0023ed
  

  masih

  authored and

  mvdan

  committed
  c0023ed

  Copy full SHA for c0023ed

• Consistently accept CAR v1 or v2 on blockstore.OpenReadOnly

  Show description for 2776842
  

  masih

  authored and

  mvdan

  committed
  2776842

  Copy full SHA for 2776842

• index: add support for null paddings in Generate

  Show description for 6765c84
  mvdan

  committed
  6765c84

  Copy full SHA for 6765c84

• Implement read or generate index from CAR accepting v1 or v2

  Show description for fc494d4
  

  masih

  authored and

  mvdan

  committed
  fc494d4

  Copy full SHA for fc494d4

• Replace redundant use of RW mutex where a simple mutex would do

  Show description for a6e8bc1
  

  masih

  authored and

  mvdan

  committed
  a6e8bc1

  Copy full SHA for a6e8bc1

• Implement ReadWrite blockstore resumption

  Show description for 2a5f894
  

  masih

  authored and

  mvdan

  committed
  2a5f894

  Copy full SHA for 2a5f894

• Avoid creating files on resumption inReadWrite blockstore

  Show description for 8c08fbf
  

  masih

  authored and

  mvdan

  committed
  8c08fbf

  Copy full SHA for 8c08fbf

• Remove resumption option for automatic resumption when file exists

  Show description for eca650e
  

  masih

  authored and

  mvdan

  committed
  eca650e

  Copy full SHA for eca650e

• Fix bug in ReadOnly blockstore GetSize

  Show description for 7373fe2
  

  masih

  authored and

  mvdan

  committed
  7373fe2

  Copy full SHA for 7373fe2

• Implement more tests for ReadOnly blockstore

  Show description for ba7ed7c
  

  masih

  authored and

  mvdan

  committed
  ba7ed7c

  Copy full SHA for ba7ed7c

• Assert write operations on ReadOnly blockstore panic

  Show description for 224cd8f
  

  masih

  authored and

  mvdan

  committed
  224cd8f

  Copy full SHA for 224cd8f

• Unexport unecesserry index APIs and integrate with mulitcodec

  Show description for 2b593c1
  

  masih

  authored and

  mvdan

  committed
  2b593c1

  Copy full SHA for 2b593c1

• Remove redundant seek before read or generate index

  Show description for cb2b58d
  

  masih

  authored and

  mvdan

  committed
  cb2b58d

  Copy full SHA for cb2b58d

• Move index generation functionality to root v2 package

  Show description for 24b8253
  

  masih

  authored and

  mvdan

  committed
  24b8253

  Copy full SHA for 24b8253

• Support resumption in ReadWrite blockstore for finalized files

  Show description for 96e8614
  

  masih

  authored and

  mvdan

  committed
  96e8614

  Copy full SHA for 96e8614

• index error should return not found

  

  aarshkshah1992

  authored and

  mvdan

  committed
  62be0f3

  Copy full SHA for 62be0f3

• Remove dependency to bufio.Reader in internal carv1 package

  Show description for cc1e449
  

  masih

  authored and

  mvdan

  committed
  cc1e449

  Copy full SHA for cc1e449

• Implement index generation using only io.Reader

  Show description for 6b085bc
  

  masih

  authored and

  mvdan

  committed
  6b085bc

  Copy full SHA for 6b085bc

• Unexport unused Writer API until it is re-implemented

  Show description for 1a14cef
  

  masih

  authored and

  mvdan

  committed
  1a14cef

  Copy full SHA for 1a14cef

• use the now-merged cid.CidFromReader

  Show description for a2cd464
  mvdan

  committed
  a2cd464

  Copy full SHA for a2cd464

• make all "Open" APIs use consistent names

  Show description for 4827ee3
  mvdan

  committed
  4827ee3

  Copy full SHA for 4827ee3

• Improve test coverage for ReadWrite blockstore

  Show description for dbdb742
  

  masih

  authored and

  mvdan

  committed
  dbdb742

  Copy full SHA for dbdb742

• Use consistent import and bot CID versions in testing

  Show description for 25dc000
  

  masih

  authored and

  mvdan

  committed
  25dc000

  Copy full SHA for 25dc000

• Fix index not found error not being propagated

  Show description for 7751d8b
  

  masih

  authored and

  mvdan

  committed
  7751d8b

  Copy full SHA for 7751d8b

• Remove duplicate test CARv1 file

  Show description for fd226df
  

  masih

  authored and

  mvdan

  committed
  fd226df

  Copy full SHA for fd226df

• update go-cid to get the CidFromReader panic fix

  Show description for 6434cd8
  mvdan

  committed
  6434cd8

  Copy full SHA for 6434cd8

• Refactor utility io conversions into one internal package

  Show description for f3fc595
  

  masih

  authored and

  mvdan

  committed
  f3fc595

  Copy full SHA for f3fc595

• unify options and add more blockstore options

  Show description for 7d8f54f
  mvdan

  committed
  7d8f54f

  Copy full SHA for 7d8f54f

• Implement examples and tests for index package

  Show description for 0aefa5b
  

  masih

  authored and

  mvdan

  committed
  0aefa5b

  Copy full SHA for 0aefa5b

• Remove memory intensive writer and add warping tests

  Show description for 0f79c3e
  

  masih

  authored and

  mvdan

  committed
  0f79c3e

  Copy full SHA for 0f79c3e

• Rename CarV1 with Data in API to be consistent with spec

  Show description for aa62719
  

  masih

  authored and

  mvdan

  committed
  aa62719

  Copy full SHA for aa62719

• Replace a file if exists on index.Save

  Show description for 64bb51b
  masih

  committed
  64bb51b

  Copy full SHA for 64bb51b

• Remove index.Save API

  Show description for e45d3b2
  masih

  committed
  e45d3b2

  Copy full SHA for e45d3b2

• Add example blockstore examples

  Show description for ae4ddd4
  masih

  committed
  ae4ddd4

  Copy full SHA for ae4ddd4

• make Close safe, not letting Finalize block forever either

  

  mvdan

  authored and

  masih

  committed
  c06b4f2

  Copy full SHA for c06b4f2