Code

feat: basic fsspec writing#1016

Code

Merged

lobis merged 78 commits intomainscikit-hep/uproot5:mainfrom fsspec-writingscikit-hep/uproot5:fsspec-writingCopy head branch name to clipboardNov 15, 2023

+281-74

Commits

Commits on Nov 2, 2023

• use paramiko instead of sshfs

  lobis

  committed
  2869480

  Copy full SHA for 2869480

• use specified port

  lobis

  committed
  b7ca2c1

  Copy full SHA for b7ca2c1

• test default handler behaviour

  lobis

  committed
  631b4bd

  Copy full SHA for 631b4bd

• default to fsspec instead of error if scheme not found

  lobis

  committed
  f8e298f

  Copy full SHA for f8e298f

• attempt to close socket

  lobis

  committed
  84c1b94

  Copy full SHA for 84c1b94

• fix ci

  lobis

  committed
  e56e337

  Copy full SHA for e56e337

• Revert "fix ci"

  Show description for 075434d
  lobis

  committed
  075434d

  Copy full SHA for 075434d

• broader exception

  lobis

  committed
  3786573

  Copy full SHA for 3786573

• also handle socket exception

  lobis

  committed
  108ff79

  Copy full SHA for 108ff79

• get user robust

  lobis

  committed
  c49aedb

  Copy full SHA for c49aedb

• enable github test with skip if api limit is hit (so we sometimes test it, but it never fails due to api limits). TODO: update exception class

  lobis

  committed
  35f5d8c

  Copy full SHA for 35f5d8c

• add memory filesystem test

  lobis

  committed
  d8ca663

  Copy full SHA for d8ca663

• add zip and tar tests

  lobis

  committed
  c38fb86

  Copy full SHA for c38fb86

Commits on Nov 3, 2023

• Merge branch 'fsspec-ssh-paramiko' into fsspec-tests

  lobis

  authored
  24b350a

  Copy full SHA for 24b350a

• fix memory test

  lobis

  committed
  fb3f99b

  Copy full SHA for fb3f99b

• Merge branch 'main' into fsspec-tests

  lobis

  authored
  242dd16

  Copy full SHA for 242dd16

• zip/tar tests

  lobis

  committed
  c31c888

  Copy full SHA for c31c888

• Merge remote-tracking branch 'origin/fsspec-tests' into fsspec-tests

  Show description for fea35b8
  lobis

  committed
  fea35b8

  Copy full SHA for fea35b8

Commits on Nov 6, 2023

• add test for writing with fsspec without any integration (file-like object)

  lobis

  committed
  bfc6844

  Copy full SHA for bfc6844

• update docstrings, add some type hints

  lobis

  committed
  9e6ecc1

  Copy full SHA for 9e6ecc1

• add helper method to produce sink from a path or file-like object

  lobis

  committed
  705087f

  Copy full SHA for 705087f

• add fsspec writing test for integration

  lobis

  committed
  a2ab17d

  Copy full SHA for a2ab17d

• helper function to check if file-like object

  lobis

  committed
  6b7ee9f

  Copy full SHA for 6b7ee9f

• return file-like object

  lobis

  committed
  320a8bb

  Copy full SHA for 320a8bb

• explicit argument

  lobis

  committed
  4650833

  Copy full SHA for 4650833

• rename test

  lobis

  committed
  ef37ccf

  Copy full SHA for ef37ccf

• create empty file

  lobis

  committed
  88e883e

  Copy full SHA for 88e883e

• rename test

  lobis

  committed
  b395176

  Copy full SHA for b395176

• pass storage options to fsspec

  lobis

  committed
  5c5a297

  Copy full SHA for 5c5a297

• test ssh and memory writing

  lobis

  committed
  f05c2c1

  Copy full SHA for f05c2c1

Commits on Nov 7, 2023

• split fsspec tests into reading and writing

  lobis

  committed
  5ba427b

  Copy full SHA for 5ba427b

• loosen the check for file-like object

  lobis

  committed
  14c6457

  Copy full SHA for 14c6457

• add test for file update

  lobis

  committed
  0a53fd2

  Copy full SHA for 0a53fd2

• file update test

  lobis

  committed
  fe1933f

  Copy full SHA for fe1933f

• properly truncate the file

  lobis

  committed
  add7501

  Copy full SHA for add7501

• refactor how sink is created (no changes)

  lobis

  committed
  d7d2ad6

  Copy full SHA for d7d2ad6

• annotation to avoid warning

  lobis

  committed
  0fd103a

  Copy full SHA for 0fd103a

• close file if sink initialization fails

  lobis

  committed
  c7b55ba

  Copy full SHA for c7b55ba

• sink will handle fsspec in a similar way to local path for open and close

  lobis

  committed
  a20517c

  Copy full SHA for a20517c

• rename to reading

  lobis

  committed
  9d55aa5

  Copy full SHA for 9d55aa5

• check if test works

  lobis

  committed
  ae57e81

  Copy full SHA for ae57e81

• missing import

  lobis

  committed
  51902e0

  Copy full SHA for 51902e0

• remove parent dirs

  lobis

  committed
  3d0a37a

  Copy full SHA for 3d0a37a

• fix zip tar tests

  lobis

  committed
  bd93a7f

  Copy full SHA for bd93a7f

• skip github if api limits hit

  lobis

  committed
  2520553

  Copy full SHA for 2520553

• attempt to fix windows paths

  lobis

  committed
  cb994a6

  Copy full SHA for cb994a6

• use more complex uri with object in zip test

  lobis

  committed
  469c3cb

  Copy full SHA for 469c3cb

• debug

  lobis

  committed
  e472a37

  Copy full SHA for e472a37

• paths

  lobis

  committed
  b8aec91

  Copy full SHA for b8aec91

• add new test case to object url split

  lobis

  committed
  8a4cc84

  Copy full SHA for 8a4cc84

• add new failing test case: TODO make it work

  lobis

  committed
  b1e1f5d

  Copy full SHA for b1e1f5d

• revert debug changes

  lobis

  committed
  2cddc78

  Copy full SHA for 2cddc78

• working in new test case

  lobis

  committed
  52c8db6

  Copy full SHA for 52c8db6

• merge and fix conflicts

  lobis

  committed
  8dccae3

  Copy full SHA for 8dccae3

• merge and fix conflicts

  lobis

  committed
  433d7e3

  Copy full SHA for 433d7e3

Commits on Nov 8, 2023

• modified where file is truncated

  lobis

  committed
  a286901

  Copy full SHA for a286901

• revert file-like check

  lobis

  committed
  ddec905

  Copy full SHA for ddec905

• revert is-file-like check

  lobis

  committed
  63b4377

  Copy full SHA for 63b4377

• unified rb+ mode (fsspec uses "rb+" not "r+b" it can be confusing sometimes)

  lobis

  committed
  83f9675

  Copy full SHA for 83f9675

• add http write test (not implemented error check)

  lobis

  committed
  f37ec6b

  Copy full SHA for f37ec6b

Commits on Nov 11, 2023

• Merge remote-tracking branch 'origin/main' into fsspec-writing

  Show description for f22ab6b
  lobis

  committed
  f22ab6b

  Copy full SHA for f22ab6b

Commits on Nov 13, 2023

• Merge branch 'main' into fsspec-writing

  lobis

  authored
  776f3c9

  Copy full SHA for 776f3c9

• skip test for debugging

  lobis

  committed
  545e244

  Copy full SHA for 545e244

• understand test failure

  lobis

  committed
  38ac90c

  Copy full SHA for 38ac90c

• use r+b instead of rb+

  lobis

  committed
  24102d2

  Copy full SHA for 24102d2

• use new sink

  lobis

  committed
  f6c4602

  Copy full SHA for f6c4602

• writing memory test

  lobis

  committed
  07fdbd5

  Copy full SHA for 07fdbd5

• correctly truncate file with fsspec

  lobis

  committed
  b56ce77

  Copy full SHA for b56ce77

Commits on Nov 14, 2023

• use fsspec to get parent dir

  lobis

  committed
  fc6c83d

  Copy full SHA for fc6c83d

• aiohttp import skip

  lobis

  committed
  2502026

  Copy full SHA for 2502026

• merge main

  lobis

  committed
  c3547f3

  Copy full SHA for c3547f3

• isstr

  lobis

  committed
  3682817

  Copy full SHA for 3682817

• use ports instead of string

  lobis

  committed
  958c0c8

  Copy full SHA for 958c0c8

• improve path:obj split to handle chained protocols (protocol1::protocol2://)

  lobis

  committed
  699f1c7

  Copy full SHA for 699f1c7

• working url chain to some extent

  lobis

  committed
  0187fd8

  Copy full SHA for 0187fd8

• attempt to fix ci

  lobis

  committed
  380f07a

  Copy full SHA for 380f07a

• cleanup

  lobis

  committed
  dad051c

  Copy full SHA for dad051c

• attempt to fix ci

  lobis

  committed
  db3b8b6

  Copy full SHA for db3b8b6