[Rust] ObjectStore::read_dir method #911
Conversation
|
Will add cloud storage tests into integration tests. |
| @@ -176,6 +176,20 @@ impl ObjectStore { | |||
| ObjectWriter::new(self, path).await | |||
| } | |||
|
|
|||
| /// Read a directory (start from base directory) and returns all sub-paths in the directory. | |||
| pub async fn read_dir(&self, dir_path: impl Into<Path>) -> Result<Vec<String>> { | |||
There was a problem hiding this comment.
Should this return Result<Vec<Path>> instead for downstream processing?
rust/src/io/object_store.rs
Outdated
| .iter() | ||
| .map(|cp| cp.filename().map(|s| s.to_string()).unwrap_or_default()) | ||
| .chain(output.objects.iter().map(|o| o.location.to_string())) | ||
| .map(|s| Ok(Path::parse(s)?.filename().unwrap().to_string())) |
There was a problem hiding this comment.
is this returning just the shortname or the full path? At the object store level, maybe it makes more sense to get the full paths? and then the lancedb layer can take just the shortname before the .lance as the table name?
There was a problem hiding this comment.
This returns the filename, which is the same behavior as the OS readdir() call. LanceDB uses the readdir() semantic, right? For example, to list what tables are available.
|
also windows failure |
Change it back to return |
To support scan directory on local and cloud storage.