Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Drop flatgeobuf dependency in geozero crate #235

Merged
merged 1 commit into from
Aug 30, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 3 additions & 5 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -15,10 +15,10 @@ readme = "README.md"
license = "MIT OR Apache-2.0"

[workspace.dependencies]
# This should point to the last published version
# This is a major hack, due to some code (like geozero tests!) relying on flatgeobuf crate,
# which in turn relies on geozero itself.
# Used in geozero-cli and geozero-bench
# geozero version must be in sync with flatgeobuf version!
geozero = { version = "0.13.0", default-features = false }
flatgeobuf = "4.3.0"

async-trait = "0.1"
byteorder = { version = "1.4.3", default-features = false }
Expand All @@ -30,7 +30,6 @@ dbase = "0.4"
diesel = { version = "2.1.0", default-features = false, features = ["postgres"] }
dup-indexer = "0.3"
env_logger = "0.10.0"
flatgeobuf = "4.0.0"
futures-util = "0.3.28"
gdal = { version = "0.16", default-features = false }
gdal-sys = "0.9"
Expand All @@ -43,7 +42,6 @@ hex = "0.4"
kdbush = "0.2"
log = "0.4.19"
lyon = "1.0.1"
polylabel = "2.5"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm guessing you've removed this example because it uses FGB. We could convert it to use geojson, but I'm guessing you've judged it to be not worth the upkeep?

That seems fine, just making sure I understand.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Exactly. This example should go either into the flatgeobuf or the polylabel crate. I tried to use geojson, but I had the same problem with the missing feature iterator.

postgis = "0.9.0"
postgres = "0.19"
postgres-types = "0.2"
Expand Down
25 changes: 3 additions & 22 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -70,23 +70,6 @@ println!("{}", fgb.to_json()?);
Full source code: [geojson.rs](./geozero/tests/geojson.rs)


Read FlatGeobuf data as geo-types geometries and calculate label position with [polylabel-rs](https://github.com/urschrei/polylabel-rs):
```rust,ignore
let mut file = BufReader::new(File::open("countries.fgb")?);
let mut fgb = FgbReader::open(&mut file)?.select_all()?;
while let Some(feature) = fgb.next()? {
let name: String = feature.property("name").unwrap();
if let Ok(Geometry::MultiPolygon(mpoly)) = feature.to_geo() {
if let Some(poly) = &mpoly.0.iter().next() {
let label_pos = polylabel(&poly, &0.10).unwrap();
println!("{name}: {label_pos:?}");
}
}
}
```
Full source code: [polylabel.rs](./geozero/tests/polylabel.rs)


## PostGIS usage examples

Select and insert geo-types geometries with rust-postgres. Requires the `with-postgis-postgres` feature:
Expand Down Expand Up @@ -189,7 +172,7 @@ geometry.process(&mut vertex_counter, GeometryType::MultiPolygon)?;
```
Full source code: [geozero-api.rs](./geozero/tests/geozero-api.rs)

Find maximal height in 3D polygons:
Find maximal height in 3D points:
```rust,ignore
struct MaxHeightFinder(f64);

Expand All @@ -205,10 +188,8 @@ impl GeomProcessor for MaxHeightFinder {
}

let mut max_finder = MaxHeightFinder(0.0);
while let Some(feature) = fgb.next()? {
let geometry = feature.geometry().unwrap();
geometry.process(&mut max_finder, GeometryType::MultiPolygon)?;
}
points.process_geom(&mut max_finder)?;
assert_eq!(max_finder.0, 457.1);
```
Full source code: [geozero-api.rs](./geozero/tests/geozero-api.rs)

Expand Down
11 changes: 2 additions & 9 deletions geozero/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -55,11 +55,9 @@ sqlx = { workspace = true, optional = true }
wkt = { workspace = true, optional = true }

[dev-dependencies]
flatgeobuf.workspace = true
geo.workspace = true
hex.workspace = true
kdbush.workspace = true
polylabel.workspace = true
postgres.workspace = true
seek_bufread.workspace = true
sqlx = { workspace = true, features = ["runtime-tokio-native-tls", "macros", "time", "postgres", "sqlite"] }
Expand All @@ -81,7 +79,7 @@ required-features = ["with-gdal"]
[[test]]
name = "geojson"
path = "tests/geojson.rs"
required-features = ["with-geojson"]
required-features = ["with-geojson", "with-wkt"]

[[test]]
name = "geopackage"
Expand All @@ -101,7 +99,7 @@ required-features = ["with-geo", "with-geojson"]
[[test]]
name = "geozero-api"
path = "tests/geozero-api.rs"
required-features = []
required-features = ["with-wkt"]

[[test]]
name = "kdbush"
Expand All @@ -113,11 +111,6 @@ name = "mvt"
path = "tests/mvt.rs"
required-features = ["with-mvt", "with-geo", "with-geojson"]

[[test]]
name = "polylabel"
path = "tests/polylabel.rs"
required-features = ["with-geo"]

[[test]]
name = "postgis"
path = "tests/postgis.rs"
Expand Down
41 changes: 10 additions & 31 deletions geozero/tests/geojson.rs
Original file line number Diff line number Diff line change
@@ -1,38 +1,17 @@
use flatgeobuf::{FgbReader, HttpFgbReader};
use geozero::geojson::GeoJsonWriter;
use geozero::ProcessToJson;
use seek_bufread::BufReader;
use std::fs::File;
use std::io::BufWriter;
use geozero::{wkt::Wkt, ToJson};
type Result<T> = std::result::Result<T, Box<dyn std::error::Error>>;

#[test]
fn fgb_to_geojson() -> Result<()> {
let mut filein = BufReader::new(File::open("tests/data/countries.fgb")?);
let mut fgb = FgbReader::open(&mut filein)?.select_bbox(8.8, 47.2, 9.5, 55.3)?;
let json = fgb.to_json()?;
fn wkt_to_geojson_feature() -> Result<()> {
let wkt = Wkt("MULTIPOLYGON (((40 40, 20 45, 45 30, 40 40)),
((35 10, 45 45, 15 40, 10 20, 35 10),
(20 30, 35 35, 30 20, 20 30)))");
let json = wkt.to_json()?;

println!("{json}");
assert_eq!(
&json[0..215],
r#"{
"type": "FeatureCollection",
"name": "countries",
"features": [{"type": "Feature", "properties": {"id": "DNK", "name": "Denmark"}, "geometry": {"type": "MultiPolygon", "coordinates": [[[[12.690006,55.609991],[12.0"#
&json,
r#"{"type": "MultiPolygon", "coordinates": [[[[40,40],[20,45],[45,30],[40,40]]],[[[35,10],[45,45],[15,40],[10,20],[35,10]],[[20,30],[35,35],[30,20],[20,30]]]]}"#
);
Ok(())
}

#[allow(dead_code)]
// #[tokio::test]
async fn http_fbg_to_json() -> Result<()> {
let url = "https://flatgeobuf.org/test/data/countries.fgb";
let mut fgb = HttpFgbReader::open(url)
.await?
.select_bbox(8.8, 47.2, 9.5, 55.3)
.await?;

let mut fout = BufWriter::new(File::create("countries.json")?);
let mut json = GeoJsonWriter::new(&mut fout);
fgb.process_features(&mut json).await?;

Ok(())
}
61 changes: 30 additions & 31 deletions geozero/tests/geozero-api.rs
Original file line number Diff line number Diff line change
@@ -1,8 +1,6 @@
use flatgeobuf::{FallibleStreamingIterator as _, FeatureProperties as _, FgbReader, GeometryType};
use geozero::error::Result as GeozeroResult;
use geozero::{ColumnValue, CoordDimensions, GeomProcessor, PropertyProcessor};
use seek_bufread::BufReader;
use std::fs::File;
use geozero::wkt::Wkt;
use geozero::{CoordDimensions, GeomProcessor, GeozeroGeometry};

struct VertexCounter(u64);

Expand All @@ -17,14 +15,13 @@ impl GeomProcessor for VertexCounter {

#[test]
fn vertex_counter() -> Result<()> {
let mut filein = BufReader::new(File::open("tests/data/countries.fgb")?);
let mut fgb = FgbReader::open(&mut filein)?.select_bbox(8.8, 47.2, 9.5, 55.3)?;
let feature = fgb.next()?.unwrap();
let geometry = feature.geometry().unwrap();
let wkt = Wkt("MULTIPOLYGON (((40 40, 20 45, 45 30, 40 40)),
((35 10, 45 45, 15 40, 10 20, 35 10),
(20 30, 35 35, 30 20, 20 30)))");

let mut vertex_counter = VertexCounter(0);
geometry.process(&mut vertex_counter, GeometryType::MultiPolygon)?;
assert_eq!(vertex_counter.0, 24);
wkt.process_geom(&mut vertex_counter)?;
assert_eq!(vertex_counter.0, 13);

Ok(())
}
Expand Down Expand Up @@ -60,22 +57,19 @@ impl GeomProcessor for MaxHeightFinder {
}

#[test]
#[ignore]
fn max_height_finder() -> Result<()> {
let mut filein = BufReader::new(File::open(
"tests/data/geoz_lod1_gebaeude_max_3d_extract.fgb",
)?);
pka marked this conversation as resolved.
Show resolved Hide resolved
let mut fgb = FgbReader::open(&mut filein)?.select_all()?;
let points = Wkt("MULTIPOINTZ (10 40 200.0, 40 30 150.0, 20 20 457.1, 30 10 0.0)");
let mut max_finder = MaxHeightFinder(0.0);
while let Some(feature) = fgb.next()? {
let geometry = feature.geometry().unwrap();
geometry.process(&mut max_finder, GeometryType::MultiPolygon)?;
}
points.process_geom(&mut max_finder)?;
assert_eq!(max_finder.0, 457.1);

Ok(())
}

/*
// Broken: this example does work with Flatgeobuf, but not GeoJson
// Disabled, since we don't want to have a circular dependency between flatgeobuf and geozero

struct FeatureFinder;

impl PropertyProcessor for FeatureFinder {
Expand All @@ -86,20 +80,25 @@ impl PropertyProcessor for FeatureFinder {

#[test]
fn feature_finder() -> Result<()> {
let mut filein = BufReader::new(File::open("tests/data/countries.fgb")?);
let mut fgb = FgbReader::open(&mut filein)?.select_all()?;
let mut filein = BufReader::new(File::open("tests/data/countries.geojson")?);
let mut json = GeoJsonLineReader::new(&mut filein);

let mut finder = FeatureFinder {};
while let Some(feature) = fgb.next()? {
let found = feature.process_properties(&mut finder);
if found.is_err() || found.unwrap() {
break;
}
}
let feature = fgb.cur_feature();
let props = feature.properties()?;
assert_eq!(props["id"], "DNK".to_string());
assert_eq!(props["name"], "Denmark".to_string());
// process_properties is not public. We should have feature iterators for all `GeozeroDatasource`s!
// json.process_properties(&mut finder);
//
// Using FgbReader:
// while let Some(feature) = fgb.next()? {
// let found = feature.process_properties(&mut finder);
// if found.is_err() || found.unwrap() {
// break;
// }
// }
// let feature = fgb.cur_feature();
// let props = feature.properties()?;
// assert_eq!(props["id"], "DNK".to_string());
// assert_eq!(props["name"], "Denmark".to_string());

Ok(())
}
*/
27 changes: 0 additions & 27 deletions geozero/tests/polylabel.rs

This file was deleted.

15 changes: 11 additions & 4 deletions geozero/tests/svg.rs
Original file line number Diff line number Diff line change
@@ -1,10 +1,6 @@
use flatgeobuf::{FgbReader, Header};
use geozero::geojson::GeoJsonReader;
use geozero::svg::SvgWriter;
use geozero::ProcessToSvg;
use seek_bufread::BufReader;
use std::fs::File;
use std::io::Write;
type Result<T> = std::result::Result<T, Box<dyn std::error::Error>>;

#[test]
Expand All @@ -23,6 +19,16 @@ fn json_to_svg() -> Result<()> {
Ok(())
}

/*
// FlatgeoBuf to SVG conversion
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you want to keep this, can you leave a comment for us-in-the-future explaining why it's here but commented out?

// Disabled, since we don't want to have a circular dependency between flatgeobuf and geozero


use flatgeobuf::{FgbReader, Header};
use geozero::svg::SvgWriter;
use seek_bufread::BufReader;
use std::io::Write;

fn invert_y(header: &Header) -> bool {
if let Some(crs) = header.crs() {
if crs.code() == 4326 {
Expand Down Expand Up @@ -69,3 +75,4 @@ fn fgb_to_svg() -> Result<()> {

Ok(())
}
*/
Loading