diff --git a/doc/decisions/database_plant_hierarchy.md b/doc/decisions/database_plant_hierarchy.md index cdb46ebff..a4b767ed2 100644 --- a/doc/decisions/database_plant_hierarchy.md +++ b/doc/decisions/database_plant_hierarchy.md @@ -34,6 +34,8 @@ See the [PSQL documentation](https://www.postgresql.org/docs/current/ddl-inherit > Table inheritance is typically established when the child table is created, using the INHERITS clause of the CREATE TABLE statement. +> Rust Diesel isn't intended for that. To only select data from a specific table, and not include all child tables, we would need to use the `FROM ONLY` keyword, which is not implemented in Rust Diesel. + So the inheritance is useful to deal with complex DDL structure on the startup, but will not help us to avoid bulk operations e.g. updating a column for every `variety` in the entire `genus` ### One table per taxonomy rank and one for concrete plants. @@ -71,7 +73,7 @@ Cons: Then we need to check for each column value if there is a higher rank that already defines the same value. Only if we can't find a match the value should be written. -### All ranks in one table. +### All ranks in one table [Example](example_migrations/normalized-plants-and-ranks) @@ -85,6 +87,46 @@ Cons: - Almost everything in the plants table needs to be nullable. - More complex insert and update logic. +### One table per taxonomy rank and one for concrete plants. + View and custom insert/update/delete functionality + +[Example](example_migrations/one-table-per-taxonomy-view-functions) + +It's similar to `One table for taxonomy ranks and one for concrete plants` We are extending it with a view and custom functions to reduce insert and update complexity in the backend and scraper. + +#### compared to [All ranks in one table](#all-ranks-in-one-table) + +Could allow us to override properties of a plant, but only under certain conditions: + +- We still need to implement a view and custom insert/update events. + Without these, we wouldn't be able to distinguish between a custom overwrite and a standard value. + This limitation makes property overrides on every level not possible. + +- There are a couple of possible approaches for FKs: + + - Each plant has multiple foreign keys representing family, genus, and species. + However, this would often lead to empty fields and potential confusion, especially when the genus of a plant and its own species are linked to a different genus. + + - Alternatively, a plant could have a single foreign key representing a plant in a higher-order category, from which it would inherit properties. However, this approach could lead to: + a) Circular references, where plants inherit from each other indefinitely. + b) Extremely long inheritance chains, with a single plant inheriting properties from numerous others. + +Pros: + +- Inserting new plants is easy. We only need to implement minor backend changes. +- Properties overrides can be done on every level. +- It's the only proposed solution that allows us to override the properties of a plant without significantly increasing the complexity of the backend. + +Cons: + +- More complex insert and update logic. + When a species/variety is added or updated, the columns can't just be set. + First, we need to make sure all higher levels are in the table. + Then we need to check for each column value if there is a higher rank that already defines the same value. + Only if we can't find a match, the value should be written. + - We can offset this issue by implementing insert/update functions. + Since they are going to be complicated, long-term maintainability may be an issue. +- Almost everything in the plants and parent tables needs to be nullable. (Is this a downside?) + ## Decision - We go with the last option "All ranks in one table" as described [in our documentation](../database/hierarchy.md). diff --git a/doc/decisions/example_migrations/one-table-per-taxonomy-view-functions/2023-03-09-194135_plant_relations/README.md b/doc/decisions/example_migrations/one-table-per-taxonomy-view-functions/2023-03-09-194135_plant_relations/README.md new file mode 100644 index 000000000..02dc8ac17 --- /dev/null +++ b/doc/decisions/example_migrations/one-table-per-taxonomy-view-functions/2023-03-09-194135_plant_relations/README.md @@ -0,0 +1 @@ +# example diff --git a/doc/decisions/example_migrations/one-table-per-taxonomy-view-functions/2023-03-09-194135_plant_relations/down.sql b/doc/decisions/example_migrations/one-table-per-taxonomy-view-functions/2023-03-09-194135_plant_relations/down.sql new file mode 100644 index 000000000..3a50ca7a9 --- /dev/null +++ b/doc/decisions/example_migrations/one-table-per-taxonomy-view-functions/2023-03-09-194135_plant_relations/down.sql @@ -0,0 +1,9 @@ +-- This file should undo anything in `up.sql` +-- since this is WIP, i will keep this empty for now. +DROP TABLE cultivars CASCADE; +DROP TABLE varieties CASCADE; +DROP TABLE species CASCADE; +DROP TABLE genera CASCADE; +DROP TABLE families CASCADE; + +DROP VIEW plants_view; diff --git a/doc/decisions/example_migrations/one-table-per-taxonomy-view-functions/2023-03-09-194135_plant_relations/test_data.sql b/doc/decisions/example_migrations/one-table-per-taxonomy-view-functions/2023-03-09-194135_plant_relations/test_data.sql new file mode 100644 index 000000000..3647f7b9e --- /dev/null +++ b/doc/decisions/example_migrations/one-table-per-taxonomy-view-functions/2023-03-09-194135_plant_relations/test_data.sql @@ -0,0 +1,536 @@ +-- Insert into family +INSERT INTO families (id, name, property1) VALUES ( + 1, 'Family1', 'Family1Property1' +); +INSERT INTO families (id, name, property1) VALUES (2, 'Family2', null); + +-- Insert into genus +INSERT INTO genera (id, name, property1, family_id) VALUES ( + 1, 'Genus1', 'Genus1Property1', 1 +); +INSERT INTO genera (id, name, property1, family_id) VALUES ( + 2, 'Genus2', null, 1 +); +INSERT INTO genera (id, name, property1, family_id) VALUES ( + 3, 'Genus3', 'Genus3Property1', 2 +); +INSERT INTO genera (id, name, property1, family_id) VALUES ( + 4, 'Genus4', null, 2 +); + +-- Insert into species +INSERT INTO species (id, name, property1, genus_id) VALUES ( + 1, 'Species1', 'Species1Property1', 1 +); +INSERT INTO species (id, name, property1, genus_id) VALUES ( + 2, 'Species2', null, 1 +); +INSERT INTO species (id, name, property1, genus_id) VALUES ( + 3, 'Species3', 'Species3Property1', 2 +); +INSERT INTO species (id, name, property1, genus_id) VALUES ( + 4, 'Species4', null, 2 +); + +INSERT INTO species (id, name, property1, genus_id) VALUES ( + 5, 'Species5', 'Species5Property1', 3 +); +INSERT INTO species (id, name, property1, genus_id) VALUES ( + 6, 'Species6', null, 3 +); +INSERT INTO species (id, name, property1, genus_id) VALUES ( + 7, 'Species7', 'Species7Property1', 4 +); +INSERT INTO species (id, name, property1, genus_id) VALUES ( + 8, 'Species8', null, 4 +); + +-- Insert into variety +INSERT INTO varieties ( + id, name, unique_name, common_name_en, common_name_de, property1, species_id +) VALUES ( + 1, + 'Variety1', + 'UniqueVariety1', + ARRAY['CommonEn1'], + ARRAY['CommonDe1'], + 'Variety1Property1', + 1 +); +INSERT INTO varieties ( + id, name, unique_name, common_name_en, common_name_de, property1, species_id +) VALUES (2, 'Variety2', 'UniqueVariety2', null, null, null, 1); +INSERT INTO varieties ( + id, name, unique_name, common_name_en, common_name_de, property1, species_id +) VALUES ( + 3, + 'Variety3', + 'UniqueVariety3', + ARRAY['CommonEn3'], + ARRAY['CommonDe3'], + 'Variety3Property1', + 2 +); +INSERT INTO varieties ( + id, name, unique_name, common_name_en, common_name_de, property1, species_id +) VALUES (4, 'Variety4', 'UniqueVariety4', null, null, null, 2); + +INSERT INTO varieties ( + id, name, unique_name, common_name_en, common_name_de, property1, species_id +) VALUES ( + 5, + 'Variety5', + 'UniqueVariety5', + ARRAY['CommonEn5'], + ARRAY['CommonDe5'], + 'Variety5Property1', + 3 +); +INSERT INTO varieties ( + id, name, unique_name, common_name_en, common_name_de, property1, species_id +) VALUES (6, 'Variety6', 'UniqueVariety6', null, null, null, 3); +INSERT INTO varieties ( + id, name, unique_name, common_name_en, common_name_de, property1, species_id +) VALUES ( + 7, + 'Variety7', + 'UniqueVariety7', + ARRAY['CommonEn7'], + ARRAY['CommonDe7'], + 'Variety7Property1', + 4 +); +INSERT INTO varieties ( + id, name, unique_name, common_name_en, common_name_de, property1, species_id +) VALUES (8, 'Variety8', 'UniqueVariety8', null, null, null, 4); + +INSERT INTO varieties ( + id, name, unique_name, common_name_en, common_name_de, property1, species_id +) VALUES ( + 9, + 'Variety9', + 'UniqueVariety9', + ARRAY['CommonEn9'], + ARRAY['CommonDe9'], + 'Variety9Property1', + 5 +); +INSERT INTO varieties ( + id, name, unique_name, common_name_en, common_name_de, property1, species_id +) VALUES (10, 'Variety10', 'UniqueVariety10', null, null, null, 5); +INSERT INTO varieties ( + id, name, unique_name, common_name_en, common_name_de, property1, species_id +) VALUES ( + 11, + 'Variety11', + 'UniqueVariety11', + ARRAY['CommonEn11'], + ARRAY['CommonDe11'], + 'Variety11Property1', + 6 +); +INSERT INTO varieties ( + id, name, unique_name, common_name_en, common_name_de, property1, species_id +) VALUES (12, 'Variety12', 'UniqueVariety12', null, null, null, 6); + +INSERT INTO varieties ( + id, name, unique_name, common_name_en, common_name_de, property1, species_id +) VALUES ( + 13, + 'Variety13', + 'UniqueVariety13', + ARRAY['CommonEn13'], + ARRAY['CommonDe13'], + 'Variety13Property1', + 7 +); +INSERT INTO varieties ( + id, name, unique_name, common_name_en, common_name_de, property1, species_id +) VALUES (14, 'Variety14', 'UniqueVariety14', null, null, null, 7); +INSERT INTO varieties ( + id, name, unique_name, common_name_en, common_name_de, property1, species_id +) VALUES ( + 15, + 'Variety15', + 'UniqueVariety15', + ARRAY['CommonEn15'], + ARRAY['CommonDe15'], + 'Variety15Property1', + 8 +); +INSERT INTO varieties ( + id, name, unique_name, common_name_en, common_name_de, property1, species_id +) VALUES (16, 'Variety16', 'UniqueVariety16', null, null, null, 8); + +-- Insert into cultivar +INSERT INTO cultivars ( + unique_name, + common_name_en, + common_name_de, + property1, + species_id, + variety_id +) VALUES ( + 'UniqueCultivar1', + ARRAY['CommonEnC1'], + ARRAY['CommonDeC1'], + 'Cultivar1Property1', + 1, + null +); +INSERT INTO cultivars ( + unique_name, + common_name_en, + common_name_de, + property1, + species_id, + variety_id +) VALUES ('UniqueCultivar2', null, null, null, 1, null); +INSERT INTO cultivars ( + unique_name, + common_name_en, + common_name_de, + property1, + species_id, + variety_id +) VALUES ( + 'UniqueCultivar3', + ARRAY['CommonEnC3'], + ARRAY['CommonDeC3'], + 'Cultivar3Property1', + 2, + null +); +INSERT INTO cultivars ( + unique_name, + common_name_en, + common_name_de, + property1, + species_id, + variety_id +) VALUES ('UniqueCultivar4', null, null, null, 2, null); +INSERT INTO cultivars ( + unique_name, + common_name_en, + common_name_de, + property1, + species_id, + variety_id +) VALUES ( + 'UniqueCultivar5', + ARRAY['CommonEnC5'], + ARRAY['CommonDeC5'], + 'Cultivar5Property1', + null, + 3 +); +INSERT INTO cultivars ( + unique_name, + common_name_en, + common_name_de, + property1, + species_id, + variety_id +) VALUES ('UniqueCultivar6', null, null, null, null, 3); +INSERT INTO cultivars ( + unique_name, + common_name_en, + common_name_de, + property1, + species_id, + variety_id +) VALUES ( + 'UniqueCultivar7', + ARRAY['CommonEnC6'], + ARRAY['CommonDeC6'], + 'Cultivar7Property1', + null, + 4 +); +INSERT INTO cultivars ( + unique_name, + common_name_en, + common_name_de, + property1, + species_id, + variety_id +) VALUES ('UniqueCultivar8', null, null, null, null, 4); + +INSERT INTO cultivars ( + unique_name, + common_name_en, + common_name_de, + property1, + species_id, + variety_id +) VALUES ( + 'UniqueCultivar9', + ARRAY['CommonEnC9'], + ARRAY['CommonDeC9'], + 'Cultivar9Property1', + 1, + null +); +INSERT INTO cultivars ( + unique_name, + common_name_en, + common_name_de, + property1, + species_id, + variety_id +) VALUES ('UniqueCultivar10', null, null, null, 1, null); +INSERT INTO cultivars ( + unique_name, + common_name_en, + common_name_de, + property1, + species_id, + variety_id +) VALUES ( + 'UniqueCultivar11', + ARRAY['CommonEnC11'], + ARRAY['CommonDeC11'], + 'Cultivar11Property1', + 2, + null +); +INSERT INTO cultivars ( + unique_name, + common_name_en, + common_name_de, + property1, + species_id, + variety_id +) VALUES ('UniqueCultivar12', null, null, null, 2, null); +INSERT INTO cultivars ( + unique_name, + common_name_en, + common_name_de, + property1, + species_id, + variety_id +) VALUES ( + 'UniqueCultivar13', + ARRAY['CommonEnC13'], + ARRAY['CommonDeC13'], + 'Cultivar13Property1', + null, + 3 +); +INSERT INTO cultivars ( + unique_name, + common_name_en, + common_name_de, + property1, + species_id, + variety_id +) VALUES ('UniqueCultivar14', null, null, null, null, 3); +INSERT INTO cultivars ( + unique_name, + common_name_en, + common_name_de, + property1, + species_id, + variety_id +) VALUES ( + 'UniqueCultivar15', + ARRAY['CommonEnC15'], + ARRAY['CommonDeC15'], + 'Cultivar15Property1', + null, + 4 +); +INSERT INTO cultivars ( + unique_name, + common_name_en, + common_name_de, + property1, + species_id, + variety_id +) VALUES ('UniqueCultivar16', null, null, null, null, 4); + +INSERT INTO cultivars ( + unique_name, + common_name_en, + common_name_de, + property1, + species_id, + variety_id +) VALUES ( + 'UniqueCultivar17', + ARRAY['CommonEnC17'], + ARRAY['CommonDeC17'], + 'Cultivar17Property1', + 1, + null +); +INSERT INTO cultivars ( + unique_name, + common_name_en, + common_name_de, + property1, + species_id, + variety_id +) VALUES ('UniqueCultivar18', null, null, null, 1, null); +INSERT INTO cultivars ( + unique_name, + common_name_en, + common_name_de, + property1, + species_id, + variety_id +) VALUES ( + 'UniqueCultivar19', + ARRAY['CommonEnC19'], + ARRAY['CommonDeC19'], + 'Cultivar19Property1', + 2, + null +); +INSERT INTO cultivars ( + unique_name, + common_name_en, + common_name_de, + property1, + species_id, + variety_id +) VALUES ('UniqueCultivar20', null, null, null, 2, null); +INSERT INTO cultivars ( + unique_name, + common_name_en, + common_name_de, + property1, + species_id, + variety_id +) VALUES ( + 'UniqueCultivar21', + ARRAY['CommonEnC21'], + ARRAY['CommonDeC21'], + 'Cultivar21Property1', + null, + 3 +); +INSERT INTO cultivars ( + unique_name, + common_name_en, + common_name_de, + property1, + species_id, + variety_id +) VALUES ('UniqueCultivar22', null, null, null, null, 3); +INSERT INTO cultivars ( + unique_name, + common_name_en, + common_name_de, + property1, + species_id, + variety_id +) VALUES ( + 'UniqueCultivar23', + ARRAY['CommonEnC23'], + ARRAY['CommonDeC23'], + 'Cultivar23Property1', + null, + 4 +); +INSERT INTO cultivars ( + unique_name, + common_name_en, + common_name_de, + property1, + species_id, + variety_id +) VALUES ('UniqueCultivar24', null, null, null, null, 4); + +INSERT INTO cultivars ( + unique_name, + common_name_en, + common_name_de, + property1, + species_id, + variety_id +) VALUES ( + 'UniqueCultivar25', + ARRAY['CommonEnC25'], + ARRAY['CommonDeC25'], + 'Cultivar25Property1', + 1, + null +); +INSERT INTO cultivars ( + unique_name, + common_name_en, + common_name_de, + property1, + species_id, + variety_id +) VALUES ('UniqueCultivar26', null, null, null, 1, null); +INSERT INTO cultivars ( + unique_name, + common_name_en, + common_name_de, + property1, + species_id, + variety_id +) VALUES ( + 'UniqueCultivar27', + ARRAY['CommonEnC27'], + ARRAY['CommonDeC27'], + 'Cultivar27Property1', + 2, + null +); +INSERT INTO cultivars ( + unique_name, + common_name_en, + common_name_de, + property1, + species_id, + variety_id +) VALUES ('UniqueCultivar28', null, null, null, 2, null); +INSERT INTO cultivars ( + unique_name, + common_name_en, + common_name_de, + property1, + species_id, + variety_id +) VALUES ( + 'UniqueCultivar29', + ARRAY['CommonEnC29'], + ARRAY['CommonDeC29'], + 'Cultivar29Property1', + null, + 3 +); +INSERT INTO cultivars ( + unique_name, + common_name_en, + common_name_de, + property1, + species_id, + variety_id +) VALUES ('UniqueCultivar30', null, null, null, null, 3); +INSERT INTO cultivars ( + unique_name, + common_name_en, + common_name_de, + property1, + species_id, + variety_id +) VALUES ( + 'UniqueCultivar31', + ARRAY['CommonEnC31'], + ARRAY['CommonDeC31'], + 'Cultivar31Property1', + null, + 4 +); +INSERT INTO cultivars ( + unique_name, + common_name_en, + common_name_de, + property1, + species_id, + variety_id +) VALUES ('UniqueCultivar32', null, null, null, null, 4); diff --git a/doc/decisions/example_migrations/one-table-per-taxonomy-view-functions/2023-03-09-194135_plant_relations/test_scrips.sql b/doc/decisions/example_migrations/one-table-per-taxonomy-view-functions/2023-03-09-194135_plant_relations/test_scrips.sql new file mode 100644 index 000000000..cc43afebb --- /dev/null +++ b/doc/decisions/example_migrations/one-table-per-taxonomy-view-functions/2023-03-09-194135_plant_relations/test_scrips.sql @@ -0,0 +1,13 @@ +SELECT + id, + unique_name, + common_name_de, + common_name_en, + variety_name, + family_name, + genus_name, + species_name, + property1 +FROM plants_view; + +--todo insert, update, delete diff --git a/doc/decisions/example_migrations/one-table-per-taxonomy-view-functions/2023-03-09-194135_plant_relations/up.sql b/doc/decisions/example_migrations/one-table-per-taxonomy-view-functions/2023-03-09-194135_plant_relations/up.sql new file mode 100644 index 000000000..491ce648d --- /dev/null +++ b/doc/decisions/example_migrations/one-table-per-taxonomy-view-functions/2023-03-09-194135_plant_relations/up.sql @@ -0,0 +1,165 @@ +CREATE SEQUENCE plants_id_seq +AS INTEGER +START WITH 1 +INCREMENT BY 1 +NO MINVALUE +NO MAXVALUE +CACHE 1; + +-- Create the "family" table +CREATE TABLE families ( + id INTEGER PRIMARY KEY, + name VARCHAR NOT NULL, + property1 TEXT, + CONSTRAINT family_name_key UNIQUE (name) +); + +-- Create the "genus" table +CREATE TABLE genera ( + id INTEGER PRIMARY KEY, + name VARCHAR NOT NULL, + property1 TEXT, + family_id INTEGER REFERENCES families (id), + CONSTRAINT genus_name_key UNIQUE (name) +); + +-- Create the "species" table +CREATE TABLE species ( + id INTEGER PRIMARY KEY, + name VARCHAR NOT NULL, + property1 TEXT, + genus_id INTEGER REFERENCES genera (id), + CONSTRAINT species_name_key UNIQUE (name) +); + +-- Create the "variety" table +CREATE TABLE varieties ( + id INTEGER DEFAULT nextval('plants_id_seq') PRIMARY KEY, + name VARCHAR NOT NULL, + unique_name TEXT NOT NULL, + common_name_en TEXT [], + common_name_de TEXT [], + property1 TEXT, + species_id INTEGER REFERENCES species (id), + CONSTRAINT variety_name_key UNIQUE (unique_name), + CONSTRAINT variety_name_key_rename UNIQUE (name) +); + +-- Create the "cultivar" table +CREATE TABLE cultivars ( + id INTEGER DEFAULT nextval('plants_id_seq') PRIMARY KEY, + unique_name TEXT NOT NULL, + common_name_en TEXT [], + common_name_de TEXT [], + property1 TEXT, + species_id INTEGER REFERENCES species (id), + variety_id INTEGER REFERENCES varieties (id), + CONSTRAINT unique_name_key UNIQUE (unique_name), + CONSTRAINT check_variety_or_species CHECK ( + variety_id IS NULL OR species_id IS NULL + ) +); + +-- Create a view joining the tables together +-- COALESCE function accepts an unlimited number of arguments. +-- It returns the first argument that is not null. + +--todo +-- test cases vorbereiten wie select, insert, update +-- insert, update, delete schreiben. +CREATE OR REPLACE VIEW plants_view AS +SELECT + p.id, + p.unique_name, + p.common_name_en, + p.common_name_de, + v.name AS variety_name, + f.name AS family_name, + g.name AS genus_name, + s.name AS species_name, + coalesce( + p.property1, + v.property1, + s.property1, + g.property1, + f.property1 + ) AS property1 +FROM ( + SELECT + id, + unique_name, + common_name_en, + common_name_de, + property1, + species_id, + id AS variety_id + FROM varieties + UNION + SELECT + id, + unique_name, + common_name_en, + common_name_de, + property1, + species_id, + variety_id + FROM cultivars +) AS p +LEFT JOIN varieties AS v ON p.variety_id = v.id +LEFT JOIN species AS s ON p.species_id = s.id +LEFT JOIN genera AS g ON s.genus_id = g.id +LEFT JOIN families AS f ON g.family_id = f.id; + + + + + +CREATE OR REPLACE FUNCTION insert_plant_view_placeholder() +RETURNS TRIGGER AS $$ +BEGIN + -- Placeholder function, no implementation provided. + -- will insert a new item and only fill columns, if they differ from parent tables. + + -- NEW contains ALL values i have selected in the view, so this should work perfectly :) + RETURN NEW; +END; +$$ +LANGUAGE plpgsql; + +CREATE TRIGGER insert_plant_view_trigger +INSTEAD OF INSERT ON plants_view +FOR EACH ROW +EXECUTE FUNCTION insert_plant_view_placeholder(); + + + +CREATE OR REPLACE FUNCTION update_plant_view_placeholder() +RETURNS TRIGGER AS $$ +BEGIN + -- Placeholder function, no implementation provided. + -- will only update values in plantDetails if they differ from a parent table. + RETURN NEW; +END; +$$ +LANGUAGE plpgsql; + +CREATE TRIGGER update_plant_view_trigger +INSTEAD OF UPDATE ON plants_view +FOR EACH ROW +EXECUTE FUNCTION update_plant_view_placeholder(); + + +CREATE OR REPLACE FUNCTION delete_plant_view_placeholder() +RETURNS TRIGGER AS $$ +BEGIN + -- Placeholder function, no implementation provided. + DELETE FROM cultivar WHERE id = OLD.id; + DELETE FROM variety WHERE id = OLD.id; +END; +$$ +LANGUAGE plpgsql; + +CREATE TRIGGER delete_plant_view_trigger +INSTEAD OF DELETE ON plants_view +FOR EACH ROW +EXECUTE FUNCTION delete_plant_view_placeholder(); diff --git a/doc/decisions/example_migrations/one-table-per-taxonomy-view-functions/README.md b/doc/decisions/example_migrations/one-table-per-taxonomy-view-functions/README.md new file mode 100644 index 000000000..6531921f7 --- /dev/null +++ b/doc/decisions/example_migrations/one-table-per-taxonomy-view-functions/README.md @@ -0,0 +1 @@ +# One Table Per Taxonomy + View + Custom Insert, Update, Delete functions