Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support arbitrary property lookup for creating "resource" type values #162

Closed
zerocrates opened this issue Jun 26, 2019 · 16 comments · May be fixed by #171
Closed

Support arbitrary property lookup for creating "resource" type values #162

zerocrates opened this issue Jun 26, 2019 · 16 comments · May be fixed by #171

Comments

@zerocrates
Copy link
Contributor

We have arbitrary-property lookup in the module already, for finding the parent item for a media row, or finding a set to put an item row into.

We can use the same lookup functionality to allow users to create "resource" type values (internal links). Currently this only works via looking up the Omeka S internal ID for a resource, which means an extra step of doing separate imports and correlating IDs.

@pbcGIS
Copy link

pbcGIS commented Jul 19, 2019

In case the title of this issue is not clear, an alternate description: It would be useful if a new item created by this module could include among its properties a link to another item. Currently there is an option to link a new item to to an existing item-set. In the existing function, the item-set is itentified by one of its properties using the Resource-Data dropdown.

It would be great to have an additional option in this same Resource-Data dropdown that could specify an Item identifier. The result would be that the new item created by a row in the CSV file would be associated with an existing item as a linked resource.

For example: My Omeka-s site has a lot of items of class: Location each of these describes a building, address or geographic feature. Now I create a CSV import table that creates a bunch of items of class: Document. Each document may be related to a location. The resource template for documents includes a property named "Location" that accepts an existing Omeka item as a linked resource. The CSV table has a column named "Location" which, for each row includes a value which is the DublinCore:Identifier for one of my existing location items. The new item created by my CSV table includes a link to an existing Location item, and the Location item includes a reference to the new document item in its list of Linked Resources.

I believe that this feature could be added to the UI and documentation for the CSV import tool very simply (I think, if the "Item Set (by selected Property) also allowed items to be referred to. It would be up to the user to assure that the property specified contained unique references. In case of a non-unique property, the operation could link to the most recent item, similar to the Advanced Update function, or perhaps it would link to any item having that property.

linked_resource

This would be a very powerful feature for using the CSV import tool to build semantic networks. Thank you.

Daniel-KM added a commit to Daniel-KM/Omeka-S-module-CSVImport that referenced this issue Dec 2, 2019
@Daniel-KM
Copy link
Contributor

The feature is added in #171 / #172 . The linked resource are not added in the image above, but in the data type of the column (wrench). The property is defined in the tab "basic".

Note: It is no more possible to import multiple rows in batch, since the identifiers are created and checked by row. Maybe a future improvement will be to load and check all the csv first.

Daniel-KM added a commit to Daniel-KM/Omeka-S-module-CSVImport that referenced this issue Dec 9, 2019
@zerocrates
Copy link
Contributor Author

I realize now I haven't actually commented on the several associated PRs here, so I'll fix that now.

I don't think I'm in favor of the general system that's used here, where the existing property setting used for lookup for updates gets overloaded for use in this case also (and therefore promoted from Advanced to Basic). I've always envisioned this feature as working through an additional option in the "wrench" menu that becomes available when you select a "resource" data type, so it could be different for every column.

@pbcGIS
Copy link

pbcGIS commented Jul 14, 2022

Dear Zerocrates
I am setting up an application for a client that requires this "Link-to-item" capability. I would like some advice on how I might approach having this capability added to the Omeka team supported CSV Import Plugin. We have money to pay for development but would like specific advice about the Omeka team's design concerns.
We are also looking for a way to save or hard-code mappings for re-use.
Thank you. Our pilot app: CDASH: Cambridge Digital Architectural Survey and History

@zerocrates
Copy link
Contributor Author

@pbcGIS It's an interesting looking site that you have there.

Can you clarify exactly what the client's looking to do, just so we make sure we're all on the same page that this feature is required for that?

Saving of mappings has been a vague wish-list item for a while also: a few things save but we kept out some partial "saving" code from prior versions with the idea just a "save" and "apply" type system that could set all the options would be better. I've imagined this as something that would just save JSON for the form state and then would possibly reapply it client-side with JS when a user wants to reuse it.

@pbcGIS

This comment was marked as outdated.

@pbcGIS
Copy link

pbcGIS commented Jul 28, 2022

Apologies for awkward novice git-hub etiquette. Criticism is welcome.
The client's collection includes items regarding places (e.g. buildings, monuments) that have locations; and items that reflect documents related to place items. Our workflow involves creating thousands of these in batches. At the time of creation, there is no way of knowing the internal ID of the place items. It is difficult to learn/use internal IDs even after the items are created. For more info about the item creation workflow you can look at our slide show.
I would be happy to help modify Daniel's design and pull request if you advise.
I've created another modification to save and re-load mappings that I could submit as another issue or pull request if this would be helpful for the community.
Your good work is very much appreciated!

@zerocrates
Copy link
Contributor Author

My main concern with the previous PR was that it worked essentially "globally," it introduced a setting that changed how all "link" lookups for the import would be made (if I'm remembering it correctly). Instead, it seemed to me that this should be something that was a property of the mapping (so it could vary from column to column: one might map to dcterms:identifier, another to title, and so on, within the same import). I think the main question on that kind of feature was really more of a UI one, as in where we would unobtrusively put such an option in the mapping interface.

The "global" version does sidestep some of those problems, so it'd also be good to know if that kind of "advanced setting" that just changes all the link mappings to use a single specified property instead of the resource ID to look up would be sufficient for your needs.

We have made some small changes more recently to make getting at the resource IDs easier (for one, they're displayed in more places), but that obviously doesn't eliminate the usefulness of this feature.

As for the save and reload of mappings: that should definitely be discussed on its own issue or PR, so I don't want to get too much into it here. I would say there's definite interest in that kind of feature, though I'm not totally sure about the current method you're using from a quick look at the branch.

@pbcGIS
Copy link

pbcGIS commented Aug 5, 2022

Thank you for the clarification. Would the approach illustrated in the 2nd message in this thread address your concern? That approach would put a second property drop-down under the Item-Specific Data panel of the mapping form. So the procedure for mapping any given column in the source table as a foreign key linking to another item (just as the existing one does for item sets.

This may be straight-forward to implement by repeating an existing pattern and also more economical in terms of documentation. In fact, if we were willing to assume that a project that implements this feature would take care to not repeat identifiers between Item Sets and Items, maybe the item linking capability could be implemented using the existing pull-down, and the link would be made to the most recently created resource having the value for the specified property matching the value of the field in the source. This modification would not change the user interface, except for the explanatory tool-tips. And the impact on the documentation would be an extra sentence or two in the explanation of that item-specific data panel.

The following idea may be a stray tangent: I wonder if it makes sense to conceptualize item sets as ordinary items that are targeted by relations from other items. The reason that I designed CDASH Place Items this way is that it allows this type of item to serve the function of item set, but also having the possibility of having a location and associated media of its own. This approach to modeling item sets as items seems to hew closely to simple relational architecture. Note that adopting this idea does not require any change to the architecture of Omeka.

Thanks again. I can't promise that I have the skills to implement this, but agreeing on a design goal would be a useful step in in the process.

@zerocrates
Copy link
Contributor Author

zerocrates commented Aug 11, 2022

The second message in this thread, meaning your post with the screenshot?

I'm not quite sure about your description of what you planned exactly, but using that dropdown is what I'm thinking of basically. You can't just put a second dropdown under the Item Set dropdown in Item-specific data though. Because these linkages are specific to a property, the choice needs to be something that comes along with also choosing a property. I guess you could just "map" it as a separate thing that doesn't do anything on its own, but we don't generally do that. Further, it only applies when you're mapping to a "resource" data type, an option you set in the "wrench" menu. One final little issue: this should also work for other resource types so it really shouldn't be in the "Item-specific" area.

So where this should probably end up is inside the "options" sidebar you get to by clicking the wrench icon for a column. That's where other things that just affect the property mappings in particular go, so it's the right place probably.

zerocrates added a commit that referenced this issue Aug 15, 2023
This allows the "Omeka property" data type to refer to data in the sheet
that's a value of a property of an existing resource, rather than having
to have the resource's internal Omeka S ID in the sheet. Usual use-case
is using an external identifier.

(#162)
@zerocrates
Copy link
Contributor Author

I have an implementation of the "option-based" solution on the branch "resource-identifier-property".

It adds a new option named "Resource identifier property" to the options sidebar. Note that this option only takes effect if you're using the "Omeka resource" data type for the column also.

@pbcGIS
Copy link

pbcGIS commented Aug 16, 2023 via email

@zerocrates
Copy link
Contributor Author

Sure, testing from people who are interested is always welcome.

The plan is definitely to include this in a CSV Import release; we'll see how testing and so on proceeds. I don't anticipate needing major changes, but some things around the UI/UX might get adjusted somewhat.

As for saving settings: we should probably constrain the conversation here to just the linking identifier issue, and make a new one for saving/loading of settings, but I'll respond quickly here:

So you're just saving the job args and using them directly if passed, skipping the mapping entirely? I can see that would work, sure. My thinking with my more complex version is that I want to make it easy to take saved settings and modify them, and/or visually confirm they're what you want them to be. The way you're doing it is much more straightforward though in terms of implementation as it doesn't have to touch the kind of esoteric UI and scripting bits of the mapping interface, so I can see why it's appealing. I don't think we'll take that route, though. In terms of something small-ish you could do within your system, you could modify Past Imports to have a link to just download the args as a file, avoiding the copy/paste step: it should be a pretty small action to just read the args from the linked job, and you could even tack on a Content-Disposition header so the link will download rather than open for viewing.

@zerocrates
Copy link
Contributor Author

The "option-based" solution here is now done and included in the new CSV Import 2.6.0.

@pbcGIS
Copy link

pbcGIS commented Jan 25, 2024 via email

@zerocrates
Copy link
Contributor Author

Just a note, it'll take a bit (until tomorrow) for the website listings and docs and stuff to be updated for 2.6.0.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants