Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I've kept this as 3 commits for now, just so it's easier to see what changes I've made.
The first commit (88db807) is the main change, introducing a new
/data/importer/
folder which contains ahandlers
folder and animporters
folder. The newdata/importer/data/
is just a hook into the old/data/import/
code - I want to do more to move this around and separate it into two parts: data preprocessing and data storage, but it isn't necessary to get the rest of the import changes so I'm leaving this half-done for now.This restructure allows for the creation of other handlers and importers which manage other file types. I've done a lot of this locally, so I'm reasonably confident this refactor is the right direction.
The second change here removes the concept of versioning from importing. We don't actually use it at present and it just makes migrations more complex (see #4479)
Finally, I made a couple of changes to the data importer itself as it wasn't working for me properly. I found that if created_at wasn't set on the post, then an SQLITE error was being thrown but not passed all the way through (reproduce with this: https://gist.github.com/ErisDS/5eff62547f66177f8057). The changes in the 3rd commit both fix the underlying problem with errors not being passed through and also adds an extra line to the post importer which sets created_at if it is not already set (if it's set, it should be honoured, if it's not set, it should be considered to be now.
If I could get a bit of a review of these changes, that'd be grand, then I'll squash:)