include / auto include facility for .json data files

[SusanthCom]

Hi, @sintaxi @kennethormandy

// Proposed Feature Request

Thoughts

Few suggestion to make harp better in data management; especially large .json data files.
Large data files could be anything like user info, product info/price list, exported data from other software, database, spreadsheet ... so on.

As of now harp.json ( or _harp.json) and _data.json is storing all data / meta data.

Storing data in separate related files and then including them to harp.json ( or _harp.json) or _data.json helps the user to update / over write the related .json quickly (especially when exported from other source) without manually hunting the correct location and updating by manual edit.

The whole data update process could be easily automated by overwriting the required .json file and a publish command.

.cson support clubbed with this features makes things amazingly flexible and simple to use / manage / automate.

1. Include Facility

If an optional include facility is available in all .json file, it will provide a better way of organising data. Especially when the the data runs several hundreds of lines; it is easy to manage by storing in separate file rather than editing one big json file and scrolling through it to locate the data.

for example: file : harp.json

{
  "__include": "data/user-info.json", //  say: 50 users X  7 line  = 350 lines
  "__include": "data/some-other-data.json", // another data file that is big
  "__hide_json": true, // a settings flag to hide .json files even if not starting with _
  "globals": {
    "title": "Sample Title",
    "name"  : "Sample Name",
    "uri": "http://sample-url.com"
  }
}

The "__include" serve as a special key and its value points to the file, that should be merged with this .json file.

2. Auto Include Facility

As of now _data.json holds the whole data for all routes found inside a folder. This makes it again hard to manage if the data is huge and when different route within that folder depends on different set of data.

with the above include facility implemented, it will be easy to manage the above mentioned situation by just adding an include entry to _data.json.

To further enhance, auto including of .json file could be done, based on file name matching a route.

for example :
Route \blog\best-book-list.md should auto include the file \blog\_best-book-list.json ( has _ prefix ) or \blog\best-book-list.json ( NO _ prefix ) inside _data.json of that folder, if such a file exists. if both files exists, let _ version (_best-book-list.json) get the priority.

this helps to skip a manual entry to _data.json file. ( convention over configuration 😄 )

if there is a special global option like "__hide_json": true inside harp.json ( or _harp.json) or _data.json harp should not publish any .json file globally or locally (to that_data.json) in compiled output even if the name does not begin with an '_' (underscore).

This option helps to manage files more easily, since \blog\best-book-list.md and \blog\best-book-list.json will be listed near by (unlike _ prefixed one), when listed in sort by name order; there by easy switching between data and text.

Thank you