Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(docs): enrich documents with metadata #457

Merged
merged 82 commits into from
Aug 25, 2022

Conversation

ffakenz
Copy link
Contributor

@ffakenz ffakenz commented Aug 11, 2022

Fixes #456

@github-actions
Copy link

github-actions bot commented Aug 11, 2022

Unit Test Results

247 tests  ±0   241 ✔️ ±0   15m 17s ⏱️ +53s
  89 suites ±0       6 💤 ±0 
    5 files   ±0       0 ±0 

Results for commit 6a47383. ± Comparison against base commit e293d36.

♻️ This comment has been updated with latest results.

@github-actions
Copy link

github-actions bot commented Aug 11, 2022

Transactions Costs

Sizes and execution budgets for Hydra protocol transactions. Note that unlisted parameters are currently using arbitrary values and results are not fully deterministic and comparable to previous runs.

Metadata
Generated at 2022-08-25 08:38:28.580655508 UTC
Max. memory units 14000000
Max. CPU units 10000000000
Max. tx size (kB) 16384

Cost of Init Transaction

Parties Tx size % max Mem % max CPU
1 4833 10.14 4.01
2 5036 13.38 5.30
3 5242 14.92 5.88
5 5652 18.51 7.25
10 6676 27.18 10.56
46 14059 99.94 38.63

Cost of Commit Transaction

Currently only one UTxO per commit allowed (this is about to change soon)

UTxO Tx size % max Mem % max CPU
1 5771 19.85 8.02

Cost of CollectCom Transaction

Parties Tx size % max Mem % max CPU
1 13136 21.44 8.60
2 13388 36.59 14.82
3 13701 55.12 22.49
4 13985 76.00 31.17

Cost of Close Transaction

Parties Tx size % max Mem % max CPU
1 9361 8.63 3.53
2 9490 8.97 3.81
3 9690 10.14 4.42
5 9983 11.31 5.17
10 10879 16.04 7.75
30 14217 32.26 17.02
71 16308 42.04 15.91

Cost of Contest Transaction

Parties Tx size % max Mem % max CPU
1 9328 8.16 3.34
2 9493 8.94 3.79
3 9685 10.16 4.42
5 9991 11.28 5.15
10 10841 15.62 7.57
30 14189 32.02 16.92
43 16359 42.74 23.01

Cost of Abort Transaction

Parties Tx size % max Mem % max CPU
1 13495 22.50 9.35
2 14106 44.21 19.55
3 13786 49.11 20.72
4 14535 85.03 38.19
5 14359 95.27 41.49

Cost of FanOut Transaction

Involves spending head output and burning head tokens. Uses ada-only UTxO for better comparability.

UTxO Tx size % max Mem % max CPU
1 13484 10.26 4.46
2 13520 12.08 5.46
3 13620 13.68 6.37
5 13563 15.96 7.83
10 13871 24.63 12.64
50 15314 85.56 47.87
60 15538 99.74 56.26

@ffakenz
Copy link
Contributor Author

ffakenz commented Aug 12, 2022

From
image
To
image

```mdx-code-block
<DocumentMetadata
lastUpdatedAt="{{last-updated-at}}"
authorInfo="{{author-info}}"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why the author info 🤔 ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no real good answer for this.
removing 🏃

*
* This source code is licensed under the MIT license found in the
* LICENSE file in the root directory of this source tree.
*/
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't give your code to Facebook 🙃

*
* This source code is licensed under the MIT license found in the
* LICENSE file in the root directory of this source tree.
*/
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

☝️

}: Props): JSX.Element {
return (
<div className={styles.block}>
<i className={styles.info}>Last updated at: <b>{lastUpdatedAt}</b></i>
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about we make those:

  • Links to the actual commit (visualized on Github)
  • Relative times (e.g. 3 days ago)

Copy link
Contributor Author

@ffakenz ffakenz Aug 12, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

New looking style
image
wdyt?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about not showing the date at all, and making the '2 minutes ago' the link 😅? We should have done that while pairing I guess, this back and forth feels unproductive. Requirements unclear. Sorry.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Other than that, I think it's amazing :D

@ffakenz ffakenz force-pushed the feat/gh-pages/docs-custom-template/456 branch 4 times, most recently from be9be14 to 7ac068b Compare August 12, 2022 19:28
@ffakenz ffakenz changed the title feat(gh-pages): Add author, updateAt, and translateAt via custom templating to markdown pages feat(gh-pages): Add updateAt, and translateAt via custom templating to markdown pages Aug 12, 2022
@ffakenz ffakenz changed the title feat(gh-pages): Add updateAt, and translateAt via custom templating to markdown pages feat(gh-pages): Add metadata documents via custom templating Aug 12, 2022
@ffakenz ffakenz changed the title feat(gh-pages): Add metadata documents via custom templating feat(gh-pages): Enrich documents with metadata via custom templating Aug 12, 2022
@ffakenz ffakenz changed the title feat(gh-pages): Enrich documents with metadata via custom templating feat(docs): Enrich documents with metadata via custom templating Aug 12, 2022
@ffakenz ffakenz changed the title feat(docs): Enrich documents with metadata via custom templating feat(docs): Enrich documents with metadata Aug 12, 2022
@ffakenz ffakenz changed the title feat(docs): Enrich documents with metadata feat(docs): enrich documents with metadata Aug 12, 2022
sed -i "s/$PLACEHOLDER/$REPLACEMENT/g" $MD

PLACEHOLDER="{{last-translated-at}}"
REPLACEMENT=$(git --no-pager log -1 --pretty=format:'%ad' --date=local docs/i18n)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seeing this now, this is a bit more complicated actually 🤔
This will yield the date where any translation was written. But we want that to be specific to that page.

Which makes me wonder for a minute: we have possibly many translations per page (fr, jp, ...). So we have 2 options I believe:

    1. Indicate when each translation was last translated. On the one hand, this is handy to see which translation are lagging behind from the reference source. Yet on the other hand, it can become cumbersome / noisy once we start adding more languages.
    1. Instead of indicating the last-translated-at on the reference source (i.e. en), we instead put a warning on each translated page if the source was modified AFTER the translation. Something a bit visible that gives readers a clear warning "This translation may not be up-to-date, the source was changed X days ago!".

I think option (2) is also easier to handle / implement with that bash script approach, as we can simply leave last-translated-at blank for files outsides of i18n and, for thos in i18n, look for the original under the same path and indicates both date to the component. And in the component, either:

  • display only the "last updated at" relative time; or
  • display the "last updated at" relative time AND a warning if it is before the "last updated at" of the source.

Copy link
Contributor Author

@ffakenz ffakenz Aug 15, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

New looking style for i18n
image

Normal pages looks the same but they don't have "last translated at" info
image

wdyt?

@ffakenz ffakenz force-pushed the feat/gh-pages/docs-custom-template/456 branch from 7ac068b to 0c4260f Compare August 15, 2022 10:20
Copy link
Member

@ch1bo ch1bo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The metadata format is suboptimal and some things can be made simpler by arranging the data slightly different in a hierarchical way + avoiding redundancy in the data structure. Furthermore, working with dates and displaying things can be made simpler using a time library like momentjs.

But, done is better than perfect.. so I can approve if you don't want to look into my suggestions.

docs/.gitignore Outdated
@@ -7,6 +7,7 @@
# Generated files
.docusaurus
.cache-loader
/static/metadatas.json
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Data is already plural. So there is no such think like datas.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

True.
I wanted to distinguish (in code) a single document metadata from the full set in code.
Any suggestions for renaming?
is "documents-metadata" a better fit?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

renamed to docs-metadata.json
hope this is a better fit.

```

```mdx-code-block
<DocumentMetadata />
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assume you have considered this, but can't hurt to ask: Do we have a global template or so to not need to repeat this for each page?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed I consider it but haven't started to work on that just yet but I think it's the right way to move forward.
Any suggestion on the approach?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have not found any way to achieve this by using docusaurus or any plugin.
Do you have any alternative?

@@ -0,0 +1,5 @@
#!/usr/bin/env bash
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see this file used anywhere. Was this an earlier approach?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, It was part of an earlier approach.
but it is also part of current approach to tackle the site-matadata.
still working to improving this approach, any suggestion?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, It was part of an earlier approach.
but it is also part of current approach to tackle the site-matadata.
still working to improving this approach, any suggestion?


async function main() {

Utils.getItems("**/*.md", { "ignore": ['**/node_modules/**'] })
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why the Utils. prefix on these local functions?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Extracting all behaviors to this Utils object makes the main function more readable, imo.

What do you think would be a better approach?

.replace('/current/', '/')
.replace('/index', '')

// @TODO rename to docusaurus-plugin-content-docs-docs
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we address this TODO (not sure exactly what it says)?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if we rename the directory of docusaurus-plugin-content-docs to docusaurus-plugin-content-docs-docs then this if/else can be removed.
do u agree we should move on with this?

lastUpdatedAt,
relativeTimeSince,
commitHash
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of storing the metadata for the source and its translations next to each other, it would be much easier to process (and not need to enumerate known languages later) if we store it in a hierarchy.

For example, instead of:

  "docs/getting-started": {
    "lastUpdatedAt": "Fri Mar 18 18:40:15 2022",
    "relativeTimeSince": "5 months ago",
    "commitHash": "c81338744"
  },
   "fr/docs/getting-started": {
    "lastUpdatedAt": "Sat May 21 09:28:11 2022",
    "relativeTimeSince": "3 months ago",
    "commitHash": "7ebee872d"
  },
  "ja/docs/getting-started": {
    "lastUpdatedAt": "Thu Jul 7 15:33:21 2022",
    "relativeTimeSince": "6 weeks ago",
    "commitHash": "ff12ae270"
  },

This format:

  "docs/getting-started": {
     "source": {
       "lastUpdatedAt": "Fri Mar 18 18:40:15 2022",
       "relativeTimeSince": "5 months ago",
       "commitHash": "c81338744"
     },
     "fr": {
       "lastUpdatedAt": "Sat May 21 09:28:11 2022",
       "relativeTimeSince": "3 months ago",
       "commitHash": "7ebee872d"
     },
     "ja": {
       "lastUpdatedAt": "Thu Jul 7 15:33:21 2022",
       "relativeTimeSince": "6 weeks ago",
       "commitHash": "ff12ae270"
     },

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

More comments on the metadata format:

  • lastUpdatedAt and relateiveTimeSince are quite redundant and the latter tends to go out of date. No need to store it, but compute it when rendering using one of the many javascript libraries (e.g. https://momentjs.com/)

  • The time format of lastUpdatedAt seems to be localized and I would suggest to use a more canonical format like ISO 8601. You can get that usually quite easy with just another flag.

  • We should store the full hash in commitHash as the truncated version can theoretically collide (more easily).

Last commit hash: <a href={link}><b>{commitHash}</b></a>
</i>
, renderLastTranslatedAt: (documentPath: string, lastTranslatedAt: string) => {
const languages = ['fr', 'ja'] //@TODO move to config
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See comment above. Grouping languages per page on collecting metadata will not require any configuration.

) ? obj : acc
})
}
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using a library like momentjs this whole code here would be quite simple. For example

`${moment(translationChangedAt).from(sourceChangedAt, true)} behind source`

would give you a relative time to the source like "3 days behind source".

@ffakenz ffakenz force-pushed the feat/gh-pages/docs-custom-template/456 branch from 3f67b6e to 3118289 Compare August 17, 2022 06:49
@ghost
Copy link

ghost commented Aug 22, 2022

When trying to build the doc I got this failure:

$ yarn start
yarn run v1.22.17
$ docusaurus start
[INFO] Starting the development server...
[SUCCESS] Docusaurus website is running at http://localhost:3000/head-protocol/.

✖ Client
  Compiled with some errors in 10.57s



Module not found: Error: Can't resolve '@site/static/docs-metadata.json' in '/home/curry/hydra-poc/docs/src/components/DocumentMetadata'
client (webpack 5.70.0) compiled with 1 error

Any clues?

@ch1bo
Copy link
Member

ch1bo commented Aug 22, 2022

When trying to build the doc I got this failure:

$ yarn start
yarn run v1.22.17
$ docusaurus start
[INFO] Starting the development server...
[SUCCESS] Docusaurus website is running at http://localhost:3000/head-protocol/.

✖ Client
  Compiled with some errors in 10.57s



Module not found: Error: Can't resolve '@site/static/docs-metadata.json' in '/home/curry/hydra-poc/docs/src/components/DocumentMetadata'
client (webpack 5.70.0) compiled with 1 error

Any clues?

Seems like the enrich-document-metadata step is not part of the yarn start. It's only on yarn build.

ffakenz and others added 7 commits August 23, 2022 14:03
Add author, updateAt, and translateAt via templating to markdown pages.

The idea is to add these templates to all of the relevant markdown documentation pages,
so that upon running the script during the CI step
the Github Page is enriched with the commit information.
@ffakenz
Copy link
Contributor Author

ffakenz commented Aug 23, 2022

@abailly-iohk nice recommendation.
added the enrichment step as part of yarn start

ffakenz and others added 19 commits August 23, 2022 20:53
Also reformat file automatically using newly installed prettier, seems
default layout is not great...
Also reformat file automatically using newly installed prettier, seems
default layout is not great...
We introduced a wrapper over EditThisPage
by running this command:
```sh
yarn run swizzle @docusaurus/theme-classic EditThisPage -- --wrap
```
Now the wrapper imports the component and voila, its available
everywhere
They are very hard to link the document url and file paths.
We had to do a hack in order to map the document
file-path to the document url-path,
but this unlocks the possibility to use the component
inside BlogPostItem wrapper
Copy link
Member

@ch1bo ch1bo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks much better now, let's get this merged.

@ch1bo ch1bo force-pushed the feat/gh-pages/docs-custom-template/456 branch from 6fa7684 to bec4e97 Compare August 25, 2022 08:24
Copy link
Member

@ch1bo ch1bo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now its' perfect.. ship it 🚢

@ch1bo ch1bo merged commit bdd0ce0 into master Aug 25, 2022
@ch1bo ch1bo deleted the feat/gh-pages/docs-custom-template/456 branch August 25, 2022 08:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
4 participants