Sync readme.md from github too for descriptions

[jerone]

Since OUJS search Github for .user.js files and we also support Markdown, should it also search for readme.md files in the same folder as the userscripts are found and use that for the description field...

[sizzlemctwizzle]

Sounds great. This will work fine for importing, but how should we handle the webhook. Listen for changes to the README? Treat it the same way we treat scripts (webhook just overwrites changes on OUJS)? Ignore changes on GH if it's been changed on OUJS? If a user tries to change that section on OUJS, should we send them to GH to edit it?

I definitely want this in some form, but it wasn't in my initial plans for the first three milestones and I don't know if I'll have time to squeeze it in so I'm assigning it to the fourth milestone.

[jerone]

Definitely a feature for later.

I like the way how GreaseFork works with sync. Give the user access to the sync feature.

[studgeek]

Also see #96 where I suggest defining a common script file structure with GreasyFork also.

[TriMoon]

I don't agree with using the readme.md for the scripts.
Because people could have more than one userscript inside their GH-repo designated for their userscripts.
The readme.md would only explain things inside the repo, but individual scripts could have totaly different descriptions inside....

[Zren]

Easy enough.

scriptInfoMdFilename = None
if numScripts(repo) == 1:
    scriptInfoMdFilename = '/ReadMe.md' # Case Insensitive

# /scripts/1234214.user.js
# /scripts/1234214.md
scriptMdFilename = scriptPath + scriptNameWithoutFileExt + '.md'
if fileExists(scriptMdFilename):
    scriptInfoMdFilename = scriptMdFilename

With that, you could have a repo with only 1 script, and have both a ReadMe.md and a scriptName.md for the synced script info.

[sizzlemctwizzle]

With that

I agree. I think that is the best way to handle this.

[Martii]

Did we decide on a naming syntax for this?

'/ReadMe.md'

This could work... but wanted to note the user-content accelerator arrows will fail on GH with the current README.md ... they are quite complex at times so it's probably not suggested to handle this in our pages.

Slapping some other naming suggestions (maybe more at a later point)

• README.oujs.md
• README.OUJS.md

[reek]

HI all,

I vote 👍 for adding this feature, it will render a great service to all those who have userscript hosted on different plattform

Regards
Reek

[TimidScript]

Great idea 👍
I just posted this and that's silly compared to this one.

[Martii]

Since I've been twiddling with about DOC pages as of late I still have to try something with adding some predefined links to documents and see how GH md and our md parser handles duplicates so the accelerator arrows may not be an issue. e.g. [toTop]: #body could be appended onto the tail of the .md versus whatever GH's fragment is... in short we might be able to reuse README.md if this works... although wiki url references will bust.

---

Yes our md processor picks the last link reference. So perhaps, typing out loud, an example with GH wiki's:

[toTop]: #wiki-body
[toContent]: #wiki-body
[toBottom]: #wiki-footer

and then appending on OUJS:

[toTop]: #body
[toContent]: #user-content
[toBottom]: #footer

overrides the default GH anchors.

So we could hardcode [⬆][toTop] [⬇][toBottom] [⇪][toContent] into our docs easily and support those or similar named references.

They look like this when no reference links are present of these names for GH:

[⬆][toTop] [⬇][toBottom] [⇪][toContent]

We can probably generate these on the fly on OUJS and just ignore GH too which would make it more cross-site ready in case there's a storm brewing on how they are named... and/or just keep it originally site named as I mentioned before to eliminate any potential conflicts. Looks like README.md rendered here on GH doesn't have a bottom and GH itself doesn't seem to have one either.

[Zren]

This is my next project.

UI

• Blue info box is hidden by default. Is shown/hidden when clicking the checkbox.
• Padded the top of the sidebar so the panels are aligned.
• Moved the save button so it will be at the bottom on mobile.
• Subtlely indicate that we're using github markdown with a label. I'd use the markdown icon if there was a 16px version somewhere It fits fine on mobile right now though.

Edit: New Screenshot

Ignore the extra .user.js in the ScriptFileName.user.js.md, wrote that before rereading this thread.

http://i.imgur.com/xKT2nwA.png

DB

  // Sync
  syncAboutSourceUrl: { type: String },
  syncAbout: { type: Boolean, default: false },
  // syncScript: { type: Boolean, default: true },

syncScript is there in case we ever want to have the option of sync just the description... meh.
syncAboutSourceUrl will be used to tell the script author where the about page was last synced from. Probably should add an aboutUpdated field too. Maybe a lastSynced field instead.

Code

Gotta do this next. But it'll check for

/readme\.md/i then dirname + scriptNameWithoutExt + 'md'.

[jerone]

@Zren commented on 3 sep. 2014 19:33 CEST:

This is my next project.

Awesome! Waiting for this feature when I first suggested it; wish I had that amount of skills with node.js.

Subtlely indicate that we're using github markdown with a label. I'd use the markdown icon if there was a 16px version somewhere It fits fine on mobile right now though.

This comment box has an Markdown icon.

[Martii]

Gotta do this next. But it'll check for
/readme\.md/i then dirname + scriptNameWithoutExt + 'md'.

I've noticed that GH is case insensitive for README.md and ReadMe.md and so on... so that should be a consideration I think.

Moving the save button below the groups might encourage ppl to use it more. :) +1 there.

---

I'd use the markdown icon if there was a 16px version somewhere

#186 but let me finish up with the serving first. This could be next in the chain after serving is handled properly.

[Zren]

I've noticed that GH is case insensitive for README.md and ReadMe.md and so on... so that should be a consideration I think.

Learn to Regex foo.

http://regex101.com/r/cB9vF7/1

https://xkcd.com/208/

This comment box has an Markdown icon.

Eh, true. I'll probably dump it in /public/img/.

[Martii]

Learn to Regex foo.

When someone offers you a tip for your declaration of assignment (even though you still haven't assigned yourself) you don't make it appear to be condescending by telling them something about learning re (regular expressions)... this isn't a noteworthy teamwork trait. I have been using re for at least a few decades, including a lot of the different implementations, but thanks for the cross-tip links for those that may find it useful. I was being polite for everyone's sake with the "I think" part as I know you will run into this. I prefer to use my noggin first (including a debugger) and then if I'm having an issue search the web for reference material, use the available add-ons and then finally use an online re tester. :) In short please refrain from inappropriate responses that was definitely perceived as bad attitude... a simple "thank you" would have sufficed. Thanks.

[Zren]

I guess you missed the case insensitive flag in /readme\.md/i then. Thought i was indoctrinating a new regex user.

[Martii]

Yes I did miss that. It's the beginning of the month and I'm super busy and not as focused as I would like to be at the moment so I apologize if it was redundant to what you had already stated.

[Zren]

The first version of this feature is only going to check for the scriptname.md + readme.md when the script source is updated. There's currently no way to query if a Script belongs to a repo without parsing every js file in the repo for it's script name.

[jerone]

@Zren commented on 16 sep. 2014 20:31 CEST:

The first version of this feature is only going to check for the scriptname.md + readme.md when the script source is updated. There's currently no way to query if a Script belongs to a repo without parsing every js file in the repo for it's script name.

So before updating the .user.js people have to be sure they commit their readme.md files first?

A solution for later is to use timed update checks (e.g. every 24 hours). Thats what GreasyFork & myUserJS do.

[Martii]

A solution for later is to use timed update checks (e.g. every 24 hours).

Wouldn't it make more sense to have this done with the webhook on demand instead of draining the clouds resources with timers?

[Zren]

Decided that since this is an opt in feature, the user might as well enter the required information.

https://github.com/Zren/OpenUserJS.org/compare/pushdescription

It's working. Just need to consider edge cases (case sensitivity on the lookup) and clean up the code.

[Martii]

So are you going to let anyone sync from anywhere on GH?... say "Load someone elses readme.md to a spoofed hack of account x?"

[Zren]

What's the point in authenticating for this if every other site just syncs with a url?

[Martii]

What's the point in authenticating for this if every other site just syncs with a url?

Just off the top of my head...

1. To be better than the average site.
2. To help prevent spammers and malware from appearing disguised as someone elses project. A very common issue on the site formerly known as USO.

You're screenshots might be useful for GH organizations but I can also think of some potential abuse there if someone doesn't have GH configured properly... and our own collaboration flags (metadata keys) if there's an error. Just some random thoughts.

[Martii]

Don't get me wrong it does solve a few things but also may introduce a vulnerability or two with administration. Looks neat though... perhaps with some refinement down to the GH account since we do store that info per GH account... just some more random thoughts. :)

[jerone]

@Zren commented on 15 okt. 2014 08:31 CEST:

https://github.com/Zren/OpenUserJS.org/compare/pushdescription

It's working. Just need to consider edge cases (case sensitivity on the lookup) and clean up the code.

@Zren
Wondering what the state is of this feature is?
It looks great and would be a huge benefit for people hosting their readme somewhere else.

[Zren]

Better UI or worse?

[Martii]

Better UI or worse?

A few questions come to mind:

• Are you getting rid of the global import e.g. does any author have to make a stub first then sync?
• From reading your summary you disable any local editing... What happens if I need to make a temporary changelog note on OUJS without actually visiting GH's md editor? e.g. this is probably not a desired effect. GH sync can always overwrite as it does now. Food for thought.
• Why do you name your branch "pd#" e.g. "pd5" when the CONTRIBUTING.md states you should be using an Issue-### whenever possible? What does "pd" mean?

I won't go into the code itself yet but there are some useful things in it. Again it contains huge chained changes and non-distinct. Cherry picking ideally should be distinct individual commits but when they aren't distinct enough someone will have to be watching and then pull distinct items out just to get something PR'd with no attribution other than a commit summary... at least in my experience with git/GH that is.

[Zren]

Are you getting rid of the global import e.g. does any author have to make a stub first then sync?

They currently already have to make a script first before the webhook will work. The paginated import works. Not sure about the unlinked one that's suppose to be removed in #468

Edit: Or did you mean the current webhook action where it will determine the script by the name in the userscript metadata block? Because that's still in there and mentioned with "automatically filled out on git push". I could remove that input field altogether instead of having it disabled, but that might make it look you can only sync the description and not the source code. That's one of the reasons why I'm iffy about the last change. On the one hand, I want to move the syncing stuff into the edit script page, but on the other, we don't need to fill out data for syncing the source code.

From reading your summary you disable any local editing... What happens if I need to make a temporary changelog note on OUJS without actually visiting GH's md editor?

Uncheck the button to disable syncing the description, and to allow editing the textbox. Jerone asked me to disable the md textbox when syncing. I just noticed a bug when doing this though. A disabled input/textarea doesn't have it's value sent in the form data when submitted. Our current route handler https://github.com/OpenUserJs/OpenUserJS.org/blob/master/controllers/script.js#L399 will bork if the about field isn't there. So I'd have to refactor the entire script.edit route handler to break up into deleteScript, scriptEditMetadataPage, and it's POST handler.

Why do you name your branch "pd#" e.g. "pd5" when the CONTRIBUTING.md states you should be using an issue-### whenever possible? What does "pd" mean?

pushdescription attempt number 5. The original was pushdescription but somewhere along the way I got annoyed at typing it out in full. I don't name my branches after issue numbers since you will have no fucking clue what it does without looking it up on github or checking it out first.

Cherry picking

I merged all my previous commits into 1 since I had to rewrite them anyways and it's easier to amend a single commit instead of the second last commit or further back in history.

[Martii]

Or did you mean

I mean https://openuserjs.org/users/username/github/repos route... e.g. I don't need to make a stub on OUJS first and I can just push to my repo first then go import that to OUJS. Without examining your branch first the screenshot appears to imply from first impression that one has to have a script stub on OUJS first then it will allow syncing of both script source and the user data. Does that make more sense on what I'm asking? Please remember that I'm working on other things too so I don't always have a chance to do a checkout. This is why I ask for "screenies" and usually some sort of outline in an issue, then pr which usually closes that issue. We all are usually in the same boat working on various projects including this one... so that process helps tremendously and is why it was outlined in CONTRIBUTING.md (and not by me but approved because it works)

Thanks for the explanation of the other points.

[jerone]

Again, I did NOT unassign you. Keep up the great work.

@Zren commented on 7 dec. 2014 06:18 CET:

Better UI or worse?

Much better looking!

Wondering if the sync block can be moved to the right sidebar?

Great to see the path to the repo for both script and readme to be the same.

Will review better when I have a change to test it.

[Zren]

It could be put on the right, but it might get a little too squished before it goes into mobile mode. Just realized synching will come before grouos on mobile though. Bleh. Whatever.

Started refactoring the script.edit route handler last night and realized the thing is evil incarnate. Going to stash my changes today and remove the disable mdeditor when synching code for now.

[Martii]

grouos

Seriously... what?

[jerone]

In an attempt to clean up my created issues that have not been processed or updated over a year, I'm closing this issue. If this issue is still relevant, please reopen another issue.