Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

merge repos: move wiki documentation into source folders #958

Closed
axd1967 opened this issue Feb 2, 2020 · 25 comments · Fixed by #1182
Closed

merge repos: move wiki documentation into source folders #958

axd1967 opened this issue Feb 2, 2020 · 25 comments · Fixed by #1182

Comments

@axd1967
Copy link
Contributor

axd1967 commented Feb 2, 2020

Is your feature request related to a problem? Please describe.

As a developer I want to read the Linux build instructions off-line. Now I need to go to the related GitHub wiki to find those instructions. Also, it is likely that those instructions change over time, which forces me to keep an eye on that wiki.

Describe the solution you'd like

Move the wiki documentation (16 files of about 100kb in total) into a subdirectory in the source tree.

Advantages

  1. Build instructions evolve with source code evolutions, which also allows to build old versions of Stellarium without having to find out what version of the instructions correspond to what version of Stellarium. (this problem will not be solved for past Stellarium versions, but for new versions.) A typical example is the evolution of the RPi instructions. A critical example is the series of changes documented in this wiki diff that are related to source code changes 2931ffe, ad2e894, b185a58 . These doc wiki changes belong in the source code repository and are critical for correct developer work and communication.
  2. Code changes in the documentation directory inform developers of anything that changed in e.g. the compilation instructions. (Such changes could e.g. be related to updates in e.g Qt libraries, to one-time changes to be applied to the compilation environment, etc)
  3. I will also automatically (via diffs) be informed of new as well as solved "Common Problems for the current version". Observe that the issues reported in that page are closely related to code base evolution - in other words, this is related to versioning.
  4. There is only one repository to track.

Disadvantages

  1. GitHub users cannot contribute to documentation pages without submitting a pull request via their own fork (a process that is supported by GitHub). This is not a real issue because the history of the wiki shows that the vast majority of changes were done by developers, and some changes were part of efforts that also involved changes in source code. Also, this will prevent random/spam changes such as this .
  2. There is no fancy table of content side panel

Describe alternatives you've considered
There is no alternative but to regularly consult the documentation wiki.

Additional context
As an example, #211 (comment) should no longer occur because a developer - assumming he/she is aware of this entry - could solve an issue (as was done in ?) and at the same time remove such an entry.

axd1967 added a commit to axd1967/stellarium that referenced this issue Feb 2, 2020
@gzotti
Copy link
Member

gzotti commented Feb 2, 2020

That relates to a general decision of what should the wiki contain or not. When we were on Sourceforge, the wiki was part of the documentation, even the user guide was only online for several years. After a few hiccups at SF it was hard to search for information. While the project was on Launchpad, we continued to use the old wiki, and it had all the dev. documentation, slowly falling into obsolescence. Now, we try to keep the info related to building the current version accurate, on the current wiki on GH, and want to move content from the old wiki to either the User Guide, the new wiki, or remove it if it is obsolete. However, this is a long-time project, and the two part-time developers of this project are overwhelmed by dozens of requests per week on totally different ends from translation to oculars, astrophoto requirements and ideas for yet another markup file format. Therefore the whole thing is kept at a minimum effort. You are probably the first person to analyze the history of the wiki. I don't have time for such investigations.

I was not aware that apparently everybody can freely add to the github wiki until our Brazilian freak spammer demonstrated this.

At this point, I am unsure what to do, or what should be placed in either source tree of the wiki. Adding building instructions may be useful.

@axd1967
Copy link
Contributor Author

axd1967 commented Feb 2, 2020

This uncertainty of adding to source tree or wiki actually adds an argument to bundle everything under one repo...

I am working on a merge of both, up to you to accept or not; but I think it is a better solution.

@axd1967
Copy link
Contributor Author

axd1967 commented Feb 2, 2020

@gzotti most current wiki pages are .md but some use the .mediawiki format which has a very valuable extra: a table of content (example here).

I'm a fan of .rst (also because TOC is possible) but if I have to choose between .md and .mediawiki I'd go for .mediawiki because of that TOC possibility, so a next step would be to convert .md to .mediawiki.Apparently a .mediawiki page is rendered slightly different when outside of the wiki environment: see here.

What do you think?

@gzotti
Copy link
Member

gzotti commented Feb 2, 2020

What data do you allow us project owners to keep in our own wiki? The sheer availability of a wiki indicates a potential value to the project. Or should it just contain one message: "Look into the source tree."

Most possible users will want to read it in markup style in a web browser, and here the current wiki is fine. It allows e.g. apt or (c)make command lines to be be applied via copy/paste. I have never heard of .rst, which does however not mean anything, because permanently moving and reformatting one-time-read text from a to b to c to d into as many markup formats as this world has invented is indeed not my interest. Is there a widely used way to read a mediawiki file offline and outside the wiki, on all 3 operating systems supported by Stellarium? We have moved some old files off the old wiki and were happy to keep the format because of the TOC, which .md does not provide. If dev docs now should go into the repo, I see plaintext or - at best - .md as the only possible formats. Sans TOC. Just keep it short enough to not need a TOC.

Maybe the most efficient way to proceed is to just collate the 3 build instructions plus Linux build requirements into a single recommended document, BUILDING.md, in the project's root. Weave build requirements in as seems fit. This then hopefully gets updated as the project proceeds: Qt versions, external dependencies, etc., and can later be restored by software archaeologists to any point in time in the regular git workflow.

@axd1967
Copy link
Contributor Author

axd1967 commented Feb 2, 2020

OK!

axd1967 added a commit to axd1967/stellarium that referenced this issue Feb 2, 2020
…ium#958

wiki pages: version d0b60c4094b424de91e91846ea9a0818d183df18
- linux build dependencies
- compiling on linux
- compiling macos
- compiling windows
- RPi
- code testing
@axd1967
Copy link
Contributor Author

axd1967 commented Jul 23, 2020

it's a pity that this has stalled.

ideally, all documentation (including wiki) should be on GitHub (note that GH behaves as a wiki), so that every change in the sourcecode is relfected in updated wiki pages.

now, contributing developers are often too lazy to update the associated wiki pages, which then become outdated.

most devs know that it is more fun to contribute the code than to update the docs, but the current situation is not making matters more easy.

remember: ideally code and docs should be updated in the same branch. but now, updating docs and updating the wiki pages is a chore.

@axd1967
Copy link
Contributor Author

axd1967 commented Jul 23, 2020

browsers can show nicely formatted pages that live in the code. it suffices to designate a folder inside the code as containing everything that is currently living in the wiki; anyone who visits the main github page canb be redirected to the corrsponding folder to find the documentation.

@alex-w
Copy link
Member

alex-w commented Jul 30, 2020

ideally, all documentation (including wiki) should be on GitHub (note that GH behaves as a wiki), so that every change in the sourcecode is relfected in updated wiki pages.

I know you are hate the wiki, but your sentension "every change in the source code is relfected in updated wiki pages" is not reflected the reality, because this is non sense.

now, contributing developers are often too lazy to update the associated wiki pages, which then become outdated.

You may write a very good docs (important note: not just copy various outdated docs into one bloated document!) - this task is not required coding - just an understanding the cmake scripts.

remember: ideally code and docs should be updated in the same branch. but now, updating docs and updating the wiki pages is a chore.

Criticism just for the sake of criticism does not help us.

@axd1967
Copy link
Contributor Author

axd1967 commented Jul 30, 2020

It's not a matter of hating the wiki (I never stated that anywhere, please show me where I write that I hate the wiki).

The wiki is fine.

The issue is that wiki and code are not tied together, and the current situation is not encouraging developers to maintain the docs. If everything is in the same branch, it is more likely that they will make an effort to keep docs in line with code. Also, this enhances the quality of the code, because code changes and docs stay together (even if a branch is never accepted and/or merged).

Also, the entire GitHub behaves like a wiki, so anyone could do exactly the same thing without the GitHub wiki. (here is an example).

axd1967 added a commit to axd1967/stellarium that referenced this issue Jul 30, 2020
just a deonstration that any GitHub page can act as a wiki page (see also Stellarium#958 (comment))
@axd1967
Copy link
Contributor Author

axd1967 commented Jul 30, 2020

... continued:

So to avoid having to keep code and wiki synchronised (because we really don't want outdated instructions in the wiki, as keepng both updated is a relatively painful manual process) we need to move as much relevant information as possible from wiki to some directory inside the code.

Developers don't need fancy formats; as long as the text format is readable (.md, .txt), that should be fine.

Which leaves the question: what can the wiki still be used for?

@axd1967
Copy link
Contributor Author

axd1967 commented Jul 30, 2020

A very nice example of what belongs as documentation in code (rather than in the wiki) is #194

What we see there is a description of how to build the Android version:

image

That descriptions belongs in the code! It is comparable to instructions to use Stellarium on a RPi, which also belongs somewhere in the code (maybe in a separate branch, I'm not sure).

@alex-w
Copy link
Member

alex-w commented Jul 30, 2020

The issue is that wiki and code are not tied together, and the current situation is not encouraging developers to maintain the docs.

Wrong! The docs always writing by special people - document writers. Yes, document writer may be developer too, but as rule this is different peoples.

If everything is in the same branch, it is more likely that they will make an effort to keep docs in line with code.

Wrong! Because after moving documents into source tree any docs writer should be work with technical tool - git. This step increasing skill level for docs writers and this reduced probability make an effort to keep docs in actual state. The docs already was in source tree and we got increasing authors for docs after moving the docs into wiki. You proposed reduce number of authors.

Also, this enhances the quality of the code, because code changes and docs stay together (even if a branch is never accepted and/or merged).

Wrong! Enhancement the quality of the code can be only in case, when docs writing within code (hello doxygen). Any docs outside the code (instructions for building is always outside the code!) does not toches quality of code.

So to avoid having to keep code and wiki synchronised (because we really don't want outdated instructions in the wiki, as keepng both updated is a relatively painful manual process) we need to move as much relevant information as possible from wiki to some directory inside the code.

Right now you proposed create an another copy of documents, who will be outdated since birth.

Just for protocol: this is my last touches any docs for Stellarium because your dictating tone killed any my wishes writing a documents. Please writing documents by yourself.

@axd1967
Copy link
Contributor Author

axd1967 commented Jul 30, 2020

And finally: assume instructions would be added to the wiki (see #194 (comment)), the problem is that those comments are useless as long as the code has not been merged! So the wiki would contain needless instructions. This is another example of bad syncing between code base and wiki, and for me yet another example that the wiki approach is not healthy from a software engineering point of view.

@axd1967
Copy link
Contributor Author

axd1967 commented Jul 30, 2020

Wrong! The docs always writing by special people - document writers. Yes, document writer may be developer too, but as rule this is different peoples.

Problem is that you don't have the luxury of having those people - you have to do all this. So in the worst case, the developer will not add documentation, yet that doc could be added during the review process (agreed, this will slow down the release - but is that wrong?)

Wrong! Because after moving documents into source tree any docs writer should be work with technical tool - git.

Correct.

This step increasing skill level for docs writers and this reduced probability make an effort to keep docs in actual state. The docs already was in source tree and we got increasing authors for docs after moving the docs into wiki. You proposed reduce number of authors.

Maybe you are confusing with the User Manual? I'm talking about the basic instructions that are currently in the wiki, and written for other developers. I'm not really talking about end user documentation - although the same logic applies there too.

Wrong! Enhancement the quality of the code can be only in case, when docs writing within code (hello doxygen). Any docs outside the code (instructions for building is always outside the code!) does not toches quality of code.

I'm not talking aboiut Doxygen. I'm referring to instructions in the wiki.

And "instructions for building is always outside the code" is wrong. in many software projects there are embedded README, COMPILING, etc files that explain how to buiold te code.

Right now you proposed create an another copy of documents, who will be outdated since birth.

No. I am proposing to move the bulk of documentation that now lives in the wiki.

Just for protocol: this is my last touches any docs for Stellarium because your dictating tone killed any my wishes writing a documents. Please writing documents by yourself.

Again, I'm not asking to write docs; the docs already exist in the wiki. And second, I pushed a PR to correct this situation.

@alex-w
Copy link
Member

alex-w commented Jul 30, 2020

Problem is that you don't have the luxury of having those people - you have to do all this. So in the worst case, the developer will not add documentation, yet that doc could be added during the review process (agreed, this will slow down the release - but is that wrong?)

The docs in the wiki partially was writing by advanced users and partially was writing by core developers for newbie developers and advanced users (hello SOCIS and GSOC students).

Maybe you are confusing with the User Manual? I'm talking about the basic instructions that are currently in the wiki, and written for other developers. I'm not really talking about end user documentation - although the same logic applies there too.

No, I'm talking not about User Manual.

And "instructions for building is always outside the code" is wrong. in many software projects there are embedded README, COMPILING, etc files that explain how to buiold te code.

This docs NOT in the code!

No. I am proposing to move the bulk of documentation that now lives in the wiki.

Yes, just moving some data from place A into place B. What will happen? Nothing, just outdated docs will be moved into place B.

Again, I'm not asking to write docs; the docs already exist in the wiki. And second, I pushed a PR to correct this situation.

Yes, you suggested poorly linked outdated documents convert into not linked outdated documents, i.e. banal trash heap.

@alex-w
Copy link
Member

alex-w commented Jul 30, 2020

As I proposed some time early - please write the good structured documents.

@axd1967
Copy link
Contributor Author

axd1967 commented Jul 30, 2020

sigh...

it's not because the current wiki documentation is wrong, that it shouldn't move to git...

I think I wrote it elsewhere (can't find it now) that this process should go in two phases:

  1. move whatever exists in the wiki to the code
  2. THEN clean up

Because the main issue here is NOT the quality of the docs, but the disparity between code and wiki.

Now by requiring to clean up the code first (which I can't because I cannot rewrite and review the entire existing doc and therefore I won't) you are blocking this problem.

What's wrong with just MOVING wiki pages FROM wiki TO code?

The essence is to get rid of the wiki!

If you oblige someone to ALSO clean up the docs, this will NEVER HAPPEN. Is that what you want?

I believe there is bad will here, made worse b y language issues.

@axd1967
Copy link
Contributor Author

axd1967 commented Jul 30, 2020

The docs in the wiki partially was writing by advanced users and partially was writing by core developers for newbie developers and advanced users (hello SOCIS and GSOC students).

As a result, a developer that has no access to the wiki is missing information.
Secondly, even "newbie developers" can read simple text files.

And "instructions for building is always outside the code" is wrong. in many software projects there are embedded README, COMPILING, etc files that explain how to build te code.

This docs NOT in the code!

But do you agree that build instructions should live together with the code?

No. I am proposing to move the bulk of documentation that now lives in the wiki.

Yes, just moving some data from place A into place B. What will happen? Nothing, just outdated docs will be moved into place B.

That is phase 2. But the essence is to BUNDLE everything so that it can be versioned correctly. Yes, the wiki also has versioning, but that is not synchronised with the code.

Yes, you suggested poorly linked outdated documents convert into not linked outdated documents, i.e. banal trash heap.

I see your point, and you are right: moving the pages will break the linking. I already corrected part of this, and I expect that further small changes will be done by the community; I DID MY BEST.

Can someone else please read #960 (comment) ?

@alex-w
Copy link
Member

alex-w commented Jul 30, 2020

  1. move whatever exists in the wiki to the code
  2. THEN clean up

Who will do it? You? If answer "No" then we do not need it, if answer "Yes", then why you do not actualise docs in same PR?

Because the main issue here is NOT the quality of the docs, but the disparity between code and wiki.

How exactly moving docs from directory A into directory B will improve the code or docs?

Now by requiring to clean up the code first (which I can't because I cannot rewrite and review the entire existing doc and therefore I won't) you are blocking this problem.

O_O Seriously? I don't want doing your work and I won't it!

What's wrong with just MOVING wiki pages FROM wiki TO code?

Because this give minuses and zero pluses.

The essence is to get rid of the wiki!

For what?

@alex-w
Copy link
Member

alex-w commented Jul 30, 2020

As a result, a developer that has no access to the wiki is missing information.

Apparently modern developers doesn't have internet access and cannot copy and paste text into file from browser.

But do you agree that build instructions should live together with the code?

No. Instructions may be live together with the code, but this is not required.

That is phase 2. But the essence is to BUNDLE everything so that it can be versioned correctly. Yes, the wiki also has versioning, but that is not synchronised with the code.

2 days ago I removed the file, which was "synchronised with the code" and "be versioned correctly" from your vision, but it was fully outdated 10 years ago already.

I see your point, and you are right: moving the pages will break the linking. I already corrected part of this, and I expect that further small changes will be done by the community; I DID MY BEST.

Community will be doing nothing. You are started this work, please do it up to finish by yourself.

@gzotti
Copy link
Member

gzotti commented Jul 30, 2020

Uhh, please calm down a bit, both. This energy is better used in the content.

Moving the documentation to the source tree to keep it updated was the motivation for me to move the User Guide there, so to add any relevant change immediately into the Guide as well.

The wiki does indeed not offer a way to say "show me the wiki in its entirety as of June 2018". You can see older versions of single documents, if anybody really cared. If the building instructions (BI) would change frequently, and if building old versions would be a task that many people were doing, there would have been a benefit if the building instructions would have been in the source tree since 2005 or so, yes, indeed. I cannot however remember people ever asking "how would I build Stellarium 0.11 today with MSVC2005 on my trusty Windows 2003 Server" (or with gcc 2.6 on Ubuntu 12.04 or whatever). The common use case is "How to build Stellarium from current sources, with currently available tools, on currently available computers, now". For this, the wiki serves well-formatted instructions which are reasonably accurate. On Ubuntu, it's even just copy-paste of dependencies and cmake/make command line, this can hardly be made simpler.

However, I think the BI have been modified significantly about 3 times in the last 10 years (one involved move from SF wiki to GH wiki), with a few minor amendments in between (e.g. adding libgps or some ssh details), so the rate of change is low.
Current Qt5.15 will be the last Qt version before a major rewrite will be necessary. The current 2 devs are indeed unwilling and not interested to invest several hours if not days of their spare time better used elsewhere reformatting and moving building instructions for the sake of reformatting and moving building instructions to make one reader happy. If @axd1967 wants, you can modify your PR and update building instructions to be accurate as of 2020, for all 3 platforms. When the content is updated, we may decide on acceptance. Else there is IMHO no benefit. In the process, the BI can probably be restructured to avoid redundant instructions. For Windows, Qt5.15 with MSVC2019 and MSYS should be discussed, while notes on MSVC2015/17 can probably be left away, given that the community editions are free. Also, notes for and mention of Windows 7 could be omitted as it is obsolete. But can it, in practice? Millions of computers still run it, and probably thousands of Stellarium users still use it, if not XP, for their observatory computers. (At least the tools have meanwhile decided to retire XP support.)

In the process of re-analyzing how to best set up tool chains and dependencies, at best done on a freshly installed computer (don't forget to mention whatever has else to be changed PATH additions or other dependencies or conflicts?), it would be interesting to understand why building with Qt5.15 takes much longer than before and why the translations use crazy amounts of RAM. I don't know if there is any need discussing older Qt versions? For Linux, probably Ubuntu 18.04LTS and 20.04LTS are to be discussed as reference platforms. What about ArchLinux, SuSE, Mandrake, and whatever else is out there? I have no idea. My RPi instructions from 2018 or so may still be interesting, but are meanwhile probably in practice obsolete as the version on the current Raspberry OS (new name for Raspbian) can now be built just following the Linux path, and has the necessary updates in the graphics libraries. I don't know if there is any real use to keep the older instructions, as I always update ASAP or install fresh cards, but maybe somebody wants to keep his Raspbian from 2018 unchanged for reasons unknown to us. And I even know less about building on macOS.

We simply don't have an army of testers and deputy mantainers who would go through everything 4 times a year and then repair the instructions where needed. When we find something is out of date (usually when installing the tools on a new computer), we fix it. On the other hand, we also have kept and moved instructions from the SF wiki that involved tips for the Qt4-based versions 0.10...0.12, just because I thought it's interesting from a historical point of view. There just has never been any structure designed for keeping information for obsolete versions. And there was never anybody who so determinedly policed the developers how and where they should maintain their building instructions.

@axd1967
Copy link
Contributor Author

axd1967 commented Aug 1, 2020

The main idea was indeed more to have an up-to-date and synced documentation for developers, rather than to be able to find back how to build an old version (although that's fine and perfectly feasible with the proposed approach).

Few people will ask for old building instructions; the real issue here was to realise that the building instructions change over time, and versioning correctly deals with this. Indeed, the User Guide is a perfect example of this approach.

Most developers can build Stellarium as is using the present state of the instructions, wherever these live; I guess very few want to build old versions (that would live on separate branches, by the way). After a while, developers no longer read those instructions, which the start to slowly rot away / get out of sync, especially when small updates must be done to keep building running smooth. And sometimes they will update the wiki.

(But it is rather sad to see a sound software engineering practice be reduced to a "policing practice".)

As an example of this problem, at this very moment I compiled 0.20.2 on Ubuntu 20.04 because of a dead mouse in Stellarium (I assume some system update broke something); but currently, the building instructions are missing (I can see that work is being done on that in a different branch - but this again illustrates the problem.).

@alex-w
Copy link
Member

alex-w commented Aug 1, 2020

The next obvious iteration for solve the problem: no docs - no problems 🤣

@gzotti
Copy link
Member

gzotti commented Aug 1, 2020

Yes, we have removed the BI from the wiki, for your personal one-person private satisfaction. They are now in the source tree. Of course not in the (meanwhile slightly outdated) 0.20.2 source tree. To build this, go to the wiki, and see that you must read BUILDING.md in the (online) source tree effective from now. The wiki will continue to provide current information by not progressing further. And now, we shall close this.

@axd1967
Copy link
Contributor Author

axd1967 commented Jul 5, 2021

this is not for my "personal one-person private satisfaction" but to simplify things and make them chronologically correct and acceptable from a software engineering POV.

for_wishlist

@Atque @gzotti @alex-w @treaves @A-j-K

@Stellarium Stellarium locked as resolved and limited conversation to collaborators Jul 5, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
3 participants