Removal of 'core' branch #313
Comments
|
@tmk What features are you planning to develop? |
|
for tmk_core? no specific plan, mainly bug fixes, pull requests and TODOs. |
|
It seems to me that you have a lot of work ahead of you! If ever feel like you need any help, just let me know. I have thought of some good features and I believe I have the C skills to implement them. If it turns out that I don't, I can always share my ideas with you. |
|
This change makes sense, but you may want to change the name of this repo from tmk_keyboard, to avoid confusion. Something like tmk_firmware or even tmk_collection might make more sense. By the way, GitHub automatically redirects renamed repositories, so no worries about breaking links. |
|
hm, I don't thnk the name of repo 'tmk_keyboard' is a problem, because this repo embraces my keyboard related firmwares. But nice to know that how-to. Github help pages are great resources we must check occasionally :D |
|
@tmk To which repository do you prefer that I send pull requests? Also, can you please set up a place (a dedicated issue?) where people can have you approve of improvements before they begin working on them? I don't want to waste my time making an improvement that you don't end up merging. |
|
@Eric-L-T, he says in the OP here that main development will continue in this repo (tmk_keyboard), and that tmk_core will be considered the "stable" release. I also like the idea of an approval queue, although I think all that can happen right here in GitHub Issues. Just create an "IDEA" label, which we can use to propose new features. Then if @tmk likes the idea, he just assigns the issue to that person (or whoever wants to take it on) and changes the labels accordingly. |
|
@tmk What are your thoughts? |
|
@eltang, sorry for late reply. |
|
@tmk Sounds good. I think you will like the improvements I've made. Unfortunately, I haven't uploaded all my code to GitHub yet, but I hope to be able to soon. By the way, how often do you check GitHub? |
|
@eltang I usually check notification mails from github in real time manner but I have no time or no idea sometimes, then just forget to reply until rechecking lists. Looking forward to your repo update! |
|
@tmk Sorry that I haven't updated my repo yet. I'm new to git and the changes that I've made so far are all over the place, some committed, some not. I'm working on organizing them. Just a quick question: is there a reason that you haven't merged any of QMK's features into TMK on your own? I find that at least some of them may be worth taking a look at. |
|
@eltang no problem, take time. I sometime check QMK's activities and watch some intresting features, you will see cherry-pick from there in some future. |
|
@eltang. Here's my view on why the QMK and TMK have got so separated.
For me the situation is really sad, @tmk has done a great work of doing all the hard work in the beginning. Work that wouldn't have got down with a community project. And now the QMK firmware is heading in a different direction, where people will forget @tmk totally. Therefore I think we should do everything we can to get these two projects combined again. It will probably mean that we should do yet another split, splitting out the hardware specific parts of the core, and the software side, which handles the layout stuff. This way the development on the "hardware" side could be much more controlled, while the software side more community drive. Personally I would also like to see, a little bit more control on the layout side, than what seems to be happening on the QMK side. I have no personal experiences with that firmware, but my bet is that things frequently gets broken, because of the number of contributors the frequency of adding new features. Therefore I would like to see complete unit tests there. To see that all the mapping stuff really works with all the new features that are constantly added. It would also help in developing those features, since you don't need to flash the keyboard in order to test it. |
|
@fredizzimo That's a very interesting comment. I will have to think about it some more. I can say for now that many of your observations and hypotheses are correct. I'll make some more specific responses later. |
|
@fredizzimo You've made some really good points here. I think this project's user base has far outgrown anything that @tmk originally intended. This has created some growing pains as more and more people have been asking for features that meet their specific needs. In the meantime, the Issues/PR tabs are an absolute mess. "Feature request: mouse jiggling," are you serious? From what I can tell, QMK seems more interested in catering to these whims. Meanwhile, TMK has historically focused on quality code and stability, with extensibility as a secondary goal (e.g., macros and functions are there, but not really built out very well yet). The way I see it, the TMK project's current greatest needs are as follows:
Organization/managementTMK needs a clear set of guidelines instructing contributors on where to post issues, which kinds of PRs will be accepted, etc., which can then be referred to by @tmk (and potentially other maintainers) to keep the Issues queue clean. For example, a rule that states that all PRs must reference an existing, open issue which is tagged as ACCEPTED or similar...this will prevent any whining by someone whose PR was rejected after they spent three hours working on a feature nobody wanted. Meanwhile, pull requests do need to be dealt with more quickly, to avoid the problem of contributed code going stale. This obviously means that testing needs to happen swiftly as well. This is all a lot of work, but I think the project has enough interested parties to shoulder the load. All it would take is for @tmk to create a GitHub organization for TMK, create a development or community branch and assign one or more GitHub users as maintainers for that branch. This setup keeps the core master branch stable, and still allows hardcore users to access the latest contributed features. Better documentationSome of you may have noticed that TMK's documentation is incomplete. This is a huge issue because a feature is worthless if nobody knows about it. But it's not just the scope of documentation which is lacking, but to some degree the language barrier is an issue, since @tmk is ESL. I can speak from experience that the most rapidly-accepted PRs for this project are for the documentation. I recently contributed a rewording and expansion of the Keymap docs, and it was accepted by @tmk the very next day along with a heartfelt thanks. The problem here, and with every project, is that documentation is not "sexy." People would rather write actual code and implemente really cool new features. But documentation gets forgotten about, or left as a perennial @todo. I'd like to humbly suggest that all contributors take a step back from the codebase, and spend some time contributing to the documentation. It's very easy to write in Markdown, and you'll be surprised how quickly you can work through a README section. Even if all you do is clean up a bit of ESL grammar, you're doing a huge service for @tmk and the community, and you're putting down earnest towards the project which lends you credibility and respect the next time you submit a PR. A clear feature roadmapThis one may or may not be something others can help with, but it's primarily a task for @tmk. We all need to know what the long-term goals and intentions are for this project. Zooming in from there, contributors need to know which features and changes take priority, ideally in the context of a timeline. Versioning would be a huge help here, as well. Next stepsI have outlined some of these ideas in #324, to which @tmk responded favorably. I just think the guy is a bit overwhelmed, and maybe a little burnt out. He's a trailblazer and a "lone ranger" type, who as @fredizzimo stated above, has accomplished something here with the creation of TMK which few other people could have done. Now the project has reached the point where it can either embrace the interest from outside contributors, or remain as-is and eventually see that interest dwindle as those contributors move on to other projects. If @tmk chooses to embrace contributions, here are some suggested next steps:
I have tremendous respect and admiration for @tmk, which is why I feel compelled to help with this project however I can, as do many others here. I think now the ball is in his court. |
|
@jackhumbert @ezuk @IBNobody @NoahAndrews You guys are the main developers of QMK. We would love your input here. |
|
@tmk I sent you my first pull request of many to come. |
|
@jackhumbert Please read this discussion and post your response. I am trying to figure out how I should contribute to both TMK and QMK at the same time. Once we figure things out I'll get the rest of my code on GitHub. |
|
Since this is the most recent issue on the subject, I will reply here, but it's really a continuation of #173 and #324. I have been thinking about the separation of the core and the keyboard code, and about submodules versus subtrees(With a daytime job as a software architect I could not stop myself from doing it). And here comes my view on it. BackgroundIn #173, @tmk expressed his concerns about not being able to test all the different keyboards, and that he frequently breaks things. As a result he wanted to split the keyboard code (tmk_keyboard, and third-party projects) and the library_code (tmk_core). Initially he tried to have a core branch, which he then decided to replace by subtrees, at least for his own tmk_keyboard project, other projects can use submodules, or a combination. Git based solutionsFirst let's look at the two Git based solutions, which are the current way of doing things. If you are familiar with them, you can skip the following sections, there's nothing new there. And the TL;DR version is that both of the solutions have big problems. SubmodulesBefore I started to work on the support for Infinity Ergodox, I didn't really have any experience with submodules. However during the past few months I have really been learning about their use and limitations, as my repository contains 5 submodules on the top level, and one more inside the tmk_serial_link library. For me submodules feel quite intuitive, when working alone, if you think of them as separate repositories. You can go into your submodule directory and run normal git commands from inside there. The parent repository stores a reference to the correct version of the subrepository, so you have to remember to make a dummy commit before other people can see the changes. One thing that I didn't realise first is that you can add remotes and fetch from them like for normal repositories, so there's no need to mess with the .gitmodules for local work. Submodules don't work well when you switch between branches though, you always have to remember to run ´git submodule update´ afterwards. And things get really ugly when you switch between branches that don't have the submodule at all. If you just want to push your changes to Github things still work reasonably well. You have to fork all repositories to your own Github and push your submodule updates to that. When doing pull requests things start to get complicated. You have to
If you want people to pull from your repositories, rather than the upstream ones, you also have to modify the .gitmodules to point to your Github location. But if you also want to contribute to the upstream, you would have to use relative paths. All this is probably manageable, but very error prone for contributors. There's also a lot of unnecessary work. For users, I gave the following instructions, which should work for most cases, but still it's probably too complicated for many. This repository uses submodules, so after you:
You should run
These are only some of the complications with git submodules, so I can't really recommend them for everyone, for my repository they have been working OK so far. SubtreesIt's probably easier to work with subtrees than submodules, at least for normal users. However the documentation is not good. For example the Git book, does not contain any information about subtrees. Subtrees is also not fully officially part of Git. It's a contrib command, which might not be installed on all systems. I'm also afraid of merge conflicts as reported here and here. But I haven't tested those cases myself. Therefore if we continue to use subtrees, they have to at least be very carefully tested, in order to see that they really work when there are multiple people contributing, and working across many different branches. The CommunitySplit of a community that belongs togetherThe other problem with the chosen solution, besides the technical problems with both submodules and subtrees, is that it fractions the community. Each keyboard type will form it's own community, and all discussions, many of which are really keyboard independent will happen in those project's issue lists. Normal users might not even be aware that the tmk_core library exists. I'm of course mostly thinking of QMK here, and I can't tell for sure, but I have the feeling that if the project would have been official, and part of tmk_keyboard, then the contributors would have contributed to here, instead of having their own fork. Nothing would of course have prevented a complete fork from happening, in that case either. A Github organizationA Github organization has been proposed many times, and that could certainly help with the community split effect, also as already mentioned, an organization don't necessarily mean that the TMK firmware has to be a complete team effort. But as others, especially @njbair has said, TMK keyboard has grown enough, and have a potential to become really great, if it was more of a team project. In absolute numbers the community might be quite small, but considering that a big percent of the users are programmers, there's a huge potential for great contributors. It doesn't have to mean that @tmk looses all control, just look at how Linux is developed, Linus has all the power to do anything, when he needs to and when he wants to. I think the problems could be solved in a different wayThe reason for the splitAs I mentioned earlier, the main reasons for the split were the following
These problems should not be big enough to warrant a split of the code. In the following sections I will tackle each of the problems separately. 1. @tmk does not have all the keyboards to testAlmost no bigger software relies on the same people to test and write the code. I'm sure that we can find people voluntarily testing beta versions of the software on different kinds of hardware before the release is marked as a stable one. Implement a branching workflow, which includes testingMy approach for this is quite simple, use Git branches with different level of stability. I wouldn't go as far as applying the Gitflow branching model, which @njbair suggested, because I think it is an overkill for a case like this. Instead I propose something similar to Gitlab Flow, specifically the "Environment branches with GitLab flow" variant. We can't apply it directly though, since that would mean that users would have to checkout the production branch instead of master. That's needlessly complicated for especially non-programmers, that still would like to use the tmk_keyboard. So instead I suggest the following three branches
Once the release is merged to master, the process can start over with the next release. I really think that this simple workflow would fix the testing problems, without much overhead. Provided that we can find testers for all keyboards. In reality we might not even need testers for all the keyboards, in most cases it's probably enough to test all the different hardware controllers. The testing need can quite easily be determined by investigating which changes are included in a release. 2. @tmk frequently introduced bugs for keyboards he wasn't working onThis is something that happens, no matter how careful you are, however a number of things can be done so that it's more unlikely to affect other people.
3. Refactoring is difficultI don't think refactoring is easier by putting the responsibility to the keyboard maintainers. It will take a lot of time from them, because they fully have to understand what needs to be refactored and how. Also usually the need would come as a surprise when they fetch a new version. Having all the code available, before making any changes, also makes it easier to see if some part of functionality is used in a way that is totally incompatible with the change you are planning to do. In that case it might be better to take a different approach, which doesn't break existing usage. Instead, the API should be planned so that the need for refactoring is minimized. Also for very big refactorings, as separate branch could be created, and the work distributed between different keyboard maintainers. When everything is building and working again, the branch could be merged to develop. 4. There is much work to deal with patches for a lot of different keyboardsGithub organizations should make it possible to have many maintainers. So each keyboard can still have a main maintainer that primarily deals with code concerning only that keyboard. If the code only touches that keyboard, it's fine, and he can accept the merge request himself. But if there are changes to the core, then he can ask @tmk and other maintainers for their opinion. Of course if he sees issues, he comments on them, and have them fixed even before @tmk need to see the patches. We could even hide the process completely from @tmk by having separate forks for the keyboard. Not forks that normal people use, but forks for developers. The maintainer then first accepts pull requests to his fork, and then issues pull request to the main development branch. This is similar how Linux works. I don't suggest that we do this though, as it's probably not needed. Instead we could have some conventions for naming of the pull requests(like adding tags), so that @tmk knows which ones he don't need to concentrate that much on. If a maintainer needs help of others, we could use @ mentions. I also think the way @tmk handles pull requests makes things unnecessarily complex, but for himself, but also for the contributor.
KeymapsI think one thing that the QMK version of the firmware has proven, is how important it is to also share keymaps. Keymaps could use a simplified contribution model. Where pull request to the master is directly allowed, and almost automatically accepted(but only if the code touches nothing else). People could be named as maintainer for keymaps, but without direct power to accept pull request. Instead their comment on the pull request would be needed, like a simple thumbs up, so that @tmk or someone else could accept it. One problem with the current keymap structure is that it's hard to both have your own keymap that you want to version control on Github, and at the same time contribute other code to the main repository. For this I propose a simple solution, we add a user_keymap folder to .gitignore. Inside that the user can freely create a new repository, which he can push separately. He could even keep his keymap totally local to his machine, hopefully with some form of backup. External depenenciesSome keyboards currently depends on ChibiOs, where the latest version requires two separate repositories, ChibiOS, and ChibiOS-Contrib. For those we should either use submodules, or subtrees could also work. If we use Submoudles, I think the instructions I gave above would work, but we could even automate the git fetching and incorporate it to the make files. Other projects, well at least mine, have two extra libraries, tmk_serial, and tmk_visualizer. These libraries can be shared between similar keyboards. For example tmk_serial would probably be useful if other split keyboards are made, with the same principle as the Infinity Ergodox. While the tmk_visualizer is a generic visualization library for keyboards with LED and LCD, something which I suspect will be much more common in the future, at least for the Infinity series of keyboards. I suggest that we also include those libraries in the same repository, but that tmk_core in no way depends on them (the tmk_api has to be good enough), so that the keyboard projects freely can choose what to use and what to not. Separate maintainers could be named for those libraries, and @tmk does not necessarily need to be involved at all. TL;DRSo, let's bring all keyboard projects, and the library together into one repository again. Including the ones that are external right now, like different Ergodox projects (which should at least partly be combined), mine, QMK, and the @cub-uanic firmware for the original version, which has had problems to find maintainers. There's also support for some of other's of course, @tmk has his version of WhiteFox, which @flabbergast is improving in his branch. There are probably many more that I'm not aware of. Linux, Google and Facebook are other examples of keeping all the code in the same repository. If Linus can do it for Linux, we can do it for tmk_keyboard. |
|
I read half way and gave up this engrish reading session for today. I'll read the rest later, it may take weeks probably :) Thanks, good reading. I just checked "TL;DR" section and was surprised at this. I don't know how this bottom line comes yet, I'll know that after reading. looking forward to it.
I was working on my TODOs these days according to my plans, I'll keep these going anyway. This is Git repo no source won't be lost and I can recover anything even if I change my mind. |
|
Don't worry, take your time, my English is not the best either, as it's my second/third language, so that probably makes the text harder to read than it should be. And of course you don't have to agree with me, you can and should do things the way you think are the best. |
|
I would personally prefer removing tmk_core from the tmk_keyboard repo and use submodules. This removes the confusion as towards which repo should one do pull requests. I think it also simplifies the workflow for developers and people using git. For people not using git, this would just mean downloading and unpacking one more zip file. I don't see a good reason why to try to keep everything in one repo. I would also argue for using tags in the tmk_core repo. This makes it easy to refer to a particular version of core API in external projects; and one can also generate an URL to download a zip of a particular release from github. Having an established way to refer to particular versions of API {a.k.a. releases} should also make it "easier" to make API changes, since the external projects should always refer to a particular release of tmk_core {or if they don't they're expected to keep up with changes in master/develop}. Finally, I would be for creating one more 'community' repository {e.g. tmk_community} for keyboard projects and keymaps that are not necessarily maintained by @tmk, with more people having write access. tmk_keyboard would contain @tmk's official keyboards and converters. |
|
Removed |
I have plan to remove core branch of tmk_keyboard repo, instead I setup tmk_core repo for the same purpose. Sorry for this confusion.
I'm going to develop mainly under tmk_core directory(git-subtree'd) in master branch of tmk_keyboard and I'll update tmk_core repo occasionally. So you can think tmk_core repo as more stable, hopefully :D
https://github.com/tmk/tmk_core
The text was updated successfully, but these errors were encountered: