Merge in changes from upstream mkdocs-material repository

[jbms]

This also adds a script used for performing the update.

[jbms]

I have done some manual testing by browsing the documentation of this package itself. It is possible some bugs slipped through, though.

It may be challenging for you to review the changes to the files from mkdocs-materials, but I also made a few changes to the Python files as well, notably to remove the references to basic.css and documentation_options.js that previously led to spurious console messages.

[jbms]

Note: I did try to add sphinx-immaterial: comments to the places where I made changes to the mkdocs-material files, for reference when merging in changes in the future.

[2bndy5]

modifications seems to affect the arrow in the nav menu for nested menus.

vs the expected

[jbms]

Thanks for spotting the issue with the arrow in the nav menus.

In mkdocs-material there is this justify-content: space-between style that was added:
https://github.com/squidfunk/mkdocs-material/blob/4f641cc0ee7befd440956e53b8798899482f6ed4/src/assets/stylesheets/main/layout/_nav.scss#L96

Perhaps to address this very issue that you mention --- not sure since the commit message is rather vauge.

However, that style causes the C/E/M/etc. icons next to API documentation headings in the table of contents to display badly --- it adds space between the icon and the text. I also noticed that in the documentation of mkdocs-material itself, there is a demonstration of the "icons" feature which is similar (see table of contents):
https://squidfunk.github.io/mkdocs-material/reference/

The mkdocs-material documentation is produced with the non-public "insiders" version of the package, for which I don't have the source code, but from chrome dev tools I saw that the justify-content: space-between style is no longer present --- presumably it was removed to address the same issue I noticed with our icons.

Therefore I removed it for this upgrade as well --- but I didn't think to check on the arrows. I will see what I can do to fix that.

[2bndy5]

The HTML structure is different...

upstream

<a href="./" class="md-nav__link md-nav__link--active">
  <span class="md-ellipsis"> Reference </span>
</a>
<label class="md-nav__link md-nav__link--active" for="__nav_4">
  <span class="md-nav__icon md-icon"></span>
</label> 

in which the .md-nav__link .md-ellipsis has flex-grow=1 applied in css.

Compare that to this PR's doc bulid

<label class="md-nav__link" for="__nav_3">
  Examples and Uses
  <span class="md-nav__icon md-icon"></span>
</label>

The item's text is not wrapped in a span.md-ellipsis, so flex-grow=1 isn't applied.

I'd probably start with looking at the html template.

[2bndy5]

wrapping the text in a ellipsis class would also help solve the existing problem of long page titles in the nav menu

[2bndy5]

I managed to power through everything. Looks fine so far; I have to go more in depth on the

• base.html
• nav-item.html
• _nav.scss

[jbms]

I wrapped the titles in <span class="md-ellipsis"> and added the flex-grow style so that the md-nav__icon elements are properly right-aligned. I also added some additional fixes to make the ellipsis actually shown.

However, we might still want to tweak the overflow behavior. If you play with the current mkdocs-material documentation website in devtools to make one of the left or right toc entries very long, you will see that it wraps rather than truncates with ellipses. Possibly that would actually be better.

One issue though with wrapping is that it doesn't work very well if you have a long title without any spaces, such as in API documentation. We could allow wrapping at any character, but that may be ugly. Or we could apply different styles depending on if there are any spaces, but I don't know if we want to do that.

[2bndy5]

make one of the left or right toc entries very long, you will see that it wraps rather than truncates with ellipses

I noticed the furo theme does this also (which seems to be conceptually based on mkdocs-material or just material design in general). I just got done overhauling the breathe ext's docs for the furo theme. I would've recommended this theme, but there's still #14 ...

One issue though with wrapping is that it doesn't work very well if you have a long title without any spaces, such as in API documentation. We could allow wrapping at any character, but that may be ugly.

Initially I would favor an ellipsis, but we could try to inject a <wbr> after _, -, :, and . chars. Although I'm not sure if we can do that for text inside a <a> element.

[2bndy5]

I'll be the first to admit that I don't fully grasp all the changes here, but I'm going on faith in that the theme's docs aren't broken.

FWIW, I did skim through all 246 files changed, and nothing stood out.

[jbms]

I'm verifying that it also doesn't break the tensorstore docs and then will merge it.

[mhostetter]

FWIW, I tested this PR with my library also. The only difference I noticed was the right-side TOC adds "Table of contents" and an extra level of indentation. Basically the title heading is added to the right-side TOC.

Before:

After:

This seems slightly different than Mkdocs for Material. The page's title heading does not appear on the right-side TOC here.

[2bndy5]

@mhostetter Thanks for the feedback! its always best to do some field testing. I think you can now modify the secondary ToC's heading behavior in new conf.py options. Its possible that the mkdocs-material site is using this new feature as well. Those mkdocs visuals you're comparing are literally of the same commit that this PR is merging changes from.

Personally, I like the indent changes. I'm excited about the tables' borders being much more distinctive now.

[jbms]

@mhostetter Thanks for pointing out the toc issue --- there is now a confg option toc_title_is_page_title to control that, but there is also a bug which I am working on now.

[2bndy5]

It looks like I'll need to update my CSS overrides about changing the default admonition colors also

[jbms]

One thing would be to confirm that the toc still displays correctly if you have a sphinx document with multiple top-level headings. I'm not sure I would particularly recommend doing that but I think @2bndy5 made use of that possibility.

[jbms]

Regarding the table header styling --- that seems to just be a change made in mkdocs-material itself. I think I agree that the old styling was nicer in how it emphasized the table header, but not sure we want to go down the rabbit hole trying to tweak such things.

It is a bit unfortunate to have to tweak the md-nav__link styling, since those styles are actually extremely complicated, with different rules for different screen widths. Hopefully in the future when mkdocs-material merges back in the Insiders changes related to that we can reduce the divergence.

[2bndy5]

Regarding the table header styling --- that seems to just be a change made in mkdocs-material itself

Oh I didn't think to check that. That's fine; it would be a simple css rule in the docs that I use this theme for.

One thing would be to confirm that the toc still displays correctly if you have a sphinx document with multiple top-level headings. I'm not sure I would particularly recommend doing that but I think @2bndy5 made use of that possibility.

I can revert one of my docs to test this, but I have hacked most of the doc pages to only use 1 top level heading. I'll try it and report back...

I also included in this PR a fix to restore the import of autodoc_property_type --- it appears that was removed in #10, presumably accidentally.

yeah, I don't recall doing that on purpose. Sorry about that.

[2bndy5]

Ok. About the wbr suggestion, Ithought we were talking about the main nav menu

With using 1 top level heading the secondary ToC looks a bit weird

raw html output for weird word wrapping

<nav class="md-nav md-nav--secondary">
    <label class="md-nav__title" for="__toc">
        <span class="md-nav__icon md-icon"></span>
        Table of contents
    </label>
    <ul class="md-nav__list" data-md-component="toc">
        <li class="md-nav__item">
            <a href="#common-problems" class="md-nav__link">
                <span class="md-ellipsis">Common Problems</span>
            </a>
            <nav class="md-nav" aria-label="Common Problems">
                <ul class="md-nav__list">
                    <li class="md-nav__item">
                        <a href="#attribute-dependency" class="md-nav__link">
                            <span class="md-ellipsis">Attribute dependency</span>
                        </a>
                    </li>
                    <li class="md-nav__item">
                        <a href="#fifo-capacity" class="md-nav__link">
                            <span class="md-ellipsis">FIFO Capacity</span>
                        </a>
                    </li>
                    <li class="md-nav__item">
                        <a href="#pipes-vs-addresses-vs-channels" class="md-nav__link">
                            <span class="md-ellipsis">Pipes vs Addresses vs Channels</span>
                        </a>
                        <nav class="md-nav" aria-label="Pipes vs Addresses vs Channels">
                            <ul class="md-nav__list">
                                <li class="md-nav__item">
                                    <a href="#pipes" class="md-nav__link">
                                        <span class="md-ellipsis">Pipes</span>
                                    </a>
                                </li>
                                <li class="md-nav__item">
                                    <a href="#addresses" class="md-nav__link">
                                        <span class="md-ellipsis">Addresses</span>
                                    </a>
                                </li>
                                <li class="md-nav__item">
                                    <a href="#channels" class="md-nav__link">
                                        <span class="md-ellipsis">Channels</span>
                                    </a>
                                </li>
                            </ul>
                        </nav>
                    </li>
                    <li class="md-nav__item">
                        <a href="#settings-that-must-match" class="md-nav__link">
                            <span class="md-ellipsis">Settings that must Match</span>
                        </a>
                        <nav class="md-nav" aria-label="Settings that must Match">
                            <ul class="md-nav__list">
                                <li class="md-nav__item">
                                    <a href="#settings-that-do-not-need-to-match" class="md-nav__link">
                                        <span class="md-ellipsis">Settings that do not need to Match</span>
                                    </a>
                                </li>
                            </ul>
                        </nav>
                    </li>
                </ul>
            </nav>
        </li>
        <li class="md-nav__item">
            <a href="#about-the-lite-version" class="md-nav__link">
                <span class="md-ellipsis">About the lite version</span>
            </a>
        </li>
        <li class="md-nav__item">
            <a href="#testing-nrf24l01-pa-lna-module" class="md-nav__link">
                <span class="md-ellipsis">Testing n<wbr>RF24L01+PA+LNA module</span>
            </a>
            <nav class="md-nav" aria-label="Testing nRF24L01+PA+LNA module">
                <ul class="md-nav__list">
                    <li class="md-nav__item">
                        <a href="#the-setup" class="md-nav__link">
                            <span class="md-ellipsis">The Setup</span>
                        </a>
                    </li>
                    <li class="md-nav__item">
                        <a href="#results-ordered-by-pa-level-settings" class="md-nav__link">
                            <span class="md-ellipsis">Results <wbr>(ordered by pa_<wbr>level settings)</span>
                        </a>
                    </li>
                    <li class="md-nav__item">
                        <a href="#conclusion" class="md-nav__link">
                            <span class="md-ellipsis">Conclusion</span>
                        </a>
                    </li>
                </ul>
            </nav>
        </li>
    </ul>
</nav>

With multiple top level headings, the output is still truncated at the second top level heading found (just as before)

The fact that the truncated part ends up showing in the main nav menu is a new "feature" indeed 😆

[jbms]

Currently the left nav menu and the right nav menu are handled basically the same way --- are you suggesting one should wrap and the other should truncate? Truncation isn't really great since there isn't even a way to resize the window to have it visible --- perhaps this should be a theme option though.

I am not sure what is going on with the extra spacing between words and apparent reordering --- perhaps you can investigate that more --- or can it be reproduced with the sphinx-immaterial docs themselves?

As far as n<wbr>RF24L01+PA+LNA --- obviously it would be better not to break after the initial n. That is caused by the regular expression intended to break camel-case identifiers since in tensorstore there are some long camel case identifiers that need to be broken to fit. Perhaps it can be tweaked a bit. It might also be good to break after +.

[2bndy5]

My initial guess is that the jinja syntax isn't trimming the whitespace around its statements. This is typically done with {%- set var 1 -%}, but its been a while since I looked up jinja docs for the various mechanisms that trim whitespace.

Treating the secondary ToC the same as the global ToC is what I was expecting, but the wbr doesn't seem to be making it in the global ToC (concerning long API names). I haven't really investigated that yet either. I see the theme's docs global ToC is word wrapping fine, but the long winded API in the secondary ToC isn't wrapping.

[jbms]

It appears that it wraps properly in Chrome but not in Firefox --- I guess the styles will need some additional tweaking to also work in Firefox.

Still not clear what is going on with the weird spacing and reordering of terms in your documentation, though maybe something is causing those wbr elements to have an unusual style.

Note that the wbr elements are being added by nav_adapt.py --- there is no jinja involved.

[2bndy5]

Some of the wbr are actually showing up as <wbr style="">. I've never tried putting a style on a wbr tag, so I'll try to look out for that when I get into it. Thanks for the lead about nav_adapt.py. I can't promise I'll have something to report promptly, but hopefully by tomorrow.

[2bndy5]

I found a fix for the wbr problem:

.md-nav__link .md-ellipsis {
  display: block; /* inheriting flex is the problem */
}

I still have to verify that this doesn't break anything else...

[2bndy5]

I also noticed that the menu title is wrapped in a way that subsequent lines are hidden behind the menu items.

This is happening with chrome and firefox. It seems to be related to

@screen and (max-width: 76.1875em) {
  .md-nav--primary .md-nav__title {
    line-height: 2.4rem;
  }
}

[2bndy5]

@jbms would it be too deviant to include the css map files with the theme while we're still fleshing out the kinks? I know the dart-sass pkg can do this, but I'm unsure if this theme is using a different sass compiler.

[jbms]

I think the theme already includes .map files in the wheels uploaded to PyPi. Also for development, you can run: npm run start and it will build a non-minified version.

[2bndy5]

My mistake, they are present in the docs builds, but the browsers don't seem to be using them. Both chorme and firefox still say all css originates from line 1 of main.sha.min.css. Turns out the paths used in the map files are relative to the theme's root (location of install). This basically renders them obsolete for distribution.

[2bndy5]

I found a fix for word wrapping in nav menu title

.md-nav--primary .md-nav__title {
  /* was height: 5.6rem; */
  min-height: 5.6rem;
  /* line-height: px2rem(48px); in scss */
  line-height: 2.4rem;
}

This makes the title completely visible, but the line height (a rule specific to tablet breakpoint) makes it look spacious

Removing it sets the line height to 1.5 which looks much better

This has side effects on titles that don't need to word wrap

2.4 rem	1.5

---

I'll commit the fix for visual word wrapping but the line height seems to be a rabbit hole I don't want to bother with (for deviation reasons).

[jbms]

Thanks --- with your fixes the nav items indeed appear to wrap correctly in Firefox and Chrome.

In the reduced width mode where the nav menu is in the drop-down menu, I see the issue with the excessive line height that you mentioned.

[jbms]

I added one additional rule to ensure the .md-nav__title elements are truncated on the mobile/tablet layout. That matches the upstream mkdocs-material behavior, and avoids complications of messing with the line height. While in some cases it might be preferred to wrap, because that section of the drop down menu is not scrollable, truncating may be better to avoid the menu becoming unusable if the title is too long.

I also added some example pages with long titles to this theme's documentation for testing purposes.

[2bndy5]

lgtm

[2bndy5]

I'd like to do another rebase to pull in fixes about mermaid.js (released as part of v8.2.5 10 days ago) in hopes of supporting an alternative to graphviz directive. Squidfunk's release cycle is very active; there have been 3 patch releases since this rebase.

[jbms]

Sounds good, do you want to attempt it? I'm guessing there haven't been a lot of changes yet that will conflict.

You will have to rerun npm install to update the package-lock.json file.

[2bndy5]

I wouldn't mind attempting it. In fact I'll probably make a section in the CONTRIBUTING.md to outline the necessary steps and considerations.

You will have to rerun npm install to update the package-lock.json file.

Seen as how I've never tried to merge a commit from a separate repo, I'm eager to have any additional advice you may have.

[jbms]

To be clear this is not at all a normal merge. If you aren't familiar with normal git merging then this will be extra confusing, but anyway let's see how it goes.

In a normal git merge scenario, there is a common ancestor commit (as far as git is concerned) between the two branches being merged. However, I have not set up this repository to actually have mkdocs-material commits in the history for various reasons:

• The mkdocs-material repository frequently checks in new versions of the generated html, css, and javascript files, as well as all of the icons. That significantly bloats the repository size, and also would lead to merge conflicts if we tried to merge. (If this repository directly used mkdocs-material commits as parents of our commits, then all of the mkdocs-mateiral repository history would also be in our history.)

Normally git does a merge using the 3-way merge algorithm: it locates the common ancestor commit, so you have:

A----B
  \---C

So suppose our main branch is C, A is the common ancestor, and B is the new version of the other branch we want to merge in.

For every file in the repository, git will consider 3 versions: the version in A, the version in B, and the version in C. If all 3 versions are the same, then the merge result will be that single version. If the version in B is the same as the version in A (i.e. modified only in C) then the version from C will be chosen automatically for the merge. Likewise if the version in C is the same as A (i.e. modified only in C), then the version from B will be chosen automatically. If all 3 versions are different, then git will attempt to merge the changes in B and C. This uses a configurable merge driver, but by default will attempt to merge automatically and if it fails to merge a portion of the file will leave conflict markers:

<<<<<< HEAD
// version from head
======
// Version from other branch
>>>>>>>> other-branch

Additionally, it will put the git index into a special state to indicate that this file has a conflict. To complete the merge you have to manually edit the file to the desired state (deleting the conflict markers) and then git add the file to mark it completed.

Once resolving all conflicts you can do git merge --continue. Instead of adding the conflict markers you can also have git use an interactive merge tool.

With the merge_from_mkdocs_material.py script there is a similar process:

Since the git history does not indicate the common ancestor in the mkdocs-material repository, the script instead relies on the MKDOCS_MATERIAL_MERGE_BASE file.

The script creates a temporary mkdocs-material repository, checks out the MKDOCS_MATERIAL_MERGE_BASE version, copies the changes from our repository into it and makes a temporary commit, then does the 3-way merge in that temporary repository, leaving conflict markers in files that have conflicts. It then copies back the merged result, including any conflict markers, into our repository. It will also update the MKDOCS_MATERIAL_MERGE_BASE file. Finally, any files that merged without conflicts are staged (added to the git index). Any remaining files that are listed as "modified" in the working tree are the files with merge conflicts and they will contain conflict markers. However, since this isn't a real merge, the git index will not indicate that these files have conflicts. But you will in any case need to resolve the conflicts, remove the conflict markers, and then stage these files and create a commit.

As I mentioned, the merge excludes the package-json.lock file, since it makes some changes to the package.json file automatically, so you will have to run npm install after completing the merge to update the lock file.

One of the most common source of conflicts is changes to the rxjs imports in javascript files, since these all happen at the start of the file. These are pretty easy to resolve, you just need to figure out which symbols are actually used in the file.

I have made a lot of changes to the search code, so if there are any upstream changes to the search code, it may take some significant effort to resolve.