Remove some cursor meta-data awareness

[PEZ]

What has Changed?

Following up on #1587, I set out to remove more of the meta data awareness I added to the token cursor when addressing #1551. Much to my surprise none of that could actually be removed. The addition to handle metadata in cursor.backWardSecp() must be there, or we're in for a major rewrite of things.

The only place where we were considering metadata outside of this was in cursor.downList(), and was added long ago (by @bpringe) to support metadata when the debugger uses the cursor. Since I have decided that we need to move the support of metadata navigation out of the cursor primitives (except for forward/backward sexp) I then removed this from cursor.downList() as well. Then prepared to fix all test cases that now would break.

Surprise number two: No test cases broke, except the one testin that cursor.downList() could be made to skip metadata. Not even the debugger util tests that tested this. And everything seems to keep working when I try it manually. I don't quite understand why, but if I have found some potentially problematic code that can be removed without any issues, I'm happy.

Can you throw some testing on this from the debugger use, @bpringe? The unit tests you wrote seem to cover the needed bases, but anyway.

Fixes #1588

My Calva PR Checklist

I have:

• Read How to Contribute.
• Directed this pull request at the dev branch. (Or have specific reasons to target some other branch.)
• Made sure I have changed the PR base branch, so that it is not published. (Sorry for the nagging.)
• Updated the [Unreleased] entry in CHANGELOG.md, linking the issue(s) that the PR is addressing.
• Figured if anything about the fix warrants tests on Mac/Linux/Windows/Remote/Whatever, and either tested it there if so, or mentioned it in the PR.
• [ ] Added to or updated docs in this branch, if appropriate
• Tests
  • Tested the particular change
  • Figured if the change might have some side effects and tested those as well.
  • Smoke tested the extension as such.
  • Tested the VSIX built from the PR (so, after you've submitted the PR). You'll find the artifacts by clicking Show all checks in the CI section of the PR page, and then Details on the ci/circleci: build test.
• Referenced the issue I am fixing/addressing in a commit message for the pull request.
  • If I am fixing the issue, I have used GitHub's fixes/closes syntax
• Created the issue I am fixing/addressing, if it was not present.
• Formatted all JavaScript and TypeScript code that was changed. (use the prettier extension or run npm run prettier-format)
• Confirmed that there are no linter warnings or errors (use the eslint extension, run npm run eslint before creating your PR, or run npm run eslint-watch to eslint as you go).

Ping @PEZ, @bpringe, @corasaurus-hex, @Cyrik

[bpringe]

@PEZ If you try to debug this function using #dbg, and step through it to completion using Step Over, you'll see the cursor gets places in the wrong place (because it's going down into the metadata, instead of the form after it), and eventually an error occurs.

#dbg
 (defn test-metadata-symbol
   [x]
   (let [y x]
     ^{:hello "world"}
     (+ x ^{:inner "meta"} (+ 1 y))))

We should add a test for that scenario, and then either not remove the metadata check in downList, or find some other solution that works.

[bpringe]

Here's an example using #break, in which the cursor is placed in the wrong place.

(defn test-metadata-symbol
  [x]
  (let [y x]
    ^{:hello "world"}
    (+ x ^{:inner "meta"} (+ 1 #break y))))

[PEZ]

Thanks for the test cases! I'll have a look.

[PEZ]

I've extracted the paredit downList logic into a cursor function that handles metadata. Using that function in the debug breakpoint navigation makes your examples work. I added unit tests for it. Took me a little while to figure out the coor, but it was worth it. Very smart device!

The debugger still steps a bit weirdly through those functions. But it is weird in the same way it was before this PR, so I'd say we can fix that later, if it should be fixed.

[bpringe]

Awesome! Yeah, I believe I noticed the same weird behavior.

Took me a little while to figure out the coor, but it was worth it. Very smart device!

That comes from cider-nrepl. It's an interesting way to describe a position in a form starting at the top level.