[WIP] Early sketch of point-valued distributions

[mateuszbaran]

I've started some work on manifold-valued distributions, extending the Distributions.jl API.

First example of a sphere-valued distributions:

using ManifoldMuseum
using Distributions
d = ManifoldMuseum.RetractedDistribution(ManifoldMuseum.Sphere((3,)), Distributions.MvNormal([0.0, 0.0, 0.0], 1.0))
rand(d)

This part is aimed at embedded manifolds.

Ref. #3 , JuliaStats/Distributions.jl#224

[kellertuer]

Since the is not necessarily a unique retraction and for some manifolds its cheap enough to do exp – could we find a way to make the retraction an optional parameter?

Manopt and Manopt.jl usually define a default retraction (retr = retractionQR for example) and all solvers and such have an optional parameter retraction = exp). which at least for my solvers defaults to the exponential map.

I am not completely sure how to provide that in a hopefully general but efficient way, but it would be nice to have.

[mateuszbaran]

retr! used there was supposed to be the "find the nearest point on the manifold" operation (only for manifolds embedded in R^n). Calling it retraction confuses me a bit as it has little to do with exp.

Anyway, it was previously suggested that the manifold object should store the retraction type to be used, so this would solve this issue.

[sethaxen]

As I understand it, a retraction doesn't require a concept of nearness. It's simply a continuous map from the space M to its subspace E, s.t. for x in E, the map is the identity map. One can easily imagine a map from Euclidean space to the nearest point of an embedded manifold (assuming Euclidean metric) that is not smooth, while one can still formulate a retraction that isn't distance-minimizing that is smooth. One can probably formulate many such retractions, so I don't think the retraction should be contained within the manifold.

I wonder if the generalization of this is simply an induced distribution. i.e., if one provides a function that maps points on the manifold on which a distribution is defined (for Distributions.jl, usually Euclidean manifold) to points on a manifold M, then this induces a distribution on M. One could even chain induced distributions. e.g., suppose we defined an induced distribution from Euclidean space to a sphere. Then we could also define an induced distribution from the sphere to real projective space.

[mateuszbaran]

Yes, of course, a retraction doesn't have to return the closest point -- it's just my intuition for what it's usually supposed to do. At least that's what I usually do. Anyway, retraction can just as well be stored within the distribution, and we also need to have a type for embedded manifolds.

[sethaxen]

... we also need to have a type for embedded manifolds.

A manifold type? The issue there is that every Riemannian manifold can be embedded in some Rn, though we don't always know how to embed it, so the type hierarchy would introduce an arbitrary distinction.

Perhaps we could have a type for an embedding that enables us to provide any special functionality we want to get from the embedding, such as a distance-minimizing map.

[sethaxen]

Perhaps we could have a type for an embedding that enables us to provide any special functionality we want to get from the embedding, such as a distance-minimizing map.

Actually, the beginnings of this interface is provided by #12. Namely, equipping a manifold with a Euclidean metric is interpreted as equipping it with the ambient metric that comes from the embedding in some Rn. Since that embedding isn't known in general, the ambient metric isn't known unless one implements that functionality, which could include a distance-minimizing retraction.

[kellertuer]

Yes, of course, a retraction doesn't have to return the closest point

It's simply a continuous map from the space M to its subspace E, s.t. for x in E, the map is the identity map.

I'd like to think of a retraction R_x as a smooth map from TxM to M such that R_x(0_x) = x and DR_x(0_x) = Id – i.e. its a(t least a) first order approximation to the exponential map.

[mateuszbaran]

Most manifolds I work with are actually represented by isometric embedding in some R^n (quotient manifolds are probably the most notable exception but I still haven't figured out how to get horizontal and vertical parts of tangent vectors in a generic way). I think it would be OK to start with some distributions for such manifolds, the interface can be easily extended in the future.

Yes, of course, a retraction doesn't have to return the closest point

It's simply a continuous map from the space M to its subspace E, s.t. for x in E, the map is the identity map.

I'd like to think of a retraction R_x as a smooth map from TxM to M such that R_x(0_x) = x and DR_x(0_x) = Id – i.e. its a(t least a) first order approximation to the exponential map.

How would you call that function: https://github.com/JuliaNLSolvers/ManifoldProjections.jl/blob/4068a8c53ab8a330c4034dd97af84ef1f9e60f3c/src/ManifoldProjections.jl#L23 ?

[kellertuer]

Just from Line 23 without documentation: Nothing.
Looking at the actual versions of retraction! for Sphere and Stiefel a retraction up to the point that the input might not be a tangent vector to a point on that manifold. So those functions for non-tangent is something but not a retraction ;)

[mateuszbaran]

Well, the point is, I still don't have a good name for functions f: N->M such that f(g(x))=x (where g: M->N) is an embedding of M in N).

[kellertuer]

If you mean the two functions such that for example for M=S2 and N=R3
then f would be project and g be embed for me.

[sethaxen]

So to be consistent with project_tangent, perhaps it should be project_manifold (though I like project_vector and project_point better).

Can we formally document what these terms (project, retract, and embed) should mean for those implementing new manifolds? I'm still not clear on how I would select these for new manifolds.

Also, to my earlier point. It seems that any map from the domain of a distribution to the manifold induces a distribution on the manifold. What is special about the RetractedDistribution that it gets its own name, especially if the retraction function is passed as an argument?

[mateuszbaran]

I think this PR is now in a fairly decent shape and shows how new distributions would be implemented. I've changed naming from retraction to projection and added tangent space-valued distributions.

RetractedDistribution (now ProjectedDistribution) isn't particularly special, it just a simple way to get a manifold-valued distribution. I needed an example and that was my first thought. What distributions would you like to have at this point?

[mateuszbaran]

Do you have any objections to merging this? I'd prefer to not have to keep merging master to this branch.

[sethaxen]

My only objection, and it's not important, is that the function names uniform_sphere_distribution and gaussian_sphere_tvector_distribution are quite long. Since the sphere is passed as an argument, and we could in principle have these for other manifolds, couldn't we do uniform_distribution and gaussian_tvector_distribution? And since Distributions.jl uses "normal" for the normal/Gaussian distribution, perhaps we should use "normal" here?

Otherwise, I'm fine with merging.

[mateuszbaran]

Good suggestions, I'll change the names and merge.

[kellertuer]

I ran the tests and that's fine, so with the renaming I am also in favour of not keeping branches open for too long – so yes please merge.