Colorby='frame' not working

[kmsouthard]

Hello everyone!

I am fairly new to python so perhaps this is a user error, but I cannot get colorby='frame' to work for trackpy.plot_traj function.

I either get axis centered around zero and not even getting to 1 pixel or, if I try and add labels the plot disappears, and my output is just something like this:

<matplotlib.figure.Figure at 0x3b73d9550>

If I am doing something wrong or I need to check something, any help would be greatly appreciated!

[danielballan]

I wonder if you are running it on an empty data set. The colorby='frame' functionality is not heavily tested, but it still works for me. Show some sample input and output. (Did you mean to paste some output into your original post? I see nothing.)

[kmsouthard]

Oh, I did indeed forget to paste in the output.

Color by particle works perfectly so I do not think it is an empty data set.

Here is an example:

IN:
tp.plot_traj(t2, colorby='frame', label=True)
RETURNS:
Drawing multicolor lines takes awhile. Come back in a minute.
<matplotlib.figure.Figure at 0x3b73d9550>
OUT:
<matplotlib.axes._subplots.AxesSubplot at 0x372cf6a10>

So it seems like matplotlib should be returning a figure but nothing is showing up.

[kmsouthard]

Huh, ok, the outputs I pasted in were not showing up. Deleted the brackets
matplotlib.figure.Figure at 0x3b73d9550
matplotlib.axes._subplots.AxesSubplot at 0x372cf6a10

[danielballan]

@tacaswell?

On Friday, October 24, 2014, superboywonder [email protected]
wrote:

Huh, ok, the outputs I pasted in were not showing up. Deleted the brackets
matplotlib.figure.Figure at 0x3b73d9550
matplotlib.axes._subplots.AxesSubplot at 0x372cf6a10

—
Reply to this email directly or view it on GitHub
#177 (comment).

[tacaswell]

Can you upload images of what you are(n't) getting? There isn't quite enough here for me to sort this out...

[kmsouthard]

Yeah, I just reloaded my iPython notebook and tried again. This time it actually gave me an error instead of not plotting anything, but again color by particle works perfect, color by track doesn't. Here's a screenshot:

[tacaswell]

The line it is pointing to should be ax.add_collection(lc) (just drop the gca()).

[tacaswell]

And this is already fixed on the master branch.

[tacaswell]

I am a tad confused, what version of trackpy are you using and where did you get it from? Searching through the history I can not find any evidence of that line ever being there.

The 'by frame plotting' was added in c62d86c and git blame on master shows that it has not been changed sense.

[kmsouthard]

Yes! sorry about that! That was me messing around trying to fix the problem myself. I have the most recent version of trackpy. I kept the original file of plots.py as a backup and reverted to it. Now I am getting the same thing as before, with incorrectly sized axes and no tracks being plotted:

[danielballan]

Can we see t2.head(50)? Make a gist.

[kmsouthard]

<script src="https://gist.github.com/superboywonder/f5aaa3c61fed9a8bee4e.js"></script>

First time making a gist, let me know if that wasn't what you were looking for.

[kmsouthard]

https://gist.github.com/superboywonder/f5aaa3c61fed9a8bee4e#file-gistfile1-txt

[tacaswell]

MWE (that works)

import numpy as np
import pandas as pd
from itertools import product
centers = product(range(10, 100, 10), repeat=2)

# make some synthetic data
N = 25
turns = 3
r = np.linspace(0, 2.5, N)
theta = np.linspace(0, 2 * np.pi * turns, N)
frame = np.arange(N)
x = np.sin(theta) * r
y = np.cos(theta) * r

# skeleton of data frame
df = pd.DataFrame({'x': [], 'y': [], 'frame': [], 'particle': []})


# load the synthetic data
for n, (x0, y0) in enumerate(centers):
    l_df = pd.DataFrame({'x': x + x0, 'y': y + y0, 'frame': frame, 'particle': n})
    df = df.append(l_df)


trackpy.plots.plot_traj(df, colorby='frame')

[danielballan]

I notice possible "gaps" in the trajectories. (There is data for Frame 2
and Frame 4 but apparently not for Frame 3.) This might be the issue -- I
will have to return to this later.

On Mon, Oct 27, 2014 at 2:11 PM, Thomas A Caswell [email protected]
wrote:

MWE:

import numpy as npimport pandas as pdfrom itertools import productcenters = product(range(10, 100, 10), repeat=2)

make some synthetic dataN = 25turns = 3r = np.linspace(0, 2.5, N)theta = np.linspace(0, 2 * np.pi * turns, N)frame = np.arange(N)x = np.sin(theta) * ry = np.cos(theta) * r

skeleton of data framedf = pd.DataFrame({'x': [], 'y': [], 'frame': [], 'particle': []})

load the synthetic datafor n, (x0, y0) in enumerate(centers):

l_df = pd.DataFrame({'x': x + x0, 'y': y + y0, 'frame': frame, 'particle': n})
df = df.append(l_df)

trackpy.plots.plot_traj(df, colorby='traj')

—
Reply to this email directly or view it on GitHub
#177 (comment).

[kmsouthard]

There very likely a lot of gaps in the trajectories. I am performing tracking on quantum dots, which inherently blink stochastically.

@tacaswell that code works for me.

[tacaswell]

It also isn't the gaps, if you drop every-other in the code above

# load the synthetic data
for n, (x0, y0) in enumerate(centers):
    l_df = pd.DataFrame({'x': x + x0, 'y': y + y0, 'frame': frame, 'particle': n})
    df = df.append(l_df.loc[::2])
    # mask = np.random.rand(N) < .95
    # df = df.append(l_df.iloc[mask])

still works fine so it isn't the gaps...

[kmsouthard]

One difference I notice between the synthetic data and the actual data is that there are 2 columns corresponding to 'frame' in the actual data. Format looks a little confusing in the gist, so here is a screenshot.

[kmsouthard]

I played around a little more and the issue seems to be related to the number of frames that are analyzed.

f = tp.batch(frames[:100], 11, minmass=10000, threshold=500) #~100 features per frame
t = tp.link_df(f, 5, memory=3)
tp.plot_traj(t, colorby='frame')

that code works, but if I change the number of frames to 200 the problem comes back. I am performing tracking on a video of 600 frames.

[danielballan]

I suspect we are overtaxing matplotlib and/or your computer. Matplotlib uses many optimizations make make line-drawing fast and memory-efficient. But multi-color lines are not so well optimized -- this is why I have trackpy print a message warning the user that it will take awhile.

As a crude workaround, you could make several plots and superimpose them. Color-by-frame is a nice feature to have, but it's not easy to implement it in matplotlib, and the way I did it is the best way I know of. For general use, I recommend colorby='traj'.

[MTone92]

I am a newbie for python and trackpy. I also encountered a similar problem. Instead of not plotting the lines with tp.plot_traj(my_track, colorby='frame'), the plotted lines are in the same color. I manually checked the code in trackpy/plots.py where the function plot_traj is defined. Under the if colorby == 'frame':, if some particles don't appear in all frames, the plotted lines are in the same color, since the number of plotted segmentations is less that the number of colors supplied to the function lc.set_array(color_numbers).
I tested some cases with matching the number of segments and number of colors in a stand-alone plot.py file with replacing the if colorby == 'frame': part by

if colorby == 'frame':
        x = traj.set_index([t_column, 'particle'])['x'].unstack()
        y = traj.set_index([t_column, 'particle'])['y'].unstack()
        color_numbers = traj[t_column].values/float(traj[t_column].max())
        norm = plt.Normalize(color_numbers.min(), color_numbers.max())
        color_max = float(traj[t_column].max())
        logger.info("Drawing multicolor lines takes awhile. "
                    "Come back in a minute.")
        for particle in x:
            x_series = x[particle].dropna()
            y_series = y[particle].dropna()
            frames = np.array(x_series.index)
            frames_map = frames/color_max
            points = np.array([x_series.values, y_series.values]).T.reshape(-1, 1, 2)
            segments = np.concatenate([points[:-1], points[1:]], axis=1)
            lc = LineCollection(segments, cmap=cmap, norm=norm)
            lc.set_array(frames_map)
            ax.add_collection(lc)
            ax.set_xlim(x.apply(np.min).min(), x.apply(np.max).max())
            ax.set_ylim(y.apply(np.min).min(), y.apply(np.max).max())

I am not sure if it is a correct way to do it, but it solved my problem.
I am a newbie. I may be wrong, but I still hope this can help someone with the similar issue.

[tacaswell]

@MTone92 Can you please open a pull request with that change?