Q: Cleaning stimulus channel without preloading everything

[kingjr]

Is there a way to clean a specific channel (e.g. stimulus channel) so as to do subsequent processing that do not require loading the entire dataset (e.g. find_events, Epochs etc) without having to preload the entire raw data?
e.g.

raw = Raw(raw_fname)
raw.pick_types(stim=True, meg=False)
stim_data = my_cleaning_function(raw.get_data())
events = find_events(stim_data)
epochs = Epochs(raw, events)

[larsoner]

You have a call to find_events with an array, which I don't think we support (but could I guess), so you'd need something like:

raw = mne.io.read_raw_fif(...)
idx = raw.ch_names.index(stim_channel)
stim_data = my_cleaning_function(raw[idx][0])
raw_stim = mne.io.RawArray(stim_data, mne.create_info(1, raw.info['sfreq'], stim_channel)
events = mne.find_events(raw_stim)
events[:, 0] += raw.first_samp

[kingjr]

Why can't we just pick channel type if data isn't loaded?

[larsoner]

If that's the question, we should close as dup of #1766

To do it right it would complicate the logic inside the reader, and would take a decent amount of work. It's possible but not easy. A simple hack version can be constructed by returning a RawArray but this is not as clean and breaks class-specific functionality.

[larsoner]

Actually now that #3896 by @jaeilepp is in, the right ideas are there at least. We'd want to generalize the code so it wasn't class-specific anymore, though, which would (again) take some thinking.

[kingjr]

ok for closing in favour of #1766.

It may complicate the life of the header function, but frankly, here it's crazy. Cleaning trigger channel is eeg/ecog 101.

[larsoner]

Cleaning trigger channel is eeg/ecog 101.

I'm not sure what you mean? AFAIK people usually re-code events after extracting events from TTL voltages rather than change the TTL voltages to get appropriate events via event extraction.

[larsoner]

... at least that's the MNE way

[kingjr]

That's not trigger cleaning, it's event selection and (re)coding. It is common to have trigger artifacts that need signal processing.

[agramfort]

what do you suggest? just allow an ndarray in find_events?

[kingjr]

That could be an option yes.

…

On 18 January 2017 at 08:41, Alexandre Gramfort ***@***.***> wrote: what do you suggest? just allow an ndarray in find_events? — You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub <#3914 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AEp7DJMFbeRZK582cKOVxM3FlTx9BUWrks5rThakgaJpZM4LmXB4> .

[agramfort]

and you find this: raw = mne.io.read_raw_fif(...) idx = raw.ch_names.index(stim_channel) stim_data = my_cleaning_function(raw[idx][0]) raw_stim = mne.io.RawArray(stim_data, mne.create_info(1, raw.info['sfreq'], stim_channel) events = mne.find_events(raw_stim) events[:, 0] += raw.first_samp too complicated? I guess not for you but colleagues?

[kingjr]

Imagine guessing 1) reading a channel by using an index of raw 2) go find the function to create a new raw data in i/o, 3) understand create_info, 4) think about 'first_samp'. IMO Fieldtrip/eeglab etc users are used to arrays; this forces an arguably heavy and opaque class.

…

On 18 January 2017 at 15:33, Alexandre Gramfort ***@***.***> wrote: and you find this: raw = mne.io.read_raw_fif(...) idx = raw.ch_names.index(stim_channel) stim_data = my_cleaning_function(raw[idx][0]) raw_stim = mne.io.RawArray(stim_data, mne.create_info(1, raw.info['sfreq'], stim_channel) events = mne.find_events(raw_stim) events[:, 0] += raw.first_samp too complicated? I guess not for you but colleagues? — You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub <#3914 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AEp7DGyIvCkJdA0Km1P_IRV4ceuMadngks5rTncRgaJpZM4LmXB4> .

[agramfort]

ok ok !