exception in CNTK 208 Tutorials

[mohamad-hasan-sohan-ajini]

Hi

CNTK 208 example is about acoustic modeling using CTC loss function, which introduces forward_backward criterion node. Running ipython instruction to 3'rd code batch is fine. But running 4'th one (Train and Save the Model) cause an exception as bellow:

RuntimeError Traceback (most recent call last)
in ()
2 for mb in range(mbs_per_epoch):
3 minibatch = train_data_reader.next_minibatch(mbsize, input_map = train_input_map)
----> 4 trainer.train_minibatch(minibatch)
5 progress_printer.update_with_trainer(trainer, with_metric = True)
6
/home/aspl/anaconda3/envs/cntk-py27/lib/python2.7/site-packages/cntk/train/trainer.pyc in train_minibatch(self, arguments, outputs, device)
168 if contains_minibatch_data:
169 updated = super(Trainer, self).train_minibatch_overload_for_minibatchdata(
--> 170 arguments, device)
171 else:
172 updated = super(Trainer, self).train_minibatch(arguments,
/home/aspl/anaconda3/envs/cntk-py27/lib/python2.7/site-packages/cntk/cntk_py.pyc in train_minibatch_overload_for_minibatchdata(self, *args)
2498
2499 def train_minibatch_overload_for_minibatchdata(self, *args):
-> 2500 return _cntk_py.Trainer_train_minibatch_overload_for_minibatchdata(self, *args)
2501
2502 def train_minibatch(self, *args):
RuntimeError: SetDataLocation [CPUMatrix]: Cannot migrate the matrix between devices because it is a view.
[CALL STACK]
[0x7f705e3066dc] + 0x53c6dc
[0x7f705b4d16bd] Microsoft::MSR::CNTK::Matrix:: SetDataLocation (Microsoft::MSR::CNTK::CurrentDataLocation, Microsoft::MSR::CNTK::MatrixType) const + 0x20d
[0x7f705b4e55e1] Microsoft::MSR::CNTK::Matrix:: _transferFromDeviceToDevice (int, int, bool, bool) const + 0x2f1
[0x7f705b4ea52e] Microsoft::MSR::CNTK::Matrix:: TensorOp (float, Microsoft::MSR::CNTK::Matrix const&, float, Microsoft::MSR::CNTK::ElementWiseOperator, Microsoft::MSR::CNTK::ElementWiseOperator, std::array<unsigned long,2ul> const&, Microsoft::MSR::CNTK::SmallVector const&, std::array<Microsoft::MSR::CNTK::SmallVector,2ul> const&, Microsoft::MSR::CNTK::SmallVector const&, std::array<Microsoft::MSR::CNTK::SmallVector,2ul> const&) + 0x6e
[0x7f705b519f8e] Microsoft::MSR::CNTK::TensorView:: DoUnaryOpOf (float, Microsoft::MSR::CNTK::TensorView const&, float, Microsoft::MSR::CNTK::ElementWiseOperator, Microsoft::MSR::CNTK::ElementWiseOperator) + 0x1de
[0x7f705e4e0700] CNTK::Accumulator:: Update (std::shared_ptrCNTK::Value const&, CNTK::DeviceDescriptor const&) + 0x2a0
[0x7f705e4ced51] CNTK::Trainer:: UpdateTrainingProgress (unsigned long, std::shared_ptrCNTK::Value const&, std::shared_ptrCNTK::Value const&, CNTK::DeviceDescriptor const&) + 0x61
[0x7f705e4d455f] CNTK::Trainer:: TrainMinibatch (std::unordered_map<CNTK::Variable,CNTK::MinibatchData,std::hashCNTK::Variable,std::equal_toCNTK::Variable,std::allocator<std::pair<CNTK::Variable const,CNTK::MinibatchData>>> const&, std::unordered_map<CNTK::Variable,std::shared_ptrCNTK::Value,std::hashCNTK::Variable,std::equal_toCNTK::Variable,std::allocator<std::pair<CNTK::Variable const,std::shared_ptrCNTK::Value>>>&, CNTK::DeviceDescriptor const&) + 0xbf
[0x7f705e4d46d0] CNTK::Trainer:: TrainMinibatch (std::unordered_map<CNTK::Variable,CNTK::MinibatchData,std::hashCNTK::Variable,std::equal_toCNTK::Variable,std::allocator<std::pair<CNTK::Variable const,CNTK::MinibatchData>>> const&, CNTK::DeviceDescriptor const&) + 0xa0
[0x7f705ee6d1f6] + 0x21b1f6
[0x7f7075187c92] PyEval_EvalFrameEx + 0x7762
[0x7f7075189a2e] PyEval_EvalCodeEx + 0x89e
[0x7f7075188a55] PyEval_EvalFrameEx + 0x8525
[0x7f7075189a2e] PyEval_EvalCodeEx + 0x89e
[0x7f7075188a55] PyEval_EvalFrameEx + 0x8525
[0x7f7075189a2e] PyEval_EvalCodeEx + 0x89e
[0x7f7075189b42] PyEval_EvalCode + 0x32
[0x7f7075187d6d] PyEval_EvalFrameEx + 0x783d
[0x7f7075189a2e] PyEval_EvalCodeEx + 0x89e
[0x7f7075188a55] PyEval_EvalFrameEx + 0x8525
[0x7f7075189a2e] PyEval_EvalCodeEx + 0x89e
[0x7f7075188a55] PyEval_EvalFrameEx + 0x8525
[0x7f7075189a2e] PyEval_EvalCodeEx + 0x89e
[0x7f7075188a55] PyEval_EvalFrameEx + 0x8525
[0x7f7075189a2e] PyEval_EvalCodeEx + 0x89e
[0x7f7075188a55] PyEval_EvalFrameEx + 0x8525
[0x7f7075189a2e] PyEval_EvalCodeEx + 0x89e
[0x7f7075188a55] PyEval_EvalFrameEx + 0x8525
[0x7f7075189a2e] PyEval_EvalCodeEx + 0x89e
[0x7f7075188a55] PyEval_EvalFrameEx + 0x8525
[0x7f7075189a2e] PyEval_EvalCodeEx + 0x89e
[0x7f70751054a8] + 0x794a8
[0x7f70750d5d23] PyObject_Call + 0x53
[0x7f7075186797] PyEval_EvalFrameEx + 0x6267
[0x7f7075189a2e] PyEval_EvalCodeEx + 0x89e
[0x7f7075188a55] PyEval_EvalFrameEx + 0x8525
[0x7f7075189a2e] PyEval_EvalCodeEx + 0x89e
[0x7f7075189b42] PyEval_EvalCode + 0x32
[0x7f70751aa050] PyRun_FileExFlags + 0xb0
[0x7f70751aa22f] PyRun_SimpleFileExFlags + 0xef
[0x7f70751bfb74] Py_Main + 0xca4
[0x7f70743b9f45] __libc_start_main + 0xf5
[0x400649]

please take a look at this tutorial. @vmazalov
regards

[vmazalov]

You probably try to run the tutorial from an older build. This tutorial relies on a change that was merged to the master branch a couple days ago, but is not in the official release yet. Please try to pull latest from master and build. There is an E2E test that guards from the type of exceptions you mentioned:
https://github.com/Microsoft/CNTK/blob/master/Tests/EndToEndTests/CNTKv2Python/Tutorials/CNTK_208_Speech_Connectionist_Temporal_Classification_test.py