Training fails due to missing AVX on Intel Avoton Atom

[Hypfer]

Using the :latest docker image of rhasspy (2.5.8), I'm unable to use deepspeech on my Supermicro A1SAi-2750F-based NAS/Server.

Logs, CPU Information, etc.

The training process crashes constantly due to an illegal instruction as can be seen in the container logs:

/usr/lib/rhasspy/rhasspy-asr-deepspeech-hermes/bin/rhasspy-asr-deepspeech-hermes: line 15:  1943 Illegal instruction     (core dumped) python3 -m rhasspyasr_deepspeech_hermes "$@"
2021-01-04 20:41:44,605 INFO exited: speech_to_text (exit status 132; not expected)
2021-01-04 20:41:45,611 INFO spawned: 'speech_to_text' with pid 1960
2021-01-04 20:41:46,613 INFO success: speech_to_text entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
/usr/lib/rhasspy/rhasspy-asr-deepspeech-hermes/bin/rhasspy-asr-deepspeech-hermes: line 15:  1964 Illegal instruction     (core dumped) python3 -m rhasspyasr_deepspeech_hermes "$@"
2021-01-04 20:41:47,151 INFO exited: speech_to_text (exit status 132; not expected)
2021-01-04 20:41:48,156 INFO spawned: 'speech_to_text' with pid 1981
2021-01-04 20:41:49,158 INFO success: speech_to_text entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)

as well as the kernel message buffer:

[1195865.031441] traps: python3[4994] trap invalid opcode ip:7fcb985ddea5 sp:7ffd9a6f3f90 error:0 in libdeepspeech.so[7fcb96cfe000+210f000]
[1195867.581137] traps: python3[5312] trap invalid opcode ip:7feb5157bea5 sp:7ffcc419dcf0 error:0 in libdeepspeech.so[7feb4fc9c000+210f000]
[1195870.121306] traps: python3[5335] trap invalid opcode ip:7f4963031ea5 sp:7fff874ebc60 error:0 in libdeepspeech.so[7f4961752000+210f000]
[1195872.662783] traps: python3[5665] trap invalid opcode ip:7f2d0edf0ea5 sp:7ffcad221520 error:0 in libdeepspeech.so[7f2d0d511000+210f000]
[1195875.221799] traps: python3[5692] trap invalid opcode ip:7f181c2ddea5 sp:7ffc03f976a0 error:0 in libdeepspeech.so[7f181a9fe000+210f000]

Looking at /proc/cpuinfo, we get this:

processor       : 0
vendor_id       : GenuineIntel
cpu family      : 6
model           : 77
model name      : Intel(R) Atom(TM) CPU  C2750  @ 2.40GHz

[...]

flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm sse4_1 sse4_2 movbe popcnt tsc_deadline_timer aes rdrand lahf_lm 3dnowprefetch cpuid_fault epb ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid tsc_adjust smep erms dtherm ida arat

If I were to guess, I'd say that this is probably caused by the Atom not supporting AVX which iirc was als the culprit when I've tried Mozillas DeepSpeech on initial release.

It looks like it should be possible to rebuild without AVX, however that would add a performance penalty for those who do in fact have an AVX-capable CPU

While writing this issue, I've noticed that this issue was mentioned at least once in the community and there's another similar issue here: rhasspy/rhasspy#150