-
Notifications
You must be signed in to change notification settings - Fork 1
/
Copy pathVergleiche_SVC_IEMOvsMSP
287 lines (222 loc) · 9.9 KB
/
Vergleiche_SVC_IEMOvsMSP
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
IEMO vs MSP
IEMO classes: angry(0), excited(2), neutral(6), sad(8)
merged excited(2) and happy(5):
precision recall f1-score support
A 0.75 0.35 0.47 327
H 0.46 0.48 0.47 502
N 0.50 0.53 0.52 537
S 0.50 0.69 0.58 294
accuracy 0.51 1660
macro avg 0.55 0.51 0.51 1660
weighted avg 0.54 0.51 0.50 1660
[[113 138 47 29]
[ 33 240 173 56]
[ 4 124 287 122]
[ 1 21 68 204]]
pred: (array(['A', 'H', 'N', 'S'], dtype='<U1'), array([151, 523, 575, 411]))
test: (array(['A', 'H', 'N', 'S'], dtype='<U1'), array([327, 502, 537, 294]))
train: (array(['A', 'H', 'N', 'S'], dtype='<U1'), array([ 776, 1134, 1171, 790]))
..............................................................
MSP classes: angry(A), happy(H), neutral(N), sad(S)
removed Preperation data (P-folder):
precision recall f1-score support
A 0.46 0.12 0.19 220
H 0.44 0.24 0.31 358
N 0.51 0.92 0.66 718
S 1.00 0.00 0.01 252
accuracy 0.50 1548
macro avg 0.60 0.32 0.29 1548
weighted avg 0.57 0.50 0.40 1548
[[ 26 50 144 0]
[ 13 87 258 0]
[ 12 47 659 0]
[ 6 15 230 1]]
pred: (array(['A', 'H', 'N', 'S'], dtype='<U1'), array([ 57, 199, 1291, 1]))
test: (array(['A', 'H', 'N', 'S'], dtype='<U1'), array([220, 358, 718, 252]))
train: (array(['A', 'H', 'N', 'S'], dtype='<U1'), array([ 534, 914, 1595, 567]))
using only half the data annotated neutral:
precision recall f1-score support
A 0.39 0.10 0.16 231
H 0.40 0.54 0.46 371
N 0.45 0.52 0.48 365
S 0.43 0.42 0.43 234
accuracy 0.43 1201
macro avg 0.42 0.40 0.38 1201
weighted avg 0.42 0.43 0.40 1201
[[ 24 131 41 35]
[ 20 200 107 44]
[ 11 114 188 52]
[ 6 51 78 99]]
pred: (array(['A', 'H', 'N', 'S'], dtype='<U1'), array([ 61, 496, 414, 230]))
test: (array(['A', 'H', 'N', 'S'], dtype='<U1'), array([231, 371, 365, 234]))
train: (array(['A', 'H', 'N', 'S'], dtype='<U1'), array([523, 901, 792, 585]))
--------------------------------------------------------------
--------------------------------------------------------------
compare two classes:
angry vs happy
IEMO (excited merged with happy)
precision recall f1-score support
A 0.71 0.32 0.44 326
H 0.67 0.91 0.77 496
accuracy 0.68 822
macro avg 0.69 0.62 0.61 822
weighted avg 0.69 0.68 0.64 822
[[105 221]
[ 43 453]]
pred: (array(['A', 'H'], dtype='<U1'), array([148, 674]))
test: (array(['A', 'H'], dtype='<U1'), array([326, 496]))
train: (array(['A', 'H'], dtype='<U1'), array([ 777, 1140]))
MSP
precision recall f1-score support
A 0.71 0.15 0.25 222
H 0.66 0.96 0.79 386
accuracy 0.67 608
macro avg 0.69 0.56 0.52 608
weighted avg 0.68 0.67 0.59 608
[[ 34 188]
[ 14 372]]
pred: (array(['A', 'H'], dtype='<U1'), array([ 48, 560]))
test: (array(['A', 'H'], dtype='<U1'), array([222, 386]))
train: (array(['A', 'H'], dtype='<U1'), array([532, 886]))
______________________________________________________________
angry vs neutral
IEMO
precision recall f1-score support
A 0.84 0.61 0.70 327
N 0.79 0.92 0.85 517
accuracy 0.80 844
macro avg 0.81 0.77 0.78 844
weighted avg 0.81 0.80 0.79 844
[[199 128]
[ 39 478]]
pred: (array(['A', 'N'], dtype='<U1'), array([238, 606]))
test: (array(['A', 'N'], dtype='<U1'), array([327, 517]))
train: (array(['A', 'N'], dtype='<U1'), array([ 776, 1191]))
MSP
precision recall f1-score support
A 0.73 0.21 0.33 225
N 0.79 0.97 0.87 696
accuracy 0.79 921
macro avg 0.76 0.59 0.60 921
weighted avg 0.78 0.79 0.74 921
[[ 48 177]
[ 18 678]]
pred: (array(['A', 'N'], dtype='<U1'), array([ 66, 855]))
test: (array(['A', 'N'], dtype='<U1'), array([225, 696]))
train: (array(['A', 'N'], dtype='<U1'), array([ 529, 1617]))
______________________________________________________________
angry vs sad
IEMO
precision recall f1-score support
A 0.90 0.85 0.87 330
S 0.85 0.91 0.88 327
accuracy 0.88 657
macro avg 0.88 0.88 0.88 657
weighted avg 0.88 0.88 0.88 657
[[279 51]
[ 31 296]]
pred: (array(['A', 'S'], dtype='<U1'), array([310, 347]))
test: (array(['A', 'S'], dtype='<U1'), array([330, 327]))
train: (array(['A', 'S'], dtype='<U1'), array([773, 757]))
MSP
precision recall f1-score support
A 0.81 0.68 0.74 241
S 0.71 0.83 0.77 231
accuracy 0.75 472
macro avg 0.76 0.75 0.75 472
weighted avg 0.76 0.75 0.75 472
[[163 78]
[ 39 192]]
pred: (array(['A', 'S'], dtype='<U1'), array([202, 270]))
test: (array(['A', 'S'], dtype='<U1'), array([241, 231]))
train: (array(['A', 'S'], dtype='<U1'), array([513, 588]))
______________________________________________________________
happy vs neutral
IEMO (merged)
precision recall f1-score support
H 0.72 0.59 0.65 502
N 0.65 0.77 0.71 502
accuracy 0.68 1004
macro avg 0.69 0.68 0.68 1004
weighted avg 0.69 0.68 0.68 1004
[[295 207]
[115 387]]
pred: (array(['H', 'N'], dtype='<U1'), array([410, 594]))
test: (array(['H', 'N'], dtype='<U1'), array([502, 502]))
train: (array(['H', 'N'], dtype='<U1'), array([1134, 1206]))
MSP
precision recall f1-score support
H 0.72 0.22 0.34 375
N 0.70 0.95 0.80 701
accuracy 0.70 1076
macro avg 0.71 0.59 0.57 1076
weighted avg 0.70 0.70 0.64 1076
[[ 83 292]
[ 33 668]]
pred: (array(['H', 'N'], dtype='<U1'), array([116, 960]))
test: (array(['H', 'N'], dtype='<U1'), array([375, 701]))
train: (array(['H', 'N'], dtype='<U1'), array([ 897, 1612]))
______________________________________________________________
happy vs sad
IEMO (merged)
precision recall f1-score support
H 0.87 0.80 0.84 488
S 0.74 0.83 0.78 328
accuracy 0.81 816
macro avg 0.80 0.81 0.81 816
weighted avg 0.82 0.81 0.81 816
[[391 97]
[ 57 271]]
pred: (array(['H', 'S'], dtype='<U1'), array([448, 368]))
test: (array(['H', 'S'], dtype='<U1'), array([488, 328]))
train: (array(['H', 'S'], dtype='<U1'), array([1148, 756]))
MSP
precision recall f1-score support
H 0.80 0.79 0.80 387
S 0.67 0.68 0.67 241
accuracy 0.75 628
macro avg 0.73 0.73 0.73 628
weighted avg 0.75 0.75 0.75 628
[[307 80]
[ 78 163]]
pred: (array(['H', 'S'], dtype='<U1'), array([385, 243]))
test: (array(['H', 'S'], dtype='<U1'), array([387, 241]))
train: (array(['H', 'S'], dtype='<U1'), array([885, 578]))
______________________________________________________________
neutral vs sad
IEMO
precision recall f1-score support
N 0.78 0.80 0.79 521
S 0.66 0.64 0.65 317
accuracy 0.74 838
macro avg 0.72 0.72 0.72 838
weighted avg 0.74 0.74 0.74 838
[[416 105]
[115 202]]
pred: (array(['N', 'S'], dtype='<U1'), array([531, 307]))
test: (array(['N', 'S'], dtype='<U1'), array([521, 317]))
train: (array(['N', 'S'], dtype='<U1'), array([1187, 767]))
MSP
precision recall f1-score support
N 0.74 1.00 0.85 693
S 0.00 0.00 0.00 247
accuracy 0.74 940
macro avg 0.37 0.50 0.42 940
weighted avg 0.54 0.74 0.63 940
[[693 0]
[247 0]]
pred: (array(['N'], dtype='<U1'), array([940]))
test: (array(['N', 'S'], dtype='<U1'), array([693, 247]))
train: (array(['N', 'S'], dtype='<U1'), array([1620, 572]))
using only half the data annotated neutral:
precision recall f1-score support
N 0.67 0.87 0.76 348
S 0.69 0.39 0.50 245
accuracy 0.67 593
macro avg 0.68 0.63 0.63 593
weighted avg 0.68 0.67 0.65 593
[[304 44]
[149 96]]
pred: (array(['N', 'S'], dtype='<U1'), array([453, 140]))
test: (array(['N', 'S'], dtype='<U1'), array([348, 245]))
train: (array(['N', 'S'], dtype='<U1'), array([809, 574]))