forked from priyaraut55/scst
-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathREADME_in-tree
758 lines (602 loc) · 29.8 KB
/
README_in-tree
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
iSCSI SCST target driver
========================
ISCSI-SCST is a deeply reworked fork of iSCSI Enterprise Target (IET)
(http://iscsitarget.sourceforge.net). Reasons of the fork were:
- To be able to use full power of SCST core.
- To fix all the problems, corner cases issues and iSCSI standard
violations which IET has.
See for more info http://iscsi-scst.sourceforge.net.
Usage
-----
See in http://iscsi-scst.sourceforge.net/iscsi-scst-howto.txt how to
configure iSCSI-SCST.
If you want to use Intel CRC32 offload and have corresponding hardware,
you should load crc32c-intel module. Then iSCSI-SCST will do all digest
calculations using this facility.
In 2.0.0 usage of iscsi-scstd.conf as well as iscsi-scst-adm utility is
obsolete. Use the sysfs interface facilities instead.
The flow of iSCSI-SCST inialization should be as the following:
1. Load of SCST and iSCSI-SCST kernel modules with necessary module
parameters, if needed.
2. Start iSCSI-SCST service.
3. Configure targets, devices, LUNs, etc. either using scstadmin
(recommended), or using the sysfs interface directly as described below.
It is recommended to use TEST UNIT READY ("tur") command to check if
iSCSI-SCST target is alive in MPIO configurations.
Also see SCST README file how to tune for the best performance.
CAUTION: Working of target and initiator on the same host isn't fully
======= supported. See SCST README file for details.
Sysfs interface
---------------
Root of SCST sysfs interface is /sys/kernel/scst_tgt. Root of iSCSI-SCST
is /sys/kernel/scst_tgt/targets/iscsi. It has the following entries:
- None, one or more subdirectories for targets with name equal to names
of the corresponding targets.
- IncomingUser[num] - optional one or more attributes containing user
name and password for incoming discovery user name. Not exist by
default and can be added through "mgmt" entry, see below.
- OutgoingUser - optional attribute containing user name and password
for outgoing discovery user name. Not exist by default and can be
added through "mgmt" entry, see below.
- iSNSServer - contains name or IP address of iSNS server with optional
"AccessControl" attribute, which allows to enable iSNS access
control. Empty by default.
- allowed_portal[num] - optional attribute, which specifies, on which
portals (target's IP addresses) this target will be available. If not
specified (default) the target will be available on all all portals.
As soon as at least one allowed_portal specified, the target will be
accessible for initiators only on the specified portals. There might
be any number of the allowed_portal attributes. The portals
specification in the allowed_portal attributes can be a simple
DOS-type patterns, containing '*' and '?' symbols. '*' means match
all any symbols, '?' means match only any single symbol. For
instance, "10.170.77.2" will match "10.170.7?.*". Additionally, you
can use negative sign '!' to revert the value of the pattern. For
instance, "10.170.67.2" will match "!10.170.7?.*". See examples
below.
- enabled - using this attribute you can enable or disable iSCSI-SCST
accept new connections. It allows to finish configuring global
iSCSI-SCST attributes before it starts accepting new connections. 0
by default.
- open_state - read-only attribute, which allows to see if the user
space part of iSCSI-SCST connected to the kernel part.
- per_portal_acl - if set, makes iSCSI-SCST work in the per-portal
access control mode. In this mode iSCSI-SCST registers all initiators
in SCST core as "initiator_name#portal_IP_address" pattern, like
"iqn.2006-10.net.vlnb:ini#10.170.77.2" for initiator
iqn.2006-10.net.vlnb connected through portal 10.170.77.2. This mode
allows to make particular initiators be able to use only particular
portals on the target and don't see/be able to connect through
others. See below for more details.
- trace_level - allows to enable and disable various tracing
facilities. See content of this file for help how to use it.
- version - read-only attribute, which allows to see version of
iSCSI-SCST and enabled optional features.
- mgmt - main management entry, which allows to configure iSCSI-SCST.
Namely, add/delete targets as well as add/delete optional global and
per-target attributes. See content of this file for help how to use
it.
Each iSCSI-SCST sysfs file (attribute) can contain in the last line mark
"[key]". It is automatically added mark used to allow scstadmin to see
which attributes it should save in the config file. You can ignore it.
Each target subdirectory contains the following entries:
- ini_groups - subdirectory defining initiator groups for this target,
used to define per-initiator access control. See SCST core README for
more details.
- luns - subdirectory defining LUNs of this target. See SCST core
README for more details.
- sessions - subdirectory containing connected to this target sessions.
- IncomingUser[num] - optional one or more attributes containing user
name and password for incoming user name. Not exist by default and can
be added through the "mgmt" entry, see above.
- OutgoingUser - optional attribute containing user name and password
for outgoing user name. Not exist by default and can be added through
the "mgmt" entry, see above.
- Entries defining default iSCSI parameters values used during iSCSI
parameters negotiation. Only entries which can be changed or make
sense are listed there.
- QueuedCommands - defines maximum number of commands queued to any
session of this target. Default is 32 commands.
- NopInInterval - defines interval between NOP-In requests, which the
target will send on idle connections to check if the initiator is
still alive. If there is no NOP-Out reply from the initiator in
NopInTimeout seconds, the corresponding connection will be closed. Default
is 30 seconds. If it's set to 0, then NOP-In requests are disabled.
- NopInTimeout - defines the maximum time in seconds a NOP-In request
can wait for response from initiator, otherwise the corresponding
connection will be closed. Default is 30 seconds.
- RspTimeout - defines the maximum time in seconds a command can wait for
response from initiator, otherwise the corresponding connection will
be closed. Default is 90 seconds.
- enabled - using this attribute you can enable or disable iSCSI-SCST
accept new connections to this target. It allows to finish
configuring it before it starts accepting new connections. 0 by
default.
- redirect - allows to temporarily or permanently redirect login to the
target to another portal. Discovery sessions will not be impacted,
but normal sessions will be redirected before security negotiation.
The destination should be specified using format "<ip_addr>[:port] temp|perm".
IPv6 addresses need to be enclosed in [] brackets. To remove
redirection, provide an empty string. For example:
echo "10.170.77.2:32600 temp" >/sys/kernel/scst_tgt/targets/iscsi/iqn.2006-10.net.vlnb:tgt/redirect
will temporarily redirect login to portal 10.170.77.2 and port 32600.
- tid - TID of this target.
Subdirectory "sessions" contains one subdirectory for each connected
session with name equal to name of the connected initiator.
Each session subdirectory contains the following entries:
- One subdirectory for each TCP connection in this session. ISCSI-SCST
supports 1 connection per session, but the session subdirectory can
contain several connections: one active and other being closed.
- Entries defining negotiated iSCSI parameters. Only parameters which
can be changed or make sense are listed there.
- initiator_name - contains initiator name
- sid - contains SID of this session
- reinstating - contains reinstatement state of this session
- force_close - write-only attribute, which allows to force close this
session. This is the only writable session attribute.
- active_commands - contains number of active, i.e. not yet or being
executed, SCSI commands in this session.
- commands - contains overall number of SCSI commands in this session.
- thread_pid - Process IDs (PIDs) of the iscsi{wr,rd} kernel threads that
process the SCSI commands for this session.
Each connection subdirectory contains the following entries:
- cid - contains CID of this connection.
- ip - contains IP address of the connected initiator.
- state - contains processing state of this connection.
Each initiator group subdirectory contains:
- per_sess_dedicated_tgt_threads - if set, each iSCSI session has
dedicated, i.e. not shared with other sessions, pool of the
iscsi{wr,rd} kernel threads. Useful to control per-session CPU
affinity to improve performance. Default: not set.
See SCST README for info about other attributes.
Below is a sample script, which configures 1 virtual disk "disk1" using
/disk1 image and one target iqn.2006-10.net.vlnb:tgt with all default
parameters:
#!/bin/bash
modprobe scst
modprobe scst_vdisk
echo "add_device disk1 filename=/disk1; nv_cache=1" >/sys/kernel/scst_tgt/handlers/vdisk_fileio/mgmt
service iscsi-scst start
echo "add_target iqn.2006-10.net.vlnb:tgt" >/sys/kernel/scst_tgt/targets/iscsi/mgmt
echo "add disk1 0" >/sys/kernel/scst_tgt/targets/iscsi/iqn.2006-10.net.vlnb:tgt/luns/mgmt
echo 1 >/sys/kernel/scst_tgt/targets/iscsi/iqn.2006-10.net.vlnb:tgt/enabled
echo 1 >/sys/kernel/scst_tgt/targets/iscsi/enabled
Below is another sample script, which configures 1 real local SCSI disk
0:0:1:0 and one target iqn.2006-10.net.vlnb:tgt with all default parameters:
#!/bin/bash
modprobe scst
modprobe scst_disk
echo "add_device 0:0:1:0" >/sys/kernel/scst_tgt/handlers/dev_disk/mgmt
service iscsi-scst start
echo "add_target iqn.2006-10.net.vlnb:tgt" >/sys/kernel/scst_tgt/targets/iscsi/mgmt
echo "add 0:0:1:0 0" >/sys/kernel/scst_tgt/targets/iscsi/iqn.2006-10.net.vlnb:tgt/luns/mgmt
echo 1 >/sys/kernel/scst_tgt/targets/iscsi/iqn.2006-10.net.vlnb:tgt/enabled
echo 1 >/sys/kernel/scst_tgt/targets/iscsi/enabled
Below is an advanced sample script, which configures more virtual
devices of various types, including virtual CDROM and 2 targets, one
with all default parameters, another one with some not default
parameters, incoming and outgoing user names for CHAP authentication,
and special permissions for initiator iqn.2005-03.org.open-iscsi:cacdcd2520,
which will see another set of devices. Also this sample configures CHAP
authentication for discovery sessions and iSNS server with access
control.
#!/bin/bash
modprobe scst
modprobe scst_vdisk
echo "add_device disk1 filename=/disk1; nv_cache=1" >/sys/kernel/scst_tgt/handlers/vdisk_fileio/mgmt
echo "add_device disk2 filename=/disk2; blocksize=4096; nv_cache=1" >/sys/kernel/scst_tgt/handlers/vdisk_fileio/mgmt
echo "add_device blockio filename=/dev/sda5" >/sys/kernel/scst_tgt/handlers/vdisk_blockio/mgmt
echo "add_device nullio" >/sys/kernel/scst_tgt/handlers/vdisk_nullio/mgmt
echo "add_device cdrom" >/sys/kernel/scst_tgt/handlers/vcdrom/mgmt
service iscsi-scst start
echo "192.168.1.16 AccessControl" >/sys/kernel/scst_tgt/targets/iscsi/iSNSServer
echo "add_attribute IncomingUser joeD 12charsecret" >/sys/kernel/scst_tgt/targets/iscsi/mgmt
echo "add_attribute OutgoingUser jackD 12charsecret1" >/sys/kernel/scst_tgt/targets/iscsi/mgmt
echo "add_target iqn.2006-10.net.vlnb:tgt" >/sys/kernel/scst_tgt/targets/iscsi/mgmt
echo "add disk1 0" >/sys/kernel/scst_tgt/targets/iscsi/iqn.2006-10.net.vlnb:tgt/luns/mgmt
echo "add cdrom 1" >/sys/kernel/scst_tgt/targets/iscsi/iqn.2006-10.net.vlnb:tgt/luns/mgmt
echo "add_target iqn.2006-10.net.vlnb:tgt1" >/sys/kernel/scst_tgt/targets/iscsi/mgmt
echo "add_target_attribute iqn.2006-10.net.vlnb:tgt1 IncomingUser1 joe2 12charsecret2" >/sys/kernel/scst_tgt/targets/iscsi/mgmt
echo "add_target_attribute iqn.2006-10.net.vlnb:tgt1 IncomingUser joe 12charsecret" >/sys/kernel/scst_tgt/targets/iscsi/mgmt
echo "add_target_attribute iqn.2006-10.net.vlnb:tgt1 OutgoingUser jim1 12charpasswd" >/sys/kernel/scst_tgt/targets/iscsi/mgmt
echo "No" >/sys/kernel/scst_tgt/targets/iscsi/iqn.2006-10.net.vlnb:tgt1/InitialR2T
echo "Yes" >/sys/kernel/scst_tgt/targets/iscsi/iqn.2006-10.net.vlnb:tgt1/ImmediateData
echo "8192" >/sys/kernel/scst_tgt/targets/iscsi/iqn.2006-10.net.vlnb:tgt1/MaxRecvDataSegmentLength
echo "8192" >/sys/kernel/scst_tgt/targets/iscsi/iqn.2006-10.net.vlnb:tgt1/MaxXmitDataSegmentLength
echo "131072" >/sys/kernel/scst_tgt/targets/iscsi/iqn.2006-10.net.vlnb:tgt1/MaxBurstLength
echo "32768" >/sys/kernel/scst_tgt/targets/iscsi/iqn.2006-10.net.vlnb:tgt1/FirstBurstLength
echo "1" >/sys/kernel/scst_tgt/targets/iscsi/iqn.2006-10.net.vlnb:tgt1/MaxOutstandingR2T
echo "CRC32C,None" >/sys/kernel/scst_tgt/targets/iscsi/iqn.2006-10.net.vlnb:tgt1/HeaderDigest
echo "CRC32C,None" >/sys/kernel/scst_tgt/targets/iscsi/iqn.2006-10.net.vlnb:tgt1/DataDigest
echo "32" >/sys/kernel/scst_tgt/targets/iscsi/iqn.2006-10.net.vlnb:tgt1/QueuedCommands
echo "add disk2 0" >/sys/kernel/scst_tgt/targets/iscsi/iqn.2006-10.net.vlnb:tgt1/luns/mgmt
echo "add nullio 26" >/sys/kernel/scst_tgt/targets/iscsi/iqn.2006-10.net.vlnb:tgt1/luns/mgmt
echo "create special_ini" >/sys/kernel/scst_tgt/targets/iscsi/iqn.2006-10.net.vlnb:tgt1/ini_groups/mgmt
echo "add blockio 0 read_only=1" >/sys/kernel/scst_tgt/targets/iscsi/iqn.2006-10.net.vlnb:tgt1/ini_groups/special_ini/luns/mgmt
echo "add iqn.2005-03.org.open-iscsi:cacdcd2520" >/sys/kernel/scst_tgt/targets/iscsi/iqn.2006-10.net.vlnb:tgt1/ini_groups/special_ini/initiators/mgmt
echo 1 >/sys/kernel/scst_tgt/targets/iscsi/iqn.2006-10.net.vlnb:tgt/enabled
echo 1 >/sys/kernel/scst_tgt/targets/iscsi/iqn.2006-10.net.vlnb:tgt1/enabled
echo 1 >/sys/kernel/scst_tgt/targets/iscsi/enabled
The resulting overall SCST sysfs hierarchy with an initiator connected to
both iSCSI-SCST targets will look like:
/sys/kernel/scst_tgt
|-- devices
| |-- blockio
| | |-- blocksize
| | |-- exported
| | | `-- export0 -> ../../../targets/iscsi/iqn.2006-10.net.vlnb:tgt1/ini_groups/special_ini/luns/0
| | |-- filename
| | |-- handler -> ../../handlers/vdisk_blockio
| | |-- nv_cache
| | |-- read_only
| | |-- removable
| | |-- resync_size
| | |-- size_mb
| | |-- t10_dev_id
| | |-- threads_num
| | |-- threads_pool_type
| | |-- type
| | `-- usn
| |-- cdrom
| | |-- exported
| | | `-- export0 -> ../../../targets/iscsi/iqn.2006-10.net.vlnb:tgt/luns/1
| | |-- filename
| | |-- handler -> ../../handlers/vcdrom
| | |-- size_mb
| | |-- t10_dev_id
| | |-- threads_num
| | |-- threads_pool_type
| | |-- type
| | `-- usn
| |-- disk1
| | |-- blocksize
| | |-- exported
| | | `-- export0 -> ../../../targets/iscsi/iqn.2006-10.net.vlnb:tgt/luns/0
| | |-- filename
| | |-- handler -> ../../handlers/vdisk_fileio
| | |-- nv_cache
| | |-- o_direct
| | |-- read_only
| | |-- removable
| | |-- resync_size
| | |-- size_mb
| | |-- t10_dev_id
| | |-- type
| | |-- usn
| | `-- write_through
| |-- disk2
| | |-- blocksize
| | |-- exported
| | | `-- export0 -> ../../../targets/iscsi/iqn.2006-10.net.vlnb:tgt1/luns/0
| | |-- filename
| | |-- handler -> ../../handlers/vdisk_fileio
| | |-- nv_cache
| | |-- o_direct
| | |-- read_only
| | |-- removable
| | |-- resync_size
| | |-- size_mb
| | |-- t10_dev_id
| | |-- threads_num
| | |-- threads_pool_type
| | |-- threads_num
| | |-- threads_pool_type
| | |-- type
| | |-- usn
| | `-- write_through
| `-- nullio
| |-- blocksize
| |-- exported
| | `-- export0 -> ../../../targets/iscsi/iqn.2006-10.net.vlnb:tgt1/luns/26
| |-- handler -> ../../handlers/vdisk_nullio
| |-- read_only
| |-- removable
| |-- size_mb
| |-- t10_dev_id
| |-- threads_num
| |-- threads_pool_type
| |-- type
| `-- usn
|-- handlers
| |-- vcdrom
| | |-- cdrom -> ../../devices/cdrom
| | |-- mgmt
| | |-- trace_level
| | `-- type
| |-- vdisk_blockio
| | |-- blockio -> ../../devices/blockio
| | |-- mgmt
| | |-- trace_level
| | `-- type
| |-- vdisk_fileio
| | |-- disk1 -> ../../devices/disk1
| | |-- disk2 -> ../../devices/disk2
| | |-- mgmt
| | |-- trace_level
| | `-- type
| `-- vdisk_nullio
| |-- mgmt
| |-- nullio -> ../../devices/nullio
| |-- trace_level
| `-- type
|-- sgv
| |-- global_stats
| |-- sgv
| | `-- stats
| |-- sgv-clust
| | `-- stats
| `-- sgv-dma
| `-- stats
|-- targets
| `-- iscsi
| |-- IncomingUser
| |-- OutgoingUser
| |-- enabled
| |-- iSNSServer
| |-- iqn.2006-10.net.vlnb:tgt
| | |-- DataDigest
| | |-- FirstBurstLength
| | |-- HeaderDigest
| | |-- ImmediateData
| | |-- InitialR2T
| | |-- MaxBurstLength
| | |-- MaxOutstandingR2T
| | |-- MaxRecvDataSegmentLength
| | |-- MaxXmitDataSegmentLength
| | |-- NopInInterval
| | |-- QueuedCommands
| | |-- RspTimeout
| | |-- enabled
| | |-- ini_groups
| | | `-- mgmt
| | |-- luns
| | | |-- 0
| | | | |-- device -> ../../../../../devices/disk1
| | | | `-- read_only
| | | |-- 1
| | | | |-- device -> ../../../../../devices/cdrom
| | | | `-- read_only
| | | `-- mgmt
| | |-- per_portal_acl
| | |-- redirect
| | |-- rel_tgt_id
| | |-- sessions
| | | `-- iqn.2005-03.org.open-iscsi:cacdcd2520
| | | |-- 10.170.75.2
| | | | |-- cid
| | | | |-- ip
| | | | `-- state
| | | |-- DataDigest
| | | |-- FirstBurstLength
| | | |-- HeaderDigest
| | | |-- ImmediateData
| | | |-- InitialR2T
| | | |-- MaxBurstLength
| | | |-- MaxOutstandingR2T
| | | |-- MaxRecvDataSegmentLength
| | | |-- MaxXmitDataSegmentLength
| | | |-- active_commands
| | | |-- commands
| | | |-- force_close
| | | |-- initiator_name
| | | |-- luns -> ../../luns
| | | |-- reinstating
| | | `-- sid
| | `-- tid
| |-- iqn.2006-10.net.vlnb:tgt1
| | |-- DataDigest
| | |-- FirstBurstLength
| | |-- HeaderDigest
| | |-- ImmediateData
| | |-- IncomingUser
| | |-- IncomingUser1
| | |-- InitialR2T
| | |-- MaxBurstLength
| | |-- MaxOutstandingR2T
| | |-- MaxRecvDataSegmentLength
| | |-- MaxXmitDataSegmentLength
| | |-- OutgoingUser
| | |-- NopInInterval
| | |-- QueuedCommands
| | |-- RspTimeout
| | |-- enabled
| | |-- ini_groups
| | | |-- mgmt
| | | `-- special_ini
| | | |-- initiators
| | | | |-- iqn.2005-03.org.open-iscsi:cacdcd2520
| | | | `-- mgmt
| | | `-- luns
| | | |-- 0
| | | | |-- device -> ../../../../../../../devices/blockio
| | | | `-- read_only
| | | `-- mgmt
| | |-- luns
| | | |-- 0
| | | | |-- device -> ../../../../../devices/disk2
| | | | `-- read_only
| | | |-- 26
| | | | |-- device -> ../../../../../devices/nullio
| | | | `-- read_only
| | | `-- mgmt
| | |-- per_portal_acl
| | |-- redirect
| | |-- rel_tgt_id
| | |-- sessions
| | | `-- iqn.2005-03.org.open-iscsi:cacdcd2520
| | | |-- 10.170.75.2
| | | | |-- cid
| | | | |-- ip
| | | | `-- state
| | | |-- DataDigest
| | | |-- FirstBurstLength
| | | |-- HeaderDigest
| | | |-- ImmediateData
| | | |-- InitialR2T
| | | |-- MaxBurstLength
| | | |-- MaxOutstandingR2T
| | | |-- MaxRecvDataSegmentLength
| | | |-- MaxXmitDataSegmentLength
| | | |-- active_commands
| | | |-- commands
| | | |-- force_close
| | | |-- initiator_name
| | | |-- luns -> ../../ini_groups/special_ini/luns
| | | |-- reinstating
| | | `-- sid
| | `-- tid
| |-- mgmt
| |-- open_state
| |-- trace_level
| `-- version
|-- threads
|-- trace_level
`-- version
Advanced initiators access control
----------------------------------
ISCSI-SCST allows you to optionally control visibility and accessibility
of your target and its portals (IP addresses) to remote initiators. This
control includes both the target's portals SendTargets discovery as well
as regular LUNs access.
This facility supersedes the obsolete initiators.[allow,deny] method,
which is going to be removed in one of the future versions.
This facility is available only in the sysfs build of iSCSI-SCST.
By default, all portals are available for the initiators.
1. If you want to enable/disable one or more target's portals for all
initiators, you should define one ore more allowed_portal attributes.
For example:
echo 'add_target_attribute iqn.2006-10.net.vlnb:tgt allowed_portal 10.170.77.2' >/sys/kernel/scst_tgt/targets/iscsi/mgmt
will enable only portal 10.170.77.2 and disable all other portals
echo 'add_target_attribute iqn.2006-10.net.vlnb:tgt allowed_portal 10.170.77.2' >/sys/kernel/scst_tgt/targets/iscsi/mgmt
echo 'add_target_attribute iqn.2006-10.net.vlnb:tgt allowed_portal 10.170.75.2' >/sys/kernel/scst_tgt/targets/iscsi/mgmt
will enable only portals 10.170.77.2 and 10.170.75.2 and disable all
other portals.
echo 'add_target_attribute iqn.2006-10.net.vlnb:tgt allowed_portal 10.170.7?.2' >/sys/kernel/scst_tgt/targets/iscsi/mgmt
will enable only portals 10.170.7x.2 and disable all other portals.
echo 'add_target_attribute iqn.2006-10.net.vlnb:tgt allowed_portal !*' >/sys/kernel/scst_tgt/targets/iscsi/mgmt
will disable all portals.
2. If you want to want to allow only only specific set of initiators be
able to connect to your target, you should don't add any default LUNs
for the target and create for allowed initiators a security group to
which they will be assigned.
For example, we want initiator iqn.2005-03.org.vlnb:cacdcd2520 and only
it be able to access target iqn.2006-10.net.vlnb:tgt:
echo 'add_target iqn.2006-10.net.vlnb:tgt' >/sys/kernel/scst_tgt/targets/iscsi/mgmt
echo 'create allowed_ini' >/sys/kernel/scst_tgt/targets/iscsi/iqn.2006-10.net.vlnb:tgt/ini_groups/mgmt
echo 'add dev1 0' >/sys/kernel/scst_tgt/targets/iscsi/iqn.2006-10.net.vlnb:tgt/ini_groups/allowed_ini/luns/mgmt
echo 'add iqn.2005-03.org.vlnb:cacdcd2520' >/sys/kernel/scst_tgt/targets/iscsi/iqn.2006-10.net.vlnb:tgt/ini_groups/allowed_ini/initiators/mgmt
echo 1 >/sys/kernel/scst_tgt/targets/iscsi/iqn.2006-10.net.vlnb:tgt/enabled
Since there will be no default LUNs for the target, all initiators other
than iqn.2005-03.org.vlnb:cacdcd2520 will be blocked from accessing it.
Alternatively, you can create an empty security group and filter out in
it all initiators except the allowed one:
echo 'add_target iqn.2006-10.net.vlnb:tgt' >/sys/kernel/scst_tgt/targets/iscsi/mgmt
echo 'add dev1 0' >/sys/kernel/scst_tgt/targets/iscsi/iqn.2006-10.net.vlnb:tgt/luns/mgmt
echo 'create denied_inis' >/sys/kernel/scst_tgt/targets/iscsi/iqn.2006-10.net.vlnb:tgt/ini_groups/mgmt
echo 'add !iqn.2005-03.org.vlnb:cacdcd2520' >/sys/kernel/scst_tgt/targets/iscsi/iqn.2006-10.net.vlnb:tgt/ini_groups/denied_inis/initiators/mgmt
echo 1 >/sys/kernel/scst_tgt/targets/iscsi/iqn.2006-10.net.vlnb:tgt/enabled
3. If you want to enable/disable one or more target's portals for
particular initiators, you should set per_portal_acl attribute to 1 and
specify SCST access control to those initiators. If an SCST security
group doesn't have any LUNs, all the initiator, which should be assigned
to it, will not see this target and/or its portal. For example:
(We assume that an empty group "BLOCKING_GROUP" is already created by for
target iqn.2006-10.net.vlnb:tgt by command (see above for more information):
"echo 'create BLOCKING_GROUP' >/sys/kernel/scst_tgt/targets/iscsi/iqn.2006-10.net.vlnb:tgt/ini_groups/mgmt)
echo 'add iqn.2005-03.org.vlnb:cacdcd2520#10.170.77.2' >/sys/kernel/scst_tgt/targets/iscsi/iqn.2006-10.net.vlnb:tgt/ini_groups/BLOCKING_GROUP/initiators/mgmt
will block access of initiator iqn.2005-03.org.vlnb:cacdcd2520 to
target iqn.2006-10.net.vlnb:tgt portal 10.170.77.2.
Another example:
echo 'add iqn.2005-03.org.vlnb:cacdcd2520*' >/sys/kernel/scst_tgt/targets/iscsi/iqn.2006-10.net.vlnb:tgt/ini_groups/BLOCKING_GROUP/initiators/mgmt
will block access of initiator iqn.2005-03.org.vlnb:cacdcd2520 to
all target iqn.2006-10.net.vlnb:tgt portals.
Troubleshooting
---------------
If you have any problems, start troubleshooting from looking at the
kernel and system logs. In the kernel log iSCSI-SCST and SCST core send
their messages, in the system log iscsi-scstd sends its messages. In
most Linux distributions both those logs are put to /var/log/messages
file.
Then, it might be helpful to increase level of logging. For kernel
modules you should make the debug build by enabling CONFIG_SCST_DEBUG.
If after looking on the logs the reason of your problem is still unclear
for you, report to SCST mailing list [email protected].
Work if target's backstorage or link is too slow
------------------------------------------------
In some cases you can experience I/O stalls or see in the kernel log
abort or reset messages. It can happen under high I/O load, when your
target's backstorage gets overloaded, or working over a slow link, when
the link can't serve all the queued commands on time,
To workaround it you can reduce QueuedCommands parameter for the
corresponding target to some lower value, like 8 (default is 32).
Also see SCST README file for more details about that issue and ways to
prevent it.
Performance advices
-------------------
1. If you use Windows XP or Windows 2003+ as initiators, you can
consider to decrease TcpAckFrequency parameter to 1. See
http://support.microsoft.com/kb/328890/ or google for "TcpAckFrequency"
for more details.
2. See how to get the maximum throughput from iSCSI, for instance, at
http://virtualgeek.typepad.com/virtual_geek/2009/01/a-multivendor-post-to-help-our-mutual-iscsi-customers-using-vmware.html.
It's about VMware, but its recommendations apply to other environments
as well.
3. ISCSI initiators built in pre-CentOS/RHEL 5 reported to have some
performance problems. If you use it, it is strongly advised to upgrade.
4. If you are going to use your target in an VM environment, for
instance as a shared storage with VMware, make sure all your VMs
connected to the target via *separate* sessions, i.e. each VM has own
connection to the target, not all VMs connected using a single
connection. You can check it using SCST sysfs interface. If you
miss it, you can greatly loose performance of parallel access to your
target from different VMs. This isn't related to the case if your VMs
are using the same shared storage, like with VMFS, for instance. In this
case all your VM hosts will be connected to the target via separate
sessions, which is enough.
5. Many dual port network adapters are not able to transfer data
simultaneously on both ports, i.e. they transfer data via both ports on
the same speed as via any single port. Thus, using such adapters in MPIO
configuration can't improve performance. To allow MPIO to have double
performance you should either use separate network adapters, or find a
dual-port adapter capable to to transfer data simultaneously on both
ports. You can check it by running 2 iperf's through both ports in
parallel.
6. Since network offload works much better in the write direction, than
for reading (simplifying, in the read direction often there's additional
data copy) in many cases with 10GbE in a single initiator-target pair
the initiator's CPU is a bottleneck, so you can see the initiator can
read data on much slower rate, than write. You can check it by watching
*each particular* CPU load to find out if any of them is close to 100%
load, including IRQ processing load. Note, many tools like vmstat give
aggregate load on all CPUs, so with 4 cores 25% corresponds to 100% load
of any single CPU.
7. For high speed network adapters it can be better if you configure
them to serve connections, e.g., from initiator on CPU0 and from
initiator Y on CPU1. Then you can bind threads processing them also to
CPU0 and CPU1 correspondingly using cpu_mask attribute of their targets
or security groups. In NUMA-like configurations it can signficantly
boost IOPS performance.
8. See SCST core's README for more advices. Especially pay attention to
have io_grouping_type option set correctly.
Compilation options
-------------------
There are the following compilation options, that could be commented
in/out in the kernel's module Makefile:
- CONFIG_SCST_DEBUG - turns on some debugging code, including some logging.
Makes the driver considerably bigger and slower, producing large amount of
log data.
- CONFIG_SCST_TRACING - turns on ability to log events. Makes the driver
considerably bigger and leads to some performance loss.
- CONFIG_SCST_EXTRACHECKS - adds extra validity checks in the various places.
- CONFIG_SCST_ISCSI_DEBUG_DIGEST_FAILURES - simulates digest failures in
random places.
Credits
-------
Thanks to:
* Ming Zhang <[email protected]> for fixes
* Krzysztof Blaszkowski <[email protected]> for many fixes
* Alexey Kuznetsov <[email protected]> for comments and help in
debugging
* Tomasz Chmielewski <[email protected]> for testing and suggestions
* Bart Van Assche <[email protected]> for a lot of help
Vladislav Bolkhovitin <[email protected]>, http://scst.sourceforge.net