Skip to main content

Linux下三种配置iscsi的方法

linux下面配置iscsi有多种方法,用户态的有tgt,内核态的有scst和lio,如果只是普通的使用来说,没有太大的区别,对于某些特殊的场景就存在区别了

比如tgt不支持pacemaker里面的 Persistent (SCSI-3) Reservations,而scst支持 tgt不支持vmware的vaai的某些属性,而lio支持 比如scst还支持srpt的功能,而tgt也不支持

所以在某些场景下,我们需要根据自己的需要来选择服务,总体上来说iscsi算是一个中间的组件,并不影响数据本身,只是一个对外的服务,满足需求即可

tgt的配置

因为tgt是支持librbd的,也是支持krbd的,所以这个地方存在两种配置,默认的tgt是不支持rbd的,需要自己重新打包支持开启

[root@lab103 ~]# tgtadm --lld iscsi --mode system --op show | grep rbd

打包的文档见这里:打包一个支持rbd的tgt

安装打好的包

[root@lab103 ~]# rpm -ivh scsi-target-utils-*
Preparing... ################################# [100%]
Updating / installing...
1:scsi-target-utils-1.0.84-4.el7 ################################# [ 50%]
2:scsi-target-utils-rbd-1.0.84-4.el################################# [100%]

检查librdb的支持情况

[root@lab103 ~]# tgtadm --lld iscsi --mode system --op show | grep rbd
rbd (bsoflags sync:direct)

支持librbd的配置方式

修改配置文件

[root@lab103 ~]# cat /etc/tgt/conf.d/sample.conf
<target iqn.2022-01.com.example:server.target1>
bs-type rbd
backing-store rbd/testrbd
</target>

启动服务

[root@lab103 ~]# systemctl start tgtd

查看生成的服务

[root@lab103 ~]# tgt-admin -s
Target 1: iqn.2022-01.com.example:server.target1
System information:
Driver: iscsi
State: ready
I_T nexus information:
LUN information:
LUN: 0
Type: controller
SCSI ID: IET 00010000
SCSI SN: beaf10
Size: 0 MB, Block size: 1
Online: Yes
Removable media: No
Prevent removal: No
Readonly: No
SWP: No
Thin-provisioning: No
Backing store type: null
Backing store path: None
Backing store flags:
LUN: 1
Type: disk
SCSI ID: IET 00010001
SCSI SN: beaf11
Size: 32212 MB, Block size: 512
Online: Yes
Removable media: No
Prevent removal: No
Readonly: No
SWP: No
Thin-provisioning: No
Backing store type: rbd
Backing store path: rbd/testrbd
Backing store flags:
Account information:
ACL information:
ALL

可以看到这里多了个

Backing store type: rbd
Backing store path: rbd/testrbd

客户端登录后查看磁盘

[root@lab101 ~]# parted -s /dev/sdc print
Error: /dev/sdc: unrecognised disk label
Model: IET VIRTUAL-DISK (scsi)
Disk /dev/sdc: 32.2GB
Sector size (logical/physical): 512B/4194304B
Partition Table: unknown
Disk Flags:

可以看到物理扇区的大小为4194304B,这个在很多环境,rac,vmware都可能不识别,需要修改扇区的大小,这个是在tgt的配置文件里面控制的

修改配置文件

[root@lab103 ~]# cat /etc/tgt/conf.d/sample.conf
<target iqn.2022-01.com.example:server.target1>
bs-type rbd
backing-store rbd/testrbd
lbppbe 0
</target>

这个lbppbe 0就是关键配置,设置以后客户端的连接后看到的磁盘扇区就是512b的了

[root@lab101 ~]# parted -s /dev/sdc print
Error: /dev/sdc: unrecognised disk label
Model: IET VIRTUAL-DISK (scsi)
Disk /dev/sdc: 32.2GB
Sector size (logical/physical): 512B/512B
Partition Table: unknown
Disk Flags:

支持krbd的配置方式

修改配置文件

<target iqn.2022-01.com.example:server.target1>
backing-store /dev/rbd/rbd/testrbd
</target>

启动服务

[root@lab103 ~]# systemctl start tgtd

查看生成的服务

[root@lab103 ~]# tgt-admin -s
Target 1: iqn.2022-01.com.example:server.target1
System information:
Driver: iscsi
State: ready
I_T nexus information:
LUN information:
LUN: 0
Type: controller
SCSI ID: IET 00010000
SCSI SN: beaf10
Size: 0 MB, Block size: 1
Online: Yes
Removable media: No
Prevent removal: No
Readonly: No
SWP: No
Thin-provisioning: No
Backing store type: null
Backing store path: None
Backing store flags:
LUN: 1
Type: disk
SCSI ID: IET 00010001
SCSI SN: beaf11
Size: 32212 MB, Block size: 512
Online: Yes
Removable media: No
Prevent removal: No
Readonly: No
SWP: No
Thin-provisioning: No
Backing store type: rdwr
Backing store path: /dev/rbd/rbd/testrbd
Backing store flags:
Account information:
ACL information:
ALL

这个地方注意下看下客户端的映射的盘的扇区大小

[root@lab101 ~]# parted -s /dev/sdc print
Error: /dev/sdc: unrecognised disk label
Model: IET VIRTUAL-DISK (scsi)
Disk /dev/sdc: 32.2GB
Sector size (logical/physical): 512B/512B
Partition Table: unknown
Disk Flags:

LIO配置

使用命令行配置

创建一个block设备

[root@lab103 ~]# targetcli /backstores/block/ create disk01 /dev/rbd/rbd/testrbd
Created block storage object disk01 using /dev/rbd/rbd/testrbd.

查看当前配置

[root@lab103 ~]# targetcli ls /
o- / ............................................................ [...]
o- backstores ................................................. [...]
| o- block ..................................... [Storage Objects: 1]
| | o- disk01 [/dev/rbd/rbd/testrbd (30.0GiB) write-thru deactivated]
| | o- alua ...................................... [ALUA Groups: 1]
| | o- default_tg_pt_gp .......... [ALUA state: Active/optimized]
| o- fileio .................................... [Storage Objects: 0]
| o- pscsi ..................................... [Storage Objects: 0]
| o- ramdisk ................................... [Storage Objects: 0]
o- iscsi ............................................... [Targets: 0]
o- loopback ............................................ [Targets: 0]

创建一个target

[root@lab103 ~]# targetcli /iscsi/ create iqn.2022-01.org.linux-iscsi.lab103.x8664:sn.192f7d06da4a
Created target iqn.2022-01.org.linux-iscsi.lab103.x8664:sn.192f7d06da4a.
Created TPG 1.
Global pref auto_add_default_portal=true
Created default portal listening on all IPs (0.0.0.0), port 3260.

创建一个lun

[root@lab103 ~]# targetcli  /iscsi/iqn.2022-01.org.linux-iscsi.lab103.x8664:sn.192f7d06da4a/tpg1/luns create /backstores/block/disk01
Created LUN 0.

设置target的属性

[root@lab103 ~]# targetcli  /iscsi/iqn.2022-01.org.linux-iscsi.lab103.x8664:sn.192f7d06da4a/tpg1 set attribute authentication=0 demo_mode_write_protect=0 generate_node_acls=1 cache_dynamic_acls=1

保存配置文件

[root@lab103 ~]# targetcli /  saveconfig
Last 10 configs saved in /etc/target/backup/.
Configuration saved to /etc/target/saveconfig.json

启动target

[root@lab103 ~]# systemctl start target

scst的配置

配置fileio模式

[root@lab103 ~]# cat /etc/scst.conf
HANDLER vdisk_fileio {
DEVICE disk01 {
filename /dev/rbd/rbd/testrbd
nv_cache 1
}
}

TARGET_DRIVER iscsi {
enabled 1
TARGET iqn.2022-05.net.vlnb:tgt {
LUN 0 disk01
enabled 1
}
}

配置blockio模式

[root@lab103 ~]# cat /etc/scst.conf
HANDLER vdisk_blockio {
DEVICE disk01 {
filename /dev/rbd/rbd/testrbd
nv_cache 1
}
}

TARGET_DRIVER iscsi {
enabled 1
TARGET iqn.2022-05.net.vlnb:tgt {
LUN 0 disk01
enabled 1
}
}

启动服务

[root@lab103 ~]# systemctl start scst