Saturday, January 31, 2015

How to Make Fibre Channel Storage Available to Oracle Solaris

How to Make Fibre Channel Storage Available to Oracle Solaris
With the Sun ZFS Storage Appliance

Introduction

Oracle Solaris 10 and Oracle Solaris 11 come with a Fibre Channel initiator stack you can configure to integrate Fibre Channel (FC) LUNs presented by the Sun ZFS Storage Appliance into the Oracle Solaris environment. This article describes how to configure the Oracle Solaris Fibre Channel stack and how to configure the Sun ZFS Storage Appliance to set up FC LUNs for access by Oracle Solaris servers. These configurations can be accomplished using the browser user interface (BUI).
The following assumptions have been made in this article:
  • The root password for the Sun ZFS Storage Appliance is known.
  • The IP address or host name of the Sun ZFS Storage Appliance is known.
  • The network used by the Sun ZFS Storage Appliance has been configured.
  • The Sun ZFS Storage Appliance has pools configured with sufficient free space available.
  • The root password for the Oracle Solaris server is known.
  • The Sun ZFS Storage Appliance is already connected to a Fibre Channel switch.
  • The appropriate zoning has been configured on the FC switch to allow the Oracle Solaris host to access the Sun ZFS Storage Appliance.
Configuring the Oracle Solaris FC Stack
Identifying the Sun ZFS Storage Appliance FC WWNs
  1. Enter an address in the address field of a Web browser that includes the IP address or host name of the Sun ZFS Storage Appliance as shown in the following URL:

https://<ip-address or host name>:215
  1. Figure 1
  1. Enter a username and password and click LOGIN.

  1. Click on Configuration > SAN > Fibre Channel Ports.

  1. Figure 2
Identifying the Oracle Solaris Host HBA WWNs
  1. To find the WWNs for the host, enter:

root@solaris:~# cfgadm -al -o show_FCP_dev
Ap_Id Type Receptacle Occupant Condition
[...]
c8 fc-fabric connected unconfigured unknown
c9 fc-fabric connected unconfigured unknown
root@solaris:~#
  1. If the HBA ports are not being used to access any other FC-connected devices, the following command can be used to determine the WWNs.

root@solaris:~# prtconf -vp | grep port-wwn
port-wwn: 210000e0.8b89bf8e
port-wwn: 210100e0.8ba9bf8e
root@solaris:~#
root@solaris:~# luxadm -e dump_map /dev/cfg/c8
Pos Port_ID Hard_Ad Port WWN Node WWN Type
0 2000 0 xxxxxxxxxxxxxxxxx xxxxxxxxxxxxxxxx 0x0 (Disk
device)
[...]
3 25600 0 2100000e08b89bf8e 200000e08b89bf8e 0x1f (Unknown
type, Host Bus Adapter)
root@solaris:~#
  1. From the output, you can see that c8 has the WWN 21:00:00:00:e0:8b:89:bf:8e and c9 has WWN21:01:00:e0:8b:a9:bf:8e.

root@solaris:~# cfgadm -al -o show_FCP_dev c8 c9
Ap_Id Type Receptacle Occupant Condition
[...]
c8 fc-fabric connected configured unknown
c8::210100e08bb2a1cf unknown connected unconfigured unknown
c9 fc-fabric connected configured unknown
c9::210000e08b92a1cf unknown connected unconfigured unknown
root@solaris:~#
Configuring the Sun ZFS Storage Appliance Using the Browser User Interface
Defining an FC Target Group
  1. Click Configuration > SAN to display the Storage Area Network (SAN) screen, as shown in Figure 3.
  1. Click the Targets tab at the right and then select Fibre Channel Ports at the top of the left panel, as shown in Figure 3.
  1. Place the mouse in the Fibre Channel Ports box and a Move icon (move icon) will appear in the far left, as shown in Figure 3.

  1. Figure 3
  1. Click on the Move icon and drag the box to the Fibre Channel Target Groups box, as demonstrated in Figure 4.

  1. Figure 4
  1. Drop the entry in the orange box to create a new target group. The group is created and named automatically as targets-n where nis an integer. See the example targets-0 in Figure 5.

  1. Figure 5
  1. Move the cursor over the entry for the new target group. Two icons appear to the right of the Fibre Channel Target Groups box, as shown in Figure 6.

  1. Figure 6
  1. To rename the new target group targets-0, click the Edit icon (edit icon) to display the dialog box shown in Figure 7.

  1. Figure 7
  1. In the Name field, replace the default name with your preferred name for the new FC target group and click OK. For this example, the name FC-PortGroup replaces targets-0. In this window, you can also add the second FC target port by clicking in the box to the left of the WWN you are choosing. In the example in Figure 7, this second port is identified as PCIe 1: Port 2.
  1. Click OK to save the changes.
  1. Click APPLY. The changes are shown in the Fibre Channel Target Groups panel in Figure 8. 

  1. Figure 8
Defining an FC Initiator
  1. Click Configuration > SAN to display the Storage Area Network (SAN) screen shown in Figure 9.
  1. Click the Initiators tab at the right and then select Fibre Channel Initiators at the top of the left panel, as shown in Figure 9.

  1. Figure 9
  1. Click the plus icon icon to the left of Fibre Channel Initiators to display the New Fibre Channel Initiator dialog box shown in Figure 10.

  1. Click on one of the WWNs (if they are displayed) at the bottom of the dialog box to prepopulate the World Wide Name or type the appropriate WWN in to the World Wide Name box.
  1. Enter a more meaningful symbolic name in the Alias box.

  1. Figure 10
  1. Click OK.
  1. Repeat the preceding steps for the other WWN(s) that refer to the Oracle Solaris host.
Defining the FC Initiator Group
  1. Select Configuration > SAN to display the Storage Area Network (SAN) screen.
  1. Select the Initiators tab at the right and then click Fibre Channel Initiators at the top of the left panel.
  1. Place the cursor over the entry for one of the FC initiators created in the previous section. The Move icon (move icon) appears to the left of the entry, as shown in Figure 11.

  1. Figure 11
  1. Click the Move icon and drag it to the Fibre Channel Initiator Groups panel on the right. A new entry appears (highlighted in yellow) at the bottom of the Fibre Channel Initiators Groups panel shown in Figure 12.

  1. Figure 12
  1. Move the cursor over the new entry box and release the mouse button. A new FC initiator group is created with the nameinitiators-n, where n is an integer, as shown in Figure 13.
  1. Move the cursor over the entry for the new initiator group. Several icons appear to the right of the target initiator group box, as shown in Figure 13.

  1. Figure 13
  1. Click the Edit icon (edit icon) to display the dialog box shown in Figure 14.
  1. In the Name field, replace the new initiator group's default name with a chosen name and click OK. For this example, the name sol-server is used.

  1. Figure 14
  1. Click APPLY on the SAN configuration screen to confirm all the modifications, as shown in Figure 15.

  1. Figure 15
Defining a Sun ZFS Storage Appliance Project
  1. Select Shares > Projects to display the Projects screen, as shown in Figure 16. 

  1. Figure 16
  1. Click the plus icon icon to the left of Projects at the top of the left panel to display the Create Project dialog box shown in Figure 17.

  1. Figure 17
  1. To create a new project, enter a name for the project and click APPLY. A new project appears in the Projects list in the left panel.
  1. Select the new project to view the components that comprise the project, as shown in Figure 18.

  1. Figure 18
Defining a Sun ZFS Storage Appliance LUN
  1. Select Shares > Projects to display the Projects screen.
  1. In the Projects panel at the left, select the project. Then select LUNs at the top of the panel at the right, as shown in Figure 19.

  1. Figure 19
  1. Click the plus icon icon to the left of LUNs to display the Create LUN dialog box shown in Figure 20.
  1. Enter values as appropriate to set up the LUN. For this example, set Name to DocArchive1, Volume size to 64 G, and select theThin provisioned checkbox. Set Target Group to the FC target group FC-PortGroup and set Initiator Group to sol-server. SetVolume block size to 32k because the volume will hold an Oracle Solaris ZFS file system.

  1. Figure 20
  1. Click APPLY to create the LUN and make it available to the Oracle Solaris server.
Configuring the LUN for Use by the Oracle Solaris Server
  1. Initiate an Oracle Solaris FC session with the Sun ZFS Storage Appliance, as shown in Listing 1. Since the LUN was created prior to initiating the FC session, it will be automatically enabled.

  1. Listing 1. Initiating an Oracle Solaris FC session
root@solaris:~# cfgadm -al c8 c9
root@solaris:~# cfgadm -c configure c8::210100e08bb2a1cf
root@solaris:~# cfgadm -c configure c9::210000e08b92a1cf
root@solaris:~# cfgadm -al -o show_FCP_dev c8 c9
Ap_Id Type Receptacle Occupant Condition
c8 fc-fabric connected configured unknown
c8::210100e08bb2a1cf,0 disk connected configured unknown
c9 fc-fabric connected configured unknown
c9::210000e08b92a1cf,0 disk connected configured unknown
root@solaris:~#
  1. Verify access to the FC LUN, as shown in Listing 2.

  1. Listing 2. Verifying access to the FC LUN
root@solaris:~# devfsadm -c ssd
root@solaris:~# tail /var/adm/messages
[...]
Nov 30 06:30:00 solaris last message repeated 2 times
Nov 30 06:32:22 solaris scsi: [ID 799468 kern.info] ssd42 at
scsi_vhci0: name g600144f0f05e906c00004ed6096d0001, bus address
g600144f0f05e906c00004ed6096d0001
Nov 30 06:32:22 solaris genunix: [ID 936769 kern.info] ssd42 is
/scsi_vhci/ssd@g600144f0f05e906c00004ed6096d0001
Nov 30 06:32:22 solaris scsi: [ID 107833 kern.warning] WARNING:
/scsi_vhci/ssd@g600144f0f05e906c00004ed6096d0001 (ssd42):
Nov 30 06:32:22 solaris Corrupt label; wrong magic number
Nov 30 06:32:22 solaris genunix: [ID 408114 kern.info]
/scsi_vhci/ssd@g600144f0f05e906c00004ed6096d0001 (ssd42) online
Nov 30 06:32:22 solaris genunix: [ID 834635 kern.info]
/scsi_vhci/ssd@g600144f0f05e906c00004ed6096d0001 (ssd42)
multipath status: degraded, path
/pci@1e,600000/SUNW,qlc@2/fp@0,0 (fp1) to target address:
w210100e08bb2a1cf,0 is online Load balancing: round-robin
Nov 30 06:32:22 solaris scsi: [ID 107833 kern.warning] WARNING:
/scsi_vhci/ssd@g600144f0f05e906c00004ed6096d0001 (ssd42):
Nov 30 06:32:22 solaris Corrupt label; wrong magic number
Nov 30 06:32:49 solaris genunix: [ID 834635 kern.info]
/scsi_vhci/ssd@g600144f0f05e906c00004ed6096d0001 (ssd42)
multipath status: optimal, path
/pci@1e,600000/SUNW,qlc@2,1/fp@0,0 (fp3) to target address:
w210000e08b92a1cf,0 is online Load balancing: round-robin
[...]
  1. Format the LUN as shown in Listing 3.

  1. Listing 3. Formatting the LUN
root@solaris:~# format
Searching for disks...done
c1t600144F0F05E906C00004ED6096D0001d0: configured with capacity of 63.93GB
AVAILABLE DISK SELECTIONS:
[...]
4. c1t600144F0F05E906C00004ED6096D0001d0 <SUN-SunStorage7210-1.0 cyl 2078 alt 2 hd 254 sec 254>
/scsi_vhci/ssd@g600144f0f05e906c00004ed6096d0001
[...]
Specify disk (enter its number): 4
selecting c1t600144F0F05E906C00004ED6096D0001d0
[disk formatted]
Disk not labeled. Label it now? y
FORMAT MENU:
disk - select a disk
type - select (define) a disk type
partition - select (define) a partition table
current - describe the current disk
format - format and analyze the disk
repair - repair a defective sector
label - write label to the disk
analyze - surface analysis
defect - defect list management
backup - search for backup labels
verify - read and display labels
save - save new disk/partition definitions
inquiry - show vendor, product and revision
volname - set 8-character volume name
!<cmd> - execute <cmd>, then return
quit
format> q
  1. Build an Oracle Solaris ZFS file system on the prepared LUN by creating a new ZFS pool, adding this device to the ZFS pool, and creating the ZFS file system, as shown in the example in Listing 4.

  1. Listing 4. Building an Oracle Solaris ZFS file system
root@solaris:~# zfs createzpool create docarchive1 \
c1t600144F0F05E906C00004ED6096D0001d0
root@solaris:~# zfs list
NAME USED AVAIL REFER MOUNTPOINT
docarchive1 72K 62.5G 21K /docarchive1
rpool 25.4G 41.5G 97K /rpool
rpool/ROOT 22.3G 41.5G 21K legacy
[...]
root@solaris:~# zfs create docarchive1/index
root@solaris:~# zfs create docarchive1/data
root@solaris:~# zfs create docarchive1/logs
root@solaris:~# zfs list
NAME USED AVAIL REFER MOUNTPOINT
docarchive1 159K 62.5G 24K /docarchive1
docarchive1/data 21K 62.5G 21K /docarchive1/data
docarchive1/index 21K 62.5G 21K /docarchive1/index
docarchive1/logs 21K 62.5G 21K /docarchive1/logs
rpool 25.4G 41.5G 97K /rpool
rpool/ROOT 22.3G 41.5G 21K legacy
[...]
Conclusion
by Andrew Ness, February 2012
Introduction
Oracle Solaris 10 and Oracle Solaris 11 come with a Fibre Channel initiator stack you can configure to integrate Fibre Channel (FC) LUNs presented by the Sun ZFS Storage Appliance into the Oracle Solaris environment. This article describes how to configure the Oracle Solaris Fibre Channel stack and how to configure the Sun ZFS Storage Appliance to set up FC LUNs for access by Oracle Solaris servers. These configurations can be accomplished using the browser user interface (BUI).
The following assumptions have been made in this article:
To identify the Sun ZFS Storage Appliance and the Oracle Solaris server to each other, the FC World Wide Number (WWN) for each must be registered with the other. You must determine the WWNs for some forms of FC zoning implemented on the FC switch.
The FC WWN of the host is used to identify the host to the Sun ZFS Storage Appliance and is needed to complete the configuration procedure in this article.
The WWNs are taken from the FC Host Bus Adapters (HBAs) installed in the Oracle Solaris host and the Sun ZFS Storage Appliance.
In order to configure the Oracle Solaris FC stack, you will need to know the WWNs of the Sun ZFS Storage Appliance. In a traditional dual-fabric Storage Area Network (SAN), the Sun ZFS Storage Appliance will have at least one FC port attached to each fabric. Thus, you must determine at least two FC WWNs.
First, you need to establish a management session with the Sun ZFS Storage Appliance.
  1. The login dialog box is displayed.
  1. Figure 1. Sun ZFS Storage Appliance login
  1. Once you have successfully logged in to the BUI, you can identify the WWNs through the Configuration tab.
  1. Figure 2. Fibre Channel port configuration for SAN
    The FC ports installed in the Sun ZFS Storage Appliance are shown. Since there is only one discovered port per HBA channel, this must be the HBA channel itself.
    In the preceding case, port 1 has the WWN 21:00:00:e0:8b:92:a1:cf and port 2 has the WWN 21:01:00:e0:8b:b2:a1:cf.
    The FC channel ports should be set to Target in the list box to the right of each FC port box. If this is not the case, the FC ports may be in use for another purpose. Do not change the setting until you have investigated the reason. (One possible reason may be for NDMP backups.)
If the Oracle Solaris host is already attached to the FC switch by the appropriate cables, use the following command to identify the WWNs.
  1. From this output, the controller numbers you need are c8 and c9. You can also see that both ports are attached to an FC switch, as the port type is fc-fabric. Next, interrogate these controllers to determine what WWNs are discovered.
  1. Alternatively, if FC devices are being accessed, the following command will show the FC HBA WWNs.
    The last entry shown as type 0x1f (Unknown type, Host Bus Adapter) gives the appropriate WWN under the port WWN entry. Repeat the command substituting /dev/cfg/c8 for the other controllers identified in step 1.
  1. The Sun ZFS Storage Appliance HBA and the Oracle Solaris host HBA WWNs can then be used to configure any FC switch zoning.
    Once this has been completed, you can verify the proper zoning by running the following command:
    You can now see the WWNs presented by the Sun ZFS Storage Appliance that are accessible by the Oracle Solaris host.
As a unified storage platform, the Sun ZFS Storage Appliance supports access to block-protocol LUNs using iSCSI and Fibre Channel protocols. This section describes how to use the Sun ZFS Storage Appliance BUI to configure the Sun ZFS Storage Appliance to recognize the Oracle Solaris host and present FC LUNs to it.
A target group is created on the Sun ZFS Storage Appliance to define the ports and the protocol by which the LUN to be presented to the Oracle Solaris server is accessed. For this example, an FC target group is created.
To define an FC target group on the Sun ZFS Storage Appliance, complete these steps:
  1. Figure 3. Selecting the Fibre Channel port
  1. Figure 4. Dragging the Fibre Channel port entry
  1. Figure 5. Newly created Fibre Channel target group
  1. Figure 6. Selecting the FC target group
  1. Figure 7. Renaming the FC Target Group
  1. Figure 8. New FC target group with changes applied
An FC initiator is defined to allow access from one or more servers to particular volumes. Permissions to volumes should be configured so that a minimum number of FC initiators are allowed to access a particular volume. If more than one host can write to a given volume concurrently and a non-shared file system is used, inconsistencies may occur in file system caches on the hosts that can ultimately lead to corruption of the on-disk image. Typically a single initiator is given access to a volume, unless a specialized cluster file system is being used.
The FC initiator serves to define the "host" from the point of view of the Sun ZFS Storage Appliance. In a traditional dual-fabric SAN, the host will be defined by at least two FC initiators. The FC initiator definition contains the host WWNs. To identify the Oracle Solaris server to the Sun ZFS Storage Appliance, the Oracle Solaris FC initiator WWNs must then be registered with the appliance by completing the following steps.
  1. Figure 9. Selecting the SAN configuration
  1. If the zoning has been configured on the FC switches, the WWNs of the Oracle Solaris host should be displayed (assuming they are not assigned to an alias already).
  1. Figure 10. New Fibre Channel initiator definition
Related FC initiators are combined into logical groups to allow single commands to execute on multiple FC initiators, for example, assigning LUN access to all FC initiators in a group with one command. For the following example, the FC initiator group will contain two initiators. Note that in a cluster, where multiple servers are treated as a single logical entity, the initiator group may contain many more initiators.
To create an FC initiator group, complete these steps:
  1. Figure 11. Displaying the Move icon for the new FC initiator
  1. Figure 12. Creating the FC initiator group
  1. Figure 13. Selecting the Fibre Channel initiator group
  1. In this dialog box, you can add the other FC initiator(s) to the group by clicking the checkbox to the left of the WWN.
  1. Figure 14. Renaming and completing the FC initiator group
  1. Figure 15. Completed FC initiator configuration
To group related volumes, you can define a project in the Sun ZFS Storage Appliance. A project allows inheritance of properties for file systems and LUNs presented from the project. A project also allows you to apply quotas and reservations.
To create a project, complete the steps below:
  1. Figure 16. Viewing a project
  1. Figure 17. Create Project dialog box
  1. Figure 18. Displaying the new project sol-project
Next, you will create a LUN from an existing pool of storage resources that the Oracle Solaris server will access. In the following example, a thin-provisioned 64-GB LUN called DocArchive1 is created.
The target group will be the FC target group, FC-PortGroup, which was created in the section Defining an FC Target Group, to ensure that this LUN can be accessed using the FC protocol. The initiator group defined in the section Defining the FC Initiator Group, sol-server, will be used to ensure that only the server(s) defined in the sol-server group can access this LUN. (In this example, this group contains only one server.)
To create a target group, complete the following steps:
  1. Figure 19. Selecting Shares > Project > LUNs
  1. Figure 20. New LUN dialog box
Now that the LUN is prepared and available to the FC initiator group, the LUN must be configured for use by the Oracle Solaris server by completing the following steps:
  1. In this example, the multipath status is shown as degraded initially because only one path had been identified at that point. Further on, the multipath status changes to optimal due to the presence of multiple paths to the volume.
    The disk device is now similarly available to an internal server disk.
  1. The final two lines of the output from the df(1) command show that approximately 64 GB of new space is now available.
This article has described how to configure the FC initiator stack supplied with Oracle Solaris 10 and Oracle Solaris 11 to enable an Oracle Solaris server to access Fibre Channel LUNs presented by the Sun ZFS Storage Appliance.

Posted by at

No comments:

Post a Comment

Subscribe to: Post Comments (Atom)