Volumes PDF

Preview

Summary

Volumes...

Description

v5.3 | May 2021 | BP-2049

BEST PRACTICES

Nutanix Volumes

Copyright Copyright 2021 Nutanix, Inc. Nutanix, Inc. 1740 Technology Drive, Suite 150 San Jose, CA 95110 All rights reserved. This product is protected by U.S. and international copyright and intellectual property laws. Nutanix and the Nutanix logo are registered trademarks of Nutanix, Inc. in the United States and/or other jurisdictions. All other brand and product names mentioned herein are for identification purposes only and may be trademarks of their respective holders.

Nutanix Volumes

Contents 1. Executive Summary.......................................................................................... 5 2.Nutanix Enterprise Cloud Overview..........................................................7 Nutanix HCI Architecture............................................................................................................................. 8

3.Nutanix Volumes Overview........................................................................ 10 4.Supported Client Environments................................................................14 5.Volume Group Connectivity....................................................................... 15 iSCSI Target Redirection............................................................................................................................. 15 Challenge Handshake Authentication Protocol (CHAP).................................................................21

6.Acropolis Dynamic Scheduling.................................................................22 7. Networking for iSCSI.................................................................................... 23 Networking Segmentation......................................................................................................................... 23

8.Configuring Windows Clients for Connectivity..................................26 Start the iSCSI Service and Set the Startup Type to Automatic............................................... 26 Set Firewall Rules to Allow iSCSI Traffic............................................................................................ 26 Get the IQN from the Virtual Machine(s)............................................................................................26 Create the Nutanix Volume Group........................................................................................................ 27 Create an iSCSI Discovery Portal........................................................................................................... 27 Connect to Discovered Targets and Persist the Connections.................................................... 28 How to Configure Red Hat Enterprise Linux (RHEL) Clients for Connectivity..................... 28

9. Best Practices...................................................................................................31 Client Tuning Recommendations............................................................................................................33 Linux Client Tuning Example................................................................................................................... 34

10. Thin Provisioning ........................................................................................ 35 SCSI UNMAP....................................................................................................................................................35

11.Volume Group Data Protection............................................................... 38 12. Conclusion....................................................................................................... 40 Appendix........................................................................................................................ 41 Exclude Nodes for Nutanix Volumes Placement.............................................................................. 41 Logon Redirection........................................................................................................................................ 42 iSCSI and Multipath I/O (MPIO)..............................................................................................................43 Linux multipath.conf Example...................................................................................................................51 About Nutanix.................................................................................................................................................52 List of Figures..............................................................................................................................................................53 List of Tables............................................................................................................................................................... 54

Nutanix Volumes

1. Executive Summary The Nutanix enterprise cloud is hypervisor-agnostic and can support any application. To support these key advantages, Nutanix distributed storage delivers storage using multiple protocols, including Network File System (NFS), Server Message Block (SMB), and Internet Small Computer System Interface (iSCSI). Nutanix Volumes is enterprise-class, software-defined storage that exposes storage resources directly to virtualized guest operating systems or physical hosts using the iSCSI protocol. Nutanix Volumes facilitates support for several use cases: • iSCSI for Microsoft Exchange Server. › Volumes allows Microsoft Exchange Server environments to use iSCSI as the primary storage protocol. • Shared storage for Linux-based clusters and Windows Server Failover Clustering (WSFC). › Volumes supports SCSI-3 persistent reservations for shared storage-based Windows clusters, which are commonly used with Microsoft SQL Server and clustered file servers. • Shared storage for Oracle RAC environments. • Bare-metal environments. › Volumes enables server hardware separate from the Nutanix environment to consume Acropolis distributed storage resources, so you can leverage existing server hardware investments against Nutanix storage resources. Workloads not targeted for virtualization can also use the storage fabric. Nutanix has long supported virtualized workloads and has more recently delivered a native scale-out file serving capability called Nutanix Files and an object-storage feature called Nutanix Objects. With Volumes, Files, and Objects, Nutanix supports use cases across virtualized and physical server workloads, enabling infrastructure consolidation for enterprise cloud environments.

© 2021 Nutanix, Inc. All rights reserved|5

Nutanix Volumes

Table 1: Document Version History

Version Number

Published

Notes

1.0

June 2016

Original publication.

2.0

January 2017

Updated for AOS 5.0.

2.1

February 2017

Updated supported clients.

2.2

May 2017

Updated for AOS 5.1 and added a section on thin provisioning.

3.0

December 2017

Updated for AOS 5.5.

4.0

October 2018

Updated for AOS 5.9. Updated product naming and the Client Tuning Recommendations and SCSI UNMAP sections.

5.0

August 2019

Updated for AOS 5.11. Updated recommendations.

5.1

February 2020

Updated supported clients for AOS 5.11.2.

5.2

June 2020

Updated jumbo frame recommendations.

5.3

May 2021

Updated for SCSI-3 PR and Traffic Isolation IP Pool.

© 2021 Nutanix, Inc. All rights reserved|6

Nutanix Volumes

2. Nutanix Enterprise Cloud Overview Nutanix delivers a web-scale, hyperconverged infrastructure solution purposebuilt for virtualization and both containerized and private cloud environments. This solution brings the scale, resilience, and economic benefits of web-scale architecture to the enterprise through the Nutanix enterprise cloud platform, which combines the core HCI product families—Nutanix AOS and Nutanix Prism management—along with other software products that automate, secure, and back up cost-optimized infrastructure. Available attributes of the Nutanix enterprise cloud OS stack include: • Optimized for storage and compute resources. • Machine learning to plan for and adapt to changing conditions automatically. • Intrinsic security features and functions for data protection and cyberthreat defense. • Self-healing to tolerate and adjust to component failures. • API-based automation and rich analytics. • Simplified one-click upgrades and software life cycle management. • Native file services for user and application data. • Native backup and disaster recovery solutions. • Powerful and feature-rich virtualization. • Flexible virtual networking for visualization, automation, and security. • Cloud automation and life cycle management. Nutanix provides services and can be broken down into three main components: an HCI-based distributed storage fabric, management and operational intelligence from Prism, and AHV virtualization. Nutanix Prism furnishes one-click infrastructure management for virtual environments running on AOS. AOS is hypervisor agnostic, supporting two third-party hypervisors

© 2021 Nutanix, Inc. All rights reserved|7

Nutanix Volumes

—VMware ESXi and Microsoft Hyper-V—in addition to the native Nutanix hypervisor, AHV.

Figure 1: Nutanix Enterprise Cloud OS Stack

Nutanix HCI Architecture Nutanix does not rely on traditional SAN or network-attached storage (NAS) or expensive storage network interconnects. It combines highly dense storage and server compute (CPU and RAM) into a single platform building block. Each building block delivers a unified, scale-out, shared-nothing architecture with no single points of failure. The Nutanix solution requires no SAN constructs, such as LUNs, RAID groups, or expensive storage switches. All storage management is VM-centric, and I/ O is optimized at the VM virtual disk level. The software solution runs on nodes from a variety of manufacturers that are either entirely solid-state storage with NVMe for optimal performance or a hybrid combination of SSD and HDD storage that provides a combination of performance and additional capacity. The storage fabric automatically tiers data across the cluster to different classes of storage devices using intelligent data placement algorithms. For best

© 2021 Nutanix, Inc. All rights reserved|8

Nutanix Volumes

performance, algorithms make sure the most frequently used data is available in memory or in flash on the node local to the VM. To learn more about Nutanix enterprise cloud software, visit the Nutanix Bible and Nutanix.com.

© 2021 Nutanix, Inc. All rights reserved|9

Nutanix Volumes

3. Nutanix Volumes Overview The Nutanix distributed storage fabric is an enterprise-class storage service that provides data redundancy, erasure coding (EC-X), compression, deduplication, data locality, and disaster recovery services for all data stored on the Nutanix platform. Volumes provides iSCSI block access to this storage pool. Nutanix designed Volumes as a scale-out storage solution where every Controller VM (CVM) in a cluster could present storage over iSCSI. This solution allows an individual application to access the entire cluster, if needed, to scale out for high performance. Volumes automatically manages high availability to ensure that upgrades or failures are nondisruptive to applications. Storage allocation and assignment for Volumes is implemented through volume groups (VGs). A VG is a collection of one or more disks (essentially virtual disks) in a Nutanix storage container. Volumes presents these disks to both VMs and physical servers, or hosts. Volumes disks inherit the properties (replication factor, compression, erasure coding, and so on) of the container you create them in. By default, these disks are thin-provisioned. Because we use iSCSI as the protocol for presenting storage with Volumes, hosts obtain access based on either the iSCSI-qualified name (IQN) or the IP addresses of the host. The system uses IQNs or IP addresses as an allowlist and attaches them to a VG to permit access by a given host. Once a host has access to a VG, the VG is discovered as one or more iSCSI targets. When it connects to the iSCSI targets, the host discovers the disks as SCSI disk devices. The following figure shows these relationships.

© 2021 Nutanix, Inc. All rights reserved|10

Nutanix Volumes

Figure 2: Volumes Relationships

You can grow a VG dynamically by adding new disks or expanding existing disks online. You can use VGs for iSCSI connectivity in your Nutanix cluster, whether the underlying hypervisor is ESXi, Hyper-V, or AHV. Starting in AOS 5.17, we introduced support for SCSI-3 Persistent Reservations when you use directattached volume groups, which makes it easier to quickly set up applicationlevel clustering on AHV with Volumes. Some clustered applications, like WSFC, use SCSI-3 Persistent Reservations to coordinate access to storage by enabling multiple instances of the application cluster to write to a shared drive. This new functionality makes this process even easier to configure by creating a VG and connecting to a VM in Prism, which you can then access from the guest VM as a SCSI device with access to Persistent Reservations. Multiple external hosts can also share disks in a VG for the purposes of shared storage clustering, as shown in the following figure. A common scenario for using shared storage is in WSFC. You can use Prism to attach multiple external initiators to a VG by default. Ensure that you have an appropriate clustering technology (like WSFC) or a clustered file system in place before you share disks across multiple hosts.

© 2021 Nutanix, Inc. All rights reserved|11

Nutanix Volumes

Figure 3: Shared Volume Group

You can use VGs in conjunction with traditional hypervisor vDisks. For example, some VMs in a Nutanix cluster may use .vmdk- or .vhdx-based storage on NFS or SMB, while other hosts use VGs as their primary storage. VMs using VGs have their start and OS drives presented with hypervisor vDisks. Note: Nutanix recommends starting an OS from iSCSI using Volumes. The Nutanix Support Portal has a list of qualified NICs, HBAs, and operating systems that support starting from iSCSI using Volumes.

You can manage VGs from Prism or from a preferred CLI, such as Acropolis command line interface (aCLI), nCLI, or PowerShell. In Prism, the Storage page allows you to create and monitor VGs, as shown in the following figure.

© 2021 Nutanix, Inc. All rights reserved|12

Nutanix Volumes

Figure 4: Prism Volume Group Page

© 2021 Nutanix, Inc. All rights reserved|13

Nutanix Volumes

4. Supported Client Environments The following table shows which clients we have qualified and which specific AOS release that support began with.

Table 2: Clients and Applications Qualified by AOS Release Version

Minimum AOS Release Version

Supported Client OS

Qualified Applications

RHEL 6 and 7

4.7

Oracle Linux 6 and 7

Oracle RAC

CentOS 6 and 7

Microsoft SQL Server

Windows 2008 R2

Microsoft Exchange Server

Windows 2012 R2

5.0

IBM AIX 7.1 and 7.2 (on POWER) SLES 11 and 12

5.0.2, 5.1 5.5

Solaris 11 Windows 2016

5.9

Linux Guest Clustering

Windows 2019 5.11.2

RHEL 7.6 RHEL 7.7

© 2021 Nutanix, Inc. All rights reserved|14

Nutanix Volumes

5. Volume Group Connectivity A Nutanix cluster can take advantage of all CVMs when it presents VG-based storage to hosts. Only one CVM can host each disk at a time, and all primary storage access for that disk occurs through the hosting CVM. This setup means that where the disks are hosted can affect network consumption and the balance of CVM usage in a cluster. Our connectivity method uses iSCSI redirection to control target path management for disk load balancing and path resiliency. The following sections explore iSCSI redirection. We provide additional information regarding the legacy use of Multipath I/O (MPIO) in the appendix.

iSCSI Target Redirection The preferred method for external connectivity to VGs doesn’t use MPIO for storage load balancing or storage path resiliency. Instead of configuring host iSCSI client sessions to connect directly to CVMs, we use an iSCSI data services IP address. This data services IP acts as a discovery portal and initial connection point. Only one CVM can own the data services address at a time. If the owner goes offline, the address moves between CVMs, ensuring that it’s always available. You can configure the data services address under the Cluster Details section of Prism.

© 2021 Nutanix, Inc. All rights reserved|15

Nutanix Volumes

Figure 5: Configuring iSCSI Data Services IP Address

Once you have established a connection to a target through the iSCSI data services IP, the system redirects the iSCSI client to the CVM that hosts the disk for the specified target. The Nutanix cluster uses a heuristic to balance target ownership evenly across CVMs in the cluster. All CVMs are potential targets, including new CVMs added during cluster expansion. The following figure shows the connection process, and we provide a more detailed diagram in the appendix.

© 2021 Nutanix, Inc. All rights reserved|16

Nutanix Volumes

Figure 6: iSCSI Redirection for Initial Connection

The logon redirection occurs on a per-target basis. A VG represents one or more virtual targets. Virtual targets enable a single VG to export multiple iSCSI targets that can be redirected to different CVMs. By default, a VG is configured to have 32 virtual targets. With the default of 32, the number of virtual targets a client sees depends on the number of disks in the VG. The following figure illustrates an example in which 3 disks reside in a VG; therefore, a client sees three virtual targets. If a VG has 32 or more disks, then client discovery displays 32 virtual targets. Names for virtual targets start with the VG name and end with a virtual target number. For example, a VG named volumesvg1 with 2 disks could have virtual targets named volumesvg1-7d64abbe-0e5c-49b6-97e2-449e6db08f54-tgt0 and volumesvg1-7d64abbe-0e5c-49b6-97e2-449e6db08f54-tgt1.

© 2021 Nutanix, Inc. All rights reserved|17

Nutanix Volumes

Figure 7: VG Virtual Targets

If an active CVM goes offline because of a failure or planned maintenance, the system disconnects any active sessions against that CVM, which triggers the iSCSI client to log on again. The new logon occurs through the iSCSI data services IP, which redirects the session to a healthy CVM. The next figure shows this process, and we provide a more detailed diagram in the appendix.

© 2021 Nutanix, Inc. All rights reserved|18

Nutanix Volumes

Figure 8: iSCSI Redirection on Failure

Each iSCSI target has a preferred CVM. The preferred CVM is configured automatically as part of the load balancing process. If a preferred CVM goes offline and then returns to operation, the iSCSI session may fail back. In the case of a failback, the client is logged off and redirected to the a...