How to create a RAIDZ2 pool with a hot spare

Continuing with the same scenarios, we would be using

Disk 2, Disk 3, Disk 4, Disk 5 and Disk 6 to create a RAIDZ2 pool. Each disk is 2.00GB each. We would be later using Disk 1 as a spare in the pool. First lets go ahead and create the pool. Here's how it is done:

root@opensolaris:~# zpool create plrdz2 c4d1 c5t0d0 c5t1d0 c5t2d0 c5t3d0
root@opensolaris:~#

Lets check this.

root@opensolaris:~# zpool status plrdz2
pool: plrdz2
state: ONLINE
scrub: none requested
config:

NAME STATE READ WRITE CKSUM
plrdz2 ONLINE 0 0 0
raidz2 ONLINE 0 0 0
c4d1 ONLINE 0 0 0
c5t0d0 ONLINE 0 0 0
c5t1d0 ONLINE 0 0 0
c5t2d0 ONLINE 0 0 0
c5t3d0 ONLINE 0 0 0

errors: No known data errors
root@opensolaris:~#

The commmand executed successfully and thus now we have a pool named plrdz2 which is RAIDZ2. So this pool can easily tolerate 2 disk failures without giving any data loss. So if 2 disks fail for any reason, you still have your data one piece. Nothing to lose. However the pool would be in DEGRADED state. Now to add further reliability, i would go ahead and one disk as a spare disk. Which would mean that if one of the disk in the pool happens to fail, that disk would be set aside and this hot spare disk would be used to write data. Eventually your pool is safe even for 3 disk failures. Here's how we add a hot spare to our pool:

root@opensolaris:~# zpool add plrdz2 spare c3d1
root@opensolaris:~#

The command completed successfully.

root@opensolaris:~# zpool status plrdz2
pool: plrdz2
state: ONLINE
scrub: none requested
config:

NAME STATE READ WRITE CKSUM
plrdz2 ONLINE 0 0 0
raidz2 ONLINE 0 0 0
c4d1 ONLINE 0 0 0
c5t0d0 ONLINE 0 0 0
c5t1d0 ONLINE 0 0 0
c5t2d0 ONLINE 0 0 0
c5t3d0 ONLINE 0 0 0
spares
c3d1 AVAIL

errors: No known data errors

Now lets do some play. I would go ahead and physically remove Disk 4 (i.e. c5t1d0). Now lets see what happens

root@opensolaris:~#
root@opensolaris:~# zpool status plrdz2
pool: plrdz2
state: DEGRADED
status: One or more devices could not be opened. Sufficient replicas exist for
the pool to continue functioning in a degraded state.
action: Attach the missing device and online it using 'zpool online'.
see: http://www.sun.com/msg/ZFS-8000-2Q
scrub: resilver completed after 0h0m with 0 errors on Wed Apr 8 15:49:52 2009
config:

NAME STATE READ WRITE CKSUM
plrdz2 DEGRADED 0 0 0
raidz2 DEGRADED 0 0 0
c4d1 ONLINE 0 0 0 67.5K resilvered
c5t0d0 ONLINE 0 0 0 70K resilvered
spare DEGRADED 0 0 0
c5t1d0 UNAVAIL 0 868 0 cannot open
c3d1 ONLINE 0 0 0 134M resilvered
c5t2d0 ONLINE 0 0 0 67.5K resilvered
c5t3d0 ONLINE 0 0 0 67K resilvered
spares
c3d1 INUSE currently in use

errors: No known data errors
root@opensolaris:~#

If you see carefully, the system has now intelligently take the hot spare disk and is still functioning well.

Point to Note and Remember: From what i know RAIDZ pool cannot be upgraded to RAIDZ2 pool. You would need a minimum of 3 disk to create a RAIDZ2 pool.