Hello, Garrett. Please clear this news for me. I will get benefit using new SSD (supporting TRIM) for OS install? Or for L2ARC cache? Not using COMSTAR.
Your question (Vinayak) doesn't really make sense. OpenSolaris is dead, but when it was still an active project there were a number of releases at different points.
illumos started from build 147 and has continued from there.
Garrett, sorry, but I think rebuffing interested people like this was impolite. And you might scare off potential faint-hearted community members who have not yet figured out the multiple distro naming changes ;)
I hope Vinayak's question can be interpreted in illumos/OpenIndiana terms - like "in which pkg version of ZFS package is the feature included for public testing/usage?" with the answer being perhaps an URL like http://repo.openindiana.org/dev/zfs/123.456 or whatever :)
Other than that - great news and a killer feature, I guess. Thanks ;)
By the way, how does COMSTAR learn that NTFS or Linux ext* freed some block? Does it intercept TRIM notifications or whatever the virtualized OS intended for an SSD driver?
In short, would this work with any guest OS or with specific ones - like "Windows 7 and above" or "Linux kernel 2.x.y.z and above built with TRIM support"?
1. My reply to Vinayak was not meant to be be a rebuff. I honestly did not understand (and still don't) the original question -- we don't have "builds" as such for illumos, and I'm not aware of any build of OpenIndiana that has incorporated this change yet.
2. COMSTAR gets it when filesystem on the initiator notifies the underlying SCSI disk driver, which should then issue a SCSI UNMAP (which is the SCSI moral equivalent of SATA TRIM). What this means is, it will work with any initiator which issues SCSI UNMAP, which is a standard T10 request. (This includes Windows NTFS, Linux ext4, and forthcoming VMware products.A)
Great stuff. I've been hoping for this for some time.
I wonder if it is possible to build support for this into the NFS server as well in the future? I realize it's not as straight forward, but would be very useful for those of us hosting VMs over NFS rather than SCSI targets.
Thanks for the response, I suppose I should be more clear.
As you said, NFS doesn't need over-provisioning. However, when you host VMs in image files which are then stored on NFS, you run into the over-provisioning problems inside the guest VM. I suppose this can also still occur if you're hosting on a SCSI target.
I guess what I'd really hope for at some point is the ability to "tunnel" these UNMAP requests over whatever protocol in order to get it back to ZFS?
Anyway, I really appreciate the effort you've put into continuing to improve zfs.
changeset: 13297:4b9dc4ca8e9f tag: tip user: Dan McDonald <danmcd@nexenta.com> date: Fri Mar 04 13:57:09 2011 -0800 description: 701 UNMAP support for COMSTAR Reviewed by: Garrett D'Amore <garrett@nexenta.com> Reviewed by: Eric Schrock <eric.schrock@delphix.com> Reviewed by: George Wilson <gwilson@zfsmail.com> Approved by: Garrett D'Amore <garrett@nexenta.com>
This change represents a significant new feature in COMSTAR and ZFS, which will greatly benefit people use SCSI target mode functionality in situations involving over-provisioning. More on that in a minute... The feature itself was developed by Sumit Gupta for Nexenta, and is part of our upcoming 3.1 release of NexentaStor. Subsequently, Dan took ownership of that code, and working with Eric and George (who are well established ZFS gurus and had significant and useful feedback) improved it still further, and got the code into illumos proper. I believe this represents the first significant ZFS feature to go into the tree since the illumos fork, and also amply demonstrates the collaboration in the illumos community. I'm looking forward to more collaboration like this in the community. Now, what does this feature give you? Well, if you're using SCSI Target Mode (either via iSCSI or Fibre Channel or FCoE) to serve up storage to systems running NTFS or ext4, you will be able to make better use of your storage. Traditionally, when a file was deleted from a filesystem, it was mostly a matter of book-keeping in the meta data in the filesystem. There was nothing to note this in the underlying storage. With newer SSDs, and with COMSTAR, the ability to get back this notification is incredibly useful. SSDs want it to do garbage collection or other optimizations thereby improving performance. COMSTAR wants it because now when your thinly-provisioned zvol gets the notification, we can return the storage back to the pool. Prior to this change, the zvol could only grow, it could never shrink. Now, we can give storage back to the pool when you delete a file on the initiator. This is huge in environments running with a lot of VMs using thinly provisioned storage with overallocation. Anyway, this is now in illumos thanks to Nexenta, and notably, Oracle doesn't have it. Of course, they are welcome to pick up the code for it, but they will need to follow the terms of the CDDL if they choose to do so, the same as everyone else.
posted by Garrett D'Amore at 2:08 PM on Mar 4, 2011
"COMSTAR and SCSI UNMAP"
12 Comments -
Hello, Garrett.
Please clear this news for me.
I will get benefit using new SSD (supporting TRIM) for OS install?
Or for L2ARC cache?
Not using COMSTAR.
March 6, 2011 at 6:39 PM
rty: No. At the moment this is *only* for COMSTAR. Soon we'll get this for ZFS and then you will see a benefit for SSDs.
March 6, 2011 at 7:16 PM
From which build does SCSI UNMAP is implemented in opensolairs?
March 16, 2011 at 3:47 AM
From which build onwards SCSI UNMAP is supported into opensolaris?
March 16, 2011 at 3:48 AM
From which build of opensolaris SCSI UNMAP is implemented?
March 16, 2011 at 3:49 AM
Your question (Vinayak) doesn't really make sense. OpenSolaris is dead, but when it was still an active project there were a number of releases at different points.
illumos started from build 147 and has continued from there.
March 16, 2011 at 8:12 AM
Garrett, sorry, but I think rebuffing interested people like this was impolite. And you might scare off potential faint-hearted community members who have not yet figured out the multiple distro naming changes ;)
I hope Vinayak's question can be interpreted in illumos/OpenIndiana terms - like "in which pkg version of ZFS package is the feature included for public testing/usage?" with the answer being perhaps an URL like http://repo.openindiana.org/dev/zfs/123.456 or whatever :)
Other than that - great news and a killer feature, I guess. Thanks ;)
By the way, how does COMSTAR learn that NTFS or Linux ext* freed some block? Does it intercept TRIM notifications or whatever the virtualized OS intended for an SSD driver?
In short, would this work with any guest OS or with specific ones - like "Windows 7 and above" or "Linux kernel 2.x.y.z and above built with TRIM support"?
Thanks again,
//Jim Klimov
March 22, 2011 at 5:40 AM
Jim.
1. My reply to Vinayak was not meant to be be a rebuff. I honestly did not understand (and still don't) the original question -- we don't have "builds" as such for illumos, and I'm not aware of any build of OpenIndiana that has incorporated this change yet.
2. COMSTAR gets it when filesystem on the initiator notifies the underlying SCSI disk driver, which should then issue a SCSI UNMAP (which is the SCSI moral equivalent of SATA TRIM). What this means is, it will work with any initiator which issues SCSI UNMAP, which is a standard T10 request. (This includes Windows NTFS, Linux ext4, and forthcoming VMware products.A)
March 22, 2011 at 7:40 AM
Great stuff. I've been hoping for this for some time.
I wonder if it is possible to build support for this into the NFS server as well in the future? I realize it's not as straight forward, but would be very useful for those of us hosting VMs over NFS rather than SCSI targets.
March 22, 2011 at 9:51 AM
Over NFS, you wind up not needing to overprovision, so it isn't the same problem.
But we will also be doing SATA TRIM and SCSI UNMAP for local ZFS resources (and NFS/CIFS shares of those resources) soon.
March 22, 2011 at 10:09 AM
Thanks for the response, I suppose I should be more clear.
As you said, NFS doesn't need over-provisioning. However, when you host VMs in image files which are then stored on NFS, you run into the over-provisioning problems inside the guest VM. I suppose this can also still occur if you're hosting on a SCSI target.
I guess what I'd really hope for at some point is the ability to "tunnel" these UNMAP requests over whatever protocol in order to get it back to ZFS?
Anyway, I really appreciate the effort you've put into continuing to improve zfs.
March 22, 2011 at 11:52 AM
Garrett, by forthcoming VMware products, you mean VMFS 5 ?
April 4, 2011 at 3:05 PM