Post a Comment On: /dev/dump

As part of fixing some recent bugs, I integrated the following into illumos:
734 taskq_dispatch_prealloc() desired 943 zio_interrupt ends up calling taskq_dispatch with TQ_SLEEP
The interesting one is the first of these. The interface is actually called taskq_dispatch_ent(), and is private to the "consolidation" (i.e. for use within bundled code only).
What this interface provides for, however, is a way to bomb-proof your taskq dispatches, if you can arrange for the dispatching state structure (taskq_ent_t) to be allocated in advance. This means you never have to worry about the possibility of a dispatch failing due to insufficient resources.
What's even cooler, is that the cost of the dispatching is much cheaper; taskq_dispatch() was the hottest piece of code on a very busy storage server. Now it goes much much faster, because it is just twiddling some linked list pointers and sending a signal to wake up the thread processing the taskq.
More importantly, the fact that we don't ever have to sleep, or have an expensive call into the kernel memory subsystem, has solved some frustrating "stalls" in the storage subsystem.
I encourage developers working with taskqs in illumos to have a look at this interface. If you can use it, you will simplify your code, and shorten your run-paths. Both of which are good things. The only limitation: this interface is not available for dynamic taskqs. (Which makes sense since dynamic taskqs might need to allocate whole threads.)