So in the latest Drizzle trunk (and the release due in the next few days) I have introduced global buffer constraints. These constraints make sure that between all the clients there is a cap on how much RAM can be allocated to a buffer.
This was a task the Rackspace Cloud team requested be implemented and I have also seen (and subsequently lost the link to) a blog which discusses the possibility of something similar in MySQL. If anyone has seen the blog post I am talking about please let me know :)
How this is implemented
For each buffer type there is a new atomic counter which is keeping account of the allocations for each buffer type. If one of theses counters hits a cap then the query executing when the cap is hit will return an error. So far I have added this to 4 buffers which are sort buffer, join buffer, read buffer and read rnd buffer. The new options for these are:
By default all are set to 0 which means unlimited, and cannot be set lower than their associative settings (eg. sort-heap-threshold cannot be lower than sort-buffer-size).
I would typically expect this would be used most in cloud scenarios where the virtual servers could have a limited amount of RAM and swap available.