Tuesday, 15 March 2011

Drizzle - The Icing on the Cake - part 1

As I'm sure all of you know already, today marks the GA release of Drizzle7.  But what was the recipe behind Drizzle?

  1. Take the raw ingredients from previous delicious, well-tried recipes

  2. Sieve out the lumps, separate the eggs and mix

  3. Bake using many cooks in many ovens for around 3 years

  4. Drizzle the special source on top


What is Drizzle?


There are many marketing buzzwords which can describe what Drizzle is such as "A lightweight, microkernel fork of MySQL optimized for the web and cloud".  To me such things are pretty meaningless.  So, lets start at the beginning...

Several people inside MySQL saw that the code could really do with re-factoring.  At the same time they believed that the focus was heading away from it's core web based installations.  They also loved open source, and whilst MySQL is open source, community contributions can be difficult.  These people got together inside Sun Microsystems (and other companies) to create a completely open development of a fork of MySQL 6.0 called "Drizzle".  They aimed to have it easy for new developers to pick up and develop on, moved many parts out to plugins thereby making it light on resources when features are not needed.

In 2010 the original development team moved to Rackspace and several more members were hired (including me), with the aim of Drizzle being used in it's cloud based products.  Even today the amount of active community contributors is higher than the amount of developers inside Rackspace working on Drizzle.

Differences From MySQL


I have been asked many times what the differences are between MySQL and Drizzle.  This is something I could probably write a book on now.  Something that should be clear at this stage is Drizzle is not MySQL, it was MySQL over 3 years but a lot has changed since then.  Having said that, applications that use MySQL can usually be converted to use Drizzle relatively easily.  For new users to Drizzle, here are a few of the key differences:

  • Strictness - Drizzle doesn't assume what you mean (which can cause incorrectly recorded data).  For example trying to store an invalid ENUM will error instead of storing an empty value.

  • Data Types - Drizzle has removed, altered and added data types to simplify things and become closer to the SQL standard.  For example:

    • There is no TINY/SMALL/MEDIUM INT, just INT (and BIGINT).

    • There is no TINY/MEDIUM/LONG TEXT/BLOB, there is just TEXT/BLOB.

    • TIMESTAMP supports microsecond precision.

    • UUID and a true BOOLEAN type added.



  • Replication - Drizzle's replication uses Google Protocol Buffer messages so a replication reader can be written in any language in minutes.  The replication data is stored in InnoDB as part of the transaction as it is being committed so that writing the replication log is very fast.

  • Development - Drizzle is developed using a completely open development model which I will discuss in part 2.

  • Licensing - The main drizzle source is GPLv2 licensed, libdrizzle is BSD licensed and the docs (which are also included in the docs directory of the source) are CC SA 3.0 licensed.  There is no proprietary licensing for any part of Drizzle.


Compatibility With MySQL


Despite many changes there is still a great deal of compatibility with MySQL.  Drizzle speaks the MySQL protocol, so existing MySQL connectors for PHP/Perl/etc... will also connect to and query Drizzle.  The SQL syntax is still very similar to MySQL and on top of all this, drizzledump (which is very similar to mysqldump) can convert table structures and data from MySQL to Drizzle on-the-fly.

Drizzle also includes libdrizzle.  This is a BSD licensed client library written in C which can talk to MySQL and Drizzle servers, from our testing as well as the testing of developers who are integrating libdrizzle into their products it appears that libdrizzle performs better than libmysqlclient too.  Connectors for libdrizzle have been written for most widely used languages such as Python, Java, Perl and PHP.

Plugins


Drizzle uses a completely new plugin architecture so that almost everything is a plugin.  From storage engines, to functions, to protocols, to authentication and even query cache.  This makes it much simpler to switch off the parts you don't use as well as customising Drizzle for your unique application.  In total there are around 80 plugins bundled in the Drizzle source and several others available around the web.

Despite this we have tried to make this easy for most people by having the plugins that most people will use compiled and running by default.

Summary


It is almost impossible to get a feel for what Drizzle is like without trying it for yourself.  We have had some great feedback both positive and negative, and have made changes thanks to this feedback.  We are all very approachable on #drizzle on Freenode and the Drizzle mailing list.

In part 2 I will discuss the open development model behind Drizzle

10 comments:

  1. A book on Drizzle would be really appreciated by the web development community.

    Drizzle is extremely exciting.

    ReplyDelete
  2. Thanks Sammy,

    It is certainly something I have considered doing several times. Drizzle is more than a job to me, the only problem with this is finding time to write a book :)

    ReplyDelete
  3. [...] part 1 of this 3 part series I talked about what is new in the recently released Drizzle7 and what makes [...]

    ReplyDelete
  4. This is more than I knew about Drizzle, and I'm sufficiently interested. Now, how would Drizzle handle an existing Wordpress/MySQL site? If I did a db dump and imported it into Drizzle, would Wordpress be able to use it without issue? Same question about Drupal, assume it's similar, would a plugin be needed, or no?

    On a related note, working on setting up a replicant MySQL db at work today and found that Oracle broke all the old links that Google knows about to the old MySQL dev/doc MySQL site! If Oracle can't keep old links valid with mod_rewrite or something similar, I really question their commitment to any existing open source community. Come on, this is basic web-hosting 101!

    ReplyDelete
  5. Hi fak3r,

    With a Wordpress 3.1 site if you convert the schemas/data with drizzledump there is a slight changed needed to the PHP code. Basically anywhere that 0000-00-00 is used for the date needs to change to 0001-01-01 since 0000-00-00 is not a valid date in Drizzle. I am going to give a talk about this at the MySQL Users Conference and will give a much better explanation in my blog in the next couple of weeks.

    Drupal I have not tried yet but need to very soon, I will work on that and blog about it as well when I have played with it.

    I have a lot of thoughts and feelings about Oracle's handling of MySQL (since I did work for MySQL under Oracle for about 6 months), but most will stay my own ;)

    ReplyDelete
  6. Very cool, I'd be interested in trying Drizzle out with a wordpress blog sometime. It sounds like there would be a change in the Wordpress PHP code? Maybe a plugin for that would enable a global change in WP to support Drizzle. Looking forward to trying it out.

    ReplyDelete
  7. Yes, it is a very slight change in the Wordpress PHP code. I will look into seeing if I can make the patch I currently have into a plugin.

    We will be interested to know how you find it. All user feedback both positive and negative is useful to help us work towards improving things for future releases.

    Making MySQL apps compatible is a high priority for me right now :)

    ReplyDelete
  8. [...] the HUGE news of Drizzle’s first GA, I thought it appropriate that I spend some time discussing the testing [...]

    ReplyDelete
  9. [...] you can see my three-part special called “Drizzle – The Icing on the Cake”: part 1 part 2 part [...]

    ReplyDelete