Saturday, 9 February 2013

Using the Libdrizzle Binlog API

Now that we have frozen the 5.1 API of Libdrizzle I can blog about how to use parts of the API.

In this blog post I will cover connecting to a MySQL server and retrieving the binary logs.

First of all we need to connect to the MySQL server

#include <libdrizzle-5.1/libdrizzle.h>
#include <stdio.h>
#include <stdlib.h>
#include <errno.h>
#include <inttypes.h>

int main(void)
  drizzle_st *con;
  drizzle_binlog_st *binlog;
  drizzle_return_t ret;
  con= drizzle_create("localhost", 3306, "user", "pass", "", NULL);
  ret= drizzle_connect(con);
  if (ret != DRIZZLE_RETURN_OK)
    printf("Could not connect to server: %s\n", drizzle_error(con));
    return EXIT_FAILURE;

We are connecting to a local server on port 3306 with the username "user" and password "pass".  We don't need a default database so we leave that empty and set the options to NULL.

After we are connected to the MySQL server we need to initialize the binary log transfer.

  binlog= drizzle_binlog_init(con, binlog_event, binlog_error, NULL, true);

This states that we want an event to call function called "binlog_event" and an error to call a function called "binlog_error".  We could pass a pointer data to callback functions but for this example we have set it to "NULL".  The final parameter specifies whether or not we want to verify the checksums in MySQL 5.6 binary logs.

Then we need to start the binary log transfer.

  ret= drizzle_binlog_start(binlog, 0, "", 0);
  if (ret != DRIZZLE_RETURN_EOF)
    return EXIT_FAILURE;  }
  return EXIT_SUCCESS;

This states that we want to start the binary log at the first position of the first file using server ID 0.  With server ID 0 the MySQL server will end the transfer at the end of the last binary log file.  The function will return at the end of the transfer with DRIZZLE_RETURN_EOF or an error code.

All we need now are error callback and event callback.  So lets start with the error callback.

void binlog_error(drizzle_return_t ret, drizzle_st *con, void *context)

  (void) context;
  if (ret != DRIZZLE_RETURN_EOF)
    printf("Error retrieving binlog: %s\n", drizzle_error(con));

Finally we need to event callback.

void binlog_event(drizzle_binlog_event_st *event, void *context)
  (void) context;
  uint32_t length;
  printf("Timestamp: %"PRIu32"\n", drizzle_binlog_event_timestamp(event));
  printf("Type: %"PRIu8"\n", drizzle_binlog_event_type(event));
  printf("Server-id: %"PRIu32"\n", drizzle_binlog_event_server_id(event));
  printf("Next-pos: %"PRIu32"\n", drizzle_binlog_event_next_pos(event));

  length= drizzle_binlog_event_length(event);
  printf("Length: %"PRIu32"\n", length);
  data= drizzle_binlog_event_data(event);
  printf("Data: 0x");
  for (i=0; i<length; i++)
    printf("%02X ", data[i]);


On every event received this function is called and will basically spit out the event details.

For a further example see the source of the drizzle_binlogs tool in the Drizzle Tools tree.


  1. The client is LGPL, right? I might have to investigate turning FlexCDC into a C app.

    1. Better than that, it is 3-clause BSD licensed. So good with GPL and commercial, even static linking :)

    2. I am kind of torn how to proceed here. Do I write an external daemon that connects to mysql and reads the binary logs, a daemon plugin that reads the logs directly and runs inside of the server or take a look at how XtraDB cluster collects row changes and integrate with the server in that way.

      I think XtraDB cluster uses a modified replication plugin interface, so that is probably out.

      The external server has the advantage that it can collect changes from remote MySQL servers easily, but it is an external program that needs to be watched.

      A daemon plugin could only collect changes from the local database, but it will have less overhead because data won't be copied around through network buffers.

      What do you think?

    3. Personally I wouldn't use a daemon plugin to do it. You would need to be really careful that a bug (for example reading a corrupted entry) doesn't crash the whole server as there is no process isolation.

      I've not looked at XtraDB cluster yet so I don't know how it works. But I plan to some time before the summer.