tokumx news
===========

tokumx 2.0.1
------------
Release date: 2015-03-02 

General
~~~~~~~

    - This release is focused on bug fixes since 2.0.0.


Bug fixes
~~~~~~~~~

    - Fixes incorrect results from the projection operator when querying
      by primary key.
      (https://tokutek.atlassian.net/browse/MX-1311[MX-1311])

    - Enforce uniqueness of index names when creating a new index.
      (https://tokutek.atlassian.net/browse/MX-1314[MX-1314])


tokumx 2.0.0
------------
Release date: 2014-09-30

General
~~~~~~~

    - This release is focused on new features and improvements since
      1.5.1: geospatial features, Ark elections, fast updates, and
      simultaneously partitioned and sharded collections.


Highlights
~~~~~~~~~~

    - Geospatial indexing and queries have been added and are compatible
      with those in MongoDB 2.4.

    - The election algorithm used for replication failover has been
      replaced with Ark, which prevents elections from rolling back writes
      acknowledged with majority Write Concern, and makes failovers
      faster.

    - Fast updates are a new update mechanism that avoids a preliminary
      query during an update operation, when possible, boosting the speed
      of update operations by up to 10x.

    - Partitioned collections may now be sharded, in a restricted form
      useful for many time-series workloads with a suitable shard key.


New features and improvements
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

    - All geospatial indexing and query features of MongoDB 2.4 have been
      added to TokuMX.
      (https://tokutek.atlassian.net/browse/MX-166[MX-166])

    - Replication elections now use Ark, a consensus algorithm similar to
      Raft, to elect a new primary node during failover.

      This reduces the downtime experienced by clients due to failed
      failover elections.

      This also ensures that writes acknowledged with majority Write
      Concern will never be rolled back by any later failover.
      (https://tokutek.atlassian.net/browse/MX-765[MX-765])

    - Previously, updates performed a query to read the existing document,
      then made changes to the relevant indexes.  In Fractal Tree indexes,
      the query is typically much slower than the changes made by an
      update.

      TokuMX 2.0 transparently optimizes updates of the right form to
      avoid the query phase.  For updates by primary key (or `_id` by
      default) that don't modify any secondary indexes, this can result in
      a dramatic speedup.

      Please see the documentation for full details.
      (https://tokutek.atlassian.net/browse/MX-932[MX-932])

    - Reduced the amount of data written to disk when applying replicated
      operations on secondaries.
      (https://tokutek.atlassian.net/browse/MX-1133[MX-1133])

    - Replication rollback has been changed so that oplog entries are
      significantly smaller for update and delete operations.
      (https://tokutek.atlassian.net/browse/MX-1223[MX-1223],
      https://tokutek.atlassian.net/browse/MX-1238[MX-1238])

    - Partitioned collections may now be sharded, with several
      restrictions:

       - The collection must be created by the `shardCollection` command,
         and at that time, both the shard key and partition key must be
         provided.  The partition key will be used as the primary key.

       - The balancer will not move chunks for a sharded and partitioned
         collection.  Users may manually move chunks with
         `sh.moveChunk()`, however this is only recommended for empty
         chunks, as a collection initialization step.

       - Chunks are not automatically split, though users may manually
         split chunks with `sh.splitAt()`, which is recommended as a
         collection initialization step.

      Please see the documentation for full details.
      (https://tokutek.atlassian.net/browse/MX-1232[MX-1232])

    - Optimized the comparison function used to compare ObjectIDs.  This
      results in a modest performance improvement for non-covered
      queries.  (https://tokutek.atlassian.net/browse/MX-1251[MX-1251])

    - Replication rollback now writes information about the operations
      that were rolled back to a separate collection,
      `local.rollback.opdata`.

      Please see the documentation for full details.
      (https://tokutek.atlassian.net/browse/MX-1258[MX-1258])

    - After a successful primary election, the new primary writes a
      comment in its oplog noting the result of the election.  This entry
      now includes the primary's hostname.
      (https://tokutek.atlassian.net/browse/MX-1260[MX-1260])


Bug fixes
~~~~~~~~~

    - Reduced delays in replication state changes caused by unnecessary
      coupling between replication subsystems.
      (https://tokutek.atlassian.net/browse/MX-1136[MX-1136])

    - Partitioned collection administrative commands (`addPartition`,
      `dropPartition`, and `getPartitionInfo`) are now passed from
      `mongos` through to `mongod`.
      (https://tokutek.atlassian.net/browse/MX-1139[MX-1139])

    - Fixed an issue that caused `collStats` to fail on a partitioned
      collection after dropping an index.
      (https://tokutek.atlassian.net/browse/MX-1262[MX-1262])

    - Fixed `mongorestore`'s behavior when restoring a partitioned
      collection, to properly restore the partition layout as well.
      (https://tokutek.atlassian.net/browse/MX-1269[MX-1269])


tokumx 1.5.1
------------
Release date: 2014-09-10

General
~~~~~~~

    - This release is focused on bug fixes since 1.5.0.


Bug fixes
~~~~~~~~~

    - In versions of TokuMX 1.4.0 through 1.5.0, compound keys with
      integer values of absolute value larger than 2^52 in all but the
      last component of the key were ordered incorrectly.  This release
      fixes that ordering.

      This error only affected compound keys, so hashed keys are
      unaffected.

      The result of the error is that if there is user data of that size
      in any but the last field of a key, the key will be ordered
      incorrectly with respect to other keys in the index.  This can cause
      queries to return improper results if they do range queries on that
      index.

      If the affected index is the primary key of a collection, then
      before upgrading to 1.5.1, you must dump the affected collection,
      drop it, and restore it after upgrading to 1.5.1.  If the affected
      index is a secondary key, it can simply be dropped and rebuilt after
      upgrading to 1.5.1.
      (https://tokutek.atlassian.net/browse/MX-1140[MX-1140])

    - Lengthened the socket timeout used by secondaries when replicating
      large transactions.  Accumulation of garbage due to aborted
      transactions can cause these queries to take a long time, and with a
      short socket timeout, secondaries could get stuck.  This socket
      timeout is configurable with the server parameter
      `soTimeoutForReplLargeTxn`, which defaults to 10 minutes.
      (https://tokutek.atlassian.net/browse/MX-1207[MX-1207])

    - Fixed an issue where if the system ran out of disk space during
      application of a transaction on a secondary, it could cause that
      transaction to be skipped rather than causing the system to report
      the error.  In this scenario, the secondary now reports the problem
      and crashes.
      (https://tokutek.atlassian.net/browse/MX-1235[MX-1235])


tokumx 1.5.0
------------
Release date: 2014-06-18

General
~~~~~~~

    - This release is focused on the addition of a new collection type,
      `partitioned` collections, which provide excellent performance and
      flexibility for time-series workloads.


Highlights
~~~~~~~~~~

    - Partitioned collections are a new feature with a similar use case to
      `capped` collections in basic MongoDB, but with better concurrency
      and flexibility.

      Briefly, a partitioned collection is similar to a partitioned table
      in some RDBMSes.  The collection is divided physically by primary
      key into multiple underlying partitions, allowing for the
      instantaneous and total deletion of entire partitions, but it is
      treated as one logical collection from the client's perspective.

      A typical use case for this would be to partition on a `{date: 1,
      _id: 1}` primary key, and periodically add new partitions and drop
      old ones to work with the latest data in a time-series data set.
      (#1063)


New features and improvements
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

    - Further reduced the locking done during migrations, to reduce
      stalls.  (#1041, #1135)

    - Initial sync now reports its progress in terms of the # of dbs,
      collections within a db, and bytes copied, then indexes built for
      a collection, in the server log, in `db.currentOp()`, and in
      `rs.status()`.  (#1050)

    - Added a new option to allow operations that require a metadata write
      lock (such as adding an index) to interrupt operations that are
      holding a read lock.  This option can be used to avoid those write
      lock requests stalling behind a long read locked operation (such as
      a large `multi: true` update), and in turn stalling all other
      operations, including readers, behind itself.

      To use this option, your application must be ready to handle failed
      operations and retry them (transactional atomicity makes this
      somewhat simpler), then you can turn it on for all clients with
      `--setParameter forceWriteLocks=true` or with the `setParameter`
      command.  This makes all future connections interruptible while they
      have read locks, but this state can be controlled for each
      connection with the `setWriteLockYielding` command.  See
      https://github.com/Tokutek/mongo/issues/1132 for more details.
      (#1132)

    - During initial sync, there is a step that checks that all
      collections that existed at the beginning of the sync's snapshot
      transaction still exist, to maintain the snapshot's validity.  This
      step has been optimized to avoid slowness in some rare situations.
      (#1162)

    - Serial insertions have been made significantly faster with
      improvements to the storage engine.  (Tokutek/ft-index#158)

    - Update workloads on extremely large data sets (where a large portion
      of even internal nodes do not fit in main memory) have been
      optimized somewhat.  (Tokutek/ft-index#226)

    - Added per-connection control of document-level lock wait timeouts
      with a new command, `{setClientLockTimeout: <timeoutms>}`.

      This controls the same behavior as `--lockTimeout` which can also be
      controlled server-wide with `setParameter` (which now controls the
      default for all new connections).  The value should be specified in
      milliseconds.  (#1168)


Bug fixes
~~~~~~~~~

    - Cleaned up the way remote errors are handled in the distributed
      locking framework, which will reduce the impact of some rare errors
      on config servers.  (#857, #1140)

    - While a shard is draining, it will no longer be chosen to receive
      new chunks due to the sequential insert optimization, nor will it be
      chosen as the primary shard for new databases.  (#1069)

    - The `mongo2toku` tool now correctly uses the `admin` database for
      authentication to the target cluster by default.  (#1125)

    - Fixed the way packaging scripts check for transparent hugepages, and
      changed them to allow `madvise` as an acceptable setting.  (#1130)

    - Connections used to copy data for initial sync no longer time out
      improperly if some operations (such as checking for snapshot
      invalidation) take too long.  (#1147)


tokumx 1.4.3
------------
Release date: 2014-06-10

General
~~~~~~~

    - This release contains a bug fix for 1.4.2.


Bug fixes
~~~~~~~~~

    - Fixed a rare memory accounting issue caused by differently typed
      integers in indexed fields of BSON documents, that could lead to
      crashes.  (Tokutek/ft-index#258)


tokumx 1.4.2
------------
Release date: 2014-05-09

General
~~~~~~~

    - This release is focused on bug fixes since 1.4.1.

    - Please take special note of issues #1085 and #1087, if you are
      affected by them you may need to drop and re-create some indexes to
      make sure they are consistent with the primary key.


New features and improvements
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

    - Added new `mongod` server parameters that control the default
      parameters used when creating new collections and indexes:

       - `defaultCompression` (default `"zlib"`)
       - `defaultPageSize` (default `"4MB"`)
       - `defaultReadPageSize` (default `"64KB"`)

      Use the `setParameter` command line option, or the
      `getParameter`/`setParameter` commands to control these values,
      e.g. `mongod --setParameter=defaultCompression="quicklz"` or
      `db.setParameter('defaultCompression', 'quicklz')`.

      Also added new `mongorestore` command line options that, if set,
      determine the default parameters used for restoring new collections
      and indexes, unless they are specified in the `metadata.json` file.
      If the `mongorestore` options are not set, the `mongod`'s values are
      used instead:

        - `defaultCompression`
        - `defaultPageSize`
        - `defaultReadPageSize`

      These options are specified as normal options, e.g.
      `mongorestore --defaultCompression "quicklz"`.  (#377)

    - The engine-specific information in serverStatus() has been expanded,
      including some new information for existing sections, and a new
      `ft.checkpoint` section.  See the user's guide for more details.
      (Tokutek/ft-index#219, #1090)

    - Background index build operations no longer hold a read lock on
      their database.  This eliminates the possibility that another thread
      can stall the system by trying to acquire a write lock, which would
      greedily block other operations until the index build finished.
      (#1035)

    - The `--rowcount` argument to `mongostat` now works with multiple
      servers.  (#1040)

    - Added better progress reporting for index building, bulk loading
      with `mongorestore`, `reIndex` operations, and disk format version
      upgrade on startup.  This information is available in the log and in
      `db.currentOp()`.  (#1048, #1052, #1053)

    - The shell's `show collections` now reports uncompressed and
      compressed sizes like `show dbs`.  (#1102)


Bug fixes
~~~~~~~~~

    - Optimized the operation of opening a database when there are a large
      number of databases already open.  (Tokutek/ft-index#225)

    - In sharded clusters, new databases now have their primary shard
      chosen correctly, according to total uncompressed data size, rather
      than always to the first shard.  (#422)

    - Chunk migrations in sharded clusters now just warn about missing
      indexes.

      Chunk migrations used to check that the recipient shard had the same
      indexes as the donor shard, and would build any missing indexes at
      that point.  This situation could arise from a user aborted index
      build, if some shards finished building before the abort.  In this
      case, it would cause migrations to force an expensive index build at
      unpredictable times.  With this change, users should watch for these
      messages, but can now choose when to schedule the index build.
      (#1045)

    - Fixed a starvation interaction between a long-running `reIndex`
      operation and a `setShardVersion` (which is used by several sharding
      components).  (#1047)

    - Fixed the default location for `pluginsDir` in deb and rpm packages.
      (#1051)

    - Fixed a crash that could occur with large range queries and drivers
      that use connection pools.  (#1058)

    - Fixed a deadlock when upgrading the disk format version with a large
      number of databases.  (#1059)

    - Several fixes to the `mongorestore` tool, including adding an option
      to disable the bulk loader (`--noLoader`) and better error
      checking.  (#1064)

    - Merged fixes from upstream through 2.4.10:  (#1077)

      - https://jira.mongodb.org/browse/SERVER-9259[SERVER-9259]
      - https://jira.mongodb.org/browse/SERVER-11560[SERVER-11560]
      - https://jira.mongodb.org/browse/SERVER-12034[SERVER-12034]
      - https://jira.mongodb.org/browse/SERVER-12132[SERVER-12132]
      - https://jira.mongodb.org/browse/SERVER-12548[SERVER-12548]
      - https://jira.mongodb.org/browse/SERVER-12908[SERVER-12908]
      - https://jira.mongodb.org/browse/SERVER-13249[SERVER-13249]
      - https://jira.mongodb.org/browse/SERVER-13333[SERVER-13333]
      - https://jira.mongodb.org/browse/SERVER-13376[SERVER-13376]

    - Fixed a bug in `update` that could cause a background index being
      concurrently built to not reflect the result of the update if the
      building index's key was affected by the update.

      This bug existed in 1.4.0 and 1.4.1; any indexes built in the
      background with those versions should be dropped and recreated if it
      is possible that there were updates that would have affected the
      index's key while it was building.  (#1085)

    - Fixed an issue where background index builds replicated to
      secondaries did not have their metadata properly maintained by 1.4.0
      and 1.4.1 secondaries.  This can cause updates to those indexes to
      not be reflected on the secondaries' versions of those indexes,
      which is similar to #1085, but this state lasts longer than just
      during the index build as in #1085.

      This can but does not necessarily also cause those indexes to be
      inaccessible after a server reboot, and upgrading to 1.4.2 will
      clean up those indexes (they will be dropped and will report to the
      log how to rebuild them).  Any background indexes that were
      replicated to secondaries running 1.4.0 and 1.4.1, if they are not
      cleaned up by this mechanism, should be dropped and recreated.

      This can be done by starting the secondary without the `--replSet`
      option or config file directive to take it out of the set, building
      the index while it's not in the set, then starting it with the
      `--replSet` option to get it back into the set.  Before doing this,
      you should add arbiters if needed to make sure there will be enough
      other nodes in the set to keep the primary from stepping down.  This
      procedure is described in
      http://docs.mongodb.org/manual/tutorial/build-indexes-on-replica-sets/[Build indexes on replica sets].
      (#1087)

    - A unique index cannot be built in the background.  In previous
      versions such an index was turned into a foreground index, now an
      error is returned that lets the user choose what to do.  (#1091)

    - Beginning a serializable multi-statement transaction
      (`db.beginTransaction('serializable')`) now requires the `readWrite`
      role on servers running with authorization.  This prevents users
      with only the `read` role from creating serializable transactions
      and running queries that take document-level locks.  (#1112)

    - Replication secondaries now handle query errors while checking for
      rollback by retrying, rather than crashing.  (#1113)


tokumx 1.4.1
------------
Release date: 2014-03-25

General
~~~~~~~

    - This release is focused on bug fixes since 1.4.0.


New features and improvements
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

    - Improved the speed and reduced the impact of deletes after a chunk
      migration.  (#957)

    - Improved the speed of the `touch` command.  (#959)

    - TokuMX now gathers more information and tries to output it to the
      log on a crash.  (#987, #993)

    - Added `compressionRatio` section to `serverStatus` to report the
      compression ratio calculated on disk write.  (#989)

    - Added aggregate per-index stats in output of `db.collection.stats()`
      for sharded collections.  (#1019)


Bug fixes
~~~~~~~~~

    - Newly sharded databases now have their primary shard chosen as the
      most lightly loaded shard (by data size), rather than always
      choosing the first shard.  (#422, #1029)

    - Fixed the output of `show dbs` and `listDatabases` in sharded
      clusters.  (#962)

    - Fixed a bad query plan used by a secondary to find its rollback
      point in the oplog, which could cause startup to time out.  (#965)

    - Removed erroneous `system.namespaces` entries for partitioned
      collections, including the oplog.  (#966, #967)

    - Enabled the use of a hashed shard key when there is a user-defined
      primary key.  In 1.4.0, the shard key would be rejected because the
      primary key looks like a unique index whose uniqueness cannot be
      enforced, however, for a primary key it is enforced by the `_id`'s
      uniqueness assumption.  (#968)

    - Fixed linking of some commands, including `loadPlugin` and the bulk
      loader commands in `mongorestore`.  (#977, #981, #983)

    - Fixed problem opening too many files on startup, causing listening
      sockets' ids to be too high.  (#978, #984)

    - Changing storage settings (compression, page sizes, etc.) with
      `reIndex` now correctly persists the change for future opens and
      added partitions.  (#982)

    - The `getParameter` command now accepts `journalCommitInterval` as an
      alias for `logFlushPeriod`.  (#988)

    - Fixed issue with `listDatabases` and partitioned collections.
      (#997)

    - Fixed a race between aborting a transaction due to client
      disconnect, and the transaction abort due to replica set
      relinquish.  (#1003)

    - Fixed the way errors during chunk migration cleanup are handled to
      avoid a crash when a replica set primary steps down at the end of a
      migration.  (#1010)

    - Fixed an issue that could cause migrations to fail repeatedly.
      (#1011)

    - Fixed command authorization settings for enterprise plugin loading
      commands.  (#1012)

    - Fixed a bug that caused migrations to fail when the migration causes
      the recipient shard to transition an index to multiKey.  (#1025)

    - Backported a fix for SERVER-12515, a bug that corrupts the state of
      the config.chunks collection in sharded clusters when using hashed
      shard keys.  (#1030)

    - Added a retry mechanism for chunk migration commit to make it less
      brittle in the face of a flaky connection to the config servers.
      (#1032)


tokumx 1.4.0
------------
Release date: 2014-02-13

General
~~~~~~~

    - This release is focused on making the TokuMX cluster experience
      smoother.  The major improvements are in eliminating stalls that
      could appear in replica sets and sharded clusters.


Upgrade notice
~~~~~~~~~~~~~~

Existing TokuMX users running with replication, please read before
upgrading:

    - Due to some new oplog entry types, servers running earlier versions
      of TokuMX cannot replicate from a primary running TokuMX version
      1.4.0 or greater.  Therefore, when upgrading a replica set, be sure
      to upgrade all secondaries first.  Only then should you upgrade the
      primary.

      If failover happens during this period, make sure it is a 1.3
      machine that steps up to primary.  You can set the servers'
      `priority` to ensure this.  If a 1.4 machine becomes primary, it may
      begin to log operations that a 1.3 machine will not replicate, which
      would halt replication for those 1.3 secondaries.  Once a majority
      of machines have been upgraded to 1.4, it is safe to allow them to
      step up, but at this point any remaining 1.3 machines may start to
      experience replication lag, so only allow this if your cluster has
      enough capacity to serve requests only from the 1.4 machines.


Highlights
~~~~~~~~~~

    The list of minor new features and bugfixes in this release is large.
    The most prominent of them appear here:

    - Chunk migrations in sharding have been greatly simplified and
      optimized.  They no longer take as many locks, particularly
      database- and server-wide locks, which could previously cause stalls
      and pileups.  Additionally, the data transfer between shards has
      been simplified and optimized and is significantly faster and more
      reliable.  (#214, #746, #823, #824)

    - In TokuMX 1.3 and earlier, updates were represented in the storage
      layer, and in the oplog, by their pre- and post-images, which could
      be quite large compared to the actual change that occurred in the
      document (consider incrementing an integer in a large document).
      For replication, this ensures that secondaries use significantly
      less I/O for many update workloads, but costs a lot of space in the
      oplog and over the network.

      In TokuMX 1.4, updates to unindexed fields represent their updates
      on disk with the update modifier object itself, which can
      significantly reduce the amount of cache memory used by internal
      node buffers in the Fractal Tree indexing structure.

      Most updates (including to indexed fields) are represented in the
      oplog by the pre-image and the update object, which can reduce, by
      up to half, the amount of space and network bandwidth used by the
      oplog.  (#511)

    - It is now possible to define a "primary key" for a collection.  A
      primary index for a collection is defined by the primary key, is an
      index which is unique and clustering, and is the index that is used
      by all non-clustering secondary indexes to find the full document.
      The primary key for each document is therefore embedded in every
      secondary index.  The key must end in `{_id: 1}` to ensure
      uniqueness, and it is a good idea to let the `_id` be
      auto-generated.

      In some cases, if a clustering secondary index is desired, a primary
      key can be defined instead, which removes the need for a clustering
      `_id` index, which therefore can reduce storage costs even further
      for such a use case.  (#684)

    - This primary key concept is prominently featured in sharding.  Newly
      sharded, nonexistent collections with a shard key other than just
      the `_id` will be created using a primary key of the shard key
      concatenated with `{_id: 1}`.  In TokuMX 1.3 and earlier, to
      facilitate fast migrations and range queries, it was strongly
      encouraged to make the shard key clustering, but this had the
      drawback that there were automatically two clustering indexes (the
      `_id` index and the shard key).  Now, sharded collections can have a
      clustered shard key without paying the extra storage cost, and this
      behavior is the default.  (#740)

    - In TokuMX 1.3 and earlier, the oplog was a single collection that
      was trimmed by a background task, to maintain a specific amount of
      history, specified in days (`expireOplogDays`).  This background
      thread could sometimes fail to reclaim space aggressively enough, so
      in TokuMX 1.4, the oplog is instead partitioned into a set of N
      collections, one per day, and each day, the oldest partition is
      dropped according to `expireOplogDays`, which reclaims space
      immediately.  This should make the oplog occupy a smaller disk
      footprint overall, and should alleviate some locking problems that
      used to be caused by the background trimming task.

      So-called "partitioned collections" (of which the oplog is now one)
      are a generally-available feature, but they currently have
      significant limitations, including that they do not permit secondary
      indexes.  See the user's guide for more details.  (#762)


New features and improvements
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

    - The `listDatabases` command now computes a size estimate for each
      db, which also means that `show dbs` in the shell will report the
      size of each database (compressed and uncompressed), rather than
      showing "(empty)" for everything regardless of contents.  (#57)

    - Provided new implementations for `db.getReplicationInfo()`,
      `db.printReplicationInfo()`, and `db.printSlaveReplicationInfo()`,
      including compressed and uncompressed sizes.  (#223)

    - The `compression`, `pageSize`, and `readPageSize` attributes of an
      index can now be changed online.  The `db.collection.reIndex()`
      shell function now takes a second parameter, which if specified is
      an object describing the options to set.  These options take effect
      immediately for all data written after the command runs, but will
      not rewrite existing data.  To force existing data to be rewritten
      with the new options, run a `reIndex` command without specifying
      options, to rewrite all the data for that index, forcing the
      conversion.  See the user's guide for full usage details.  (#389)

    - The diagnostic tools `db.showPendingLockRequests()` and
      `db.showLiveTransactions()` now use cursors to return results, which
      means that in extreme cases, the results will no longer be truncated
      due to the maximum size of a BSON object, instead the results will
      be paginated.  This doesn't change the fact that they are a snapshot
      of the state of the system.  (#606)

    - The `count()` operation has been further optimized for simple
      interval count operations, by up to 4x in some workloads.  (#725)

    - Exhaustive scans are no longer ever chosen when a useful index
      exists for a query.  The query optimizer is a sample-based optimizer
      which would periodically retry all possible plans, including an
      exhaustive scan.  For certain update workloads, this could cause a
      major 99th percentile latency problem, and concurrency bottleneck,
      so this change alleviates that problem.  (#796)

    - The sharding subsystem used to use database- and system-wide locking
      to protect metadata and sharding-only state.  That usage of those
      locks has been refined to avoid stalls caused by long-running DML
      operations combined with unrelated administrative operations.
      (#797)

    - Added the `$allMatching` flag to `db.currentOp()` to aid in
      finding background tasks.  (#814)

    - The storage format for integers with absolute value greater than
      2^52 has been optimized to reduce the CPU cost of operations on such
      values.  This is of particular importance to users of hashed keys
      (e.g. hashed shard keys), because hash values are pseudorandom bit
      patterns and are therefore almost always larger than 2^52.  In some
      tests, up to a 20% speed improvement has been measured for CPU-bound
      workloads.  (#820)

    - Added the `compression`, `pageSize`, and `readPageSize` parameters
      to the `db.createCollection()` shell function, so the following is
      now valid:

          db.createCollection('foo',
                              {compression: 'lzma'
                               pageSize: '8MB'})

      Also, shell functions were added to wrap the `getParameter` and
      `setParameter` commands.  See `db.help()` for more.  (#854)

    - Added a server option `loaderCompressTmp` that, if on, causes the
      bulk loader (used by `mongorestore` and `mongoimport` and
      non-background index builds) to compress intermediate files before
      writing them to disk, thus using less disk space during the index
      build.  This can also be set with the `setParameter` command.
      (#858)

    - TokuMX is now built with CMake.  Therefore, the binary tarball names
      have changed slightly, though they will unpack to a similar location
      to previous tarballs.  (#869)

    - When a document-level lock request times out (the `lockTimeout`
      parameter), some debugging information is printed to the server's
      log, including a transaction identifier for the operation which was
      holding the lock that the current operation failed to acquire.  Now,
      that blocking transaction's information (the same information
      printed in `db.currentOp()` is also printed to the log, which should
      help in debugging lock timeouts in applications.  (#876)

    - Improved the latency of short range query operations.  (#882)

    - Improved the performance of workloads with high concurrency.  (#904)

    - Added information about the following to `db.serverStatus()`:

      * Cache misses of the "partial fetch" type.
      * Cache evictions.
      * Number of pending document-level lock requests.

      See the user's guide for a full description.  (#905, #906, #907)

    - Added a server parameter for `mongod`, `migrateUniqueChecks`.  If
      this setting is on, then during a migration, TokuMX will perform
      unique checks for all data in the chunk being migrated.  That is the
      default.

      These unique checks exist to prevent users from inserting different
      documents that initially are inserted to different shards, but which
      have the same `_id`, from causing issues if those documents are
      migrated to the same server.

      With this setting turned on, such a migration would fail, but the
      documents would be preserved.

      If this setting is turned off, such a migration would succeed, and
      silently overwrite the document that was there before, just in the
      `_id` index.  This would effectively overwrite that document, and
      for any secondary indexes, the newly migrated document would appear
      twice.  This can only happen when using an `_id` index as the
      primary key (the old behavior, see above), but when also using a
      shard key other than the `_id`, and simultaneously generating
      inappropriate (non-unique) values for the `_id` in the application.

      For the best performance, use the default behavior of
      `shardCollection` which will create a primary key containing the
      shard key, allow TokuMX to generate `_id` values on your
      application's behalf (use a different field for an application-level
      ID if needed), and turn _off_ `migrateUniqueChecks`.  If these
      recommendations are followed, this setting is safe and will not lead
      to corruption or data loss.  However, to be careful, the default is
      to do the unique checks, which is slower.  (#941)


Bug fixes
~~~~~~~~~

    - The `fsync` command, which has always been useless in TokuMX (its
      functionality is available with other commands: `getLastError` with
      `j` or `fsync` set to 1 flushes the transaction log, and
      `checkpoint` forces data pages to be written), has finally been
      deprecated, and is now completely a no-op.  (#180)

    - Updates with `upsert` can no longer allow bad values for the `_id`
      field (`Array`, `RegEx`, or `undefined`).  (#736)

    - Inserts into capped collections within a multi-statement transaction
      have been disabled, because they could not properly maintain the
      size of the collection.  (#852)

    - Replication initial sync no longer fails when a profile collection
      exists.  (#862)

    - All relevant bugfixes from MongoDB 2.4.9 have been incorporated.
      (#868)

      This includes:

      * http://jira.mongodb.org/browse/SERVER-5625[SERVER-5625]
      * http://jira.mongodb.org/browse/SERVER-7246[SERVER-7246]
      * http://jira.mongodb.org/browse/SERVER-9238[SERVER-9238]
      * http://jira.mongodb.org/browse/SERVER-10538[SERVER-10538]
      * http://jira.mongodb.org/browse/SERVER-10919[SERVER-10919]
      * http://jira.mongodb.org/browse/SERVER-11099[SERVER-11099]
      * http://jira.mongodb.org/browse/SERVER-11194[SERVER-11194]
      * http://jira.mongodb.org/browse/SERVER-11502[SERVER-11502]
      * http://jira.mongodb.org/browse/SERVER-11731[SERVER-11731]
      * http://jira.mongodb.org/browse/SERVER-11971[SERVER-11971]
      * http://jira.mongodb.org/browse/SERVER-11977[SERVER-11977]
      * http://jira.mongodb.org/browse/SERVER-12041[SERVER-12041]
      * http://jira.mongodb.org/browse/SERVER-12092[SERVER-12092]
      * http://jira.mongodb.org/browse/SERVER-12094[SERVER-12094]
      * http://jira.mongodb.org/browse/SERVER-12146[SERVER-12146]

    - An index can no longer become a "multi-key" index inside a
      multi-statement transaction.  An index becomes multi-key when a
      document is inserted such that the key generated for that index
      contains an array in the document, and therefore generates multiple
      keys (this is useful information for the query optimizer), but this
      only happens when the first such document is inserted.  This
      transition cannot be safely maintained if done in a multi-statement
      transaction, so it will throw an error if done.

      If you intend to use documents with array-valued key fields, as well
      as multi-statement transactions, you may wish to prepare the
      collection by inserting one such document so that the index becomes
      multi-key, then removing that document, all outside of a
      multi-statement transaction.  (#872)

    - Fixed a bug where a dropped collection could leave behind a piece of
      zombie metadata that would prevent future collections from being
      created with the same name.  If this has happened already, upgrading
      to 1.4 will clean out any such zombie metadata.  (#879, #937)

    - Fixed two bugs related to setting the size of a capped collection:

      * When setting a capped collection size to an integer larger than
        2GB, no longer reject all insertions until the server is
        restarted.
      * When setting a capped collection size to a string to be
        interpreted with a SI suffix (K, M, G), no longer reject all
        insertions after the server is restarted.

      (#934)


tokumx 1.3.3
------------
Release date: 2013-12-16

General
~~~~~~~

    - This release is focused on bug fixes since 1.3.2.


New features and improvements
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

    - Added an optimization to the execution of aggregation pipelines that
      greatly speeds up all range queries done for aggregations.  (#802)

    - The `mongo2toku` migration tool supports authentication with a
      MongoDB replica set using new options `--ruser`, `--rpass`,
      `--rauthenticationDatabase`, and `--rauthenticationMechanism`.
      (#836, #849)

    - The `mongo2toku` migration tool now prints OpTimes in human-readable
      form.  (#839)


Bug fixes
~~~~~~~~~

    - In TokuMX 1.3.2 and earlier, part of the oplog trimming process (an
      optimization step) tended to hold a read lock for long periods of
      time.  Now in 1.3.3, this component works in 4-second periods, which
      is configurable with the parameter `optimizeOplogQuantum`, which can
      be set dynamically with `setParameter`.  (#777)

      Valid values are:

      * `N > 0`: Work in periods of `N` seconds.

      * `N = 0`: Do not optimize the oplog at all.

      * `N < 0`: The 1.3.2 behavior, optimize aggressively without
        yielding.

    - Removed an unnecessary step in a replica set secondary's startup
      process that had the potential to cause long stalls and pileups.
      (#806)

    - Certain query operations sometimes spend a long time in the storage
      layer, searching through entries that were deleted but not garbage
      collected.  Added a mechanism by which such operations can be killed
      like normal operations.  (#813)

    - Fixed a memory allocation size issue that could cause crashes when
      dealing with very large leaf entries.  (#822)

    - Cursors no longer automatically timeout after 10 minutes regardless
      of continued activity, now continuing to use a cursor will preserve
      it indefinitely.  (#826)

    - Fixed some miscellaneous bugs in `mongo2toku`.  (#834, #837, #838)


tokumx 1.3.2
------------
Release date: 2013-11-21

General
~~~~~~~

    - This release is focused on bug fixes since 1.3.1.


New features and improvements
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

    - Added `tokumxstat.py`, a script for polling `db.engineStatus()`.
      (#534, #535)

    - Improved the error messages reported in the server log when
      filesystem errors occur.  (#703, #715)

    - Improved the concurrency of sequential inserts into a unique index.
      (#705)

    - Improved the concurrency of `justOne` remove operations.  (#721)


Bug fixes
~~~~~~~~~

    - Removed an overactive assert that could prevent a server from
      starting if recovery was needed.  (#650)

    - Fixed counting of oplog insert statistics on secondaries.
      Previously, operations were double-counted on secondaries.  (#689)

    - Fixed the upgrade of `system.users` collections to work when the
      server is started with `--auth`.  (#690)

    - Changed required authorization of transaction and loader commands so
      that they are available to normal users when running with `--auth`.
      (#698)

    - Fixed the semantics of `getLastError` with `j:1` or `fsync:1`.
      (#709)

    - Fixed the rollback of very large transactions after a replication
      failover.  (#716, #720)

    - Fixed a problem when dropping an index concurrently with a
      background index build.  The server now correctly serializes those
      operations.  (#745)


tokumx 1.3.1
------------
Release date: 2013-10-25

General
~~~~~~~

    - This release is focused on bug fixes since 1.3.0.

    - Please take careful note of the changed default of expireOplogDays.


New features and improvements
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

    - TokuMX now reports progress related to the purging of expired
      entries from the oplog.  (#645)

    - The default value of expireOplogDays has been changed to 14 (entries
      expire after 2 weeks), from 0 (no expiration or purging occurs).

      Therefore, if you haven't set expireOplogDays on the command line or
      in a config file and you've been running a TokuMX replica set for
      more than 14 days, when you upgrade, you may see a lot of oplog
      purging activity initially as it adjusts to the new default.  If you
      have set expireOplogDays, nothing will change when you upgrade.

      See the user's guide for a description of expireOplogDays and
      guidelines for how to choose a good value.

    - Optimized deletes that specify just the `_id` field.  (#669)

    - TokuMX now warns you if you have `cacheSize` set to <1GB, as
      performance may suffer.  (#678)


Bug fixes
~~~~~~~~~

    - Incorporated the fix for
      http://jira.mongodb.org/browse/SERVER-9283[SERVER-9283]
      (lastHeartbeatRecv flaps between invalid and valid dates).  (#660)

    - Fixed an authentication error that prevented manual `copydb` and
      `cloneCollection` commands from succeeding.  (#666)

    - Fixed a problem where setting `lockTimeout` with `setParameter`
      would succeed but claim that it had failed.  It now succeeds and
      reports success.  (#667)

    - Fixed an isolation bug with multi-statement transactions.  (#668)

    - Fixed the upgrading of the `system.users` collection to the MongoDB
      v2.4 format.  Also, version 1.3.1 repairs damage that was possible
      to do to the `system.users` collection in rare cases with version
      1.3.0.  (#673, #674)

    - Fixed a bug that prevented serializable transactions from starting
      on the primary of a replica set.  (#682)


tokumx 1.3.0
------------
Release date: 2013-10-18

General
~~~~~~~

    - This release brings TokuMX full compatibility with MongoDB v2.4,
      except for full text search and geo indexes (2d and 2dsphere).

      For a full set of changes, confer with https://jira.mongodb.org and
      MongoDB's marketing materials.  Some major features are called out
      below in addition to TokuMX-specific features and fixes.


New features and improvements
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

    - Switched the default Javascript interpreter to V8.  (#161, #497,
      #517)

    - Added the `$setOnInsert` update operator.  (#279)

    - Many numeric `"byte quantity"` parameters can now be specified with
      units, for example `"1MB"` or `"32k"` or `"6T"`.  The allowed
      suffixes are `"kmgt"`, either upper or lower case, and optionally
      also with a `"b"` or `"B"`.  All suffixes use binary quantities, not
      decimal, so `"1MB"` is `1 * 1024 * 1024`, not `1000000`.  The
      command line parameters that accept these quantities are
      `cacheSize`, `locktreeMaxMemory`, and `txnMemLimit`.  These
      parameters, if passed in a BSON object, must be passed as strings,
      and are accepted as parameters to `pageSize`, `readPageSize`, and
      the `size` for capped collections, as well as the argument to the
      `backupThrottle` command in the enterprise Hot Backup plugin.
      (#266, 536, #545)

    - Added mongo shell wrapper functions `db.showLiveTransactions()` and
      `db.showPendingLockRequests()`.  (#437, #526)

    - Merged improvements to the aggregation framework in v2.4, including
      performance optimizations and some behavioral changes.  (#516, #517)

    - Merged authorization system changes (fine-grained access control) in
      v2.4.  (#527, #549)

    - Re-enabled an optimization for updates using the `_id` index.
      (#538)

    - Added a parameter `--loaderMaxMemory` which controls the amount of
      memory used by the bulk loader, which is used by mongorestore.  The
      default value is 100MB.  (#571)

    - The `lockTimeout` parameter can now be set by the `setParameter`
      command.  (#574)

    - Merged changes to the `db.serverStatus()` command, and added several
      TokuMX-specific fields.  See the user's guide for more details.
      (#591, #626, #627)

    - Optimized generation of secondary keys, which impacts workloads with
      a large number of secondary indexes and/or indexes on array-valued
      fields.  (#600, #618)

    - Skip generation of secondary keys for indexes that are not affected
      by an update.  This has a significant positive impact on the
      performance of workloads with indexes on array-valued fields that
      don't get changed by update statements.  (#601, #617)


Bug fixes
~~~~~~~~~

    - Fixed display of date/time information in `db.engineStatus()`
      output.  (#533)

    - Avoid building unique indexes in the background.  Doing this is not
      actually an error, but if another client violates the unique
      constraint for an index that isn't finished building, the index
      build will waste a lot of time building the index only to roll it
      back later.  Unique indexes will always be built in the foreground.
      (#537)

    - Fixed command line parsing of the `--lockTimeout` option.  (#541)

    - Allow `db.dropDatabase()` when profiling is on.  (#546)

    - Fixed an issue where deletes and updates in multi-statement
      transactions can cause indexes to get out of sync.  (#562)

    - Fixed a possible deadlock caused by trying to open a collection in
      certain contexts, like `db.eval()`.  (#573)

    - Fixed a possible crash after aborting a bulk load.  (#589)

    - Prevented `mapReduce` operations from using too much server memory.
      (#593)

    - Fixed an issue where splitting a chunk may fail if many documents
      have been deleted from the chunk since it was created.  (#602)

    - Fixed a bug that causes an extra empty chunk to be generated when
      the `force` option is used with the `splitVector` command.  (#604)

    - The date fields in `db.engineStatus()` are now stored as true BSON
      Date objects, rather than formatted strings.  (#630)

    - Fixed a locking issue with the `replSetExpireOplog` command.  (#642)

    - Fixed an issue where the initial splitting of an empty collection
      for a `shardCollection` command with a hashed shard key would fail
      on clusters of 32 or more shards.  (#649)

    - Fixed the logging of "lock not granted" errors that happen on the
      ydb directory.  (#652)


tokumx 1.2.1
------------
Release date: 2013-09-18

General
~~~~~~~

    - This release adds new diagnostic and performance analysis tools, and
      bug fixes since 1.2.0.


New features and improvements
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

    - Added index usage stats.  The `db.coll.stats()` command now shows
      cumulative statistics for how each index has been used since the
      last reboot.  See the user's guide for more details.  (#285, #286,
      #501)

    - Added document-level lock debugging information.  With new commands
      `showLiveTransactions` and `showPendingLockRequests`, you can show
      the state of all document-level locks currently held in TokuMX,
      which helps diagnose why a workload might get "lock not granted" or
      "deadlock" errors.  In addition, when an operation fails for one of
      these reasons, some of this information is immediately printed to
      the error log.  See the user's guide for more details.  (#297, #437,
      #490)

    - Added mongo shell wrapper functions `db.beginTransaction()`,
      `db.commitTransaction()`, `db.rollbackTransaction()`, and
      `db.engineStatus()`.  (#488)

    - Added to `mongo2toku` persistent save/resume functionality.  When
      the tool exits, it will write the timestamp to a file, and resume
      from that file, rather than just by printing out the argument to use
      for the next invocation.  (#495)

    - Increased the default `numInitialChunks` for sharding empty
      collections on a hashed key.  This should reduce the number of
      migrations experienced during an initial load or at the beginning of
      a workload.  (#513)


Bug fixes
~~~~~~~~~

    - Fixed a deadlock that occurred when multiple clients tried to create
      the same collection concurrently.  (#477)

    - Improved the reliability and usability of mongorestore when using
      the bulk loader.  (#492, #512, #514, #515)

    - Fixed a race condition that occurred when multiple clients tried to
      open the same database concurrently.  (#499)

    - Reduced lock contention for profiling and diagnostics logging.
      (#505, #506, #508)

    - Includes a fix for a rare memory allocation error exposed by large
      keys in the fractal tree library.  (Tokutek/ft-index#70)


tokumx 1.2.0
------------
Release date: 2013-09-05

General
~~~~~~~

    - This release adds a hot backup tool as an enterprise component.


New features and improvements
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

    - The enterprise version of TokuMX now comes with a hot backup plugin
      that offers fast, non-blocking backup of a live running system.  See
      the user's guide for more details.


Bug fixes
~~~~~~~~~

    - Removed unnecessary unique checks done for updates and deletes.
      (#483)

    - Prevent overlocking on findAndModify and point update/delete
      operations.  (#485)


tokumx 1.1.2
------------
Release date: 2013-08-29

General
~~~~~~~

    - This release is focused on bug fixes since 1.1.1.


New features and improvements
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

    - The maximum amount of memory that will be used by the document-level
      locking data structures is now configurable with the parameter
      `locktreeMaxMemory=<bytes>`, which defaults to 10% of
      `cacheSize`. (#340)

    - It is now possible to get more debugging information in the log
      during a crash.  To use this, download and unpack the `-debuginfo`
      tarball along with the binary tarball, and run with
      `--gdbPath=/path/to/gdb`.


Bug fixes
~~~~~~~~~

    - Fixed a bug in background indexing and the bulk loader that caused
      indexes to be built as multi-key indexes, which causes the query
      planner to create non-optimal query plans. (#469)

    - Optimized deletes, especially those performed on the oplog. (#471)


tokumx 1.1.1
------------
Release date: 2013-08-23

General
~~~~~~~

    - This release fixes a memory leak introduced in 1.1.0


Bug fixes
~~~~~~~~~

    - Fixed a memory leak in insert/update/delete workloads.  #465


tokumx 1.1.0
------------
Release date: 2013-08-21

General
~~~~~~~

    - This release introduces background indexing support and a fast bulk
      loader for `mongoimport` and `mongorestore`.


New features and improvements
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

    - Background indexing is now supported. (#9, #372)

    - A bulk load API has been introduced (`beginLoad`, `commitLoad`, and
      `abortLoad` commands) that increases the throughput of initial bulk
      collection loading.  This API is used automatically by `mongoimport`
      and `mongorestore` when loading into a non-existent collection or
      when using the `--drop` option, and is documented in the User's
      Guide. (#287, #288, #313, #371, #386, #387)

    - Reduced the recovery logging overhead of insert/update/delete
      workloads. (#10, #370)

    - Improved the reliability of index builds. (#290, #295, #296)

    - Improved the concurrency of certain highly multi-threaded
      modification workloads. (#342, #343, #358, #432)

    - Optimized the thread that purges old entries from the oplog. (#399,
      #435, #447)

    - Improved the reliability of distributed locks used in
      sharding. (#451)


tokumx 1.0.4
------------
Release date: 2013-08-14

General
~~~~~~~

    - This release is focused on bug fixes since 1.0.3.

    - This release fixes a critical bug in replication present in all
      versions up to and including 1.0.3.  See below for details.


New features and improvements
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

    - Added a thread to do periodic optimization of the oplog, which
      should clear space from it more aggressively. (#399)

    - Added new commands `replGetExpireOplog` and `replSetExpireOplog` for
      dynamically controlling the size of the oplog. (#415)


Bug fixes
~~~~~~~~~

    - Backported https://jira.mongodb.org/browse/SERVER-1929[SERVER-1929]
      from mongodb 2.4 to prevent replica set failover thrashing. (#344)

    - Prevented the oplog purging thread from completely emptying the
      oplog during long idle periods. (#345)

    - Backported https://jira.mongodb.org/browse/SERVER-7951[SERVER-7951]
      from mongodb 2.4 to allow replace-style updates when using hashed
      shard keys. (#347)

    - Improved the concurrency model used by authentication.  This also
      fixes a deadlock that can occur most often when running with
      authentication but without any admin users declared. (#366, #375,
      #401)

    - Resolved a crash when using the web UI on a server running with
      authentication. (#378)

    - Data cleanup after a successful chunk migration no longer happens on
      a background thread.  This fixes some performance issues triggered
      by a large number of migrations piling up. (#379, #393)

    - Improved reliability of the distributed locking used by
      sharding. (#413)

    - Fixed a race condition that could result in data loss on
      secondaries, under multi-threaded workloads.  This bug can produce a
      secondary that appears to be fully synced but is missing some data,
      and if this secondary is ever elected primary, could result in the
      permanent loss of that data.  All servers should immediately be
      upgraded to 1.0.4 or greater, all secondaries should be re-synced
      from an empty state, and any possible internal consistency
      invariants should be checked. (#433)

    - Fixed a bug that prevented the oplog from getting trimmed properly
      in certain cases. (#435)

    - Backported fixes for
      https://jira.mongodb.org/browse/SERVER-10458[SERVER-10458],
      https://jira.mongodb.org/browse/SERVER-10478[SERVER-10478], and
      https://jira.mongodb.org/browse/SERVER-10498[SERVER-10498] from
      upstream.  This brings tokumx current with mongodb 2.2.6. (#436)


tokumx 1.0.3
------------
Release date: 2013-07-23

General
~~~~~~~

    - This release adds a tool for performing online data migration from
      vanilla MongoDB.


New features and improvements
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

    - Added the `mongo2toku` tool that reads and replays the vanilla
      MongoDB replication format, which allows TokuMX to sync from vanilla
      MongoDB. (#320)

    - Added a storage layer optimization that improves the processor
      caching behavior and performance of point queries and especially
      non-covering secondary range queries. (#312)

    - Improved the concurrency behavior of the storage layer when, for a
      large transaction, the commit or abort itself is expensive. (#334)


Bug fixes
~~~~~~~~~

    - Fixed a few race conditions and strange behaviors related to
      replication. (#322, #325, #328, #329)

    - Fixed a resource leak that could occur if the creation of an index
      fails for any reason. (#331)


tokumx 1.0.2
------------
Release date: 2013-07-11

General
~~~~~~~

    - This release is focused on bug fixes since 1.0.1.


New features and improvements
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

    - Backported SERVER-1752 to improve the performance of some count()
      queries by a factor of 2. (#300)


Bug fixes
~~~~~~~~~

    - Fixed a bug introduced in tokumx 1.0.1 where lock timeouts and
      deadlocks caused the connection to die unnecessarily. (#293, #301)

    - Properly handle errors that arise during operation of the bulk
      loader. (#295, #296)


tokumx 1.0.1
------------
Release date: 2013-06-26

New features and improvements
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

    - The default compression method is now zlib. (#189)


Bug fixes
~~~~~~~~~

    - Fixed a bug causing concurrent insertions on different keys in
      certain (typically nearly-sequential insertion order) scenarios to
      block.  This should improve the performance of workloads with
      multiple writers.  (#273)

    - The code that takes distributed locks in a cluster now correctly
      handles row lock error codes.  This eliminates what were previously
      rare crashes.  (#249)


tokumx 1.0.0
------------
Release date: 2013-06-19

General
~~~~~~~

    - This release is focused on bug fixes since 1.0.0-rc.0.

    - The file names used by TokuMX have been changed, so upgrading from a
      prior version (even 1.0.0-rc.0) is not possible.  Please do a dump
      and restore to upgrade.


New features and improvements
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

    - The parameters --logDir and --tmpDir now allow finer control of
      where TokuMX puts its files.  See the users' guide for details.

    - Building an index on a non-empty collection now uses a faster bulk
      loader to do the initial index build.


Bug fixes
~~~~~~~~~

    - Large transactions no longer fail during replication or chunk
      migrations. (#93, #100)

    - Fixed a deadlock that could occur during rapid chunk
      migrations. (#115)

    - Fixed a few miscellaneous bugs in replication and sharding. (#232,
      #228, #225)

    - Properly ignore the $atomic flag for queries. (#230)

    - The bug causing large multi-field $in queries to run slowly has been
      fixed. (#212)

    - Calls to db.eval() are now properly profiled. (#16)


Miscellaneous
~~~~~~~~~~~~~

    - The mongoperf tool has been deprecated. (#174)


tokumx 1.0.0-rc.0
-----------------
Release date: 2013-06-03

General
~~~~~~~

    - The major developments in this release are replication and sharding.


New features and improvements
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

    - Replication and sharding are now supported for evaluation.

    - Sharding on a hashed key is supported (backported from 2.4).

    - Sharding creates clustering indexes by default, which is a
      significant performance advantage in most cases.  If you need to
      shard on a non-clustering index, you need to specify that in
      sh.shardCollection(), for example

          sh.shardCollection("test.foo",
                             {x: 1},
                             false, // unique
                             false  // clustering
                             );

      - The JS client driver now supports the clustering flag in
        sh.shardCollection, which should only be used if you want to
        specify that you want to shard on a NON-clustering index (set it
        to false).  See above.

    - "DDL" changes (create, drop, ensureIndex, dropIndexes, rename) are
      now supported inside multi-statement transactions, and are properly
      rolled back on rollbackTransaction.

    - The C++ client driver now supports clustering indexes.  This changes
      the function signature for DBClientWithCommands::ensureIndex, please
      check your applications in light of this API change.

    - The C++ client driver has a convenience class for multi-statement
      transactions: RemoteTransaction.  Please check its documentation.

    - The local.oplog.rs collection is no longer capped, and instead can
      be trimmed to keep N days' worth of data.  Use the command line flag
      --expireOplogDays to configure this number of days (by default,
      nothing is trimmed).


Limitations
~~~~~~~~~~~

    - This version of TokuMX targets compatibility with MongoDB 2.2.4.
      Some features and bugfixes have been backported from the 2.4 series,
      but most have not.

    - The format of the local.oplog.rs collection has changed and will
      break any applications that were relying on it.  Also, it is no
      longer capped (see above).

    - On a replica set, large transactions can fail if a transaction is
      too large (due to a limit on the size of a BSON object).  This
      should not be a worry for most existing applications, but some
      commands (copydb, clone, cloneCollection), as well as updates that
      affect a large number of documents (and specify {multi: true}).
      This will be fixed soon, but for now, avoid such operations if you
      are using replication.

    - When performing an initial sync on a secondary, if the primary drops
      any collection, the initial sync may fail.  Try to avoid dropping
      collections during initial syncs of secondaries.

    - For the fastsync option, when starting a secondary, to work, the
      machine must have some data in the oplog associated with the state
      of the machine. If the oplog is empty, then a full initial sync is
      performed.

    - mongodump --oplog and mongorestore --oplogReplay are not supported
      and cannot be used to create a secondary for a replica set.

    - The commands aggregate and mapReduce may have poor concurrency with
      some workloads.  This will be fixed soon, and these commands should
      be able to be run concurrently with each other and with other
      operations.

    - Multi-statement transactions are not supported in a sharded cluster,
      and will be rejected by mongos.

    - $in queries may be slower than expected on a sharded cluster.

    - Calls to db.eval() are not properly profiled when profiling is
      enabled.  The cause for this is not understood.

    - Geospatial and Full Text Search Indexes are not supported (the '2d'
      and 'text' index types).

    - The reIndex command no longer has the same effect it used to.  In
      particular, one effect in vanilla MongoDB is that it recalculates
      whether an index is "multi-key" (which affects query planning and
      some other things), and this side effect no longer exists.  To get
      the old reIndex functionality, just drop and re-create the index
      manually.

    - Only foreground indexing is supported. Background indexing is not.
      We will add background indexing soon.

    - The mongorestore tool only works properly if it restores into a
      running mongod.  Don't try to restore into a fresh data directory,
      instead, start a mongod pointing to that directory, and then use
      mongorestore.

    - Capped collection insert/update performance may be poor compared
      to regular MongoDB if there are few or no secondary indexes.
      Additionally, the constant-size-on-disk property held by regular
      MongoDB is only loosely held by TokuMX, especially because
      collections are compressed. In generaly, users are encouraged
      to use regular collections instead of capped collections when
      possible, especially when disk size properties were a motivating
      factor. Time-to-live (TTL) collections may be a good alternative.

    - listDatabases command and "show dbs" do not show database statistics

tokumx 0.1.0
------------
Release date: 2013-04-15

General
~~~~~~~

    - This release highlights single-threaded and multi-threaded
      performance improvements.


New features and improvements
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

    - Concurrency: inserts, updates, and deletes into existing collections
      are now concurrent operations, with each other and with read
      operations. (trac #6019)

    - Multi-statement transactions: clients may now issue
      `beginTransaction`, `commitTransaction`, and `rollbackTransaction`
      commands using the `runCommand` interface, to bundle multiple
      individual statements together in the same system transaction.

    - The key format used in vanilla MongoDB's `{v:1}` indexes is now used
      in the Fractal Tree storage system.  This yields significant
      performance gains, particularly for in-memory workloads.

    - Capped collections and tailable cursors are now supported for
      experimental use only.

    - The `renameCollection` command is now supported.

    - The system allocator has been changed fully to jemalloc.


tokumx 0.0.4
------------
Release date: 2013-04-08

New features and improvements
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

    - Server crashes now produce a core file. (trac #6458)

    - Debug info is available in a separate package. (trac #6453)


Bug Fixes
~~~~~~~~~

    - The gcc and stdc++ runtime libraries are now packaged with the
      binaries to fix runtime issues on older Centos 5 machines. (trac
      #6456)


tokumx 0.0.2
------------
Release date: 2013-03-14

General
~~~~~~~

    - First evaluation release.
