backport-util-concurrent

Backport of JSR 166 (java.util.concurrent) to Java 1.2, 1.3, 1.4, and ... 5.0, and 6.0

http://dcl.mathcs.emory.edu/util/backport-util-concurrent/

Release: 3.1, for Java 6.0


Overview

This package is the backport of java.util.concurrent API, introduced in Java 5.0 and further refined in Java 6.0, to older Java platforms. The backport is based on public-domain sources from the JSR 166 CVS repository, the dl.util.concurrent package, and the Doug Lea's collections package.

The ambition of this project is to provide a concurrency library that works with uncompromised performance on all Java platforms currently in use, allowing development of fully portable concurrent applications. OK, let's get real - the traget scope is Java 1.3 and above, and some limited 1.2.

The backport is close to complete; unsupported functionality is limited to: 1) classes that rely on explicit JVM support and cannot be emulated at a satisfactory performance level on platforms before 5.0, 2) some of the functions described in the original javadoc as "designed primarily for use in monitoring in system state, not for synchronization control".

The ultimate purpose of the backport is to enable gradual migration to java.util.concurrent. In a perfect world, everybody would instantly and simultaneously upgrade to Java 6.0 once it comes out. In reality, many people are, for various reasons, still stuck to older platforms. The backport allows them to use the JSR 166 goodies without upgrading just yet. And once they are ready to switch, their concurrency code will require only very minor modifications, such as changing the import statements.

Note: Retrotranslator in an interesting alternative to using the backport directly.

Features

The following functionality of java.util.concurrent is supported in the backport:

License

This software is released to the public domain, in the spirit of the original code written by Doug Lea. The code can be used for any purpose, modified, and redistributed without acknowledgment. No warranty is provided, either express or implied.

JSR 166 functionality is tied closely to the Java 5.0+ Virtual Machine, and some aspects of it cannot be fully backported to older platforms. This section discusses these differences between the backport and JSR 166.

Package Names

Since user libraries cannot define classes in java.* packages, all the original package names have been prefixed with edu.emory.mathcs.backport. For instance, java.util.concurrent.Future becomes edu.emory.mathcs.backport.java.util.concurrent.Future.

Differences between releases

The backport is based on latest JSR 166 source code and thus includes functionality that has been added in Java 6.0. Pay attention to the "since" javadoc tags if a strict conformance with Java 5.0 is desired. Examples of "since 1.6" functionality include: deques, navigable maps and sets (including ConcurrentSkipList[Map,Set]), "newTaskFor" in AbstractExecutorService, "lazySet" in atomics, RunnableFuture and RunnableScheduledFuture, "allowCoreThreadTimeout" in ThreadPoolExecutor, "decorateTask" in ScheduledThreadPoolExecutor, MINUTES, HOURS, and DAYS in TimeUnit, and appropriate retrofits in collection classes.

Backport is developed carefully to retain link-time compatibility, i.e. it is generally safe to replace an old library JAR with a new one (with a possible exception of APIs based on beta releases, e.g. current "since 1.6" classes and methods). Serial compatibility (i.e. ability of one version to deserialize objects that has been serialized using a different version) is maintained on a best-effort basis, and not always guaranteed. Please see details below. (Note that concurrency tools are usually not intended for persistent storage anyway). Compile-time compatibility: applications using wildcard imports (e.g. java.util.* and edu.emory.mathcs.backport.java.util.*) may cease to compile with updated backport versions (containing new classes) due to import ambiguities. In such cases, you must dis-ambiguate imports (i.e. use explicit imports as appropriate) and recompile.

Notes for version 3.0-3.1: Link-time and serial compatibility is fully preserved.

Notes for version 2.2: Link-time and serial compatibility is fully preserved for "since 1.5" APIs. For "since 1.6" APIs, link-time and serial compatibility is preserved except for navigable maps and sets, which API has recently changed slightly in Java 6.0 beta.

Notes for version 2.1: Link-time compatibility is preserved fully. Serial compatibility is preserved except for the class ReentrantReadWriteLock.

Notes for version 2.0:

Differences between versions for Java 1.2 - 1.3, 1.4, 5.0, and 6.0

Within the same release, backport versions optimized for Java 1.2 - 1.3, 1.4, 5.0, and 6.0, are source-level and link-time compatible. That is, (1) sources compiled using one version will compile using another version, and (2) classes compiled with one version can be used with another version without recompiling. However, they are not serially compatible - that is, objects serialized using one version cannot be deserialized using another version.

Unsupported functionality

Detailed listing of functionality that has not been backported is presented below.

Java 5.0 Syntax

Package java.util.concurrent exploits new language features introduced in Java 5.0. In particular, most API classes are generic types. In the backport, they have been flattened to standard, non-generic classes. Still, programs linked against the backport should compile with Java 5.0 (after changing package names). Nevertheless, you may want to consider gradually switching to using generics once you make the transition to Java 5.0, since it gives better compile-time type checking.

In Condition

Method long awaitNanos(long nanosTimeout) is not supported, since the emulation cannot reliably report remaining times with nanosecond precision. Thus, it probably would be too dangerous to leave the emulated method in the Condition interface. However, the method is still available, for those who know what they are doing, in the util.concurrent.helpers.Utils class.

In ReentrantLock

the following monitoring methods are supported only for fair locks: boolean hasQueuedThreads(), int getQueueLength(), Collection getQueuedThreads(), boolean isQueued(), boolean hasWaiters(Condition), int getWaitQueueLength(Condition), Collection getWaitingThreads(Condition).

In ReentrantReadWriteLock

The backport implementation for Java 1.2 - 1.3 and 1.4 is based on dl.util.concurrent class ReentrantWriterPreferenceReadWriteLock, and thus slightly departs from java.util.concurrent that does not specify acquisition order but allows to enable/disable fairness. The backport implementation does not have a single-parameter constructor allowing to specify fairness policy; it always behaves like writer-preference lock with no fairness guarantees. Because of these characteristics, this class is compliant with JSR 166 specification of non-fair reentrant read-write locks, while the exact semantics of fair locks are not supported (and the appropriate constructor is thus not provided).

Also, the following instrumentation and status methods are not supported: Collection getQueuedWriterThreads(), Collection getQueuedReaderThreads(), boolean hasQueuedThreads(), boolean hasQueuedThread(Thread), Collection getQueuedThreads(), boolean hasWaiters(Condition), int getWaitQueueLength(Condition), Collection getWaitingThreads(Condition).

In Semaphore

Blocking atomic multi-acquires: acquire(int permits) and tryAcquire(int permits, long timeout, TimeUnit unit) are supported only for FAIR semaphores.

Platform-level functionality

To emulate System.nanoTime(), not present on JVM < 5.0, the method nanoTime() is provided in the class dl.util.concurrent.helpers.Utils. On Java 1.4.2 and above, it attempts to use high-precision timer via sun.misc.Perf (thanks to Craig Mattocks for suggesting this). On older Java platforms, or when sun.misc.Perf is not supported, it falls back to System.currentTimeMillis(). In the Java 5.0 and 6.0 version, it delegates to System.nanoTime().

Class ThreadHelpers is provided to emulate certain aspects of Thread.UncaughtExceptionHandler.

Note on nanosecond precision timing

The backport strives to honor nanosecond timeouts, if such are requested, by using two-parameter variant of Object.wait() in Java 1.2 - 1.3 and 1.4 versions. However, most Java platforms will round up the timeout to full milliseconds anyway. In Java 5.0 and 6.0 versions, the backport uses native nanosecond APIs.

Low-level concurrency classes

The following classes are not supported: LockSupport, AbstractQueuedSynchronizer, AbstractQueuedLongSynchronizer.

Rationale: these classes depend on explicit JVM support, delegating to low-level OS concurrency primitives. There seems to be no simple way of emulating them without introducing prohibitive performance overhead.

Atomic utilities

The following "atomic" utilities are not supported: Atomic[Integer,Long,Reference]FieldUpdater.

Rationale: on many platforms, these utilities cannot be emulated without a prohibitive overhead, negating their usefulness.

The backport has been around since December 2004. It has been downloaded tens of thousands of times. It has been successfully used in many open-source and commercial projects. Arguably, it has become a de-facto standard concurrency library for Java 1.4, and by now, it is very well tested by its users.

Moreover, the backport is based on a very stable and robust code base of JSR 166 and dl.util.concurrent. This partially explains why so few bugs have been reported against it. Once again, I would like to express gratitude to the JSR 166 expert group for doing such a great job, and for their commitment to open source.

Unit tests

Version 3.1 of the library passes all the relevant 1859 unit tests from TCK test package designed for java.util.concurrent (the tests of unsupported functionality were skipped).

The following unit tests have been completed (listed in the alphabetical order): AbstractExecutorServiceTest, AbstractQueueTest, ArrayBlockingQueueTest, ArrayDequeTest, AtomicBooleanTest, AtomicIntegerArrayTest, AtomicIntegerTest, AtomicLongArrayTest, AtomicLongTest, AtomicMarkableReferenceTest, AtomicReferenceArrayTest, AtomicReferenceTest, AtomicStampedReferenceTest, ConcurrentHashMapTest, ConcurrentLinkedQueueTest, ConcurrentSkipListMapTest, ConcurrentSkipListSubMapTest, ConcurrentSkipListSetTest, ConcurrentSkipListSubSetTest, CopyOnWriteArrayListTest, CopyOnWriteArraySetTest, CountDownLatchTest, CyclicBarrierTest, DelayQueueTest, EntryTest, ExchangerTest, ExecutorsTest, ExecutorCompletionServiceTest, FutureTaskTest, LinkedBlockingDequeTest, LinkedBlockingQueueTest, LinkedListTest, PriorityBlockingQueueTest, PriorityQueueTest, ReentrantLockTest, ReentrantReadWriteLockTest, ScheduledExecutorTest, ScheduledExecutorSubclassTest, SemaphoreTest, SynchronousQueueTest, SystemTest (testing Utils.nanoTime()), ThreadLocalTest, ThreadPoolExecutorTest, ThreadPoolExecutorSubclassTest, TimeUnitTest, TreeMapTest, TreeSubMapTest, TreeSetTest, TreeSubSetTest.

Stress tests

The backport is being stress-tested using the "loops" tests from JSR 166 (courtesy of Doug Lea and the JSR 166 Expert Group). The tests, included in the development source bundle, thoroughly evaluate behavior and performance of various types of locks, queues, maps, futures, and other API classes, under various conditions (contention etc.) and circumstances, including cancellation.

Despite exhaustive testing, as any software, this library may still contain bugs. If you find one, please report it, or better yet, contribute a fix.

Known problems

Note: A bug has been reported against Sun 1.4.2_04 JVMs, and fixed in 1.4.2_06 (see ID 4917709) that makes those JVMs to occassionally crash with SIGSEGV during backport stress tests, particularly MapLoops and MapChecks. It is therefore recommended to use JVM versions 1.4.2_06 or newer when using the backport (although the crashes seem to not happen also on 1.4.2_03, and perhaps on earlier JVMs). Detected in version: 2.0.

Note: due to what is apparently a bug in SUN JVM implementations for Solaris, observed on 1.4.2_03 and 1.4.2_06, the 'ExecutorsTest.testPrivilegedThreadFactory()' unit test fails with ClassNotFoundException when launched from a class path that has backport classes stored as individual files in the "classes" directory. The problem disappears when the classes are put in a JAR file. The bug is most likely related to handling context class loaders. It is therefore advised to use JAR files instead of class files when running code that explicitly or implicitly modifies context class loaders, as does privileged thread factory. Detected in version: 2.0.

Note: missed signals have been observed on Linux 2.6.3-7 kernel for SMP w/64GB support under contention and in the presence of frequent timeouts. (The bug was captured during TimeoutProducerConsumerLoops on SynchronousQueue). Apparently, this is caused by a kernel bug. The problem have been observed on several different JVMs. It does not occur on newer kernels. Detected in version: 2.0.

As evident from the above, IT IS CRUCIAL THAT YOU RUN THE STRESS TESTS on the target configuration before using the backport in a production environment. Concurrency issues are tricky, and possible bugs in JVMs, operating systems, and this software, usually won't show up until under heavy loads. Stress tests included with this distribution test this software under extreme conditions, so if they are consistently passing, there's a very good chance that everything works fine.

Changelog

Version 3.1 (Jul 5, 2006)
SVN logs: [Java 1.4, Java 1.2 - 1.3, Java 5.0, Java 6.0]

Version 3.0 (Nov 11, 2006)
SVN logs: [Java 1.4, Java 5.0, Java 1.2 - 1.3] Version 2.2 (Jun 4, 2006) [CVS log] Version 2.1 (Jan 28, 2006) [CVS log] Version 2.0_01 (Aug 3, 2005) [CVS log] Version 2.0 (Jul 6, 2005) [CVS log] Version 1.1_01 (Feb 7, 2005) [CVS log] Version 1.1 (Jan 21, 2005) [CVS log] Version 1.0_01 (Dec 28, 2004) [CVS log] Version 1.0 (Dec 1, 2004)

Documentation and Support

For more information:


Distributed Computing Laboratory, Emory University
Google, Inc.