Version 5.4 Release Notes

These are Release Notes for Celera Assembler version 5.4, released July 17th, 2009.

This distribution package provides a stable, tested, documented version of the software. The distribution is usable on most Unix-like platforms, and some platforms have pre-compiled binary distributions ready for installation.

The source code package includes full source code, Makefiles, and scripts. A subset of the kmer package (http://kmer.sourceforge.net/, version r1804), used by some modules of Celera Assembler, is included.

This package was prepared by scientists at the J. Craig Venter Institute (http://www.jcvi.org/) with funding provided by the National Institutes of Health (http://www.nih.gov/).

Full documentation can be found online at http://wgs-assembly.sourceforge.net/.

Contents

    * 1 Citation
    * 2 Compilation and Installation
    * 3 Example
    * 4 Changes
          o 4.1 Features Available Starting in 5.4
          o 4.2 Known Problems
          o 4.3 Improvements
          o 4.4 Bug Fixes
    * 5 Legal

Citation

Please cite Celera Assembler in publications that refer to its algorithm or its output. The standard citation is the original paper [Myers et al. 2000]. More recent papers describe software modifications for human genome assembly [Istrail et al. 2004; Levy et al. 2007], metagenomics assembly [Venter et al. 2004; Rusch et al. 2007], haplotype separation [Levy et al. 2007; Denisov et al. 2008], and Sanger+pyrosequencing hybrid assembly [Goldberg et al. 2006; Miller et al. 2008]. There are links to these papers, and more, in the on-line documentation (http://wgs-assembler.sourceforge.net). The standard citation is:

    * Myers et al. (2000) A Whole-Genome Assembly of Drosophila. Science 287 2196-2204. 

Compilation and Installation

The source code package needs to be compiled and installed before it can be used. The following commands will work on nearly all Unix-like platforms.

% bzip2 -dc wgs-5.4.tar.bz2 | tar -xf -
% cd wgs-5.4
% cd kmer
% sh configure.sh
% gmake install
% cd ../src
% gmake
% cd ..

Binary distributions need only be unpacked.

% bzip2 -dc wgs-5.4-Linux-amd64.tar.bz2 | tar -xf -


Example

Here is one example of how to use Celera Assembler. We download DNA sequence from the NCBI TraceDB hosted by the National Institutes of Health. We request paired-end fragment reads generated by Sanger chemistry on a bacterial genome. We convert the data to Celera Assembler format (a *.frg file). Then we launch the assembler software.

% ftp ftp ftp.ncbi.nih.gov
ftp> cd pub/TraceDB/porphyromonas_gingivalis_w83
[...]
ftp> ls -l
229 Entering Extended Passive Mode (|||50055|)
150 Opening ASCII mode data connection for file list
-r--r--r--   1 ftp      anonymous   803288 Feb 19 17:29 anc.porphyromonas_gingivalis_w83.001.gz
-r--r--r--   1 ftp      anonymous   226576 Feb 19 17:29 clip.porphyromonas_gingivalis_w83.001.gz
-r--r--r--   1 ftp      anonymous 10261474 Feb 19 17:29 fasta.porphyromonas_gingivalis_w83.001.gz
-r--r--r--   1 ftp      anonymous 18039942 Feb 19 17:29 qual.porphyromonas_gingivalis_w83.001.gz
-r--r--r--   1 ftp      anonymous  1143755 Feb 27 08:01 xml.porphyromonas_gingivalis_w83.001.gz
226 Transfer complete.
ftp> bin
ftp> prompt
ftp> mget fasta* qual* xml*
ftp> bye

Convert the NCBI files to files suitable as Celera Assembler input. The last trace-to-frg step takes about a minute. The 'ls' command demonstrates the expected output files.

% perl wgs/Linux-amd64/bin/tracedb-to-frg.pl -xml xml.porphyromonas_gingivalis_w83.001.gz
% perl wgs/Linux-amd64/bin/tracedb-to-frg.pl -lib xml.porphyromonas_gingivalis_w83.001.gz
WARNING: creating bogus distance for library 'T0611'
WARNING: creating bogus distance for library 'T0612'
WARNING: creating bogus distance for library 'T13146'
WARNING: creating bogus distance for library 'T24315'
porphyromonas_gingivalis_w83.001: frags=40039 links=17113 (85.48%)
% perl wgs/Linux-amd64/bin/tracedb-to-frg.pl -frg xml.porphyromonas_gingivalis_w83.001.gz
Found 20020 in tafrg-porphyromonas_gingivalis_w83/porphyromonas_gingivalis_w83.001.frglib.
% ls -l *.frg*
-rw-r--r--  1 bri  bri       550 Jul 15 21:54 porphyromonas_gingivalis_w83.1.lib.frg
-rw-r--r--  1 bri  bri  19119148 Jul 15 21:55 porphyromonas_gingivalis_w83.2.001.frg.bz2
-rw-r--r--  1 bri  bri    704432 Jul 15 21:54 porphyromonas_gingivalis_w83.3.lkg.frg

Run Celera Assembler with runCA.

% time perl wgs/Linux-amd64/bin/runCA -p pging -d testassembly porphyromonas_gingivalis_w83.*
[...lots of status output...]
[...about four minutes later...]
% ls -l testassembly/9-terminator/*fasta
-rw-r--r--  1 bri  bri  2373611 Jul 15 22:07 testassembly/9-terminator/pging.ctg.fasta
-rw-r--r--  1 bri  bri    39177 Jul 15 22:07 testassembly/9-terminator/pging.deg.fasta
-rw-r--r--  1 bri  bri  2392028 Jul 15 22:07 testassembly/9-terminator/pging.scf.fasta
-rw-r--r--  1 bri  bri   445876 Jul 15 22:07 testassembly/9-terminator/pging.singleton.fasta
-rw-r--r--  1 bri  bri  2692602 Jul 15 22:07 testassembly/9-terminator/pging.utg.fasta

Changes
Features Available Starting in 5.4

   1. Closure Reads
   2. Toggle and Run 

Known Problems

These may be fixed in future releases.

   1. 454 reads that are perfect prefixes of other reads are discard by sffToCA. Reads that are near-perfect prefixes of each other, and mated reads that are duplicates are not detected or discarded. Duplicate reads may skew coverage-based repeat classification, while duplicate mate pairs may lead to incorrect scaffolding.
   2. There is no explicit support for the high-coverage induced by XLR runs on small genomes. Coverage such as 80X induces combinations of sequencing errors that confound Celera Assembler. At best this leads to higher reported rates of allelic variation. At worst this leads to a fractured assembly. Sampling from high-coverage reads can yield a better assembly.
   3. There is little support for assembly of data sets with a small ratio of mate pairs to unpaired fragments. This ratio happens when XLR is applied to many fragment libraries but few paired-end libraries. This ratio impedes the assembly of unitigs into contigs and scaffolds. It leads to fewer bases in scaffolds and more bases in degenerate contigs.
   4. There is no support for bar-coded 454 data. Users with bar-coded data may use some other utility to remove the bar code sequence and partition reads by bar code into separate SFF files.
   5. There is no support for data from short-read platforms such as Illumina Solexa or ABI SOLiD.
   6. There is no support for cDNA, exon-enriched DNA, or DNA amplified with bias of any sort. 

Improvements

   1. (runCA) We improved our error reporting. Failures should now report what crashed, where it crashed, and provide information as to what happened.
   2. (sffToCA) Minor optimizations during de-duplication.
   3. (sffToCA) Add option to disable de-duplication.
   4. (sffToCA) Add option to report simple statistics on counts of fragments.
   5. (sffToCA) Simplify the interface. Clear up confusion about "finding clear ranges" and "applying clear ranges", for example, what to do when a N is encountered. The command-line usage has changed.
   6. (gatekeeper) The -L option ("search for 454 paired end linker") is gone. The functionality moved to sffToCA.
   7. (overlapper=mer) Significant performance optimizations.
   8. (consensus) We implemented many improvements to the unitig consensus code. We expect it to produce better alignments and fewer failures. A major overhaul of this module is under way.
   9. (eCR) We improved reliability of gap closure code in the Extend Clear Ranges module. In some cases, we used to close a gap but fail to rebuild the unitig multi-alignment. The code would be forced to back out the change. Fixed. 

Bug Fixes

   1. (portability) Many problems with OS-X and Solaris builds fixed.
   2. (runCA) Naming of overlap output files was limiting assemblies to less than 100 million fragments.
   3. (runCA) Files specified in the specFile would fail to be found if they contained whitespace on the line.
   4. (runCA) Slight improvements in SGE support. Now we always use the "-cwd" option to "qsub".
   5. (runCA) Add option 'shell' for setting the shell to run scripts in. Remove a few places where 'sh' was invoked directly.
   6. (runCA) Add option doChimeraDetection. This can disable the chimera module from Overlap Based Trimming.
   7. (runCA) The "createACE" option was failing if the .frg file was missing. Fixed.
   8. (gatekeeper) The -dumpfrg option was generating invalid LKG messages. Fixed.
   9. (gatekeeper) Errors in large files were being reported with negative line numbers.
  10. (sffToCA) Now checks that a user-supplied linker sequence is a valid DNA sequence. Previously, a linker name (e.g. titanium) that was misspelled on the command line would be interpreted as DNA sequence.
  11. (convert-fasta-to-v2.pl) When the last QV occurred on a line by itself, it was incorrectly converted to a CA-style quality.
  12. (overlapper) Mistakenly using "-9" optimization on gzip. Some genomes with lots of overlaps were spending enormous amounts of time in gzip instead of doing useful work. Now using "gzip -1".
  13. (overmerry) High-count seeds were mistakenly treated as low-count seeds in some cases. This would lead to olap-from-seeds testing spurious seeds and usually not finding an overlap.
  14. (consensus) All placed surrogate reads were being placed in the first instance of each surrogate in a single contig (i.e., if a contig had two instances of the same surrogate, all reads were placed in the first instance). It now places surrogate reads in the correct position.
  15. (buildPosMap) Fix a rare crash which occurred when there are no VAR records in the assembly.
  16. (buildPosMap) Close a few memory leaks in unitig and contig processing.
  17. (QC) Count VAR records in contigs separately from VAR records in degenerates.
  18. (perl) The perl interfaces in AS_PL5 were not compiling.
  19. (tracedb-to-frg.pl) A change at NCBI inserted a comma into library names which violated our input specification. Commas are now silently removed. 

Legal

Copyright 1999-2004 by Applera Corporation. Copyright 2005-2009 by the J. Craig Venter Institute. The Celera Assembler software, also known as the wgs-assembler and CABOG, is open-source and available free of charge subject to the GNU General Public License, version 2. 
