## Search Quality Benchmarking.

This package allows to benchmark search quality of a Lucene application.

In order to use this package you should provide:

- A searcher.
- Quality queries.
- Judging object.
- Reporting object.

For benchmarking TREC collections with TREC QRels, take a look at the trec package.

Here is a sample code used to run the TREC 2006 queries 701-850 on the .Gov2 collection:

        File topicsFile = new File("topics-701-850.txt");
        File qrelsFile = new File("qrels-701-850.txt");
        Searcher searcher = new IndexSearcher("index");

        int maxResults = 1000;
        String docNameField = "docname";

        PrintWriter logger = new PrintWriter(System.out,true);

        // use trec utilities to read trec topics into quality queries
        TrecTopicsReader qReader = new TrecTopicsReader();
        QualityQuery qqs[] = qReader.readQueries(new BufferedReader(new FileReader(topicsFile)));

        // prepare judge, with trec utilities that read from a QRels file
        Judge judge = new TrecJudge(new BufferedReader(new FileReader(qrelsFile)));

        // validate topics & judgments match each other
        judge.validateData(qqs, logger);

        // set the parsing of quality queries into Lucene queries.
        QualityQueryParser qqParser = new SimpleQQParser("title", "body");

        // run the benchmark
        QualityBenchmark qrun = new QualityBenchmark(qqs, qqParser, searcher, docNameField);
        SubmissionReport submitLog = null;
        QualityStats stats[] = qrun.execute(maxResults, judge, submitLog, logger);

        // print an avarage sum of the results
        QualityStats avg = QualityStats.average(stats);
        avg.log("SUMMARY",2,logger, "  ");

Some immediate ways to modify this program to your needs are:

- To run on different formats of queries and judgements provide your own Judge and Quality queries.
- Create sophisticated Lucene queries by supplying a different Quality query parser.