com.google.gwt.benchmarks.client
Class Benchmark

java.lang.Object
  junit.framework.Assert
      junit.framework.TestCase
          com.google.gwt.junit.client.GWTTestCase
              com.google.gwt.benchmarks.client.Benchmark

All Implemented Interfaces:

junit.framework.Test

public abstract class Benchmark
extends GWTTestCase

A type of GWTTestCase which specifically records performance results. Benchmarks have additional functionality above and beyond GWT's JUnit support for standard TestCases.

Reporting

In a single JUnit run, the results of all executed benchmarks are collected and stored in an XML report viewable with the benchmarkViewer.

Permutations

GWT supports test methods that have parameters. GWT will execute each benchmark method multiple times in order to exhaustively test all the possible combinations of parameter values. All of your test method parameters must be annotated with a Range annotation such as RangeField or RangeEnum. For example,

 
 public void testArrayListRemoves(
   @RangeEnum(Position.class) Position where, 
   @RangeField("insertRemoveRange") Integer size) { ... 
 }
 

Timing

• GWT automatically removes jitter from your benchmark methods by running them for a minimum period of time (150ms).
• GWT supports time limits on the maximum duration of each permutation of a benchmark method. With this feature, you can supply very high upper bounds on your ranges (such as Integer.MAX_VALUE), which future-proofs your benchmarks against faster hardware.
• GWT supports Setup and Teardown methods which separate test overhead from the actual work being benchmarked. The timings of these lifecycle methods are excluded from test results.

Notes

Please note that Benchmarks do not currently support asynchronous testing mode. Calling GWTTestCase.delayTestFinish(int) or GWTTestCase.finishTest() will result in an UnsupportedOperationException.

Examples of benchmarking in action

A simple benchmark example

public class AllocBenchmark extends Benchmark {

  private static final int numAllocs = 1000;

  @Override
  public String getModuleName() {
    return "com.google.gwt.examples.Benchmarks";
  }

  /**
   * Allocates java.lang.Object in a for loop 1,000 times.
   *
   * The current version of the compiler lifts the declaration of obj outside
   * of this loop and also does constant folding of numAllocs.
   * Also, this loop allocs the GWT JS mirror for java.lang.Object
   * NOT an empty JS object, for example.
   *
   */
  public void testJavaObjectAlloc() {
    for ( int i = 0; i < numAllocs; ++i ) {
      Object obj = new Object();
    }
  }

  /**
   * Compares GWT mirror allocations of java.lang.Object to an empty JS object.
   */
  public native void testJsniObjectAlloc1() /*-{
    for (var i = 0; i < @com.google.gwt.examples.benchmarks.AllocBenchmark::numAllocs; ++i ) {
      var obj = {}; // An empty JS object alloc
    }
  }-*/;

  /**
   * Like version 1, but also folds the constant being used in the iteration.
   */
  public native void testJsniObjectAlloc2() /*-{
    for (var i = 0; i < 1000; ++i ) {
      var obj = {}; // An empty JS object alloc
    }
  }-*/;

  /**
   * Like version 2, but hoists the variable declaration from the loop.
   */
  public native void testJsniObjectAlloc3() /*-{
    var obj;
    for (var i = 0; i < 1000; ++i ) {
      obj = {}; // An empty JS object alloc
    }
  }-*/;

  /**
   * Like version 3, but counts down (and in a slightly different range).
   */
  public native void testJsniObjectAlloc4() /*-{
    var obj;
    for (var i = 1000; i > 0; --i ) {
      obj = {}; // An empty JS object alloc
    }
  }-*/;
}

An advanced benchmark example

public class ArrayListBenchmark extends Benchmark {

  private static final int PRIME = 3001;

  /**
   * The various positions that data can be inserted into a list.
   */
  protected enum Position {

    BEGIN("at the beginning"), EXPLICIT_END("explicitly at the end"), IMPLICIT_END(
        "implicitly at the end"), VARIED("in varied locations");

    private String label;

    /**
     * Constructor for Position.
     * 
     * @param label a not null label describing this
     *          Position.
     */
    Position(String label) {
      this.label = label;
    }

    /**
     * Returns the textual description for the position.
     * 
     * @return a not null description.
     */
    @Override
    public String toString() {
      return label;
    }
  }

  protected final List explicitPositions = Arrays.asList(
      Position.BEGIN, Position.EXPLICIT_END, Position.VARIED);

  protected final IntRange insertRemoveRange = new IntRange(64,
      Integer.MAX_VALUE, Operator.MULTIPLY, 2);

  protected final IntRange baseRange = new IntRange(512, Integer.MAX_VALUE,
      Operator.MULTIPLY, 2);

  List list;

  int index = 0;

  @Override
  public String getModuleName() {
    return "com.google.gwt.emultest.EmulSuite";
  }

  /**
   * Appends size items to an empty {@code List}.
   * 
   * @param size the size of the {@code List}
   */
  @Setup("beginListAdds")
  public void testListAdds(@RangeField("baseRange")
  Integer size) {
    int num = size.intValue();
    for (int i = 0; i < num; i++) {
      list.add("hello");
    }
  }

  // Required for JUnit
  public void testListAdds() {
  }

  /**
   * Performs size gets on a {@code List} of size,
   * size.
   * 
   * @param size the size of the {@code List}
   */
  @Setup("beginListGets")
  public void testListGets(@RangeField("baseRange")
  Integer size) {
    int num = size.intValue();
    for (int i = 0; i < num; i++) {
      list.get(i);
    }
  }

  // Required for JUnit
  public void testListGets() {
  }

  /**
   * Performs size inserts at position, where, on
   * an empty List.
   * 
   * @param where Where the inserts happen
   * @param size The size of the List
   * 
   */
  @Setup("beginListInserts")
  public void testListInserts(@RangeEnum(Position.class)
  Position where, @RangeField("insertRemoveRange")
  Integer size) {
    insertIntoCollection(size, where, list);
  }

  // Required for JUnit
  public void testListInserts() {
  }

  /**
   * Performs size removes at position, where, on
   * an ArrayList of size, size.
   * 
   * @param where Where the inserts happen
   * @param size The size of the List
   */
  @Setup("beginListRemoves")
  public void testListRemoves(@RangeField("explicitPositions")
  Position where, @RangeField("insertRemoveRange")
  Integer size) {
    removeFromCollection(size, where, list);
  }

  // Required for JUnit
  public void testListRemoves() {
  }

  /**
   * Creates a new empty List.
   * 
   * @return a not null, empty List
   */
  protected List newList() {
    return new ArrayList();
  }

  void beginListAdds(Integer size) {
    list = newList();
  }

  void beginListGets(Integer size) {
    createList(size);
  }

  void beginListInserts(Position where, Integer size) {
    list = newList();
    index = 0;
  }

  void beginListRemoves(Position where, Integer size) {
    beginListInserts(where, size);
    testListInserts(where, size);
  }

  private void createList(Integer size) {
    beginListAdds(size);
    testListAdds(size);
  }

  private void insertIntoCollection(Integer size, Position where, List l) {
    int num = size.intValue();
    for (int i = 0; i < num; i++) {
      if (where == Position.IMPLICIT_END) {
        l.add("hello");
      } else if (where == Position.BEGIN) {
        l.add(0, "hello");
      } else if (where == Position.EXPLICIT_END) {
        l.add(l.size(), "hello");
      } else if (where == Position.VARIED) {
        l.add(index, "hello");
        index += PRIME;
        index %= l.size();
      }
    }
  }

  private int removeFromCollection(Integer size, Position where, List l) {
    int num = size.intValue();
    for (int i = 0; i < num; i++) {
      if (where == Position.IMPLICIT_END) {
        throw new RuntimeException("cannot remove from the end implicitly");
      } else if (where == Position.BEGIN) {
        l.remove(0);
      } else if (where == Position.EXPLICIT_END) {
        l.remove(l.size() - 1);
      } else if (where == Position.VARIED) {
        l.remove(index);
        index += PRIME;
        int currentSize = l.size();
        if (currentSize > 0) {
          index %= l.size();
        }
      }
    }
    return index;
  }
}

REPORT_PATH

public static final java.lang.String REPORT_PATH

The name of the system property that specifies the location where benchmark reports are both written to and read from. Its value is com.google.gwt.junit.reportPath. If this system property is not set, the path defaults to the user's current working directory.

See Also:

Constant Field Values

Benchmark

public Benchmark()

createStrategy

protected com.google.gwt.junit.JUnitShell.Strategy createStrategy()

Description copied from class: GWTTestCase

Creates the test strategy to use (see GWTTestCase.getStrategy()).

Overrides:

createStrategy in class GWTTestCase

runTest

protected final void runTest()
                      throws java.lang.Throwable

Runs the test via the BenchmarkShell environment. Do not override or call this method.

Overrides:

runTest in class GWTTestCase

Throws:

java.lang.Throwable

supportsAsync

protected final boolean supportsAsync()

Benchmarks do not support asynchronous mode.

Overrides:

supportsAsync in class GWTTestCase