ScaleOut ComputeServer^® Java Simple MapReduce Programmer's Guide
Simple MapReduce API

Each simple MapReduce job uses an InvocationGrid (IG), which is a set of JVM’s associated with the grid service processes on each server that is loaded with all necessary JARs, classes, files etc. If an invocation grid is not created before the job starts, then the MapReduce class will create one once MapReduce.run(TimeSpan) is called. Creating the IG is an expensive operation and when running continuous MapReduce jobs, pre-defining an invocation grid will dramatically increase performance. The following example builds upon the previous examples and demonstrates how to create an InvocationGrid for continuously running simple MapReduce jobs:

// Create the invocation grid.
InvocationGrid grid = new InvocationGridBuilder("WordCount_Grid_" + System.currentTimeMillis())
                    .addJar("/path/to/wordcount.jar")
                    .addClass(MyClass.class)
                    .setLibraryPath("simple_map_reduce")
                    .load();

// Create a NamedMap input map
NamedMap<Integer, String> inputMap = NamedMapFactory.getMap("WordCount_InputMap_" + System.currentTimeMillis());
// Create a NamedMap output map
NamedMap<String, Integer> outputMap = NamedMapFactory.getMap("WordCount_OutputMap_" + System.currentTimeMillis());

// Create a parameter object
HashMap<String, Boolean> ignoreTheseWords = new HashMap<String, Boolean>(3);
ignoreTheseWords.put("a", false);
ignoreTheseWords.put("an", false);
ignoreTheseWords.put("the", false);

for(int runNum = 0; runNum < 10; runNum++) {
        MapReduce wordcount = new MapReduce(inputMap, outputMap, WordCountMapper.class, WordCountReducer.class)
                                                .withCombiner(WordCountCombiner.class)
                                                .withParameterObject(ignoreTheseWords);

        wordcount.run(grid, TimeSpan.INFINITE_TIMEOUT);
}