The provided storage handler supports NamedMaps which use custom serialization to store data. The custom serializer class name is provided to the storage handler by setting the hserver.value.serializer property in the CREATE TABLE statement. The optional hserver.value.type property sets the value type passed by the storage handler to the CustomSerializer's setObjectClass() method. This property is used for serialization formats which do not record the object type in the stream (such as Hadoop Writables). Please refer to the ScaleOut StateServer Java Library documentation for more information about NamedMap custom serialization.
In this example we create a class which implements Hadoop’s Writable interface and stores instances of that class in the NamedMap using the WritableSerializer provided as part of the ScaleOut hServer library as a custom serializer:
public class WritableType implements Writable { private String stringProperty; private int numProperty; public int getNumProperty() { return numProperty; } public String getStringProperty() { return stringProperty; } @Override public void write(DataOutput dataOutput) throws IOException { dataOutput.writeUTF(stringProperty); dataOutput.writeInt(numProperty); } @Override public void readFields(DataInput dataInput) throws IOException { stringProperty = dataInput.readUTF(); numProperty = dataInput.readInt(); } public static void main(String argv[]) { NamedMap<IntWritable, WritableType> namedMap = NamedMapFactory.getMap("writableMap", new WritableSerializer(IntWritable.class), new WritableSerializer(WritableType.class)); IntWritable key = new IntWritable(); WritableType value = new WritableType(); for(int i = 0; i<1000; i++) { key.set(i); value.numProperty = i; value.stringProperty = "String # " + i; namedMap.put(key, value); } } }
The following statement creates the Hive table view. Notice that the custom serializer’s class name and the name of the serialized type is sent to the storage handler through TBLPROPERTIES:
hive>CREATE TABLE writableTable (stringproperty string, numproperty int) STORED BY 'com.scaleoutsoftware.soss.hserver.hive.HServerHiveStorageHandler' TBLPROPERTIES ("hserver.map.name" = "writableMap", "hserver.value.serializer"="com.scaleoutsoftware.soss.hserver.WritableSerializer", "hserver.value.type"="WritableType"); OK Time taken: 0.602 seconds
The table is now ready to be queried:
SELECT * FROM writableTable; .............................. String # 959 959 String # 963 963 String # 967 967 String # 971 971 String # 975 975 String # 979 979 String # 983 983 String # 987 987 String # 991 991 String # 995 995 String # 999 999 Time taken: 0.912 seconds, Fetched: 1000 row(s)