Table of Contents

Overview

When objects stored in the ScaleOut distributed data grid are large or subject to more frequent updates than reads, it can be advantageous to use an event-based update model rather than a conventional CRUD (Create/Read/Update/Delete) access model. Sending events directly to the ScaleOut hosts where objects reside is often more efficient than pulling an entire object to a client, modifying it, and then sending the entire object back to the StateServer host via an update.

ScaleOut's StreamServer product introduces the PostEventAsync operation, a method that allows messages to be sent directly to the server where an object resides. This enables a stream-oriented approach to handling incoming events: once the event reaches the server where an object resides, it can be analyzed and/or persisted in the associated StateServer object.

Posting an Event

Posting an event to the ScaleOut data grid involves calling the PostEventAsync method. The method takes the following parameters:

  • key: The key to the persisted object in ScaleOut service that is associated with the event. To minimize network overhead, the event will be sent directly to the ScaleOut host that contains this persisted state object. For example, if the ScaleOut service is storing stock price history objects and we are posting a price change to a stock, the ID argument here could be the ticker symbol of the stock history object associated with the price change.

  • eventInfo: An optional, arbitrary string associated with the event. This string is typically used to identify the type of event being posted if more than one kind of event is sent to and handled by the data grid. However, if the event can be entirely described by a string (say, as a JSON-serialized object) then the eventInfo string can serve as the payload of the event.

  • payload: An optional, arbitrary byte array containing the payload of the object (typically a serialized object).

  • invokeTimeout: The amount of time allowed for the PostEvent operation to complete on the server.

Sample Event Payload

Consider an application that streams stock prices as events to be stored and analyzed. The following StockQuote type would be defined in a shared class library project so that both a streaming client and server-side event handling logic can use it. In this example, these StockQuote instances will be used both as an event payload and also in the ScaleOut service as elements in an equity's price history.

using System;
using System.IO;
using MessagePack;

[MessagePackObject]
public class StockQuote
{
    [Key(0)]
    public string Ticker { get; set; }
    [Key(1)]
    public Decimal Price { get; set; }
    [Key(2)]
    public long Volume { get; set; }
    [Key(3)]
    public DateTimeOffset Timestamp { get; set; }

    // Serializes to a byte array.
    public byte[] ToBytes() =>
        MessagePackSerializer.Serialize(this);

    // Deserializes StockQuote from a byte array.
    public static StockQuote FromBytes(byte[] bytes) =>
        MessagePackSerializer.Deserialize<StockQuote>(bytes);
}
Note

This example uses the MessagePack library to serialize messages and cached objects. Any serialization library can be used in its place.

Using PostEventAsync

In this sample, the ScaleOut service is storing stock history objects, (a linked list of stock quotes), keyed by ticker symbol. Rather than using the traditional method of moving the entire history object over the network to modify it, we use PostEventAsync to push a single stock quote to the ScaleOut host that holds the stock's history.

static async Task Main(string[] args)
{
    var grid = await GridConnection.ConnectAsync("bootstrapGateways=localhost:721");
    var cb = new CacheBuilder<string, LinkedList<StockQuote>>("price histories", grid);
    cb.SetKeyEncoder(new ShortStringKeyEncoder());
    // Use MessagePack serialization
    cb.SetSerialization(
            (history, stream) => MessagePackSerializer.Serialize(stream, history),
            (stream) => MessagePackSerializer.Deserialize<LinkedList<StockQuote>>(stream)
    );
    var cache = cb.Build();

    // Simulate an incoming stock quote:
    var quote = new StockQuote()
    {
        Ticker = "MSFT",
        Timestamp = DateTimeOffset.Parse("5/21/2019 17:59 -5:00"),
        Price = 126.90m,
        Volume = 14330868
    };

    // Post the quote to the ScaleOut service holding this stock's history.
    // Here, ticker symbols are the keys to history objects the cache.
    // Use it as the key for the post operation to cause the event to be
    // raised on the ScaleOut where the MSFT history resides
    var response = await cache.PostEventAsync(key: quote.Ticker,
                                              eventInfo: "stock quote",
                                              payload: quote.ToBytes());

    switch (response.Result)
    {
        case ServerResult.Posted:
            Console.WriteLine("Event posted successfully.");
            break;
        case ServerResult.UnhandledExceptionInCallback:
            Console.WriteLine("Unhandled exception thrown from handler.");
            Console.WriteLine(Encoding.UTF8.GetString(response.ErrorData));
            break;
        default:
            break;
    }
}

Unhandled Exceptions

Avoid throwing unhandled exceptions from server-side event handling code whenever possible. In case an exception does escape from your event handler, it will be sent back and made available to the PostEventAsync caller through the ErrorData property. The representation and encoding of the error will vary depending on library that is used to handle posted events; if using this Scaleout.Client library to handle events, the error will be the full "ToString()" text representation of the exception, encoded as UTF-8.