Eviction
The size of a client cache can be limited by either a maximum number of entries or a memory limit. When the limit is reached, eviction is performed using either a least-recently-used (LRU) algorithm or by random removal of cache entries.
Available Client Caches
The type of client cache to use is a cache-level policy that can be specified either through a config file (as a string) or when building a Cache using the CacheBuilder.SetClientCache methods (as either a string or a client cache instance).
| Config Name | Class | Size Limit | Eviction Algorithm |
|---|---|---|---|
| "LRU" | LruEvictionCache | Object count | LRU |
| "Random" | RandomEvictionCache | Object count | Random |
| "LRU-MaxMemory" | LruMemoryEvictionCache | Memory Usage | LRU |
| "Random-MaxMemory" | RandomMemoryEvictionCache | Memory Usage | Random |
If a client cache is unspecified, a cache will use an LRU client cache with a limit of the 1000 most recently accessed objects.
LRU vs. Random Eviction
LRU eviction is the default strategy and should meet the needs of most applications. Objects that have not been accessed in a long time are evicted to make room for more recently accessed objects.
However, LRU eviction is susceptible to sequential flooding: if a client application repeatedly retrieves the same set of objects from the ScaleOut service in the same order, the client cache will never deliver a hit (unless it is large enough to hold all the objects in the sequence). Consider using random eviction in the following scenarios:
- Applications that frequently query the ScaleOut service to iterate over a large set of objects, and the result set does not vary much from query to query.
- Users of ScaleOut's PMI compute engine should use random eviction in their PMI invocation handler applications, since they will repeatedly iterate over the same sequence of objects.
Estimating Memory Usage
The Scaleout.Client library uses the serialized size of an object as the basis for its memory usage estimates in the "LRU-MaxMemory" and "Random-MaxMemory" client caches. This heuristic is usually accurate enough for most objects, and it provides the ability to estimate the relative size differences between objects in a cache. The actual memory usage of the cache may vary from the estimate, in which case the configured memory limit can be tuned to compensate for the inaccuracy.
Some serializers, however, can efficiently pack objects down to a very small size (Google Protocol Buffers are one such example). In this case, the client cache's memory usage estimates can be very inaccurate because instantiated objects will be much larger than their serialized form. To address this, the Scaleout.Client library allows custom serialization callbacks to return their own size estimates: the CacheBuilder provides a SetSerialization overload that allows you to register serialization callbacks that return your own memory usage estimates.