|
|
Technology
Distributed caching can help your applications unlock their full performance potential on server farms, while offloading database servers. ScaleOut Software's suite of products deliver the industry's leading distributed caching solution for .NET server farms by combining:
- fast access that outperforms other storage alternatives,
- scalability capable of handling the largest server-farm loads,
- high availability to meet stringent mission-critical requirements, and
- ease of use designed to keep development time and management to an absolute minimum.
This section explores some of the exciting technology that helps ScaleOut StateServer deliver these benefits to you.
Why Distributed Caching?
With the advent of Web and application server farms, database servers have increasingly been used to hold mission-critical but relatively short-lived and fast-changing data that must be accessible across the farm. Examples include session-state, e-commerce shopping carts, SOAP requests, and intermediate results from data grid computations. Because of their fast turnover and high update rates, these new types of data have increasingly bottlenecked database servers.
Distributed caching offers a new storage alternative that dramatically lowers cost and boosts application performance. By moving fast-changing data closer to where it is used and taking advantage of a server farm's inherent scalability, this technology significantly enhances overall application performance while making more efficient use of the data center's computational, networking, and storage resources.
Partitioned Storage for Scaled Performance
The key to achieving high performance and scalability on server farms is to distribute a workload across the servers so that all can simultaneously share a portion of the work. This reduces the delays in handling sub-tasks, and as the workload increases, servers can be added to scale the processing power of the farm.
To reap these performance benefits, ScaleOut StateServer automatically partitions all of distributed cache's stored objects across the farm and simultaneously processes access requests on all servers. This reduces access times and scales the overall throughput of the distributed cache. It also avoids "hot spots" that can arise if objects are stored on the servers where they are created.
As servers are added to the farm, ScaleOut StateServer automatically repartitions and rebalances the storage workload to scale throughput. Likewise, if servers are removed, ScaleOut StateServer coalesces stored objects on the surviving servers and rebalances the storage workload as necessary.
Integrated, Internal Caching
To keep access times fast, ScaleOut StateServer automatically caches objects on the servers where they were most recently accessed. Two levels of internal caching are employed to ensure the fastest possible access times. These caches are integrated into ScaleOut StateServer to automatically accelerate performance without involving the developer in configuring and coordinating multiple caches. One level of internal caching holds objects within the StateServer service process on each server. This speeds up repeated accesses to these objects by avoiding the networking overheads required to copy them from remote hosts.
A second level of internal caching holds deserialized objects within ScaleOut StateServer's client libraries. This cache sidesteps CPU overheads required to retrieve deserialized objects when they are repeatedly accessed from the distributed cache. Caching deserialized data dramatically lowers access times so that they are very close to "in process" speeds.
The following diagram shows an example of a client application retrieving an object from a remote server within the distributed cache. ScaleOut StateServer automatically caches the object's deserialized data with the client library on the requesting server:
Integrated Data Replication for High Availability
High availability of cached data is essential for mission-critical applications. ScaleOut StateServer ensures that cached data is never lost - even if a server in the farm fails - by replicating all cached objects on up to two additional servers. If a server goes offline or loses network connectivity, ScaleOut StateServer retrieves its objects from replicas stored on other servers in the farm, and it creates new replicas to maintain redundant storage as part of its "self-healing" process.
ScaleOut StateServer uses a fixed number of replicas to ensure that the storage capacity of the distributed cache scales as servers are added to the farm. Replicating objects to all servers would quickly overflow the memory of each server.
The following diagram depicts an object (in blue) and its two replicas (in red) stored within the distributed cache on different servers. ScaleOut StateServer automatically takes care of creating, load-balancing, and accessing replicas so that you can benefit from the cache's high availability without having to manage these details.
Traditional master/slave replication mechanisms are subject to "split brain" problems in which network outages make it impossible to keep both replicas synchronized. If this ever occurs, the distributed cache could lose updates and return "stale" data to its clients. To avoid split brain problems, ScaleOut StateServer employs patent-pending technology that provides scalable, highly available, quorum-based updating. This technology, which originated in the design of highly available server clusters, guarantees that replicated objects always have the latest updates. It transparently handles a variety of server and network failures without any intervention on the part of the developer.
Scalable Failure Detection and Recovery
A major challenge in the design of server farms is the design of a global membership mechanism that scales with the number of servers in the farm. This membership mechanism determines which servers in the farm are active and able to participate in the distributed cache. An integral aspect of this mechanism is the use of a heartbeat protocol between servers that quickly detects server or networking outages and initiates recovery.
ScaleOut StateServer uses a patent-pending, scalable, point-to-point heartbeat architecture that efficiently detects failures without flooding the server farm's network with multicast heartbeat packets. Heartbeat failures automatically trigger ScaleOut StateServer's "self-healing" technology, which quickly restores access to cache partitions and dynamically rebalances the storage load across the farm.
The following diagram depicts ScaleOut StateServer's use of heartbeat channels to detect server and networking outages:
The Benefits of a Scalable, Highly Available Architecture
ScaleOut StateServer's scalable, highly available storage architecture creates a foundation that benefits every function of the distributed cache, including:
- APIs and asynchronous event handling,
- data replication to remote distributed caches, and
- remote client access.
ScaleOut StateServer provides comprehensive APIs for managing the lifetime of stored objects, including timeouts, object dependencies, and memory reclamation of the least recently used objects. Because of its highly available storage, the distributed cache automatically ensures that these properties are preserved after server failures. Also, ScaleOut StateServer's scalable design allows the handling of object expiration to scale with the server farm by simultaneously processing asynchronous, object expiration events on all servers, as depicted by the red arrows in the following diagram. Moreover, if a server fails, its expiration events are automatically directed to surviving servers so that delivery is guaranteed.
Likewise, ScaleOut GeoServer's remote data replication uses scalable and highly available connections that take full advantage of all servers in each farm. To maximize performance and availability, all servers within both the local and remote StateServer farms participate in data replication, as shown in the following diagram:
Since all local servers participate in handling local object updates, they simultaneously replicate these updates to the remote distributed cache. This maximizes both the throughput and scalability of data replication. In addition, the servers download load-balancing information that is used to most efficiently direct object updates to individual servers in the remote farm. This information is automatically updated whenever a membership or load-balancing change occurs at the remote store. Likewise, this architecture ensures that data replication is not interrupted if servers in either the local or remote farm should fail.
ScaleOut Remote Client also leverages ScaleOut StateServer's scalable, highly available architecture to give applications the fastest possible performance. Its client libraries use multiple, simultaneous connections to the distributed cache which automatically scale networking throughput with the size of the server farm. These connections are depicted by the red arrows in the following diagram. The client libraries also maintain load-balancing information obtained from the distributed cache so that access requests are directly sent to the cache servers that store the requested objects for maximum performance.
To maintain high availability if a cache server should fail or be taken offline, the client libraries automatically re-establish a connection as necessary to other servers in the distributed cache. Likewise, if a new cache server is added, the remote client automatically detects the new server and communicates with it.
Integrated Ease of Use
ScaleOut Software has designed its products to keep the application designer's valuable development time to an absolute minimum. (ASP.NET developers can store session-state objects with no code changes using ScaleOut StateServer or ScaleOut SessionServer.) This is accomplished by creating the simplest possible view of the distributed cache for the application developer and hiding the details of object placement, performance scaling, and data replication for high availability.
To a large extent, the application developer can view the distributed cache as if it were a local cache which is accessed by the customary add, retrieve, update, and remove operations on cached objects. Object locking for synchronization across threads and servers is built into these operations and occurs automatically. To keep this view as simple as possible, almost all of the extensive distributed caching technology described above is hidden from the developer.
For example, consider a retrieve operation. When the developer uses this one-line API method to access a cached object, ScaleOut StateServer transparently takes several steps that allow the distributed cache to maintain its high performance, scalability, and high availability:
- The client library looks in a local, deserialized data cache to see if the object's data is available and current.
- The local server looks in its serialized data cache to see if the object was recently accessed or a replica is stored locally.
- If necessary, the local server determines which remote server holds the object and forwards the request to that server.
- If a server fails, the failure is detected and the server is removed from the distributed cache. The access request is automatically re-forwarded to another server that has a replica for the object.
- If a server is added and the object is load-balanced to another server, the access request is automatically re-forwarded.
- The server on which the object resides is locked for unique access by the client. The object's lock is forwarded to the replica servers in case a failure should occur. (The lock is released when the client application updates or unlocks the object.)
- The object's data is returned to the client and cached on the local server if necessary. If an internal cache is full, older objects are flushed to make room for the new object.
This discussion shows how ScaleOut StateServer's integrated design hides the many details of distributed caching so that the application developer does not have to devote time and energy to managing them. In particular, ScaleOut StateServer transparently handles the issues of server farm membership, object placement, scaling, recovery, creating and managing replicas, and handling synchronization on object access.
Summary
ScaleOut StateServer's scalable, highly available architecture enables distributed caching to provide a highly effective foundation for building scalable, mission-critical applications. It also integrates and encapsulates this powerful technology wherever possible so that the application developer can maintain focus on application design.
Distributed caching lets developers take advantage of the full potential for scaling and high availability offered by server farms. It also forms the basis for the next generation of parallel or "data grid" computing and unlocks huge performance benefits for a wide range of applications. ScaleOut Software is committed to the ongoing development of distributed caching that will continue to make these benefits a reality.
|
