Fantastic Data

Automatic Partitioning of a Common Information Space

Shared information spaces are utilized in many organization to manage workflow across organizational boundaries. When these organizations are geographically dispersed, such shared information spaces are often partitioned and stored in one or more locations to allow fast retrieval and update. Unfortunately, this partitioning is often done poorly, and instead of fast access, the users experience slow and cumbersome access to the shared information. Fantastic Data developed automatic partitioning tools to determine an optimum partitioning of the information space using an organization’s policies and measured data access to minimize a composite cost function.

The partitioning system gathers usage statistics over a period of time to produce partitioning recommendations. Data records are distributed to and stored at the recommended locations to provide efficient access by application programs. The partitioner is interfaced to a number of commercial database systems. It makes real-time corrections based on changing usage patterns for those systems that support dynamic query changes.

Nodes are clustered into groups based on the similarity of their individual rules. A composite group query is formed and used to control the dissemination of data records among the nodes. The system uses an assertion protocol that allows a single node to unilaterally impose its notion of the correct grouping and queries when it believes its input data is superior to the other nodes' data. Conflicting assertions are resolved cooperatively by the nodes. Data is distributed using multicast messages according to the group query.

The operation of the partitioning system is illustrated in this animation.

actual

Partitioning

Copyright © 2006, 2007 by Fantastic Data. All rights reserved.