Dynamic federations: the Uniform Generic Redirector (UGR) (Dynamic federations: storage aggregation using open tools and protocols)
The Unifor Generic Redirector (UGR) is the core of the Storage federation system that we designed and built to match with the existing and upcoming Grid-related data management architectures. The system is able to federate storage sites and metadata endpoints that expose a suitable data access protocol, into a transparent, high performance storage federation that exposes a unique name space. The architecture can accommodate LFN/PFN algorithmic name translations without the need of catalogues. On the other hand, if catalogues are needed, several of them can be accommodated into the same federation. The idea is to allow applications to access a globally distributed repository, to which sites participate. The applications would be able to efficiently access data that is spread through different sites, by means of a redirection mechanism that is supported by the data access protocol that is used. The focus is on standard protocols for data access, like HTTP and WebDAV, and NFS can be considered as well. The architecture and the components of such a system are anyway detached from the actual protocol that is used.
The focus of our design is on the fact that a federation may be composed by distant sites, and the redirection choices have to take this into account, without imposing the need of partitioning a federation into smaller ones on a geographical basis, or partitioning the name space.
Another point that is important for our design, is efficiency and performance for the data access and also in the browsing case, e.g. allowing an user to list the content of a federated directory in a fast and reliable way that does not impact the performance of the whole system.
Category
Scientific areas
Main features
high performance file metadata access
License
Apache License 2.0 (Apache-2.0)
Supported Operating Systems
Supported CPU Architectures
Programming languages
Build tools