SciencePAD
Published on SciencePAD (https://test-static-01.web.cern.ch)

Home > Dynamic federations: storage aggregation using open tools and protocols

Dynamic federations: storage aggregation using open tools and protocols [1]

Dynamic federations: the Uniform Generic Redirector (UGR) (Dynamic federations: storage aggregation using open tools and protocols)

The Unifor Generic Redirector (UGR) is the core of the Storage federation system that we designed and built to match with the existing and upcoming Grid-related data management architectures. The system is able to federate storage sites and metadata endpoints that expose a suitable data access protocol, into a transparent, high performance storage federation that exposes a unique name space. The architecture can accommodate LFN/PFN algorithmic name translations without the need of catalogues. On the other hand, if catalogues are needed, several of them can be accommodated into the same federation. The idea is to allow applications to access a globally distributed repository, to which sites participate. The applications would be able to efficiently access data that is spread through different sites, by means of a redirection mechanism that is supported by the data access protocol that is used. The focus is on standard protocols for data access, like HTTP and WebDAV, and NFS can be considered as well. The architecture and the components of such a system are anyway detached from the actual protocol that is used. The focus of our design is on the fact that a federation may be composed by distant sites, and the redirection choices have to take this into account, without imposing the need of partitioning a federation into smaller ones on a geographical basis, or partitioning the name space. Another point that is important for our design, is efficiency and performance for the data access and also in the browsing case, e.g. allowing an user to list the content of a federated directory in a fast and reliable way that does not impact the performance of the whole system.

 

 

Category

Web service or application [2]

 

Scientific areas

High performance computing [3]

High Energy Physics [4]

 

Main features

Storage federations [5]

Metadata federations [6]

high performance file metadata access [7]

 

License

Apache License 2.0 (Apache-2.0) [8]

 

Supported Operating Systems

Linux [9]

 

Supported CPU Architectures

 

Programming languages

 

Build tools

 

Test tools

Coordinated by EMI. EMI is partially funded by the European Commission under Grant Agreement INFSO-RI-261611

Powered by Drupal

Source URL: https://test-static-01.web.cern.ch/node/128

Links
[1] https://test-static-01.web.cern.ch/node/128
[2] https://test-static-01.web.cern.ch/taxonomy/term/335
[3] https://test-static-01.web.cern.ch/taxonomy/term/574
[4] https://test-static-01.web.cern.ch/taxonomy/term/240
[5] https://test-static-01.web.cern.ch/taxonomy/term/646
[6] https://test-static-01.web.cern.ch/taxonomy/term/647
[7] https://test-static-01.web.cern.ch/taxonomy/term/648
[8] https://test-static-01.web.cern.ch/taxonomy/term/265
[9] https://test-static-01.web.cern.ch/taxonomy/term/508