| |
Records |
Links |
|
Author  |
Fernandez Casani, A.; Orduña, J.M.; Sanchez, J.; Gonzalez de la Hoz, S. |
|
| |
Title |
A Reliable Large Distributed Object Store Based Platform for Collecting Event Metadata |
Type |
Journal Article |
| |
Year |
2021 |
Publication |
Journal of Grid Computing |
Abbreviated Journal |
J. Grid Comput. |
|
| |
Volume |
19 |
Issue |
3 |
Pages |
39 - 19pp |
|
| |
Keywords |
Grid computing; Hadoop file system; Object-Based storage |
|
| |
Abstract |
The Large Hadron Collider (LHC) is about to enter its third run at unprecedented energies. The experiments at the LHC face computational challenges with enormous data volumes that need to be analysed by thousands of physics users. The ATLAS EventIndex project, currently running in production, builds a complete catalogue of particle collisions, or events, for the ATLAS experiment at the LHC. The distributed nature of the experiment data model is exploited by running jobs at over one hundred Grid data centers worldwide. Millions of files with petabytes of data are indexed, extracting a small quantity of metadata per event, that is conveyed with a data collection system in real time to a central Hadoop instance at CERN. After a successful first implementation based on a messaging system, some issues suggested performance bottlenecks for the challenging higher rates in next runs of the experiment. In this work we characterize the weaknesses of the previous messaging system, regarding complexity, scalability, performance and resource consumption. A new approach based on an object-based storage method was designed and implemented, taking into account the lessons learned and leveraging the ATLAS experience with this kind of systems. We present the experiment that we run during three months in the real production scenario worldwide, in order to evaluate the messaging and object store approaches. The results of the experiment show that the new object-based storage method can efficiently support large-scale data collection for big data environments like the next runs of the ATLAS experiment at the LHC. |
|
| |
Address |
[Fernandez Casani, Alvaro; Sanchez, Javier; Gonzalez de la Hoz, Santiago] Univ Valencia, Inst Fis Corpuscular IFIC, Burjassot, Spain, Email: alvaro.fernandez@ific.uv.es; |
|
| |
Corporate Author |
|
Thesis |
|
|
| |
Publisher |
Springer |
Place of Publication |
|
Editor |
|
|
| |
Language |
English |
Summary Language |
|
Original Title |
|
|
| |
Series Editor |
|
Series Title |
|
Abbreviated Series Title |
|
|
| |
Series Volume |
|
Series Issue |
|
Edition |
|
|
| |
ISSN |
1570-7873 |
ISBN |
|
Medium |
|
|
| |
Area |
|
Expedition |
|
Conference |
|
|
| |
Notes |
WOS:000692413100001 |
Approved |
no |
|
| |
Is ISI |
yes |
International Collaboration |
no |
|
| |
Call Number |
IFIC @ pastor @ |
Serial |
4953 |
|
| Permanent link to this record |
| |
|
| |
|
Author |
Mendez, V.; Amoros, G.; Kaci, M. |
|
| |
Title |
A Decentralized Deployment Strategy and Performance Evaluation of LCG File Catalog Service |
Type |
Journal Article |
| |
Year |
2011 |
Publication |
Journal of Grid Computing |
Abbreviated Journal |
J. Grid Comput. |
|
| |
Volume |
9 |
Issue |
3 |
Pages |
345-354 |
|
| |
Keywords |
Distributed computing; Grid computing; Performance; Catalog |
|
| |
Abstract |
The LHC Computing Grid (LCG) leads by CERN, has solved with the LCG File Catalog (LFC) the major problem of scaling the data management catalog. However, additional performance issues should be faced to deploy a painless catalog service. With this aim, we present a decentralized LFC server configuration, and its performance evaluation compared with the traditional LFC deployment. A performance analysis is shown, including not only the catalog server, but also analysing the client side overhead. We find that the LFC service has in the clients a relevant workload of the overall service. The experimental results show that the proposed LFC deployment for servers and clients, improves the performance of the service. |
|
| |
Address |
[Munoz, VM; Vicente, GA; Kaci, M] CSIC, Mixed Inst, Inst Fis Corpuscular IFIC, Grid & E Sci Grp, Valencia 46071, Spain, Email: vmendez@ific.uv.es |
|
| |
Corporate Author |
|
Thesis |
|
|
| |
Publisher |
Springer |
Place of Publication |
|
Editor |
|
|
| |
Language |
English |
Summary Language |
|
Original Title |
|
|
| |
Series Editor |
|
Series Title |
|
Abbreviated Series Title |
|
|
| |
Series Volume |
|
Series Issue |
|
Edition |
|
|
| |
ISSN |
1570-7873 |
ISBN |
|
Medium |
|
|
| |
Area |
|
Expedition |
|
Conference |
|
|
| |
Notes |
WOS:000293973700004 |
Approved |
no |
|
| |
Is ISI |
yes |
International Collaboration |
no |
|
| |
Call Number |
IFIC @ elepoucu @ |
Serial |
721 |
|
| Permanent link to this record |
