1.5. Finding data in the Bookkeeping

Knowing how data flows through the various Gaudi applications is crucial for knowing where to look for your data.

Data are catalogued in ‘the bookkeeping’, and are initially sorted in broad groups such as ‘Delphes Events’, ‘Full detector simulation’, and according to the FCC flavor (ee, hh or eh). After this, a tree of various application and processing versions will eventually lead to the data you need.

Learning Objectives

  • Find FCC data in the central storage

  • Store FCC data in the central storage

1.5.1. Gaining access permissions

Read/write access to the shared filesystem is gained by subscribing to the relevant egroups here:

fcc-eos-read-hh
fcc-eos-read-ee
fcc-eos-read-eh

fcc-eos-write-hh
fcc-eos-write-ee
fcc-eos-write-eh

1.5.2. Finding Data

Browse the fcc directory on eos, the shared filesystem used by CERN.

tree /eos/experiment/fcc/ -L 2

You should see an output similar to:

/eos/experiment/fcc/
├── ee
│   ├── accelerator
│   ├── analyses
│   ├── datasets
│   ├── generation
│   ├── mdi
│   ├── simulation
│   ├── tutorial
│   └── utils
├── eh
│   └── datasets
├── helhc
│   ├── analyses
│   ├── generation
│   ├── utils
│   └── YR_summary
├── hh
│   ├── analyses
│   ├── CDR_script
│   ├── generation
│   ├── hcal
│   ├── simulation
│   ├── tests
│   ├── testsamples
│   ├── tutorials
│   └── utils

The file paths on eos already give an indication where to find what kind of data. In some instances, the information is very detailed:

ls /eos/experiment/fcc/ee/tutorial/

But not all datasets follow this exact format. In case of questions simply contact the software team.

On lxplus eos is mounted as a standard directory and you can use standard bash tools such as ls, cp, mv … to manipulate the files - Also to write them provided you have the correct permissions.

However, using eos like this is not recommended to transfer actual data (we speak from experience) and also not possible on machines where eos has not been mounted. The right tool here is xrdcp, which uses the xrootd protocol. It works like cp (minus some features) but all paths must be prefixed by the URL root://eospublic.cern.ch/.