1.5. Finding data in the Bookkeeping
Knowing how data flows through the various Gaudi applications is crucial for knowing where to look for your data.
Data are catalogued in ‘the bookkeeping’, and are initially sorted in broad groups such as ‘Delphes Events’, ‘Full detector simulation’, and according to the FCC flavor (ee, hh or eh). After this, a tree of various application and processing versions will eventually lead to the data you need.
Learning Objectives
Find FCC data in the central storage
Store FCC data in the central storage
1.5.1. Obtaining access permissions
In general, access to the EOS shared filesystem is obtained by subscribing to the relevant egroups here. For running the tutorials read access is sufficient, which is granted through subscription to the following e-group:
fcc-eos-access
which is open for self subscription requests.
For analysis activities, the physics groups have dedicated areas under /eos/experiment/fcc/ee/analyses_storage/
/eos/experiment/fcc/ee/analyses_storage/BSM
/eos/experiment/fcc/ee/analyses_storage/EW_and_QCD
/eos/experiment/fcc/ee/analyses_storage/flavour
/eos/experiment/fcc/ee/analyses_storage/Higgs_and_TOP
Write access to those areas is controlled by dedicated e-groups
fcc-eos-write-ee-analyses-bsm
fcc-eos-write-ee-analyses-ew-qcd
fcc-eos-write-ee-analyses-flavor
fcc-eos-write-ee-analyses-higgs-top
managed by the conveners of those physics groups, which should be contacted for subscription.
See the here for more information about FCC computing resources.
1.5.2. Finding Data
Browse the fcc directory on eos, the shared filesystem used by CERN.
tree /eos/experiment/fcc/ -L 2
You should see an output similar to:
/eos/experiment/fcc/
├── ee
│ ├── accelerator
│ ├── analyses
│ ├── datasets
│ ├── generation
│ ├── mdi
│ ├── simulation
│ ├── tutorial
│ └── utils
├── eh
│ └── datasets
├── helhc
│ ├── analyses
│ ├── generation
│ ├── utils
│ └── YR_summary
├── hh
│ ├── analyses
│ ├── CDR_script
│ ├── generation
│ ├── hcal
│ ├── simulation
│ ├── tests
│ ├── testsamples
│ ├── tutorials
│ └── utils
The file paths on eos already give an indication where to find what kind of data. In some instances, the information is very detailed:
ls /eos/experiment/fcc/ee/tutorial/
But not all datasets follow this exact format. In case of questions simply contact the software team.
On lxplus eos is mounted as a standard directory and you can use standard bash tools such as ls, cp, mv … to manipulate the files - Also to write them provided you have the correct permissions.
However, using eos like this is not recommended to transfer actual data (we speak from experience) and also not possible on machines where eos has not been mounted. The right tool here is xrdcp, which uses the xrootd protocol. It works like cp (minus some features) but all paths must be prefixed by the URL root://eospublic.cern.ch/.