5. Distributed Computing

Page contributors

Gerardo Ganis, Juraj Smiesko

This chapter covers two complementary approaches to large-scale event production for FCC analyses: EventProducer and DIRAC.

The EventProducer framework, introduced first, provides a lightweight and reproducible interface to define, submit, monitor, and manage centrally produced event samples. It integrates the different stages of the production chain from generator-level event creation to detector simulation and output handling.

In addition, this chapter documents workflows based on the DIRAC Interware system through iLCDirac, the extension developed by the Linear Collider community and also used by CALICE. These tutorials provide concrete examples of running distributed workflows on grid resources, including job submission, production management, and output retrieval.

The setup procedure needed to use the FCC resources through DIRAC is described first. Additional information about the use of iLCDirac can be found in the CLIC and ILC dedicated Wiki pages.

Unless specified otherwise, in the remainder of this section the word DIRAC refers to the iLCDirac extension mentioned above.

Contents:

• 5.1. EventProducer
  • 5.1.1. Clone and initialisation
  • 5.1.2. Generate LHE files from gripacks
  • 5.1.3. Generate LHE files directly from MG5
  • 5.1.4. Generate STDHEP files directly from Whizard + Pythia6
  • 5.1.5. Generate EDM4hep files from the LHE and decay with Pythia8
  • 5.1.6. Generate EDM4hep files from STDHEP
  • 5.1.7. Generate EDM4hep files from Pythia8
  • 5.1.8. Expert mode
    • 5.1.8.1. Updating the database
    • 5.1.8.2. Cleaning bad jobs
    • 5.1.8.3. Update the webpage
    • 5.1.8.4. Create the sample list for analyses
    • 5.1.8.5. All in One
  • 5.1.9. Produce samples with EventProducer outside of official campaign
• 5.2. Getting Started with DIRAC
  • 5.2.1. Registering to the FCC VO
  • 5.2.2. Enabling DIRAC
  • 5.2.3. Copying, browsing, accessing files
  • 5.2.4. The web portal
• 5.3. FCC DIRAC Example Workflows
  • 5.3.1. Overview of the Submission Scripts
    • 5.3.1.1. Before starting: cloning of workflows repository
  • 5.3.2. Workflow 1
    • 5.3.2.1. Purpose
    • 5.3.2.2. Output files
    • 5.3.2.3. DIRAC components involved
    • 5.3.2.4. The script, dissected
      • 5.3.2.4.1. The utility function
      • 5.3.2.4.2. Adding the --wms switch
      • 5.3.2.4.3. The DIRAC API instance
      • 5.3.2.4.4. The DIRAC Job manager
      • 5.3.2.4.5. The KKMC application instance
      • 5.3.2.4.6. The steering files for Delphes
      • 5.3.2.4.7. Completing the sandbox for Delphes
      • 5.3.2.4.8. Standalone Delphes using the DIRAC generic application
      • 5.3.2.4.9. Submitting the job to the chosen WMS
    • 5.3.2.5. Running the script on lxplus
      • 5.3.2.5.1. Local submission
      • 5.3.2.5.2. WMS submission
• 5.4. Structure for /eos/experiment/fcc/prod
  • 5.4.1. Overview