Link to paper
The full paper is available here.
You can also find the paper on PapersWithCode here.
Abstract
- Generative machine learning models enable fast event generation
- EPiC-GAN is a flexible framework based on deep sets for simulating sprays of particles
- EPiC layers do not rely on pairwise information sharing between particles
- EPiC-GAN scales well to large particle multiplicities and achieves high generation fidelity
Paper Content
Introduction
- Particle interactions are simulated for fundamental physics research.
- Generative machine learning models are used to replace or augment simulations.
- More simulations are needed due to high-luminosity upgrade of the Large Hadron Collider.
- Generative models can speed up simulations by leveraging GPU computations.
- Generative models need to learn underlying data distributions.
- Generative models are used for calorimeter shower simulations.
- Generative models used include GANs, autoencoders, normalizing flows, and score-based models.
- Generative models for HEP have focused on rigid detector geometries.
- Generative models for point cloud data use graph- and transformer-based architectures.
- Current state-of-the-art for point cloud data generation in HEP is the message-passing GAN.
- This paper introduces a generative model for point clouds that uses global attributes.
Equivariant point cloud gan
- Introduce equivariant point cloud (EPiC) layers
- Stacking multiple EPiC layers to build generator and discriminator architectures
- Novel contribution to generative modeling literature
- Implemented EPiC-GAN in Pytorch, code available on GitHub
Epic layers
- Point cloud C is a graph without edges
- Global attributes of point cloud are represented by g
- Set of points are represented by P
- EPiC layer transforms both global attributes and set of points
- EPiC layer is permutation equivariant
- Involves two consecutive computations
- Computations are learned by neural networks
- Aggregation function uses both mean and sum pooling
- Number of global attributes and number of EPiC layers are hyperparameters to optimize model
Gan architecture
- EPiC GAN feature a generator and discriminator
- Generator and discriminator consist of multiple consecutive EPiC layers
- Discriminator has two additional aggregation functions
- Generator uses input and output blocks for dimensionality expansion/reduction
- Discriminator has same input/output block structure
- GAN training objective follows Least Squares GAN approach
Case study in jet physics
- EPiC-GAN is applied to a task in particle physics
- Generating particle jets from first principles is a well-understood process
- JetNet datasets are used to compare generative models for equivariant point clouds
- EPiC-GAN architecture and training procedure are outlined
- Results are presented on JetNet30 and JetNet150 datasets
- EPiC-GAN is fast and scalable
- Interpretability of its global latent space is discussed
Jetnet datasets
- JetNet30 and JetNet150 datasets were generated with PYTHIA 8.212
- Particles from proton-proton collisions at 13 TeV were clustered with anti-k T algorithm
- Particles were normalized and centered, resulting in 3 features
- Leading 30 particles used for JetNet30, leading 150 particles used for JetNet150
- Datasets consist of 170,000 events, split into training, validation and test sets
- Datasets include gluon jets, light quark jets and top jets
- Extra pre-processing step to re-center jets based on kept particle subset
Architecture & training procedure
- Trained 6 EPiC-GANs for 2000 epochs
- Used L generator = 6 EPiC layers and L discriminator = 3 EPiC layers
- Global attribute length of dim(g ) = 10 and hidden dimensionality of 128 nodes
- Activation functions used LeakyReLU with a slope of 0.01
- Optimizer used Adam with learning rate of 10 โ4
- Input noise dimensionality set to three noise variables per point
- Features of training set standardized to follow a normal distribution with N (0, 5 2 )
- Best epoch chosen based on mean of Wasserstein-1 distance of relative jet mass distribution
- Trained EPiC-GAN 3 times for each dataset
- Further research needed on evaluation metrics for generative models
Jetnet30 results
- EPiC-GAN results compared to state-of-the-art generative model MP-GAN
- GAPT not included in comparison as it under-performs MP-GAN
- JetFlow not included as it does not allow for equivariant generation and requires explicit conditioning
Evaluation scores
- EPiC-GAN and MP-GAN performance compared using multiple evaluation scores
- EPiC-GAN and MP-GAN scores lie within margin of error for most scores
- EPiC-GAN performs slightly better for gluon W M 1 and MP-GAN performs slightly better for gluon W P 1
- MP-GAN performs better for gluon and light quark FPND, EPiC-GAN performs better for top quark FPND
- GANs perform worse than truth scores for most scores, except for gluon and light quark EFP-based scores and EPiC-GAN mass scores
- EPiC-GAN advantage lies in scaling behavior to large cardinalities
- EPiC-GAN and MP-GAN both agree well with JetNet30 datasets
- EPiC-GAN and MP-GAN both struggle to reach the fidelity of the dataset itself
Jetnet150 results
- EPiC-GAN performs well on JetNet150 datasets with up to 150 particles
- EPiC-GAN is the first to show a well performing and fast generating model on a jet dataset with such large particle multiplicity
- EPiC-GAN yields comparable results to the truth in the Wasserstein-1 distance metrics for gluon and light quark datasets
- EPiC-GAN performs a bit worse in Wasserstein-1 distance metrics for top dataset
- EPiC-GAN performs worse than the truth for the particle feature score W P 1
- EPiC-GAN reproduces JetNet150 training data very well
- EPiC-GAN generation of 150 particles is only slower by a linear factor
- EPiC-GAN reproduces particle-level distributions very well, except for ฯ rel and 20th p rel T distributions
Timing
- MP-GAN and EPiC-GAN achieve similar generative fidelity
- MP-GAN scales quadratically, EPiC-GAN scales linearly
- 500k jets were generated
- EPiC-GAN is 13x faster for 30 particles and 55x faster for 150 particles compared to Pythia
Interpretability
- Particle jets are defined by physical observables such as mass, transverse momentum, and particle multiplicity.
- EPiC-GAN uses a global attribute vector to encode physically meaningful jet features.
- Distance correlation between physical jet observables and global attribute vector is calculated from 5,000 generated events.
- After each EPiC layer, there is a higher distance correlation of global attribute vector with physical observables.
Conclusions
- Tasks in physics and other sciences often involve data best represented as point clouds or sets
- Data can have variable cardinality of measured features and data points
- Previous state-of-the-art generative models for point cloud data in HEP considered graph-and transformer-based networks
- These architectures are computationally expensive when scaled to large jet sizes
- Introduced a simple, yet high fidelity set-based alternative to graph-based generative models
- GAN framework utilizes Equivariant Point Cloud (EPiC) layers in the generator and discriminator
- EPiC-GAN is a permutation equivariant generative model that allows for variable particle multiplicity
- Computation cost scales linearly with the particle multiplicity per generated events
- Compared EPiC-GAN to the state-of-the-art network on the JetNet30 benchmark dataset
- Observed comparable generative fidelity between the two models, yet significantly faster generation time for EPiC-GAN
- EPiC-GAN scales well to large point clouds sizes
- Generation time of MP-GAN scales quadratically with the number of points
- Point clouds in particle physics can generally benefit from the simple protocol proposed in this work