Skip to content

Updates to Configurations for ICARUS MCP - #223

Merged
SFBayLaser merged 46 commits into
developfrom
feature/MCP2021Reco
Aug 17, 2021
Merged

SFBayLaser merged 46 commits into
developfrom
feature/MCP2021Reco

Conversation

@gputnam

@gputnam gputnam commented Aug 3, 2021 •

Copy link
Copy Markdown
Contributor

Updates to ICARUS MC reconstruction for the upcoming MCP.

Right now includes a bugfix in running the stage1 MC reco fcl file and includes a number of configurations to run "CAF" making.

This is intended as a draft PR to collect updates needed for the MCP.

Depends on SBNSoftware/sbncode#156.

ICARUS random number generator policy

To address the issue of CORSIKAGen (and GENIEGen?) requiring a random number when NuRandomService (perEvent policy) is not able to provide one, the policy for random number generation has been changed as follows.

  • The general default policy in ICARUS is set to random.
  • The policy for service configuration presets designed for simulation (G4, DetSim and also indirectly Reco) is set to perEvent, to preserve reproducibility.
  • Selected hybrid job configurations that perform both generation and simulation tasks (e.g. in-time cosmic ray generation) have been explicitly set to random policy.

Note that integration test configurations for event generation already explicitly pin down the random number seeds, therefore they are not expected to be broken by this change.

As an aside, I have opened a ticket with LArSoft to address the general issue with perEvent policies.

MultiTPC DetSim configurations

They are the new standard.

@gputnam
gputnam marked this pull request as draft August 3, 2021 14:09
@gputnam

gputnam commented Aug 3, 2021

Copy link
Copy Markdown
Contributor Author

Also linked: SBNSoftware/icarusalg#34

gputnam and others added 7 commits August 3, 2021 14:58
Centralized space charge configuration in G4.
Removed duplication of intime_g4 icarus configuration.
standard_detsim_icarus.fcl reintroduced; "multitpc" label should fade out.
Since `perEvent` is not in general safe, `random` was set as new default.
This commit sets the policy to `perEvent` for the standard simulation
service set (which includes G4 and DetSim). In addition, generators
using only RadioGen or SingleGen are set back to `perEvent`,
and G4 processes which include generation (e.g. "intime") are set to
`random`. Reco configuration happens to happily inherit the simulation
configuration (but Reco does not use random numbers anyway, does it??).
@PetrilloAtWork

PetrilloAtWork commented Aug 5, 2021 via email

Copy link
Copy Markdown
Member

@SFBayLaser

Copy link
Copy Markdown
Contributor

I think this is critical for the next release, not the current one (v09_28_00) and so will leave open. I think there is still more work for the MC versions of stage0 and stage1. TBC.

gputnam and others added 4 commits August 9, 2021 16:12
…ff to the cafmaker defs fcl as done for pandoraTrackGausSCE. Settings meant to be like the previous ones in reco_icarus.fcl but with the CryoW/CryoE names.
@brucehoward-physics

Copy link
Copy Markdown
Contributor

Pushing commit adding showers. Ran very small tests on a few particle gun MC events just to see that things ran basically. Possibly interestingly, in a test of a few events trying to put in SCE and SCE corr, I found very little difference in the sample having or not having the correction applied. It was not say 0 change for all showers, but in any case I wasn't sure if it should be as small a difference as it was. (just noted for future testing)

Also a few other things I believe I noticed from this test:
(1) This seemed to be running a different hit finding than I usually see. Was this intentional?
(2) I didn't scour the logs, mostly just was looking for it to have showers now. However, I think I noticed a few warnings/etc. from this test (I definitely noticed these, it's just a matter of it I noticed during this test or another thing I was running, but I think it was this...)

  • One was a warning that daqTrigger was invalid. Maybe related to using a particle gun generator fcl?
  • "Failed to get GTruth object" - looks like while making a CAF file or one of the associated producer modules

A transitional list of failing tests that still need to be fixed is also added.
CAF making configuration files are currently broken in `develop`.
@PetrilloAtWork

Copy link
Copy Markdown
Member

I have pushed configurations to generate proton-only cosmic rays with overburden based on the "standard" configuration.
Unfortunately something seems broken in my box... cosmic rays generation fails when it's time to pick CORSIKA files.

@gputnam

gputnam commented Aug 12, 2021

Copy link
Copy Markdown
Contributor Author

I have merged in two more feature branches: Tracy's to include latest updates to stage0 / stage1 processing and Jacob's to run Iker's flash matching. Should be ready! So I will make this not a draft anymore.

I do want to run a couple more tests before this is merged, but it is ready for review.

@gputnam
gputnam marked this pull request as ready for review August 12, 2021 22:35
//std::cout << " totalrms size " << totalRMS.size() << std::endl;

noise_factor=1;
// noise_factor=1; ****** not sure why this was done?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we open an issue related to this so we don't forget that it is commented out?

@jzennamo

Copy link
Copy Markdown
Contributor

This all looks awesome and I am really excited to have it merged in!

@PetrilloAtWork PetrilloAtWork left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My partial review (conflict of interest, remember?) found a couple of potential issues.

  1. Naming of a few of new proton-only configurations does not follow the existing pattern.
  2. Hard-coded numbers replicated around: suggesting centralisation.
  3. PMT decoding appears to have disappeared from one of the configurations.
  4. one of the job configurations, stage0_single_icarus.fcl, seems to fail parsing.

Also there is material for a feature request to GENIEGen (it is possible to implement it without even changing GENIEHelper, which means that we can hope to see it soon).

Comment thread fcl/caf/cafmakerjob_icarus.fcl Outdated
Comment on lines +25 to +31
#Start each new event with an empty event.
source:
{
module_type: RootInput
maxEvents: -1 # Number of events to create
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd remove this.

Suggested change
#Start each new event with an empty event.
source:
{
module_type: RootInput
maxEvents: -1 # Number of events to create
}

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed.

Comment on lines +32 to +43
outputs:
{
outpid:
{
dataTier: cafana # for grid test
streamName: out1 # for grid test
module_type: RootOutput
fileName: "pid_r%r_s%s.root"
fastCloning: false
}
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would really remove this: fastCloning deactivation is questionable, the streamName does not mean anything, the file name is not representative nor well protected against duplication, and all the thing is not used anyway.

Suggested change
outputs:
{
outpid:
{
dataTier: cafana # for grid test
streamName: out1 # for grid test
module_type: RootOutput
fileName: "pid_r%r_s%s.root"
fastCloning: false
}
}

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed the bad config -- although the block is used in one of the caf flavors so I am keeping it there.

# Because this is hardcoded, it needs to be updated for
# updates to the geometry. This is updates as of
# geometry: icarus_complete_20210527_no_overburden.gdml
physics.producers.generator.FiducialCut: "mbox: -378.49,-191.86,-904.950652270838,378.49,144.96,904.950652270838"

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
physics.producers.generator.FiducialCut: "mbox: -378.49,-191.86,-904.950652270838,378.49,144.96,904.950652270838"
physics.producers.generator.FiducialCut: "mbox: -378.49,-191.86,-904.950652270838,378.49,144.96,904.950652270838"

This is actually something we can automate.
@jzennamo Care to open a ticket on LArSoft about that?

@@ -0,0 +1,41 @@
#
# File: prodcorsika_bnb_nue_genie_protononly_icarus.fcl

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

According to the surgeon general, this one should be called prodcorsika_genie_protononly_icarus_bnb_nue.fcl.

Comment on lines -36 to +37
services.TFileService.fileName: "Supplemental-prodcorsika_genie_protononly_icarus_%tc-%p.root"
outputs.out1.fileName: "prodcorsika_genie_protononly_icarus_%tc-%p.root"
services.TFileService.fileName: "Supplemental-prodcorsika_bnb_genie_protononly_icarus_%tc-%p.root"
outputs.out1.fileName: "prodcorsika_bnb_genie_protononly_icarus_%tc-%p.root"

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Considering the existing prodcorsika_genie_protononly_icarus_numi.fcl (and prodcorsika_genie_standard_icarus_numi.fcl), this should be prodcorsika_genie_protononly_icarus_bnb.fcl.

# * `cosmgen`: generated cosmic rays from CORSIKA
#

#include "prodcorsika_genie_standard_icarus.fcl"

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can also start from prodcorsika_genie_protononly_icarus_bnb.fcl and override only the particle code.

# Because this is hardcoded, it needs to be updated for
# updates to the geometry. This is updates as of
# geometry: icarus_complete_20210527_no_overburden.gdml
physics.producers.generator.FiducialCut: "mbox: -378.49,-191.86,-904.950652270838,378.49,144.96,904.950652270838"

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two is one too many. I recommend:

Suggested change
physics.producers.generator.FiducialCut: "mbox: -378.49,-191.86,-904.950652270838,378.49,144.96,904.950652270838"
#include "genie_fiducial_cuts_icarus.fcl"
physics.producers.generator.FiducialCut: @local::icarus_allTPCbox

and fcl/gen/genie/genie_fiducial_cuts_icarus.fcl:

# 
# File:    genie_fiducial_cuts_icarus.fcl
# Purpose: hard-coded fiducial volume cuts for GENIE generation.
#

BEGIN_PROLOG

# This box includes all TPC active volumes, and just that.
# Because this is hardcoded, it needs to be updated for 
# updates to the geometry. This is updates as of 
# geometry: icarus_complete_20210527_no_overburden.gdml
icarus_allTPCbox: "mbox: -378.49,-191.86,-904.950652270838,378.49,144.96,904.950652270838"

END_PROLOG

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can include this as a future fix but it's still not clear to me that this is the best way to generate genie events in both cryostats, so I'd like to wait before optimizing the config.


## Use the following to run the full defined stage0 set of modules
physics.reco: [@sequence::icarus_stage0_multiTPC]
physics.reco: [daqTPC, @sequence::icarus_stage0_multiTPC]

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@SFBayLaser What about PMT decoding?

@gputnam

gputnam commented Aug 15, 2021

Copy link
Copy Markdown
Contributor Author

Added some commits to address Gianluca's comment and add in CRT hits/tracks to CAF-making

@gputnam

gputnam commented Aug 15, 2021

Copy link
Copy Markdown
Contributor Author

Also I have run the tests that I wanted to!

@SFBayLaser

Copy link
Copy Markdown
Contributor

trigger build

@FNALbuild

Copy link
Copy Markdown
Collaborator

CI build for LArSoft on slf7 for c7:prof is in progress -- details available through the CI dashboard

@FNALbuild

Copy link
Copy Markdown
Collaborator

CI build for LArSoft on slf7 for e20:prof is in progress -- details available through the CI dashboard

@FNALbuild

Copy link
Copy Markdown
Collaborator

✔️ CI build for LArSoft Succeeded on slf7 for c7:prof -- details available through the CI dashboard

@FNALbuild

Copy link
Copy Markdown
Collaborator

✔️ CI build for LArSoft Succeeded on slf7 for e20:prof -- details available through the CI dashboard

@FNALbuild

Copy link
Copy Markdown
Collaborator

CI build for ICARUS on slf7 for c7:prof is in progress -- details available through the CI dashboard

parent CI build details are available through the CI dashboard

@FNALbuild

Copy link
Copy Markdown
Collaborator

CI build for ICARUS on slf7 for e20:prof is in progress -- details available through the CI dashboard

parent CI build details are available through the CI dashboard

@SFBayLaser SFBayLaser left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am already familiar with the bulk of the changes and have verified they are working. There are further updates that I think I need to see in action. So I will merge this and then run through my checks on my laptop.

@FNALbuild

Copy link
Copy Markdown
Collaborator

❌ CI build for ICARUS Failed at phase unit_test ICARUS on slf7 for c7:prof - ignored warnings for build -- details available through the CI dashboard

🚨 For more details about the failed phase, check the unit_test ICARUS phase logs

parent CI build details are available through the CI dashboard

@FNALbuild

Copy link
Copy Markdown
Collaborator

❌ CI build for ICARUS Failed at phase unit_test ICARUS on slf7 for e20:prof - ignored warnings for build -- details available through the CI dashboard

🚨 For more details about the failed phase, check the unit_test ICARUS phase logs

parent CI build details are available through the CI dashboard

@SFBayLaser

Copy link
Copy Markdown
Contributor

Ok, the expected outcome of the build since we are still getting things fixed there too. So will merge.

@SFBayLaser
SFBayLaser merged commit e3438eb into develop Aug 17, 2021
@SFBayLaser
SFBayLaser deleted the feature/MCP2021Reco branch August 17, 2021 14:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Generate ICARUS neutrino events in both cryostats

7 participants