Skip to content

Data-handling CAFMaker script(s) - #303

Merged
SFBayLaser merged 8 commits into
developfrom
feature/howard_cafmaker_for_data
Dec 13, 2021
Merged

SFBayLaser merged 8 commits into
developfrom
feature/howard_cafmaker_for_data

Conversation

@brucehoward-physics

Copy link
Copy Markdown
Contributor

Add fhicl that is basically a copy of cafmakerjob_icarus.fcl but nullifies a some labels and a list of labels that belong to the simulation:

physics.producers.cafmaker.G4Label: ""
physics.producers.cafmaker.GenLabel: ""
physics.producers.cafmaker.SimChannelLabel: ""
physics.producers.cafmaker.SystWeightLabels: []

and makes it use a caf_ana_data_sequence which is the same as caf_ana_sequence removing the mcreco instance.

Tested that it ran over a few data events in v09_37 and produced a CAF file with some of the reco branches filled that I'd expect to be filled for data.

I'm tagging Gray and Chris B as reviewers here, and in fact have a question for each:

  • Chris B: the resulting CAF file had many (33?) globalTree s -- is that expected?
  • Gray: are there preliminary data calo constants that we should point this fcl too, or if not should we maybe comment that as a TODO?

…n, for data processing. Also add a fcl that uses this sequence and nullifies the simulation-specific labels.
@cjbacchus

Copy link
Copy Markdown
Contributor

There should only be one globalTree per se. ROOT does this strange auto-save thing that can make trees show up multiple times in a browser, but I don't think that's too important. Did you maybe run over 33 input runs? How many entries does the globalTree have?

@cjbacchus cjbacchus left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. I think there is probably some fat that can be trimmed.

Comment thread fcl/caf/cafmakerjob_icarus_data.fcl Outdated
Comment on lines +26 to +35
outputs:
{
outpid:
{
dataTier: cafana # for grid
streamName: out1 # for grid
module_type: RootOutput
fileName: "stage2_r%r_s%s.root"
}
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You don't actually connect the output up, right?

Suggested change
outputs:
{
outpid:
{
dataTier: cafana # for grid
streamName: out1 # for grid
module_type: RootOutput
fileName: "stage2_r%r_s%s.root"
}
}

Comment thread fcl/caf/cafmakerjob_icarus_data.fcl Outdated
}

runprod: [ @sequence::caf_ana_data_sequence, cafmaker]
stream1: [ ]

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
stream1: [ ]

Comment thread fcl/caf/cafmakerjob_icarus_data.fcl Outdated
runprod: [ @sequence::caf_ana_data_sequence, cafmaker]
stream1: [ ]
trigger_paths: [ runprod ]
end_paths: [ stream1 ]

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
end_paths: [ stream1 ]

Comment thread fcl/caf/cafmakerjob_icarus_data.fcl Outdated
Comment on lines +37 to +39
# Define and configure some modules to do work on each event.
# First modules are defined; they are scheduled later.
# Modules are grouped by type.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This comment is copy-pasted everywhere. It obviously comes from some demo fcl. I would be in favour of nuking this almost everywhere it appears.

Suggested change
# Define and configure some modules to do work on each event.
# First modules are defined; they are scheduled later.
# Modules are grouped by type.

Comment thread fcl/caf/cafmaker_defs.fcl Outdated
# TODO: rns??
]

caf_ana_data_sequence: [

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If caf_ana is the standard name elsewhere in the fcls then OK I guess, but it seems like it would be nice to disambiguate this from CAFAna. It's bad enough that some people call the CAFs "CAFAna files" :)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think @gputnam is possibly the originator of this naming scheme, but I think the name is probably because these are some of the additional modules that are run in the CAFMaker fcl but before the cafmaker process itself. That said I see the potential raised here for ambiguity between caf_ana_sequence and CAFAna.

Gray should comment if my reasoning for the title is incorrect or if he is not the originator of the name schema.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah that's correct @brucehoward-physics. I don't have strong opinions about the name so feel free to change it to whatever.

Comment thread fcl/caf/cafmakerjob_icarus_data.fcl Outdated
services:
{
ParticleInventoryService: @local::standard_particleinventoryservice
BackTrackerService: @local::standard_backtrackerservice

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Obviously we shouldn't need BackTracker for data. Do we try to instantiate one anyway? In which case this is fine but maybe the modules that do that should be fixed.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Following up -- I don't believe we should need the BackTracker. I would suggest removing this and fixing any code that fails of BackTracker or ParticleInventory are not present.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Commented below that this was just missed, my bad. I suspect it "should just work" but we'll see.

@cjbacchus

Copy link
Copy Markdown
Contributor

Fix for the multiple globalTrees in SBNSoftware/sbncode#206

@brucehoward-physics

Copy link
Copy Markdown
Contributor Author

From poking around, each copy of the globalTree seems to have global.@wgts.size() with 1 entry, seemingly at 0. They were all from 1 run, but spread out into multiple files.

The answer to most items is that I copied the cafmakerjob_icarus.fcl and only changed a few lines -- in the case of the BackTracker I clearly didn't look close enough and left that in, sorry. CAFMaker seemed to run alright with it, but better to remove it and do another test run and update the PR.

I can trim out some of the other extraneous bits while doing that.

@brucehoward-physics

Copy link
Copy Markdown
Contributor Author

Ah I missed the last comment with a proposed fix for the multiple Global Trees. Thanks for the fast turn-around.

@gputnam gputnam left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would suggest changing the list of analysis modules.

Comment thread fcl/caf/cafmaker_defs.fcl

caf_ana_data_sequence: [
# Calorimetry
pandoraCaloGausCryoE, pandoraCaloGausCryoW,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because calorimetry is not ready, my suggestion would be to exclude the Calorimetry / Low-Energy-Proton-Reco steps from the data chain.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't necessarily agree we want to turn off the calorimetry outputs, even if the calorimetry constants aren't quite ready. Wouldn't it be better to have this info for testing how things look and just somehow note for the record that these aren't fully finalized? Even if you remove them, calorimetry info will exist for some things, e.g. the showers.

Comment thread fcl/caf/cafmakerjob_icarus_data.fcl Outdated
services:
{
ParticleInventoryService: @local::standard_particleinventoryservice
BackTrackerService: @local::standard_backtrackerservice

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Following up -- I don't believe we should need the BackTracker. I would suggest removing this and fixing any code that fails of BackTracker or ParticleInventory are not present.

@brucehoward-physics

Copy link
Copy Markdown
Contributor Author

As ever "it should just work" are famous last words... In fact both services seem to be needed by one of the modules (not yet sure which).

@brucehoward-physics

Copy link
Copy Markdown
Contributor Author

The requirement for backtracker and particle inventory is baked into CAFMaker --

https://github.com/SBNSoftware/sbncode/blob/d466a8e336fa3ae950ca33ab6309e5aa3ca8e7d5/sbncode/CAFMaker/CAFMaker_module.cc#L753-L754

…(leaving the others untouched). Remove the vertex charge/stubs from that sequence. Remove some unnecessary pieces from the data cafmaker fcl. Make the label for the stubs be blank string in CAFMaker fcl given earlier mentioned update.
@brucehoward-physics

Copy link
Copy Markdown
Contributor Author

Tried to address most of the comments in the latest commit. Didn't change the name of other caf_ana_sequence instances other than the data one here. I don't even know what the feelings about changing those now are, as the workflow is already being tested.

Still to address:

  1. Whether to remove all of the calo-focused preprocessing in the data sequence. I removed the vertex charge/stub, but am not so sure we should remove the calo/chi2pid (instead suggested we make sure it's announced that the current production will not have finalized values here.)

  2. Removing the ParticleInventory and BackTracker servicers (see Issue in SBNCode and related PR... ). If those are accepted then I have to add another commit here.

And one other thing: it seems there is a CAFMaker param for FillTrueParticle - do we want to explicitly set that to false here?

@brucehoward-physics

brucehoward-physics commented Dec 2, 2021 •

Copy link
Copy Markdown
Contributor Author

Context

Per discussion at the SBN Analysis Frameworks meeting I believe we have agreement on not turning off the extra reco pieces for data and acknowledgement that we should somehow inform analyzers that some of these products (calorimetry etc.) that this isn't in it's final form for data.

Also per discussion at the meeting, the FillTrueParticle option I asked about above can be left as-is.

This PR now touches many more cafmakerjob fhicls, changing instances of caf_ana_ to caf_preprocess_ to try to make it less ambiguous with CAFAna. I tested that cafmakerjob_icarus_sce_genie_and_fluxwgt.fcl at least worked as part of testing, though I didn't check every one of the fhicls.

I removed the comment about defining modules from the cafmakerjob_icarus.fcl as well, but left the out stream here since that actually is used in the nocaf fhicl. I removed it from the data fhicl as there is no such link-up there, at least at the moment.

Especially for reviewers

I think that's basically it, except the following question: Do we also want to remove this from the cafmakerjob_icarus_data.fcl?
NuRandomService: @local::per_event_NuRandomService

Other than this, I think this is now in the hands of the reviewers @cjbackhouse and @gputnam to comment further (especially if I missed addressing something)/request further things as necessary. Thanks!

For Release Managers (Please take note!)

This PR now depends on SBNSoftware/sbncode#209

@SFBayLaser

Copy link
Copy Markdown
Contributor

trigger build

@FNALbuild

Copy link
Copy Markdown
Collaborator

✔️ CI build for LArSoft Succeeded on slf7 for e20:prof -- details available through the CI dashboard

@FNALbuild

Copy link
Copy Markdown
Collaborator

✔️ CI build for LArSoft Succeeded on slf7 for c7:prof -- details available through the CI dashboard

@FNALbuild

Copy link
Copy Markdown
Collaborator

⚠️ CI build for ICARUS Warning at phase ci_tests ICARUS on slf7 for e20:prof - ignored warnings for build -- details available through the CI dashboard

🚨 For more details about the warning phase, check the ci_tests ICARUS phase logs

parent CI build details are available through the CI dashboard

@FNALbuild

Copy link
Copy Markdown
Collaborator

⚠️ CI build for ICARUS Warning at phase ci_tests ICARUS on slf7 for c7:prof - ignored warnings for build -- details available through the CI dashboard

🚨 For more details about the warning phase, check the ci_tests ICARUS phase logs

parent CI build details are available through the CI dashboard

@SFBayLaser

Copy link
Copy Markdown
Contributor

trigger build

@FNALbuild

Copy link
Copy Markdown
Collaborator

✔️ CI build for LArSoft Succeeded on slf7 for c7:prof -- details available through the CI dashboard

@FNALbuild

Copy link
Copy Markdown
Collaborator

✔️ CI build for LArSoft Succeeded on slf7 for e20:prof -- details available through the CI dashboard

@FNALbuild

Copy link
Copy Markdown
Collaborator

⚠️ CI build for ICARUS Warning at phase ci_tests ICARUS on slf7 for e20:prof - ignored warnings for build -- details available through the CI dashboard

🚨 For more details about the warning phase, check the ci_tests ICARUS phase logs

parent CI build details are available through the CI dashboard

@FNALbuild

Copy link
Copy Markdown
Collaborator

⚠️ CI build for ICARUS Warning at phase ci_tests ICARUS on slf7 for c7:prof - ignored warnings for build -- details available through the CI dashboard

🚨 For more details about the warning phase, check the ci_tests ICARUS phase logs

parent CI build details are available through the CI dashboard

@SFBayLaser
SFBayLaser merged commit d54280c into develop Dec 13, 2021
@SFBayLaser
SFBayLaser deleted the feature/howard_cafmaker_for_data branch December 13, 2021 17:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants