Repository navigation
Data-handling CAFMaker script(s) - #303
Conversation
…n, for data processing. Also add a fcl that uses this sequence and nullifies the simulation-specific labels.
|
There should only be one globalTree per se. ROOT does this strange auto-save thing that can make trees show up multiple times in a browser, but I don't think that's too important. Did you maybe run over 33 input runs? How many entries does the globalTree have? |
cjbacchus
left a comment
There was a problem hiding this comment.
Looks good. I think there is probably some fat that can be trimmed.
| outputs: | ||
| { | ||
| outpid: | ||
| { | ||
| dataTier: cafana # for grid | ||
| streamName: out1 # for grid | ||
| module_type: RootOutput | ||
| fileName: "stage2_r%r_s%s.root" | ||
| } | ||
| } |
There was a problem hiding this comment.
You don't actually connect the output up, right?
| outputs: | |
| { | |
| outpid: | |
| { | |
| dataTier: cafana # for grid | |
| streamName: out1 # for grid | |
| module_type: RootOutput | |
| fileName: "stage2_r%r_s%s.root" | |
| } | |
| } |
| } | ||
|
|
||
| runprod: [ @sequence::caf_ana_data_sequence, cafmaker] | ||
| stream1: [ ] |
There was a problem hiding this comment.
| stream1: [ ] |
| runprod: [ @sequence::caf_ana_data_sequence, cafmaker] | ||
| stream1: [ ] | ||
| trigger_paths: [ runprod ] | ||
| end_paths: [ stream1 ] |
There was a problem hiding this comment.
| end_paths: [ stream1 ] |
| # Define and configure some modules to do work on each event. | ||
| # First modules are defined; they are scheduled later. | ||
| # Modules are grouped by type. |
There was a problem hiding this comment.
This comment is copy-pasted everywhere. It obviously comes from some demo fcl. I would be in favour of nuking this almost everywhere it appears.
| # Define and configure some modules to do work on each event. | |
| # First modules are defined; they are scheduled later. | |
| # Modules are grouped by type. |
| # TODO: rns?? | ||
| ] | ||
|
|
||
| caf_ana_data_sequence: [ |
There was a problem hiding this comment.
If caf_ana is the standard name elsewhere in the fcls then OK I guess, but it seems like it would be nice to disambiguate this from CAFAna. It's bad enough that some people call the CAFs "CAFAna files" :)
There was a problem hiding this comment.
I think @gputnam is possibly the originator of this naming scheme, but I think the name is probably because these are some of the additional modules that are run in the CAFMaker fcl but before the cafmaker process itself. That said I see the potential raised here for ambiguity between caf_ana_sequence and CAFAna.
Gray should comment if my reasoning for the title is incorrect or if he is not the originator of the name schema.
There was a problem hiding this comment.
Yeah that's correct @brucehoward-physics. I don't have strong opinions about the name so feel free to change it to whatever.
| services: | ||
| { | ||
| ParticleInventoryService: @local::standard_particleinventoryservice | ||
| BackTrackerService: @local::standard_backtrackerservice |
There was a problem hiding this comment.
Obviously we shouldn't need BackTracker for data. Do we try to instantiate one anyway? In which case this is fine but maybe the modules that do that should be fixed.
There was a problem hiding this comment.
Following up -- I don't believe we should need the BackTracker. I would suggest removing this and fixing any code that fails of BackTracker or ParticleInventory are not present.
There was a problem hiding this comment.
Commented below that this was just missed, my bad. I suspect it "should just work" but we'll see.
|
Fix for the multiple globalTrees in SBNSoftware/sbncode#206 |
|
From poking around, each copy of the globalTree seems to have The answer to most items is that I copied the I can trim out some of the other extraneous bits while doing that. |
|
Ah I missed the last comment with a proposed fix for the multiple Global Trees. Thanks for the fast turn-around. |
gputnam
left a comment
There was a problem hiding this comment.
I would suggest changing the list of analysis modules.
|
|
||
| caf_ana_data_sequence: [ | ||
| # Calorimetry | ||
| pandoraCaloGausCryoE, pandoraCaloGausCryoW, |
There was a problem hiding this comment.
Because calorimetry is not ready, my suggestion would be to exclude the Calorimetry / Low-Energy-Proton-Reco steps from the data chain.
There was a problem hiding this comment.
I don't necessarily agree we want to turn off the calorimetry outputs, even if the calorimetry constants aren't quite ready. Wouldn't it be better to have this info for testing how things look and just somehow note for the record that these aren't fully finalized? Even if you remove them, calorimetry info will exist for some things, e.g. the showers.
| services: | ||
| { | ||
| ParticleInventoryService: @local::standard_particleinventoryservice | ||
| BackTrackerService: @local::standard_backtrackerservice |
There was a problem hiding this comment.
Following up -- I don't believe we should need the BackTracker. I would suggest removing this and fixing any code that fails of BackTracker or ParticleInventory are not present.
|
As ever "it should just work" are famous last words... In fact both services seem to be needed by one of the modules (not yet sure which). |
|
The requirement for backtracker and particle inventory is baked into CAFMaker -- |
…(leaving the others untouched). Remove the vertex charge/stubs from that sequence. Remove some unnecessary pieces from the data cafmaker fcl. Make the label for the stubs be blank string in CAFMaker fcl given earlier mentioned update.
|
Tried to address most of the comments in the latest commit. Didn't change the name of other Still to address:
And one other thing: it seems there is a CAFMaker param for |
…o caf_preprocess_ ... In general (though not in every case) this is in part describing modules run in the same fhicl but before the actual CAFMaker step.
…es some changes in CAFMaker in sbncode.
ContextPer discussion at the SBN Analysis Frameworks meeting I believe we have agreement on not turning off the extra reco pieces for data and acknowledgement that we should somehow inform analyzers that some of these products (calorimetry etc.) that this isn't in it's final form for data. Also per discussion at the meeting, the This PR now touches many more I removed the comment about defining modules from the Especially for reviewersI think that's basically it, except the following question: Do we also want to remove this from the cafmakerjob_icarus_data.fcl? Other than this, I think this is now in the hands of the reviewers @cjbackhouse and @gputnam to comment further (especially if I missed addressing something)/request further things as necessary. Thanks! For Release Managers (Please take note!)This PR now depends on SBNSoftware/sbncode#209 |
|
trigger build |
|
✔️ CI build for LArSoft Succeeded on slf7 for e20:prof -- details available through the CI dashboard |
|
✔️ CI build for LArSoft Succeeded on slf7 for c7:prof -- details available through the CI dashboard |
|
🚨 For more details about the warning phase, check the ci_tests ICARUS phase logs parent CI build details are available through the CI dashboard |
|
🚨 For more details about the warning phase, check the ci_tests ICARUS phase logs parent CI build details are available through the CI dashboard |
|
trigger build |
|
✔️ CI build for LArSoft Succeeded on slf7 for c7:prof -- details available through the CI dashboard |
|
✔️ CI build for LArSoft Succeeded on slf7 for e20:prof -- details available through the CI dashboard |
|
🚨 For more details about the warning phase, check the ci_tests ICARUS phase logs parent CI build details are available through the CI dashboard |
|
🚨 For more details about the warning phase, check the ci_tests ICARUS phase logs parent CI build details are available through the CI dashboard |
Add fhicl that is basically a copy of
cafmakerjob_icarus.fclbut nullifies a some labels and a list of labels that belong to the simulation:and makes it use a
caf_ana_data_sequencewhich is the same ascaf_ana_sequenceremoving themcrecoinstance.Tested that it ran over a few data events in v09_37 and produced a CAF file with some of the reco branches filled that I'd expect to be filled for data.
I'm tagging Gray and Chris B as reviewers here, and in fact have a question for each:
globalTrees -- is that expected?TODO?