Skip to content

Take the fIsRealData that was previously hard-coded to false and set … - #209

Merged
miquelnebot merged 1 commit into
release/SBN2021Cfrom
feature/howard_cafmaker_fixBackTrackParticleInventory
Dec 13, 2021
Merged

miquelnebot merged 1 commit into
release/SBN2021Cfrom
feature/howard_cafmaker_fixBackTrackParticleInventory

Conversation

@brucehoward-physics

Copy link
Copy Markdown
Contributor

…it instead based on a fhicl parameter. Move backtracker and particle inventory service to be initialized when needed - this does have the downside of meaning it can be initiated multiple times per event. Move some of the Fill function calls to be inside a check on IsRealData, e.g. where it calls something that needed the backtracker or particle inventory...

NOTE This was pretty hastily coded up given the time, so comments/looks are appreciated. Maybe there is a better way even. I am concurrently opening an issue and tagging some folks there.

@cjbacchus

Copy link
Copy Markdown
Contributor

I'm fine with the basic idea. I think nova's equivalent variables is filled automatically by just checking for the presence of any MCTruth objects in the file.

Is it the creation of the ServiceHandle object itself that throws the exception, or the first attempt to dereference it (in this case calling .get()?). If the latter, then we could put the service definitions back to their original place and then take care not to touch them.

@brucehoward-physics

Copy link
Copy Markdown
Contributor Author

It seemed to be the creation of the service handle itself.

@brucehoward-physics

Copy link
Copy Markdown
Contributor Author

For what it's worth, I re-ran cafmaker in my test area using cafmakerjob_icarus_sce_genie_and_fluxwgt.fcl and Joseph's workflow test stage1 file in /icarus/app/users/jaz8600/2021C_Workflow_Prep/NuMI/numi_nu_cosmics/ and then compared to the caf in that area with diff_cafs

In my diff_cafs (output at /icarus/app/users/howard/icaruscode_Nov21_Release/test_cafmaker/RUN_DIFF_CAFS.txt) I'm seeing some differences but they look like they are just null results, e.g. a bunch that relate to CRT-track matching
rec.reco.trk[0].crthit.hit.plane differs: -1626884672 vs -1646519680
and fewer but still non-zero of track vars like
rec.reco.trk[0].ID differs: -1906314998 vs -1488944617
and one instance of shower completeness
rec.reco.shw[139].truth.matches[2].energy_completeness differs: 4.178704321093373e+31 vs 1.0946131984469779e+33

I can repeat the test with one of the data files I was looking at too, using standard v09_37_01 and my test area. I guess because of the difference in filling/not filling certain variables that there might be more "diffs" to parse through.

@cjbacchus

Copy link
Copy Markdown
Contributor

Looks like there are some more SR initialization cases that need to be patched, for fields that always get filled by CAFMaker for MC, but are left uninitialized in the data mode.

@brucehoward-physics

Copy link
Copy Markdown
Contributor Author

@cjbackhouse This is a sim file above. I talked with @henrylay97 and CRTHit planes and Trk IDs are in the set of instances he was looking at. I pointed out the shower one to him.

We'll see what the data one turns up to your point about additional SR initialization cases.

@PetrilloAtWork PetrilloAtWork left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Three general comments:

  1. The presence of a default value for the new parameter is questionable (not wrong: just questionable), and I suggest to think a moment about that and take a pondered decision (which maybe already happened without me knowing, which is also fine); this is the reason why I don't straight approve the pull request, but I let you proceed if you think the current choice is good.
  2. Given the usage current pattern, I would have chosen a fIsSimulation variable, since all instances I can see use !fIsRealData; of course, if in the future data-only features will be added (beam quality? calibrations?), that argument would not stand.
  3. As Bruce wrote elsewhere, ServiceHandle construction will throw if the service can't be found. To be even more explicit: all configured services are constructed at the beginning of the job and before all modules. Calling ServiceHandle only picks the pointer to the service object, which exists already. So the overhead for creating and for dereferencing a service handle is very low, and (a) the first construction of ServiceHandle is the one which needs error checking and (b) constructing handles multiple times does not trigger service reinitialization or anything beside some lookup in art service registry.

It might have also had sense to save either a pointer or an std::optional to the two backtracking service objects as class data members to be conditionally filled at module construction time, but that's matter of choice.

Finally and foremost, thanks Bruce for taking care of this.

Comment thread sbncode/CAFMaker/CAFMaker_module.cc Outdated
art::ServiceHandle<cheat::BackTrackerService> bt_serv;

id_to_ide_map = PrepSimChannels(simchannels, *geometry);
id_to_truehit_map = PrepTrueHits(hits, clock_data, *bt_serv.get());

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The .get() should not be necessary here:

Suggested change
id_to_truehit_map = PrepTrueHits(hits, clock_data, *bt_serv.get());
id_to_truehit_map = PrepTrueHits(hits, clock_data, *bt_serv);

(ServiceHandle::operator * will return a reference to the service object directly).
I suggest removal.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure - I just copied the previous function call into the loop, but I can make this change. Are the other instances of .get() similar? (I thought I spotted a few last night...)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed there were multiple and multiple *pi_serv.get(). But since they are all service handle .get() calls with the * operator, I assume they can all be changed in this fashion. If this is not the case let me know!

Comment thread sbncode/CAFMaker/CAFMakerParams.h Outdated
Atom<bool> IsRealData {
Name("IsRealData"),
Comment("Set true for a data file."),
false

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Topic for discussion: do we really want a default value here?
Beside making the transition easier (which may still be a value, but IMO should not make things harder till the end of time), I am not sure there is a good reason to assume the input to be simulation.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Happy to change but this would mean we need to add the false setting to the current workflows in ICARUS/SBND for CAFs from simulation. I wasn't sure about tampering with the default workflow, but if the sentiment is to go for it then we can 😅

@brucehoward-physics

Copy link
Copy Markdown
Contributor Author

I responded to point 1 in the comments. Ultimately I say it's not my choice, but am happy to participate in discussion! Having the default be false means that current workflows remain unchanged, but if it's better to change them for the future then that's fair.

There was already an fIsRealData parameter that was sort-of extraneous (it seemed to only be filled as a CAF file variable and was always hard-coded to false) and I have just co-opted it and tried to make it more useful to fit this purpose.

(Tagging @PetrilloAtWork directly because I will shortly be making another post with info from the diff_cafs on a data event and don't want this response to be lost...)

@brucehoward-physics

Copy link
Copy Markdown
Contributor Author

Ran a single data event through a version with default CAFMaker and just a somewhat edited fhicl, meant to be more in-line with the current state of my icaruscode PR (e.g. also having the BackTracker and ParticleInventory services). Also ran the event through my test area CAFMaker. Then used diff_cafs to compare: /icarus/app/users/howard/icaruscode_Nov21_Release/test_cafmaker_data/nominal/RUN_DIFF_CAFS.txt.

I took a very quick look and report a few things below. Are the following sorts of behaviors, or any others that people spot in that file or other testing, considered reasonable enough behavior?

Of course, most of the diffs are related to "truth" information, which makes some sense since in one we moved a lot of fills behind a check on fIsRealData meaning that things can be treated differently. From a pretty quick look, it seems they fall into ~3 classes.

  1. Seemingly null value vs. 0 or vice-versa, like rec.reco.shw[0].truth.p.parent differs: 0 vs 2141192192 or rec.reco.trk[1].truth.p.parent differs: 265537960 vs 0
  2. Similar variable but that has two seemingly null values like rec.slc[57].reco.shw[1].truth.p.parent differs: 1667330644 vs 2141192192
  3. Default value vs. nan, e.g. rec.reco.shw[4].truth.eff differs: -1.0 vs nan

The lines not matching to "truth" (grep -v "truth") show mostly the track -> CRTHit plane null differences as before, like
rec.reco.trk[1].crthit.hit.plane differs: 191717616 vs 185549280
and a few instances of the track ID type
rec.reco.trk[0].ID differs: -1426694211 vs 1592542871
and finally the very much expected
rec.hdr.ismc differs: 1 vs 0

@cjbacchus

Copy link
Copy Markdown
Contributor

The most interesting differences would be any you see between two runs over the same mode in isRealData mode. Those will be missing initializations we'd want to clean up for diff_cafs to be properly useful.

@brucehoward-physics

Copy link
Copy Markdown
Contributor Author

After adding back in the stubs, can see they appear to have similar printouts to the other stuff in terms of before/after changing how this filling happens. They also have some interesting truth parent printouts like
rec.reco.stub[4].truth.p.parent differs: 64 vs 2

To @cjbackhouse point in the most recent comment: not a bad idea. I compared the one I made for this test and one I made yesterday and filtered out the lines matching stub. The only remaining lines seem to be CRTHit and Trk ID related, so nothing "new".

@brucehoward-physics

brucehoward-physics commented Dec 2, 2021 •

Copy link
Copy Markdown
Contributor Author

Context

This comment is in the current CAFMaker around the fIsRealData param:
use this instead of evt.isRealData(), see init in [...] produce(evt)

However I believe this is vestigial and left-over from NOvA's CAFMaker (especially given that it was previously just being hard-coded to false)? For now, I've removed this comment, added a comment to the constructor, and I left this fIsRealData parameter in the code and use it but overwrite it at each new event with evt.isRealData().

These changes nullify Gianluca's question regarding defaulting a fhicl parameter since there is no longer a fhicl parameter for this.

A remaining point/question

The filling of true (g4) particles is itself inside of a check on mc_particles.isValid() here and since the related labels should be set to null for data I think this block is not hit in my tests. I did not put this block into an explicit check on fIsRealData but can do if that is better. Let me know if you think this would be better?

For the reviewers

@gputnam @cjbackhouse @PetrilloAtWork - I believe this is ready to be looked at again? Please let me know if things are left or are newly unsatisfactory or if other comments remain to be addressed (it's possible I'm missing or forgetting something!)? I did ask one question in the above as well. Thanks!

@cjbacchus

Copy link
Copy Markdown
Contributor

Yeah, I think that's a NOvA comment. We had trouble with overlays. If you start with data and then overlay MC into it, art thinks you have a data event. But you probably do want to be able to query the truth of the stuff you added. So we wound up with some kind of check whether there were any truth products in the record, rather than relying on art's answer.

@PetrilloAtWork
PetrilloAtWork self-requested a review December 3, 2021 20:04

@PetrilloAtWork PetrilloAtWork left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks ok.
The only thing I am a bit fought about is that now fIsRealData does not really need to be a class data member: it could be a variable local to produce(), with no other consequence.
It bothers me a bit because it suggests that we know which kind the data is, which is actually true only after the first event is read.
After some deliberation, I recommend that this variable be removed from the class, and a local one be used. But I will accept a refusal.

@brucehoward-physics

Copy link
Copy Markdown
Contributor Author

Sure - I'm fine either way, so I've pushed a change. Obviously I can't say if others feel strongly a certain way.

I did not re-run any test as this seems a very minor change...

@miquelnebot

Copy link
Copy Markdown
Contributor

trigger build

@FNALbuild

Copy link
Copy Markdown

✔️ CI build for LArSoft Succeeded on slf7 for c7:prof -- details available through the CI dashboard

@FNALbuild

Copy link
Copy Markdown

✔️ CI build for LArSoft Succeeded on slf7 for e20:prof -- details available through the CI dashboard

@FNALbuild

Copy link
Copy Markdown

⚠️ CI build for ICARUS Warning at phase ci_tests ICARUS on slf7 for c7:prof - ignored warnings for build -- details available through the CI dashboard

🚨 For more details about the warning phase, check the ci_tests ICARUS phase logs

parent CI build details are available through the CI dashboard

@FNALbuild

Copy link
Copy Markdown

⚠️ CI build for SBND Warning at phase ci_tests SBND on slf7 for c7:prof - ignored warnings for build -- details available through the CI dashboard

🚨 For more details about the warning phase, check the ci_tests SBND phase logs

parent CI build details are available through the CI dashboard

@FNALbuild

Copy link
Copy Markdown

⚠️ CI build for SBND Warning at phase ci_tests SBND on slf7 for e20:prof - ignored warnings for build -- details available through the CI dashboard

🚨 For more details about the warning phase, check the ci_tests SBND phase logs

parent CI build details are available through the CI dashboard

@FNALbuild

Copy link
Copy Markdown

⚠️ CI build for ICARUS Warning at phase ci_tests ICARUS on slf7 for e20:prof - ignored warnings for build -- details available through the CI dashboard

🚨 For more details about the warning phase, check the ci_tests ICARUS phase logs

parent CI build details are available through the CI dashboard

@miquelnebot

Copy link
Copy Markdown
Contributor

trigger build SBNSoftware/icaruscode#303

@FNALbuild

Copy link
Copy Markdown

✔️ CI build for LArSoft Succeeded on slf7 for e20:prof -- details available through the CI dashboard

@FNALbuild

Copy link
Copy Markdown

✔️ CI build for LArSoft Succeeded on slf7 for c7:prof -- details available through the CI dashboard

@FNALbuild

Copy link
Copy Markdown

❌ CI build for SBND Failed at phase ci_tests SBND on slf7 for c7:prof - ignored warnings for build -- details available through the CI dashboard

🚨 For more details about the failed phase, check the ci_tests SBND phase logs

parent CI build details are available through the CI dashboard

@FNALbuild

Copy link
Copy Markdown

❌ CI build for SBND Failed at phase ci_tests SBND on slf7 for e20:prof - ignored warnings for build -- details available through the CI dashboard

🚨 For more details about the failed phase, check the ci_tests SBND phase logs

parent CI build details are available through the CI dashboard

@FNALbuild

Copy link
Copy Markdown

⚠️ CI build for ICARUS Warning at phase ci_tests ICARUS on slf7 for e20:prof - ignored warnings for build -- details available through the CI dashboard

🚨 For more details about the warning phase, check the ci_tests ICARUS phase logs

parent CI build details are available through the CI dashboard

@FNALbuild

Copy link
Copy Markdown

⚠️ CI build for ICARUS Warning at phase ci_tests ICARUS on slf7 for c7:prof - ignored warnings for build -- details available through the CI dashboard

🚨 For more details about the warning phase, check the ci_tests ICARUS phase logs

parent CI build details are available through the CI dashboard

@miquelnebot

Copy link
Copy Markdown
Contributor

trigger build SBNSoftware/icaruscode#303

@FNALbuild

Copy link
Copy Markdown

✔️ CI build for LArSoft Succeeded on slf7 for c7:prof -- details available through the CI dashboard

@FNALbuild

Copy link
Copy Markdown

✔️ CI build for LArSoft Succeeded on slf7 for e20:prof -- details available through the CI dashboard

@FNALbuild

Copy link
Copy Markdown

⚠️ CI build for SBND Warning at phase ci_tests SBND on slf7 for c7:prof - ignored warnings for build -- details available through the CI dashboard

🚨 For more details about the warning phase, check the ci_tests SBND phase logs

parent CI build details are available through the CI dashboard

@FNALbuild

Copy link
Copy Markdown

⚠️ CI build for ICARUS Warning at phase ci_tests ICARUS on slf7 for e20:prof - ignored warnings for build -- details available through the CI dashboard

🚨 For more details about the warning phase, check the ci_tests ICARUS phase logs

parent CI build details are available through the CI dashboard

@FNALbuild

Copy link
Copy Markdown

⚠️ CI build for ICARUS Warning at phase ci_tests ICARUS on slf7 for c7:prof - ignored warnings for build -- details available through the CI dashboard

🚨 For more details about the warning phase, check the ci_tests ICARUS phase logs

parent CI build details are available through the CI dashboard

@FNALbuild

Copy link
Copy Markdown

⚠️ CI build for SBND Warning at phase ci_tests SBND on slf7 for e20:prof - ignored warnings for build -- details available through the CI dashboard

🚨 For more details about the warning phase, check the ci_tests SBND phase logs

parent CI build details are available through the CI dashboard

@miquelnebot
miquelnebot changed the base branch from develop to release/SBN2021C December 13, 2021 11:46
@miquelnebot
miquelnebot changed the base branch from release/SBN2021C to develop December 13, 2021 13:52
@miquelnebot
miquelnebot changed the base branch from develop to release/SBN2021C December 13, 2021 13:52
@miquelnebot
miquelnebot merged commit 367fb23 into release/SBN2021C Dec 13, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants