[cburgard]

Important

Depends on #24001 (which depends on #23840). This branch is stacked on it: please review only the last 5 commits. It stays in draft until HS3 PR #109 is merged (the HistFactory part also follows HS3 PR #124), and will be rebased when the PRs below it are merged.

Rebased onto master after the merge of #23996, including the follow-up commits added there during review (e.g. isHS3VersionBefore() instead of isLegacyHS3Version(), analyses named after their ModelConfig). The test FormulaNormSysConstraint (added in #23996) is adapted: the constraint is now written into the array distribution of its type, and its name after the round trip depends on that grouping, so the test checks that the parameter is still constrained by a Gaussian. Likewise, HistFactoryShapeFactor (added in #23840) expects the constrained gamma to name the array distribution.

This Pull request:

Implements array_dist from HS3 PR #109 in RooFit's HS3 import and export, and uses it for the constraint terms of HistFactory models, which are also listed in the likelihood from now on.

An array_dist is a product of one-dimensional distributions of one type (component_type), each over its own observable x, with the parameters given per component or in default_parameters. It is the form that is to replace barlow_beeston_lite_poisson_constraint_dist as the recommended staterror constraint of #124 (to be changed in #109 once #124 is merged; the BB-lite distribution is then going to be deprecated), and the form of the HS3 appendix on auxiliary measurements (one array of constraint terms paired with one data set of observed values).

RooFit gets no dedicated class for it: an array is split into one pdf per component, and a tagged RooProdPdf records the grouping. A vectorized implementation is left for later.

Changes or fixes:

1. Import of array_dist

What	How
Components	One pdf per component through the importer of the component type (RooPoisson, RooGaussian, RooLognormal), with default_parameters filled in where a component omits a parameter. Numbers and names may be mixed. Named after the observable: <x>_constraint.
The array	A RooProdPdf named after the array, with the attribute hs3_array_dist, so that the array is written back as array_dist.
Supported types	poisson_dist, gaussian_dist, lognormal_dist; anything else is an explicit error. A poisson_dist component without RooFit's integer flag is the continued Poisson distribution.
Validation	Errors for duplicate or numeric observables, x in default_parameters, empty components, and parameters that depend on an observable of the array (directly or through a function).

An array that no HistFactory modifier references, paired with a one-entry unbinned data set in a likelihood (the appendix form), gives the expected likelihood: one term per component.

2. Export of array_dist

A tagged RooProdPdf (imported, or built by hand) is written as array_dist: one component per factor with the fields of the standalone distribution, a field with the same value in every component once in default_parameters. The observable of a component has to be named, and no two components may share it. The tag is not written as an attribute; it is restored from the type on import.

New: RooJSONFactoryWSTool::exportAsArrayComponent(). The components are registered while the objects that use them are exported, and the arrays are written after all other objects, so that the position of an array in the document does not depend on when its components were registered (the HistFactory channels are exported before the arrays they reference, and a re-export must give the same document).

HistFactory builds its Gaussian constraints with the parameter as x and the global observable as mean, a long-known oddity. The HistFactory exporter records for every constraint which side is the observable (the one that does not depend on the modifier parameter, also for constant parameters such as Lumi), and Gaussians are written with the observable as x (the Gaussian is symmetric in the two), both as array components and as standalone gaussian_dist. Re-imported workspaces therefore have the usual orientation, and the data pairing of section 4 refers to the observable. Gaussians with a number as mean keep the parameter as x, there is nothing to swap with.

3. HistFactory import

A modifier may name an array_dist in constraints. The importer picks the component that depends on the parameter (directly or through a function, e.g. the product of a gamma and its τ) out of the array; no or several matching components is an error. A constraint shared by several channels is one object in all of them, so it is counted once, as with individually named constraints. Components of an array that no parameter uses are reported with a warning at the end of the import, since by the standard the whole array enters the likelihood.

4. HistFactory export

What	How
Grouping	One array_dist per constraint type and top-level pdf: <toplevel>_poisson_constraints, <toplevel>_gaussian_constraints, <toplevel>_lognormal_constraints. The modifiers name the array for each constrained parameter ("constraints": ["simPdf_poisson_constraints", "simPdf_poisson_constraints"]). A constraint shared by several channels is one component. A constraint that already belongs to an array (imported document) stays in that array.
Order	Channels, samples and modifiers in the order they are written; the gammas of staterror and shapesys in bin order.
Not grouped	Constraints that cannot be components: a number as observable (as in rf515_hfJSON.json), a RooLognormal with the transformed parametrization. They are written on their own as before.
Likelihood	The constraint terms of the channels of a model are listed in its likelihood: by default in aux_distributions (observed values as constant parameters, the global observables), or with config().exportConstraintData = true in distributions, each paired with a generated data set <term>_data (unbinned, one axis per observable, one entry) for terms whose observables are global observables of the ModelConfig. Constraint terms not attached to any modifier parameter are listed too, instead of being lost.
Opt-out	config().exportArrayDist = false: no array_dist is written at all. HistFactory constraints are named individually as before, a tagged product is a plain product_dist.

On import, a term listed in the likelihood that is a constraint of one of the listed HistFactory distributions is not added again (as external constraint or as a channel); its data set sets the observed values, keeps them constant, puts them into the global observables of the ModelConfig, and is not imported into the workspace.

The normfactor export now finds the constraint of a parameter among its clients (like the other modifiers), independently of which side of a Gaussian the parameter is.

Behaviour changes users may notice

• RooFit's HistFactory export changes for every constrained modifier: constraints names one array per type instead of one distribution per parameter, the arrays stand at the end of distributions, and the likelihood lists them in aux_distributions. Consumers of these documents need array_dist (HS3 0.3). config().exportArrayDist = false restores the previous form of the constraints.
• After an HS3 round trip, the constraint pdfs of a HistFactory workspace are named after their observable (nom_gamma_stat_channel1_bin_0_constraint, nom_alpha_syst1_constraint) instead of ..._constraint/...Constraint after the parameter, Gaussian constraints of normsys/histosys/normfactor parameters have the global observable as x (in every export mode), and the workspace holds one top-level tagged RooProdPdf per array. The likelihood is unchanged (tested).
• Documents written before this change, and documents without array_dist, import as before.

Checklist:

• tested changes locally
• updated the docs (if necessary)

Tests run (Linux, Ubuntu 26.04, GCC 15.2, RelWithDebInfo, -Droofit=ON -Dtesting=ON): testRooFitHS3 115 → 125 tests (10 new), testJSONInterface 1, testHistFactory 69 → 70 (1 new), all passing; each commit built and tested on its own. clang-format clean.

New tests: ArrayDistImport, ArrayDistImportErrors, ArrayDistExport, ArrayDistExportOfTaggedProduct, ArrayDistLikelihood, HistFactoryArrayDistConstraints (two channels with a shared normsys, staterror and shapesys, written with individual constraints and with arrays: same constraint terms, same NLL), HistFactoryArrayDistConstraintErrors, HistFactoryArrayDistExport (structure and order of the written arrays, NLL unchanged by the round trip, second and third export identical, opt-out), HistFactoryConstraintTermsInLikelihood (both routes, with and without arrays), TutorialRF515RoundTrip, and HS3ArrayDistRoundTrip_ROOT6_16_Combined_Model in testHistFactory (reference workspace of ROOT 6.16: model sections stable under a round trip, NLL unchanged up to a constant).

Updated to the grouped form: HistFactoryConstraintKeyMigration, HistFactoryLegacyConstraintTypeInConstraintKey, HistFactoryStatErrorConstraint, HistFactoryConstraintsByName.

AI assistance: Claude Code (claude-fable-5-1) implemented the changes, the tests and this description under review of the author. The commits carry Assisted-by: claude-code:claude-fable-5-1.

🤖 Generated with Claude Code