[ENH] ibma support - #59
Merged
Merged
Conversation
Point at NiMARE main rather than a released version. 0.21.0 is where the `groupby` estimator argument lands, where Stouffers loses `normalize_contrast_weights`, and where `aggressive_mask` starts defaulting to False; PyPI still only has 0.20.0, which has none of it. A direct reference needs hatchling's opt-in to build, so that goes in too. Both carry a note to swap back to a version specifier once 0.21.0 ships, because the PyPI publish step rejects direct references even though the build accepts them. `requests` becomes explicit because staging an image-based meta-analysis's maps needs it directly rather than through a transitive dependency. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
NiMARE resolves an image to a path and hands it to nibabel, which cannot read over HTTP, so the maps have to be local before a studyset can be analysed. Two things make this less mechanical than it sounds. Which of an image's `url` and `filename` holds the downloadable NIfTI depends on how it reached Neurostore, and both shapes are live in production: images ingested from NeuroVault put the NIfTI in `url` and a bare basename in `filename`, while images uploaded by compose put the landing page in `url` and the NIfTI in `filename`. So neither field can be trusted by position, and the choice is made on content instead -- the candidate that is both fetchable and named like a NIfTI. Choosing the bare basename would have dropped every map in an ingested studyset, which is most of what an image-based meta-analysis would want to run over. A landing page also returns HTTP 200 with HTML, so `raise_for_status` does not catch it. Downloaded bytes are checked for a gzip or NIfTI header before being written, to keep a file nibabel cannot open out of the cache, where it would fail every later run too. `value_type` arrives as a NeuroVault label rather than a NiMARE image type, and maps NiMARE has no use for are dropped -- along with a record, per analysis, of what was dropped and why, so a caller can say which maps are absent rather than only how many. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`FixedEffectsHedges` and `SampleSizeBasedLikelihood` require a `sample_sizes` entry on each analysis, `Stouffers` and `Fishers` can weight by it, and NiMARE needs it to convert a t-map into a z-map -- which is the common case, since most NeuroVault uploads are t-maps. Compose records it on the annotation note or on the study's metadata rather than on the analysis, and as a free-form string, so it has to be found, coerced and copied across before the studyset becomes a Dataset. Precedence is annotation note, then analysis metadata, then study metadata: the annotation is the most specific and the most deliberately curated. Analyses with no sample size are counted and warned about rather than failing the run, since an estimator that does not need one is unaffected. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Compose stores whatever the frontend collected and hands it straight to the estimator. Two things have to happen in between. `n_cores` is the load-bearing one. The runner injects it for every estimator and no image-based estimator accepts it, so passing it through would raise now that NiMARE rejects unrecognized keywords. `PermutedOLS` calls its equivalent `n_jobs`; the rest are serial. The rest is a better error, earlier. Validating against the estimator's own signature -- walked over the MRO, because `aggressive_mask` and `groupby` live on the base class and reach subclasses through `**kwargs` -- lets a stale specification be rejected before a studyset's worth of maps is downloaded. The message names what replaced a retired argument rather than only reporting it as unexpected. There is nothing to be backwards compatible with: compose cannot create an image-based meta-analysis yet, so no stored specification names an argument an older NiMARE had. This targets one API, the installed one. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Real studysets are uneven. A base-study search for image data on "language" returns 27 studies carrying nine distinct value_type labels between them -- z maps, t maps, beta maps, F maps, ROI masks, anatomicals. NiMARE closes the gap on its own: `IBMAWorkflow` converts what it can and `drop_invalid` discards the rest. Both steps are silent, and that is the only problem -- a meta-analysis can rest on a third of the studies the user selected and say nothing about it. On that studyset, `Stouffers` uses 39 of 53 analyses when sample sizes are recorded and 22 of 53 when they are not, because the difference decides whether 17 t-map analyses can be converted. This owns no transform rules and repeats none of NiMARE's work. It introspects the fitted estimator, which already holds both answers: `inputs_["id"]` is what survived the dropping, and `estimator.dataset.images` is the table after the transform, so diffing it against the table that was submitted says which maps were converted rather than supplied. The per-analysis verdict goes to the log and to a TSV beside the results. Note for anyone extending this: NiMARE's image table has a column per type across the whole studyset, and an absent path reads as NaN, which is truthy. Testing truthiness reports every analysis as holding every type. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Dispatch a specification whose type is IBMA to `IBMAWorkflow`. Group comparisons are rejected up front, because no image-based estimator is pairwise. Analyses are no longer merged. `combine_analyses` is right for CBMA, where the point is to pool a study's coordinates, but for IBMA it concatenates a study's images into one analysis and the conversion to a Dataset keeps only one map per type, so every extra contrast is silently discarded -- along with the study grouping the dependence correction depends on. The estimator is built before the images are fetched. NiMARE raises on an argument it does not recognize and so does the resolver, so a stale specification should not first cost a studyset's worth of downloads. The diagnostic is named rather than constructed, matching the coordinate-based branches, so the workflow's `voxel_thresh` and `cluster_threshold` are what define clusters. `n_cores` is threaded through to the workflow, which is what parallelizes the diagnostics. Two outcomes are refused rather than reported as success. A studyset with nothing usable gets NiMARE's own message with the per-analysis account appended, since there is no fitted estimator to introspect at that point. And a result that is NaN at every voxel is rejected naming the input maps responsible: NiMARE counts a voxel as valid only where it is finite and non-zero, so a single empty upload empties an aggressive mask however good the rest are. That run looks successful and the maps it would upload hold no values. Uploads now skip a map whose value is None, mirroring `save_maps`, which never writes a file for one. Reading unconditionally failed at the very end of a run, after all the work. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
An IBMA is heavy because of its inputs, not its correction: it downloads a full-resolution map per contrast, holds every one of them in memory as a single masked array, and under aggressive_mask=False fits a separate model per group of voxels sharing a validity pattern. None of that applies to a coordinate-based run, which only ever handles foci. So the specification type escalates the task size on its own, whatever the corrector says. The fallbacks to the standard task now log why. `_fetch_meta_analysis` is an unauthenticated GET with a ten second timeout, and `?nested=true` is what makes the specification a dict rather than an id string; either failing silently sized an image-based run as though it were coordinate-based, which shows up afterwards as an unexplained slowdown or OOM with nothing recording the cause. Note that this buys CPU and memory but not disk: both task definitions set `ephemeral_storage_gib` only when the context value exceeds 20, and nothing overrides it. Measured, that is not currently tight -- a 27-study run used 156 MB -- so it is left alone. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Compose cannot create an image-based meta-analysis yet, so there is no bundle to download and no way to find out what breaks by running one. `simulate_ibma_bundle.py` assembles the bundle the frontend would produce -- real Neurostore studies, real NeuroVault maps, and a specification generated from the frontend's own meta_analysis_params.json -- and pushes it through the runner. Taking the specification from the committed config rather than writing it by hand is the point: it is what surfaced the places where the config, NiMARE and this runner disagreed. `--search` goes further and builds the studyset from a base-study query, using whatever comes back rather than selecting for studies the estimator can already use. The document records the bundle's shape, what the runner does differently for IBMA, what Neurostore still needs, and what was found and fixed in NiMARE along the way, with the measurements behind each claim. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
![github-code-quality[bot]](https://avatars.githubusercontent.com/u/9919?s=60&v=4){kind=small}
| number = float(value) | ||
| except (TypeError, ValueError): | ||
| return None | ||
| if number <= 0 or number != number: # reject non-positive and NaN |
The previous commit escalated every IBMA to the large task on the assumption
that its inputs were what made it expensive. Measured, they are not. Only the
FWE path is heavy, and for an image-based specification that means PermutedOLS's
permutation test -- the one image-based estimator with FWE_enabled -- so that is
the only case that escalates now.
On the "language" studyset with aggressive_mask=False, pinned to four vCPU to
emulate the standard task:
studies images bags cores wall time peak RSS
9 14 13 4 44 s 2.25 GiB
18 26 46 4 80 s 2.56 GiB
27 39 263 4 232 s 3.10 GiB
27 39 263 8 222 s 4.99 GiB
Ten percent of a 30 GiB limit at worst, and memory is mostly fixed overhead:
2.8x the images buys 1.36x the memory, roughly a 2 GiB baseline plus ~30 MB per
image, so the standard task holds around 900 images. The array that sounded
expensive -- every map held as one masked array -- is 36 MB; the 3 GiB is the 4D
concatenation during resampling plus one estimator copy per joblib worker.
Cores do not help. Doubling to eight took 4% off the wall time and added 61% to
peak memory, since each worker copies the estimator. Stouffers fits serially, the
voxel-bag loop is sequential, and Jackknife is evidently bound by something other
than CPU. Escalating would have paid four times the task cost for nothing.
`_is_fwe` and `_corrector_method` come out as helpers because both branches now
need the FWE check, and the method lookup was buried in nested early returns.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
ef561d4 removed the branch that logged `specification_not_nested`, leaving this test asserting a log line nothing emits. The premise it was defending is not worth defending: `_fetch_meta_analysis` always passes `?nested=true`, so the specification is reliably a dict rather than an id string. The `specification_unavailable` case is different and keeps its test -- a transient fetch failure is real, and the size chosen after one is a guess. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Two rows described code that no longer exists. `_report_coverage` ran NiMARE's transform to work out what was usable before fitting; that was replaced by introspecting the fitted estimator, so the row is now `_fit_image_based` and `_describe_coverage`, neither of which transforms anything. And the sizing row still claimed every IBMA goes to the large task, which 8440240 narrowed to FWE only. The prose sections were updated when those changes were made; the table was not. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
| self.nsc_key = meta_analysis.get("run_key") | ||
|
|
||
| def apply_filter(self, studyset, annotation): | ||
| def apply_filter(self, studyset, combine=True): |
jdkent
commented
Aug 21, 2026
jdkent
left a comment
jdkent
left a comment
Member
Author
There was a problem hiding this comment.
LGTM, some of the IBMA simulation/testing will be deleted once the workflow actually exists on our server.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
this is the attempt to officially support image based meta-analysis.
there is a lot of code in this pull request, it may be merged due to time constraints, but should be combed over soon.