Update `batch_single_subject.sh` to the latest version merged into `master` in the SCT repo by joshuacwnewton · Pull Request #35 · spinalcordtoolbox/sct_tutorial_data

joshuacwnewton · 2026-03-30T16:47:05Z

Description

This PR updates the batch_single_subject.sh script so that it produces the same CSV files that batch_processing.sh produces.

Here is what the old batch_processing.sh script tested:

t2/csa_c2c3.csv-0-MEAN(area)
t2/csa_pmj.csv-0-MEAN(area)
t2/csa_pam50.csv-38-MEAN(area)
t2s/csa_gm.csv-3-MEAN(area)
t2s/csa_wm.csv-3-MEAN(area)
mt/mtr_in_wm.csv-0-MAP()
dmri/fa_in_cst.csv-0-WA()
dmri/fa_in_cst.csv-1-WA()

These files are now generated by a copy of batch_single_subject.sh in the SCT repo. batch_processing.sh is no longer used.

Moving forward

The batch_single_subject.sh script now lives in SCT. When that script is updated, we will likely need to update sct_tutorial_data too (since there is a good chance we will need to regenerate the intermediate datasets used in the tutorials). So, the workflow will go something like this:

An SCT developer updates the web tutorials (e.g. by adding a new tutorial section).
The SCT CI will flag that there is now a mismatch in the commands that are present between the web tutorials and script.
The developer will update batch_single_subject.sh.
(Optional) If the changes cause some sort of difference to the intermediate outputs (e.g. t2_seg.nii.gz), then the CI will likely fail, because the numerical results of the pipeline will have changed. So, the dev will need to open a parallel PR in the sct_tutorial_data repo.
Once the changes are finalized, we will merge the sct_tutorial_data PR and generate a new set of outputs.
Then, back in the SCT repo, we will update the sct_tutorial_data.zip URL (as well as the cached results) and merge the PR.

This process ensures that we have a single source of truth (SCT repo) and that all of the parts of the tutorials (web tutorials, script, dataset, cached results, etc.) get updated in an orderly fashion.

I will merge this PR (step 4.) then create a new release (step 5.) and open a PR on the SCT repo, which should fix any remaining discrepancies in the results, see:

Investigate batch_single_subject.sh test failure discrepancies spinalcordtoolbox#5285

This commands should produce the necessary files to run `test_batch_processing.yml`, albeit likely with different results.

The previous commit added commands to `batch_single_subject.sh` to reproduce the existing `batch_processing.sh` metrics EXACTLY. But, to do so, many outdated commands and methods were needed. This commit removes the outdated methods, and tries to reproduce the relevant metric files using only the commands that were already present within `batch_single_subject.sh`. Doing this will fundamentally change the values we produced (due to different underlying methods), but it allows us to keep the pipeline we've already developed for the SCT Course over the years. Only minimal changes to the existing commands were necessary, and the changes should be painless to port over to the SCT Course. This provides a smooth transition from having 2 scripts (batch_processing.sh and batch_single_subject.sh) to just 1 script (batch_single_subject.sh), allowing us to get rid of the old script entirely. For documentation on what exactly changed here from old -> new, refer to the extensive comments in the previous commit.

joshuacwnewton · 2026-05-22T18:23:51Z

Figure out a way to keep the two repos in sync. (If an SCT pipeline changes, we will need to update batch_single_subject.sh in tandem.)

I am working on this now. It looks like there are more deviations between the tutorials and the batch_single_subject.sh script than I had realized. I'm going to push some changes to get the two more in sync.

Diff between tutorials and script

(venv_sct) joshua@jn/3508-merge-example-datasets-and-batch-scripts >
$ [~/repos/spinalcordtoolbox] # cd /home/joshua/repos/spinalcordtoolbox
python3 testing/batch_single_subject/extract_commands.py \
  --compare \
  --script-path ~/repos/sct_tutorial_data/single_subject/batch_single_subject.sh
INFO: Found 113 RST tutorial files in /home/joshua/repos/spinalcordtoolbox/documentation/source/user_section/tutorials
INFO: Found 93 unique invocations in tutorials
INFO: Reading script from /home/joshua/repos/sct_tutorial_data/single_subject/batch_single_subject.sh
INFO: Found 82 unique invocations in script

Legend: ' ' = same, '-' = tutorials only, '+' = script only

  sct_analyze_lesion -m t2_lesion_seg.nii.gz -s t2_sc_seg.nii.gz -f label -qc ~/qc_singleSubj
  sct_analyze_lesion -m t2_lesion_seg.nii.gz -s t2_sc_seg.nii.gz -qc ~/qc_singleSubj
- sct_compute_ascor -i-SC t2_compressed_seg.nii.gz -i-canal t2_compressed_canal_seg.nii.gz -perlevel 1 -o ascor.csv                                                                     
+ sct_compute_ascor -i-SC t2_seg.nii.gz -i-canal t2_canal_seg.nii.gz -perlevel 1 -o ascor.csv                                                                                           
  sct_compute_compression -i t2_compressed_seg.nii.gz -vertfile t2_compressed_seg_labeled.nii.gz -l t2_compressed_labels-compression.nii.gz -metric diameter_AP -normalize-hc 0 -o ap_ratio.csv
  sct_compute_compression -i t2_compressed_seg.nii.gz -vertfile t2_compressed_seg_labeled.nii.gz -l t2_compressed_labels-compression.nii.gz -metric diameter_AP -normalize-hc 1 -o ap_ratio_norm_PAM50.csv
  sct_compute_mtr -mt0 mt0_reg.nii.gz -mt1 mt1.nii.gz
  sct_create_mask -i dmri_dwi_mean.nii.gz -p centerline,dmri_dwi_mean_seg.nii.gz -size 35mm
- sct_create_mask -i fmri.nii.gz -p centerline,t2_seg_reg.nii.gz -size 35mm
+ sct_create_mask -i fmri.nii.gz -p centerline,t2_seg_reg.nii.gz -size 35mm -f cylinder
+ sct_create_mask -i mt1.nii.gz -p centerline,mt1_seg.nii.gz -size 35mm -f cylinder -o mask_mt1.nii.gz                                                                                  
- sct_create_mask -i mt1.nii.gz -p centerline,mt1_seg.nii.gz -size 35mm -o mask_mt1.nii.gz
- sct_create_mask -i t1.nii.gz -p centerline,t1_seg.nii.gz -size 35mm -o mask_t1.nii.gz
- sct_create_mask -i t2.nii.gz -p centerline,t2_seg.nii.gz -size 35mm -o mask_t2.nii.gz
- sct_crop_image -i IMAGE_UNIT1 -m IMAGE_seg -dilate 30x30x5
+ sct_crop_image -i fmri_moco_mean.nii.gz -m mask_fmri.nii.gz -b 0
- sct_crop_image -i t1.nii.gz -m mask_t1.nii.gz
- sct_crop_image -i t2.nii.gz -m mask_t2.nii.gz
  sct_deepseg -h
  sct_deepseg graymatter -i t2s.nii.gz -o t2s_gmseg.nii.gz -qc ~/qc_singleSubj
- sct_deepseg lesion_ms -i t2.nii.gz -qc ~/qc_singleSubj
+ sct_deepseg lesion_ms -i t2.nii.gz -qc ~/qc_singleSubj -qc-seg t2_seg.nii.gz -single-fold
- sct_deepseg lesion_ms_mp2rage -i IMAGE_UNIT1 -qc ~/qc_singleSubj
  sct_deepseg lesion_sci_t2 -i t2.nii.gz -qc ~/qc_singleSubj
- sct_deepseg rootlets -i t2.nii.gz -o t2_rootlets.nii.gz -qc ~/qc_singleSubj
+ sct_deepseg rootlets -i t2.nii.gz -qc ~/qc_singleSubj
+ sct_deepseg sc_canal_t2 -i t2.nii.gz -qc ~/qc_singleSubj
- sct_deepseg sc_canal_t2 -i t2_compressed.nii.gz -o t2_compressed_canal_seg.nii.gz -qc ~/qc_singleSubj                                                                                 
+ sct_deepseg sc_epi -i fmri_moco_mean_crop.nii.gz -qc ~/qc_singleSubj
  sct_deepseg sc_lumbar_t2 -i t2_lumbar.nii.gz -qc ~/qc_singleSubj
- sct_deepseg sc_lumbar_t2 -install
- sct_deepseg spinalcord -i IMAGE_UNIT1 -o IMAGE_seg
  sct_deepseg spinalcord -i dmri_dwi_mean.nii.gz -qc ~/qc_singleSubj
  sct_deepseg spinalcord -i dmri_moco_dwi_mean.nii.gz -qc ~/qc_singleSubj
  sct_deepseg spinalcord -i fmri_moco_mean.nii.gz -qc ~/qc_singleSubj/
  sct_deepseg spinalcord -i mt1.nii.gz -qc ~/qc_singleSubj
- sct_deepseg spinalcord -i t1.nii.gz -qc ~/qc_singleSubj
+ sct_deepseg spinalcord -i t1.nii.gz -qc ~/qc_singleSubj/
  sct_deepseg spinalcord -i t1_smooth.nii.gz -qc ~/qc_singleSubj
- sct_deepseg spinalcord -i t2.nii.gz
+ sct_deepseg spinalcord -i t2.nii.gz -o test/t2_seg_2.nii.gz
  sct_deepseg spinalcord -i t2.nii.gz -qc ~/qc_singleSubj
  sct_deepseg spinalcord -i t2_compressed.nii.gz -qc ~/qc_singleSubj
  sct_deepseg spinalcord -i t2s.nii.gz -qc ~/qc_singleSubj
  sct_deepseg spine -i t2.nii.gz -label-vert 1 -qc ~/qc_singleSubj
  sct_deepseg spine -i t2.nii.gz -qc ~/qc_singleSubj
- sct_deepseg_lesion -i t2.nii.gz -c t2
  sct_detect_pmj -i t2.nii.gz -c t2 -qc ~/qc_singleSubj
  sct_dmri_compute_dti -i dmri_moco.nii.gz -bval bvals.txt -bvec bvecs.txt
  sct_dmri_moco -i dmri.nii.gz -m mask_dmri_dwi_mean.nii.gz -bvec bvecs.txt -qc ~/qc_singleSubj -qc-seg dmri_dwi_mean_seg.nii.gz
  sct_dmri_separate_b0_and_dwi -i dmri.nii.gz -bvec bvecs.txt
- sct_download_data -d PAM50
- sct_download_data -d manual-correction -o manual-correction
+ sct_extract_metric -i dti_FA.nii.gz -f label/atlas -l 4,5 -method wa -z 2:14 -o fa_in_cst.csv                                                                                         
  sct_extract_metric -i dti_FA.nii.gz -f label/atlas -l 51 -method map -vert 2:5 -vertfile label/template/PAM50_levels.nii.gz -perlevel 1 -o fa_in_wm.csv
  sct_extract_metric -i mtr.nii.gz -f label/atlas -method map -l 4,5 -z 5:15 -o mtr_in_cst.csv
- sct_extract_metric -i mtr.nii.gz -f label/atlas -method map -l 51 -o mtr_in_wm.csv
+ sct_extract_metric -i mtr.nii.gz -f label/atlas -method map -l 51 -vert 2:5 -o mtr_in_wm.csv                                                                                          
  sct_extract_metric -i mtr.nii.gz -f label/atlas -method map -l 53 -vert 2:4 -vertfile label/template/PAM50_levels.nii.gz -o mtr_in_dc.csv
  sct_extract_metric -i t2s.nii.gz -f t2s_gmseg.nii.gz -method bin -z 2:12 -o t2s_value.csv -append 1
  sct_extract_metric -i t2s.nii.gz -f t2s_wmseg.nii.gz -method bin -z 2:12 -o t2s_value.csv
  sct_flatten_sagittal -i t1.nii.gz -s t1_seg.nii.gz
  sct_fmri_compute_tsnr -i fmri.nii.gz
  sct_fmri_compute_tsnr -i fmri_moco.nii.gz
  sct_fmri_moco -i fmri.nii.gz -m mask_fmri.nii.gz -qc ~/qc_singleSubj -qc-seg t2_seg_reg.nii.gz
- sct_label_utils -i mt1_seg.nii.gz -create-seg-mid 4 -o label_c3c4.nii.gz
- sct_label_utils -i t2.nii.gz -create-viewer 3 -o label_c2c3.nii.gz -msg "Click at the posterior tip of C2/C3 inter-vertebral disc"                                                    
- sct_label_utils -i t2.nii.gz -create-viewer 3,4,5 -o labels_disc.nii.gz -msg "Place labels at the posterior tip of each inter-vertebral disc. E.g. Label 3: C2/C3, Label 4: C3/C4, etc."                                                                                          
+ sct_label_utils -i t2_compressed.nii.gz -create 30,152,99,1.0:30,156,118,1.0:30,157,140,1.0:31,160,159,1.0 -o t2_compressed_labels-compression.nii.gz                                 
  sct_label_utils -i t2_lumbar.nii.gz -create 27,76,187,17:27,79,80,60 -o t2_lumbar_labels.nii.gz -qc ~/qc_singleSubj
- sct_label_utils -i t2_seg_labeled.nii.gz -vert-body 3,9 -o t2_labels_vert.nii.gz
  sct_label_utils -i t2_totalspineseg_discs.nii.gz -keep 3,9 -o t2_labels_vert.nii.gz
+ sct_label_vertebrae -i t2.nii.gz -s t2_seg.nii.gz -c t2 -discfile t2_totalspineseg_discs.nii.gz                                                                                       
- sct_label_vertebrae -i t2.nii.gz -s t2_seg.nii.gz -c t2 -initlabel label_c2c3.nii.gz -qc ~/qc_singleSubj                                                                              
- sct_label_vertebrae -i t2.nii.gz -s t2_seg.nii.gz -c t2 -qc ~/qc_singleSubj
  sct_label_vertebrae -i t2_compressed.nii.gz -s t2_compressed_seg.nii.gz -c t2 -qc ~/qc_singleSubj
  sct_maths -i fmri.nii.gz -mean t -o fmri_mean.nii.gz
  sct_maths -i t2s_seg.nii.gz -sub t2s_gmseg.nii.gz -thr 0 -o t2s_wmseg.nii.gz
+ sct_process_segmentation -i t2_seg.nii.gz -anat t2.nii.gz -vert 2:3 -discfile t2_totalspineseg_discs.nii.gz -perlevel 1 -o csa_perlevel.csv                                           
- sct_process_segmentation -i t2_seg.nii.gz -anat t2.nii.gz -vert 3:4 -discfile t2_totalspineseg_discs.nii.gz -perlevel 1 -o csa_perlevel.csv                                           
- sct_process_segmentation -i t2_seg.nii.gz -discfile t2_totalspineseg_discs.nii.gz -perslice 1 -normalize-PAM50 1 -o csa_PAM50.csv                                                     
+ sct_process_segmentation -i t2_seg.nii.gz -discfile t2_totalspineseg_discs.nii.gz -perslice 1 -normalize-PAM50 1 -o csa_pam50.csv                                                     
+ sct_process_segmentation -i t2_seg.nii.gz -pmj t2_pmj.nii.gz -pmj-distance 60 -pmj-extent 30 -o csa_pmj.csv -qc ~/qc_singleSubj -qc-image t2.nii.gz                                   
- sct_process_segmentation -i t2_seg.nii.gz -pmj t2_pmj.nii.gz -pmj-distance 64 -pmj-extent 30 -o csa_pmj.csv -qc ~/qc_singleSubj -qc-image t2.nii.gz                                   
+ sct_process_segmentation -i t2_seg.nii.gz -vert 2:3 -discfile t2_totalspineseg_discs.nii.gz -o csa_c2c3.csv                                                                           
- sct_process_segmentation -i t2_seg.nii.gz -vert 3:4 -discfile t2_totalspineseg_discs.nii.gz -o csa_c3c4.csv                                                                           
  sct_process_segmentation -i t2_seg.nii.gz -z 30:35 -discfile t2_totalspineseg_discs.nii.gz -perslice 1 -o csa_perslice.csv
- sct_process_segmentation -i t2s_gmseg.nii.gz -o csa_gm.csv -perslice 1 -angle-corr 0
+ sct_process_segmentation -i t2s_gmseg.nii.gz -o csa_gm_perslice.csv -perslice 1 -angle-corr 0                                                                                         
+ sct_process_segmentation -i t2s_gmseg.nii.gz -vert 2:5 -perlevel 1 -o csa_gm.csv -centerline t2s_seg.nii.gz -centerline-exclude-missing 1                                             
- sct_process_segmentation -i t2s_wmseg.nii.gz -o csa_wm.csv -perslice 1 -angle-corr 0
+ sct_process_segmentation -i t2s_wmseg.nii.gz -o csa_wm_perslice.csv -perslice 1 -angle-corr 0                                                                                         
+ sct_process_segmentation -i t2s_wmseg.nii.gz -vert 2:5 -perlevel 1 -o csa_wm.csv -centerline t2s_seg.nii.gz -centerline-exclude-missing 1                                             
  sct_qc -i fmri_tsnr.nii.gz -d fmri_moco_tsnr.nii.gz -s fmri_moco_mean_seg.nii.gz -p sct_fmri_compute_tsnr -qc ~/qc_singleSubj/
+ sct_qc -i t2.nii.gz -s t2_labels_vert.nii.gz -p sct_label_utils -qc ~/qc_singleSubj
  sct_qc -i t2_lumbar.nii.gz -s t2_lumbar_labels.nii.gz -p sct_label_utils -qc ~/qc_singleSubj
+ sct_register_multimodal -i "${SCT_DIR}"/data/PAM50/template/PAM50_t1.nii.gz -iseg "${SCT_DIR}"/data/PAM50/template/PAM50_cord.nii.gz -d dmri_moco_dwi_mean.nii.gz -dseg dmri_moco_dwi_mean_seg.nii.gz -initwarp ../t2/warp_template2anat.nii.gz -initwarpinv ../t2/warp_anat2template.nii.gz -owarp warp_template2dmri.nii.gz -owarpinv warp_dmri2template.nii.gz -param step=1,type=seg,algo=centermass:step=2,type=seg,algo=bsplinesyn,slicewise=1,iter=3 -qc ~/qc_singleSubj                                                                                       
+ sct_register_multimodal -i "${SCT_DIR}"/data/PAM50/template/PAM50_t2.nii.gz -iseg "${SCT_DIR}"/data/PAM50/template/PAM50_cord.nii.gz -d mt1.nii.gz -dseg mt1_seg.nii.gz -m mask_mt1.nii.gz -initwarp ../t2/warp_template2anat.nii.gz -param step=1,type=seg,algo=centermass:step=2,type=seg,algo=bsplinesyn,slicewise=1,iter=3 -owarp warp_template2mt.nii.gz -qc ~/qc_singleSubj                                                                                         
+ sct_register_multimodal -i "${SCT_DIR}"/data/PAM50/template/PAM50_t2.nii.gz -iseg "${SCT_DIR}"/data/PAM50/template/PAM50_cord.nii.gz -d mt1.nii.gz -dseg mt1_seg.nii.gz -param step=1,type=seg,algo=centermass:step=2,type=seg,algo=bsplinesyn,slicewise=1,iter=3 -m mask_mt1.nii.gz -initwarp ../t2s/warp_template2t2s.nii.gz -owarp warp_template2mt.nii.gz -qc ~/qc_singleSubj                                                                                         
+ sct_register_multimodal -i "${SCT_DIR}"/data/PAM50/template/PAM50_t2s.nii.gz -iseg "${SCT_DIR}"/data/PAM50/template/PAM50_cord.nii.gz -d fmri_moco_mean.nii.gz -dseg fmri_moco_mean_seg.nii.gz -param step=1,type=seg,algo=centermass:step=2,type=seg,algo=bsplinesyn,slicewise=1,iter=3:step=3,type=im,algo=syn,metric=CC,iter=3,slicewise=1 -initwarp ../t2/warp_template2anat.nii.gz -initwarpinv ../t2/warp_anat2template.nii.gz -owarp warp_template2fmri.nii.gz -owarpinv warp_fmri2template.nii.gz -qc ~/qc_singleSubj                                         
+ sct_register_multimodal -i "${SCT_DIR}"/data/PAM50/template/PAM50_t2s.nii.gz -iseg "${SCT_DIR}"/data/PAM50/template/PAM50_wm.nii.gz -d t2s.nii.gz -dseg t2s_wmseg.nii.gz -initwarp ../t2/warp_template2anat.nii.gz -initwarpinv ../t2/warp_anat2template.nii.gz -owarp warp_template2t2s.nii.gz -owarpinv warp_t2s2template.nii.gz -param step=1,type=seg,algo=rigid:step=2,type=seg,metric=CC,algo=bsplinesyn,slicewise=1,iter=3:step=3,type=im,metric=CC,algo=syn,slicewise=1,iter=2 -qc ~/qc_singleSubj                                                            
- sct_register_multimodal -i "${SCT_DIR}/data/PAM50/template/PAM50_t1.nii.gz" -iseg "${SCT_DIR}/data/PAM50/template/PAM50_cord.nii.gz" -d dmri_moco_dwi_mean.nii.gz -dseg dmri_moco_dwi_mean_seg.nii.gz -initwarp ../t2/warp_template2anat.nii.gz -initwarpinv ../t2/warp_anat2template.nii.gz -owarp warp_template2dmri.nii.gz -owarpinv warp_dmri2template.nii.gz -param step=1,type=seg,algo=centermass:step=2,type=seg,algo=bsplinesyn,slicewise=1,iter=3 -qc ~/qc_singleSubj                                                                                       
- sct_register_multimodal -i "${SCT_DIR}/data/PAM50/template/PAM50_t2.nii.gz" -iseg "${SCT_DIR}/data/PAM50/template/PAM50_cord.nii.gz" -d mt1.nii.gz -dseg mt1_seg.nii.gz -param step=1,type=seg,algo=centermass:step=2,type=seg,algo=bsplinesyn,slicewise=1,iter=3 -m mask_mt1.nii.gz -initwarp ../t2s/warp_template2t2s.nii.gz -owarp warp_template2mt.nii.gz -qc ~/qc_singleSubj                                                                                         
- sct_register_multimodal -i "${SCT_DIR}/data/PAM50/template/PAM50_t2s.nii.gz" -d fmri_moco_mean.nii.gz -dseg t2_seg_reg.nii.gz -param step=1,type=im,algo=syn,metric=CC,iter=5,slicewise=0 -initwarp ../t2/warp_template2anat.nii.gz -initwarpinv ../t2/warp_anat2template.nii.gz -owarp warp_template2fmri.nii.gz -owarpinv warp_fmri2template.nii.gz -qc ~/qc_singleSubj     
- sct_register_multimodal -i "${SCT_DIR}/data/PAM50/template/PAM50_t2s.nii.gz" -iseg "${SCT_DIR}/data/PAM50/template/PAM50_wm.nii.gz" -d t2s.nii.gz -dseg t2s_wmseg.nii.gz -initwarp ../t2/warp_template2anat.nii.gz -initwarpinv ../t2/warp_anat2template.nii.gz -owarp warp_template2t2s.nii.gz -owarpinv warp_t2s2template.nii.gz -param step=1,type=seg,algo=rigid:step=2,type=seg,metric=CC,algo=bsplinesyn,slicewise=1,iter=3:step=3,type=im,metric=CC,algo=syn,slicewise=1,iter=2 -qc ~/qc_singleSubj                                                            
- sct_register_multimodal -i $SCT_DIR/data/PAM50/template/PAM50_t2.nii.gz -iseg $SCT_DIR/data/PAM50/template/PAM50_cord.nii.gz -d mt1.nii.gz -dseg mt1_seg.nii.gz -m mask_mt1.nii.gz -initwarp ../t2/warp_template2anat.nii.gz -param step=1,type=seg,algo=centermass:step=2,type=seg,algo=bsplinesyn,slicewise=1,iter=3 -owarp warp_template2mt.nii.gz -qc ~/qc_singleSubj     
  sct_register_multimodal -i ../t2/t2_seg.nii.gz -d fmri_mean.nii.gz -identity 1
  sct_register_multimodal -i mt0.nii.gz -d mt1.nii.gz -dseg mt1_seg.nii.gz -m mask_mt1.nii.gz -param step=1,type=im,algo=slicereg,metric=CC -x spline -qc ~/qc_singleSubj
- sct_register_multimodal -i t1_crop.nii.gz -d ../t2/t2_crop.nii.gz -param step=1,type=im,algo=dl -qc ~/qc_singleSubj -dseg ../t2/t2_seg.nii.gz                                         
- sct_register_to_template -i mt1.nii.gz -s mt1_seg.nii.gz -ldisc label_c3c4.nii.gz -ref subject -param step=1,type=seg,algo=centermassrot:step=2,type=seg,algo=bsplinesyn,slicewise=1  
  sct_register_to_template -i t2.nii.gz -s t2_seg.nii.gz -ldisc t2_labels_vert.nii.gz -c t2 -qc ~/qc_singleSubj
+ sct_register_to_template -i t2.nii.gz -s t2_seg.nii.gz -ldisc t2_labels_vert.nii.gz -qc ~/qc_singleSubj -ofolder advanced_param -c t2 -param step=1,type=seg,algo=rigid:step=2,type=seg,metric=CC,algo=bsplinesyn,slicewise=1,iter=3:step=3,type=im,metric=CC,algo=syn,slicewise=1,iter=2                                                                                     
- sct_register_to_template -i t2.nii.gz -s t2_seg.nii.gz -lrootlet t2_rootlets.nii.gz -c t2 -ofolder rootlets-reg -qc ~/qc_singleSubj                                                   
  sct_register_to_template -i t2_lumbar.nii.gz -s t2_lumbar_seg.nii.gz -ldisc t2_lumbar_labels.nii.gz -c t2 -qc ~/qc_singleSubj -param step=1,type=seg,algo=centermassrot:step=2,type=seg,algo=bsplinesyn,metric=MeanSquares,iter=3,slicewise=0:step=3,type=im,algo=syn,metric=CC,iter=3,slicewise=0
- sct_run_batch -script process_data.sh -config config.yml
- sct_run_batch -script process_data.sh -path-data data/ -path-output output_correction -jobs 3                                                                                         
  sct_smooth_spinalcord -i t1.nii.gz -s t1_seg.nii.gz
  sct_warp_template -d dmri_moco_dwi_mean.nii.gz -w warp_template2dmri.nii.gz -qc ~/qc_singleSubj
  sct_warp_template -d fmri_moco_mean.nii.gz -w warp_template2fmri.nii.gz -a 0 -qc ~/qc_singleSubj
  sct_warp_template -d mt1.nii.gz -w warp_template2mt.nii.gz -a 1 -qc ~/qc_singleSubj
+ sct_warp_template -d mt1.nii.gz -w warp_template2mt.nii.gz -qc ~/qc_singleSubj
  sct_warp_template -d t2.nii.gz -w ../t2/warp_template2anat.nii.gz
  sct_warp_template -d t2.nii.gz -w warp_template2anat.nii.gz -a 0 -qc ~/qc_singleSubj
+ sct_warp_template -d t2s.nii.gz -w warp_template2t2s.nii.gz -qc ~/qc_singleSubj

This is needed to be able to regenerate the .csv files that we display in the tutorials (done automatically in CI)

## Description This PR removes the [`batch_processing.sh`](https://spinalcordtoolbox.com/stable/user_section/getting-started.html) script (which has been a part of SCT since v1.0.3 (2014-07-30)!) and replaces it with [`batch_single_subject.sh`](https://github.com/spinalcordtoolbox/sct_tutorial_data/blob/master/single_subject/batch_single_subject.sh) (the script we use in the SCT Course and tutorials). (Similarly, it removes the (essentially duplicate) `sct_example_data` dataset and replaces it with the identical (and more commonly used in tutorials/courses) `sct_tutorial_data`.) Tandem PR: - spinalcordtoolbox/sct_tutorial_data#35 This replacement was done in phases: ### Phase 1: Reproduce batch_processing.sh metrics with `batch_single_subject.sh` script - [x] Replace `sct_example_data` (old dataset) with `sct_tutorial_data` (current dataset) - [x] Put the exact "old commands" from `batch_processing.sh` into `batch_single_subject.sh` (to reproduce the metrics precisely) - [x] Replace the "old commands" with our most recent up-to-date processing pipeline (tweaked to produce the same filenames as the old commands) - [x] Remove the now-unused `batch_processing.sh` script and update all of our infrastructure to reference `sct_tutorial_data/single_subject/batch_single_subject.sh` instead ### Phase 2: Bringing `batch_single_subject.sh` into SCT and keeping it in sync > Note: This is only partially in-scope for this PR. But, I've included it here to avoid having to re-review all of the same commands in a follow-up PR. Now that `batch_single_subject.sh` is processed by SCT's CI, there remains the question: Should the script actually live in SCT? And if so, how do we make sure we're keeping the script in sync with the web tutorials? In a recent SCT meeting, we decided that yes, it should live in the SCT repo: - [x] Make a copy of the script in the SCT repo. - [x] Write a CI workflow that compares the commands between the script and the tutorials. - [x] Update both the tutorials and the script so that they contain precisely the same commands. - [x] Update https://spinalcordtoolbox.com/stable/user_section/getting-started.html, since having a separate page for `batch_processing.sh` is no longer necessary when the Tutorials page does the same job. ### Phase 3: Parallelizing the CI (one runner per tutorial) See: - #5239 ### Phase 4: Handling future `sct_tutorial_data` releases If the script lives in SCT, and will continue to be updated in the SCT repo alongside tutorial changes, then we will eventually need to port whatever changes we make in SCT to the `sct_tutorial_data` repo. Currently, I think it's fine to do this manually + once prior to each stable release, but this is open for discussion. ## Open discussion points - All addressed in meetings, I think! ## Linked issues Fixes #3508. --------- Co-authored-by: Mathieu Guay-Paquet <mathieu.guay-paquet@polymtl.ca>

…e_subject.sh` CI (#5297) Right now, we have 2 different workflows that validate our comprehensive MRI pipeline `batch_single_subject.sh`: 1. `test-batch-processing.yml`: Runs the full script on a fresh dataset. 2. `test-batch-processing-matrix.yml`: Runs each partial sub-section of the script in isolation on cached sub-datasets. Because we made some updates to the underlying methods in recent PRs (e.g. `sc-crop` used by default), workflow 1. gets updated immediately (since the intermediate seg files are regenerated every time) while worfklow 2. was still using the old/cached seg files. As a result, there is some drift: The PR workflow, the workflow on master, the cached dataset files, and the cached metric results may not all agree. This PR follows the process outlined in spinalcordtoolbox/sct_tutorial_data#35 (comment) to get everything back into sync. After this PR, both CI runs will pass against the updated ground truth. Given the possibility of drift between the two workflows (full vs. partial), we may need to be cognizant of when an intermediate file changes in a PR context and doesn't match the outputs that get passed along to the next sub-tutorial. For example, maybe we could add a simple post-run check that compares the hash of the generated outputs to the cached outputs used for each sub-tutorial. I think this would be fairly simple to do. But this is more of a follow-up step. ## Linked issues Fixes #5285.

joshuacwnewton force-pushed the jn/3508-use-sct_tutorial_data-for-batch_processing.sh branch 2 times, most recently from 865ae41 to f8ccf1d Compare March 30, 2026 18:28

joshuacwnewton mentioned this pull request Mar 30, 2026

Replace batch_processing.sh with batch_single_subject.sh spinalcordtoolbox/spinalcordtoolbox#5185

Merged

13 tasks

joshuacwnewton added 2 commits May 22, 2026 11:35

batch_single_subject.sh: Add legacy commands from batch_processing

4b4c88e

This commands should produce the necessary files to run `test_batch_processing.yml`, albeit likely with different results.

joshuacwnewton force-pushed the jn/3508-use-sct_tutorial_data-for-batch_processing.sh branch from f8ccf1d to 2ee29f0 Compare May 22, 2026 15:38

joshuacwnewton added 4 commits June 24, 2026 11:00

batch_single_subject.sh: Copy over changes from PR #5185

23d4fce

This is needed to be able to regenerate the .csv files that we display in the tutorials (done automatically in CI)

run_script.yml: Temporarily use upstream SCT branch

32695f3

[debug] update path to csv files in GHA yml

f0b7fbc

process_csvs.py: Update from c3c4 to c2c3

12b4d82

joshuacwnewton added 2 commits August 20, 2026 11:45

batch_single_subject.sh: Use version merged to master

4b85d61

run_script_and_create_release.yml: Fix checkout step

7e39bfd

joshuacwnewton changed the title ~~Minimally update batch_single_subject.sh to produce the same metric CSVs as batch_processing.sh~~ Update batch_single_subject.sh to the version that replaced batch_processing.sh in SCT Aug 20, 2026

joshuacwnewton changed the title ~~Update batch_single_subject.sh to the version that replaced batch_processing.sh in SCT~~ Update batch_single_subject.sh to the latest version merged into master in the SCT repo Aug 20, 2026

joshuacwnewton merged commit 6f182bc into master Aug 20, 2026
2 checks passed

joshuacwnewton deleted the jn/3508-use-sct_tutorial_data-for-batch_processing.sh branch August 20, 2026 17:49

joshuacwnewton mentioned this pull request Aug 21, 2026

Update sct_tutorial_data URL and cached results to fix batch_single_subject.sh CI spinalcordtoolbox/spinalcordtoolbox#5297

Merged

5 tasks

Navigation Menu

Uh oh!

Update `batch_single_subject.sh` to the latest version merged into `master` in the SCT repo - #35

Update `batch_single_subject.sh` to the latest version merged into `master` in the SCT repo#35
joshuacwnewton merged 8 commits into
masterfrom
jn/3508-use-sct_tutorial_data-for-batch_processing.sh

joshuacwnewton commented Mar 30, 2026 •

edited

Loading

Uh oh!

joshuacwnewton commented May 22, 2026 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

joshuacwnewton commented Mar 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Moving forward

Uh oh!

joshuacwnewton commented May 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

joshuacwnewton commented Mar 30, 2026 •

edited

Loading

joshuacwnewton commented May 22, 2026 •

edited

Loading