Integrate reproducible Day 2 engineering workflow

.gitignore

@@ -161,10 +161,19 @@ cython_debug/
 
 
 # Project-related
-datasets
-
-ct-runs
-ct-evals
+datasets/
+runs/
+checkpoints/
+ct-runs/
+ct-evals/
+wandb/
+*.pkl
+*.pt
+*.pth
+*.ckpt
+*.safetensors
+*.zip
+*.pyc
 
 slurm*
 debug.sh

README.md

@@ -2,7 +2,7 @@
 
 Pytorch implementation for [Easy Consistency Tuning (ECT)](https://www.notion.so/gsunshine/Consistency-Models-Made-Easy-954205c0b4a24c009f78719f43b419cc).
 
-ECT unlocks state-of-the-art (SoTA) few-step generative abilities through a simple yet principled approach. 
+ECT unlocks state-of-the-art (SoTA) few-step generative abilities through a simple yet principled approach.
 With minimal tuning costs, ECT demonstrates promising early results and scales with training FLOPs and model sizes.
 
 Try your own [Consistency Models](https://arxiv.org/abs/2303.01469)! You only need to fine-tune a bit. :D
@@ -13,7 +13,7 @@ Try your own [Consistency Models](https://arxiv.org/abs/2303.01469)! You only ne
 
 ## Introduction
 
-This repository is organized in a multi-branch structure, with each branch offering a minimal implementation for a specific purpose. 
+This repository is organized in a multi-branch structure, with each branch offering a minimal implementation for a specific purpose.
 The current branches support the following training protocols:
 
 - `main`: ECT on CIFAR-10. Best for understanding CMs and fast prototyping.
@@ -44,7 +44,7 @@ Prepare the dataset in the EDM's format. See a reference [here](https://github.c
 
 ## Training
 
-Run the following command to tune your SoTA 2-step ECM and match Consistency Distillation (CD) within 1 A100 GPU hour. 
+Run the following command to tune your SoTA 2-step ECM and match Consistency Distillation (CD) within 1 A100 GPU hour.
 
 ```bash
 bash run_ecm_1hour.sh 1 <PORT> --desc bs128.1hour
@@ -67,7 +67,7 @@ To enable fp16 and GradScaler, add the following arguments to your script:
 bash run_ecm_1hour.sh 1 <PORT> --desc bs128.1hour --fp16=True --enable_amp=True
 ```
 
-For more information, please refer to this [PR](https://github.com/locuslab/ect/pull/13). 
+For more information, please refer to this [PR](https://github.com/locuslab/ect/pull/13).
 Full support for Automatic Mixed Precision (AMP) will be added later.
 
 ## Evaluation
@@ -139,4 +139,3 @@ Feel free to drop me an email at zhengyanggeng@gmail.com if you have additional
   year={2024}
 }
 ```
-

artifacts/day1/dataset_check.txt

@@ -0,0 +1,24 @@
+Day 1 CIFAR-10 dataset verification
+===================================
+Source: https://dataset.bj.bcebos.com/cifar/cifar-10-python.tar.gz
+Source path: /mnt/ect_project/datasets/cifar-10-python.tar.gz
+Source size: 170498071 bytes
+Source MD5: c58f30108f718f92721af3b95e74349a
+Source SHA256: 6d958be074577803d12ecdefd02955f39262c83c16fe9348329d7fe0b5c001ce
+MD5 check: PASS
+tar gzip integrity: PASS
+
+EDM ZIP path: /mnt/ect_project/datasets/cifar10-32x32.zip
+EDM ZIP size: 166000134 bytes
+EDM ZIP SHA256: 2d4056e80de1a96fe16f2f58945c6c4710ecd9fc02e3cc7aa5b50513b7cdf389
+Expected ZIP SHA256 match: PASS
+ZipFile.testzip(): None
+PNG count: 50000
+dataset.json present: True
+dataset.json label count: 50000
+Random image 00025/img00025247.png: 32x32 RGB
+Random image 00049/img00049673.png: 32x32 RGB
+Random image 00027/img00027562.png: 32x32 RGB
+ImageFolderDataset length: 50000
+DataLoader image batch shape: (4, 3, 32, 32)
+Dataset validation: PASS

artifacts/day1/environment.txt

@@ -0,0 +1,24 @@
+Day 1 environment verification
+==============================
+Python: 3.9.18
+PyTorch: 2.3.0
+CUDA runtime: 12.1
+CUDA available: True
+GPU: NVIDIA A100-PCIE-40GB
+diffusers: 0.26.3
+accelerate: 0.27.2
+huggingface_hub: 0.23.4
+
+Import checks:
+click: OK
+requests: OK
+PIL: OK
+numpy: OK
+scipy: OK
+psutil: OK
+tqdm: OK
+imageio: OK
+pyspng: OK
+diffusers: OK
+accelerate: OK
+huggingface_hub: OK

artifacts/day1/smoke_test.txt

@@ -0,0 +1,19 @@
+Day 1 bootstrap smoke-test verification
+=======================================
+Date: 2026-07-15 (Asia/Shanghai)
+Branch: leader/day1-bootstrap
+
+Checks:
+- Python source compilation: PASS
+- Project module imports in the ect Conda environment: PASS
+- CUDA availability: PASS (NVIDIA A100-PCIE-40GB)
+- training_loop enable_amp parameter: PASS
+- --enable_amp=True dry-run: PASS
+- --amp=True compatibility alias dry-run: PASS
+- --enable_gradscaler=True compatibility alias dry-run: PASS
+- --metrics none parsed as an empty metric list: PASS
+- CIFAR-10 EDM ZIP SHA256: 2d4056e80de1a96fe16f2f58945c6c4710ecd9fc02e3cc7aa5b50513b7cdf389
+- ImageFolderDataset load: PASS (50000 RGB images, 32x32, uint8)
+- Git whitespace/error check: PASS
+
+No full training or FID run was performed as part of this bootstrap smoke test.

artifacts/day2/collaborator_audit.md

@@ -0,0 +1,128 @@
+# Day 2 collaborator audit
+
+Audit date: 2026-07-15 (Asia/Shanghai)
+
+This report records the read-only audit completed before local integration. Remote refs were
+fetched and inspected, but no collaborator branch was modified, merged, rebased, reset, or
+pushed.
+
+## Public baseline
+
+- `origin/main`: `4311059770f54821d151a9b0e1f76770a5f3930e`
+- `origin/leader/day1-bootstrap`: `4e33194777a347ea5286b5ec1d5c29a58c792d29`
+- Merge base: `origin/main`
+- Relative topology: leader ahead 4, behind 0
+- Changed files versus main: 8
+- `git diff --check`: passed
+- AMP CLI: `--enable_amp`, `--amp`, and `--enable_gradscaler` map to `enable_amp`
+- AMP propagation: `enable_amp` is passed into `training_loop`
+- GradScaler order: accumulation, `unscale_`, non-finite handling, `step`, `update`
+- GradScaler state: saved in numbered/latest training states and restored on resume
+- Non-AMP path: retains ordinary `optimizer.step()`
+- `metrics=none`: parses to `[]`; periodic and final metric calls are skipped
+- Compile and help checks: passed in the `ect` environment
+- Decision: **READY_TO_MERGE**
+
+No formal training or FID/KID evaluation was run during the audit.
+
+## codex/day1-engineering
+
+- SHA: `3cb1c52fc56f01942c84d535dddddffd99c3af47`
+- Commit: `Add reproducible Day 1 training workflow`
+- Base: `origin/main`; behind the public baseline by 4 commits and ahead by 1
+- Scope: 18 changed files, 1183 insertions, 14 deletions
+- Shell, Python, JSON, and whitespace syntax checks: passed
+
+Useful engineering deliverables:
+
+- `conda-matpool.yml`
+- `setup_env.sh`
+- `prepare_data.sh`
+- `download_checkpoint.sh`
+- `scripts/check_environment.py`
+- `scripts/verify_assets.py`
+- `scripts/verify_smoke_run.py`
+- `docs/DAY1_A.md`
+
+Files that must not replace the public baseline:
+
+- `ct_train.py`
+- `training/ct_training_loop.py`
+- `env.yml`
+- `.gitignore`
+- `README.md`
+
+The collaborator environment pinned huggingface-hub 0.20.3, while the validated public
+runtime used 0.23.4. The collaborator dataset ZIP SHA256
+`45e772cbbcb4ebb8657d383557fba2fd24cb929aeaab99fe1963b1462377da9d` differs from the
+public ZIP SHA256 `2d4056e80de1a96fe16f2f58945c6c4710ecd9fc02e3cc7aa5b50513b7cdf389`.
+Both reported the same byte size. `dataset_tool.py` stores variable ZIP entry timestamps, so
+the converted ZIP digest is informational; source MD5, CRC, image/label content, dimensions,
+color mode, and project loader behavior are authoritative.
+
+The collaborator smoke used global batch 10 and ran 100 fresh plus 100 resumed optimizer
+updates. It explicitly used `--fp16=False`, did not enable GradScaler, and was produced from a
+dirty tree whose recorded SHA was main rather than the collaborator commit. It is FP32
+engineering-connectivity evidence only. No checkpoint, training state, dataset ZIP, or large
+log was committed to Git.
+
+Decision: **ACCEPT_SELECTED_FILES + REQUIRES_REBASE + REJECT_WHOLE_BRANCH**.
+
+## wk/iniBR
+
+- SHA: `e3158d83112a2fffb0796a515becee699c73d3fa`
+- Ahead 1, behind 0 relative to main
+- Raw scope: 30 files, 5375 insertions and 5375 deletions
+- Ignoring end-of-line differences produces an empty diff with exit code 0
+- Typical files changed from pure LF to pure CRLF with identical logical lines
+- Semantic changes: none
+- Classification: **CRLF_ONLY_CHANGE**
+- Decision: **DO_NOT_MERGE**
+
+The member should create a new branch from `origin/leader/day1-bootstrap`, set
+`git config core.autocrlf input`, reapply only genuine work, and avoid cherry-picking the
+line-ending conversion commit.
+
+## edwards365
+
+- SHA: `4311059770f54821d151a9b0e1f76770a5f3930e`
+- Exactly equal to `origin/main`
+- Ahead 0, behind 0, changed files 0
+- Status: **NO_REMOTE_DELIVERABLE**
+
+No claim is made about local, unpushed work.
+
+## Missing collaborator
+
+No fourth collaborator branch was present after fetching and pruning remote-tracking refs.
+The member must provide a branch name and SHA and push the deliverable before role mapping or
+integration.
+
+## Role coverage
+
+| Role | Expected work | Current branch | Status |
+|------|---------------|----------------|--------|
+| A | Engineering and environment reproduction | `codex/day1-engineering` | Clear selective deliverable |
+| B | Official fixed ECT baseline | None evidenced | Missing remote deliverable |
+| C | Adaptive t-to-r schedule | None evidenced | Missing remote deliverable |
+| D | Unified sampling, evaluation, visualization | None evidenced | Missing remote deliverable |
+
+Only Role A can be mapped from content. The other visible collaborator refs contain either no
+semantic work or no commits, and one collaborator branch is absent.
+
+## Selective integration decision
+
+ACCEPT:
+
+- the eight engineering files listed above, subject to Day 2 path and validation hardening
+
+REWRITE:
+
+- `smoke_test.sh` as `scripts/smoke_engineering_100steps.sh`
+- collaborator JSON evidence and provenance rather than importing it as frozen evidence
+
+REJECT:
+
+- direct replacement of the five protected public-baseline files
+- whole-branch merge of `codex/day1-engineering`
+- any merge of `wk/iniBR`

artifacts/day2/integration_report.md

@@ -0,0 +1,126 @@
+# Day 2 local integration report
+
+Integration date: 2026-07-15 (Asia/Shanghai)
+
+## Scope and branch
+
+- Local branch: `leader/day2-local-integration`
+- Base: `origin/leader/day1-bootstrap@4e33194777a347ea5286b5ec1d5c29a58c792d29`
+- Selective source: `origin/codex/day1-engineering@3cb1c52fc56f01942c84d535dddddffd99c3af47`
+- Commit created: no
+- Push or PR created: no
+
+The worktree versions of `ct_train.py`, `training/ct_training_loop.py`, `env.yml`, `.gitignore`,
+and `README.md` were hash-checked against the public baseline immediately after extraction and
+were identical.
+
+## Imported files
+
+- `conda-matpool.yml`
+- `setup_env.sh`
+- `prepare_data.sh`
+- `download_checkpoint.sh`
+- `scripts/check_environment.py`
+- `scripts/verify_assets.py`
+- `scripts/verify_smoke_run.py`
+- `docs/DAY1_A.md`
+
+## Rewritten or adapted files
+
+- `scripts/smoke_engineering_100steps.sh`
+  - engineering-connectivity label, never a formal ECT baseline
+  - persistent `/mnt/ect_project` defaults
+  - `metrics=none`
+  - public-baseline `--enable_amp` CLI with FP16 and AMP enabled by default
+  - explicit statement that legacy FP32 evidence did not validate GradScaler
+  - separate `--check-only` and `--dry-run` paths
+- `scripts/verify_assets.py`
+  - official CIFAR-10 tarball MD5
+  - ZIP CRC
+  - exactly 50000 PNGs and 50000 one-to-one labels
+  - all images checked as 32x32 RGB
+  - `ImageFolderDataset` length, labels, dtype, shape, and sample reads
+  - ZIP SHA256 recorded without enforcing one conversion digest
+- `scripts/check_environment.py`
+  - aligned to the protected public environment rather than collaborator `env.yml`
+  - records all observed package versions and validates imports
+  - checks the frozen core package/runtime versions and CUDA
+- `setup_env.sh`, `prepare_data.sh`, and `download_checkpoint.sh`
+  - use `/mnt/ect_project` persistent defaults
+  - write optional reports under persistent `runs/day2`
+  - preserve check-only workflows
+- `scripts/verify_smoke_run.py`
+  - records engineering-only provenance
+  - validates expected batch, FP16, AMP, and disabled formal metrics
+- `docs/DAY1_A.md`
+  - documents persistent paths and the engineering/formal-baseline boundary
+  - removes dirty-main evidence claims
+  - does not claim a completed FP16 + GradScaler training validation
+
+## Validation results
+
+All commands below passed:
+
+- `bash -n setup_env.sh`
+- `bash -n prepare_data.sh`
+- `bash -n download_checkpoint.sh`
+- `bash -n scripts/smoke_engineering_100steps.sh`
+- `python -m compileall scripts training ct_train.py` with pycache redirected outside the repo
+- `python ct_train.py --help`
+- help checks for all imported/reworked scripts
+- `bash setup_env.sh --check-only`
+- `bash prepare_data.sh --check-only`
+- `bash download_checkpoint.sh --check-only`
+- `bash scripts/smoke_engineering_100steps.sh --check-only`
+- `bash scripts/smoke_engineering_100steps.sh --dry-run`
+
+The dry-run configuration reported:
+
+- dataset size: 50000
+- global batch: 10
+- network `use_fp16`: true
+- `enable_amp`: true
+- `metrics`: empty list
+- formal FID/KID: disabled
+- output directory was not created
+
+Asset verification reported:
+
+- source tarball MD5: `c58f30108f718f92721af3b95e74349a`
+- dataset ZIP CRC: passed
+- PNG count: 50000
+- label count: 50000
+- image format: 32x32 RGB
+- `ImageFolderDataset` length: 50000
+- dataset ZIP SHA256, informational only:
+  `2d4056e80de1a96fe16f2f58945c6c4710ecd9fc02e3cc7aa5b50513b7cdf389`
+- official transfer checkpoint SHA256:
+  `4d5dcc1f1d0d41c8934ad21626eeddbdc0460182becf9fc059a0631b1eedb4da`
+
+No optimizer update, complete engineering smoke, formal training, FID, or KID was run.
+
+## Protected files rejected from import
+
+- `ct_train.py`
+- `training/ct_training_loop.py`
+- `env.yml`
+- `.gitignore`
+- `README.md`
+- collaborator `artifacts/day1/*.json`
+- collaborator `artifacts/day1/README.md`
+- entire `codex/day1-engineering` branch
+- entire `wk/iniBR` branch
+
+## Remaining issues and decisions
+
+1. The integration is intentionally uncommitted and unpushed.
+2. No dynamic FP16 + GradScaler optimizer step or GradScaler state/resume test was run; the
+   successful dry run validates configuration propagation only.
+3. The official fixed ECT baseline, adaptive schedule, and unified evaluation work still have
+   no collaborator remote deliverables.
+4. `edwards365` still points to main, and one collaborator branch is still missing.
+5. Public `env.yml` does not explicitly pin huggingface-hub 0.23.4 even though the validated
+   runtime and environment checker freeze that compatibility version. Changing `env.yml` was
+   explicitly outside this selective integration manifest and requires a separate decision.
+6. `scripts/verify_smoke_run.py` was syntax-checked but cannot be exercised end-to-end without
+   running a new 100-step engineering smoke, which was explicitly prohibited in this phase.