Skip to content

Fail --fail-under=100 when a docstring is missing - #192

Open
Eljees wants to merge 1 commit into
econchick:masterfrom
Eljees:fix/186-fail-under-100-rounding
Open

Fail --fail-under=100 when a docstring is missing#192
Eljees wants to merge 1 commit into
econchick:masterfrom
Eljees:fix/186-fail-under-100-rounding

Conversation

@Eljees

@Eljees Eljees commented Aug 14, 2026

Copy link
Copy Markdown

Fixes #186.

_get_coverage rounds the coverage percentage to the number of decimals given on --fail-under, then compares:

round_to = -decimal.Decimal(fail_under_str).as_tuple().exponent
if self.config.fail_under > round(results.perc_covered, round_to):
    results.ret_code = 1

--fail-under=100 has no decimals, so round_to is 0 and the percentage is rounded to a whole number. Once a codebase is large enough that one missing docstring is worth less than half a percent, the rounded value reaches 100 and the run passes:

total=2000  covered=1999  missing=1  perc=99.95  ->  round(99.95, 0) == 100.0  ->  ret_code 0

The fix keeps the rounded value for the comparison, except when nodes are actually missing and rounding has reached 100 — then the exact percentage is used. Nothing else changes: rounding still works as intended for --fail-under=95.5, and a genuinely complete codebase still passes.

One correction to the issue title

The threshold is 2,000 nodes, not 4,000 functions. Rounding flips at 99.5% for --fail-under=100, and interrogate counts modules and classes as nodes alongside functions, so the false pass starts at 1 missing docstring out of 2,000 nodes. I've written the test at exactly that boundary.

Measured

python -m pytest on f35a9d6:

result
baseline, clean tree 228 passed, 13 skipped, 0 failed
new test, before the fix 1 failed, 2 passed — missing=1 total=2000 perc=99.95 ret=0
new test, after the fix 3 passed
full suite, after the fix 231 passed, 13 skipped, 0 failed

The new test is parametrised over three cases: 1999/2000 with one missing docstring (the report), 1999/2000 complete (must still pass), and 10 functions with one missing (the small-codebase path that already worked).

Project hooks from .pre-commit-config.yaml at their pinned versions: black 24.3.0, isort, flake8, mypy and interrogate on itself are all clean. Note that a newer black wants to reformat the import block of coverage.py, but it does that on an untouched checkout too — it's a version drift, not this change, and I left it alone.

docs/changelog.rst gains a Fixed entry under 1.8.0 (UNRELEASED).

AI-assisted (LLM used for drafting); the runs above are mine.

_get_coverage rounds the coverage percentage to the number of decimals
given on --fail-under before comparing. With 2,000 nodes and a single
missing docstring, 99.95% rounds up to 100.0 and the run passes, which
is the opposite of what --fail-under=100 asks for.

Keep the rounded value for the comparison, except when nodes are
actually missing and rounding has reached 100 - then compare against
the exact percentage.

Fixes econchick#186
@Eljees
Eljees force-pushed the fix/186-fail-under-100-rounding branch from 4d65896 to 609b2b3 Compare August 14, 2026 14:00
@Eljees

Eljees commented Aug 14, 2026

Copy link
Copy Markdown
Author

Both red checks here are pre-existing, and I can show that for each.

Read the Docs fails on every open pull request in this repository, including ones I have nothing to do with — #188, #189 and #190 all carry the same failed build.

pre-commit.ci fails in the pyupgrade hook, and the failure is a crash inside pyupgrade itself rather than a finding about the code:

File ".../pyupgrade/_main.py", line 297, in _fix_tokens
    tokenize.cookie_re.match(token.src)
TypeError: cannot use a bytes pattern on a string-like object

The pinned pyupgrade is v3.15.2 and pre-commit.ci now builds hooks on Python 3.14, where tokenize.cookie_re is a bytes pattern. It therefore crashes on any file it is handed. The clearest evidence is #190: it changes a single line of documentation and touches no Python at all, and pre-commit.ci fails there too. The runs that still show green (#188, #189) are from before the toolchain moved to 3.14. A pre-commit autoupdate would clear it, but that is a separate change and not mine to make here.

On my side the hooks are clean at their pinned versions. Running each id from .pre-commit-config.yaml against the two files this PR touches:

hook
interrogate pass
black 24.3.0 pass
pyupgrade 3.15.2 (on Python 3.12) pass
isort pass
flake8 pass
trailing whitespace, end of file, debug statements, check toml, check yaml pass
mypy pass

Test suite: 231 passed, 13 skipped, 0 failed — the baseline is 228 passed, 13 skipped, 0 failed.

@Eljees

Eljees commented Aug 14, 2026

Copy link
Copy Markdown
Author

Closing this as a duplicate of #193, which carries the same change and the same test. Both were opened within seconds of each other on my side, which should not have happened — apologies for the noise.

#193 is the one to review. Its description also gets the root cause right, where this one does not: --fail-under is a float, so str(100.0) is "100.0" and round_to is 1, not 0 as I wrote here.

For what it is worth on the red checks here: docs/readthedocs.org and pre-commit.ci currently fail on every open PR in this repository, including #190 (a one-line docs typo fix) and pre-commit.ci's own #180, so they are not coming from either of these branches.

@Eljees Eljees closed this Aug 14, 2026
@Eljees

Eljees commented Aug 14, 2026

Copy link
Copy Markdown
Author

Reopened. My previous comment had it backwards: #192 is the one to keep, and #193 is now closed. Sorry for the churn.

@Eljees Eljees reopened this Aug 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

False pass when --fail-under=100 and codebase has >4k functions

1 participant