fix!: mask generation should use sha1 for rsa-oaep-mgf1p - #140
Open
elbuo8 wants to merge 14 commits into
Open
Conversation
elbuo8
commented
Jul 31, 2026
elbuo8
force-pushed
the
fix/rsa-oaep-mgf1
branch
from
July 31, 2026 20:20
11da941 to
52a395d
Compare
Node's crypto cannot set the MGF1 digest separately from the OAEP digest, so implement EME-OAEP over the raw RSA primitive. Verified against OpenSSL-generated ciphertext in both directions. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…sZero/isOne Addresses code-review findings for Task 1: Finding 1 (Important): Hoist key parsing and ciphertext-length validation above the try-catch so operational failures (bad PEM, wrong key type) surface immediately rather than masquerading as OAEP decode errors. Only the modular exponentiation remains masked so that the RFC 8017 7.1.2 checks stay indistinguishable from outside. Finding 2 (Important): Add four tests that pin each RFC 8017 §7.1.2 rejection guard (leading byte, lHash mismatch, no separator, non-zero PS). Mutation testing confirms each guard is now covered. Minor: Replace isZero/isOne ternaries with branch-free arithmetic form to eliminate data-dependent branching in the separator scan. Minor: Derive k from the fixture key instead of hardcoding 256, so max- length test cases remain correct if the key changes. All 77 tests passing. OpenSSL interop verified. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
An absolute XPath anchored under KeyInfo/EncryptedKey missed documents using EncryptedData/KeyInfo/RetrievalMethod, silently defaulting oaepHash to sha1. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
BREAKING CHANGE: the rsa-oaep-mgf1p identifier fixes MGF1 to SHA-1 per XML-Enc 1.1 section 5.5.2, and DigestMethod selects only the OAEP message digest. Ciphertext produced by this library with keyEncryptionDigest sha256 or sha512 (v3.1.0 through v5.0.0) used MGF1 matching the digest and no longer decrypts; it was never interoperable with compliant peers.
BREAKING CHANGE: encrypting with keyEncryptionDigest sha256 or sha512 under rsa-oaep-mgf1p now wraps the key with MGF1-SHA1, as the identifier requires. Peers that adapted to the previous non-compliant output must switch to the xmlenc11#rsa-oaep identifier with keyEncryptionMgf.
- Parse XML to detect MGF elements instead of string matching base64 - Pin wrapped key length to 32 bytes (aes256-gcm key size) Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Adds the XML-Enc 1.1 key transport identifier whose xenc11:MGF child selects the mask generation function, which is how a digest other than SHA-1 for MGF1 is expressed. Unknown MGF URIs are rejected rather than defaulting to SHA-1. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
XML-Enc 1.1 5.5.2 permits OAEPparams as the base64 PSourceAlgorithm value. Documents carrying one previously failed with an opaque OAEP decoding error. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
semantic-release derives the version from the commit history; naming 5.1.0 in prose was both unverifiable and wrong for a breaking change.
- OAEPparams now emits as e:OAEPparams (xenc namespace), not bare OAEPparams (inherited xmldsig namespace). Schema-validating peers require it in xenc per EncryptionMethodType. - Test updated to assert namespaceURI and element ordering via DOM. - MGF_ALGORITHMS now defined once in lib/mgf-algorithms.js. Template derives short→URI map, ensuring sha1 emits xmlenc11#mgf1sha1 (not the legacy alias). Drift now caught: unmapped short names throw instead of emitting bare Algorithm="name". - Removed || keyEncryptionMgf fallback that silently serialized non-URIs. Unmapped values now fail loudly. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Add two tests to verify publicEncryptOaep randomizes the OAEP seed: 1. Twenty encryptions of identical plaintext produce distinct ciphertexts. 2. The maskedSeed itself varies (extracted via RSA_NO_PADDING decrypt). This pins IND-CPA security — a constant seed would make OAEP deterministic and leak plaintext equality. The mutation test confirms: replacing crypto.randomBytes(hLen) with Buffer.alloc(hLen) drops the count from 100 passing to 98 passing (2 failing), and restoring it returns to 100. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
MGF_URI_FOR_EMIT was derived from a map that also held the decrypt-only xmlenc#MGF1withSHA1 alias, with first-occurrence-wins deciding which URI sha1 emits. Reordering the literal would have silently started emitting a non-normative URI with every test still green. The emit map now derives from the canonical 5.5.2 list alone, and a test pins that every emitted value is an xmlenc11#mgf1* URI. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
elbuo8
force-pushed
the
fix/rsa-oaep-mgf1
branch
from
August 3, 2026 13:45
52a395d to
7e94135
Compare
CoreyGMartin
approved these changes
Aug 4, 2026
adamjmcgrath
changed the title
Comment on lines
+83
to
+85
| return callback(new Error('keyEncryptionMgf is only supported with http://www.w3.org/2009/xmlenc11#rsa-oaep; ' | ||
| + options.keyEncryptionAlgorithm + ' fixes the mask generation function')); | ||
| } |
Contributor
There was a problem hiding this comment.
nit: Can the message be framed a little differently? Something along the lines of,
keyEncryptionMgf is only supported with keyEncryptionAlgorithm:http://www.w3.org/2009/xmlenc11#rsa-oaep; The provided keyEncryptionAlgorithm + options.keyEncryptionAlgorithm + ' defines the mask generation function'
| return callback(new Error('keyEncryptionMgf is only supported with http://www.w3.org/2009/xmlenc11#rsa-oaep; ' | ||
| + options.keyEncryptionAlgorithm + ' fixes the mask generation function')); | ||
| } | ||
|
|
Contributor
There was a problem hiding this comment.
Should we add a similar validation for options.keyEncryptionOaepParams as we have for keyEncryptionMgf above?
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
By submitting a PR to this repository, you agree to the terms within the Auth0 Code of Conduct. Please see the contributing guidelines for how to create and submit a high-quality PR for this repo.
Description
Encrypted xml using the rsa-oaep-mgf1p key-transport algorithm with a SHA-256 (or SHA-512) DigestMethod fail to decrypt with error:02000079:rsa routines::oaep decoding error, even when the decryption key is correct.
For XML Encryption Syntax and Processing the XML-Enc 1.1 spec (§5.5.2) fixes the mask generation function to MGF1-SHA1; the DigestMethod selects only the OAEP message digest. Node's crypto.privateDecrypt cannot set MGF1 independently of the
OAEP digest — it forces MGF1 to match the message digest. So when the digest is SHA-256, Node attempts MGF1-SHA256 and rejects the (spec-compliant) MGF1-SHA1 ciphertext. This PR fixes this issue.
References
Testing
Checklist