Rework TLS to OpenSSL-native style (memory BIOs, explicit status codes)#8002
Rework TLS to OpenSSL-native style (memory BIOs, explicit status codes)#8002Copilot wants to merge 8 commits into
Conversation
Co-authored-by: achamayou <4016369+achamayou@users.noreply.github.com>
Co-authored-by: achamayou <4016369+achamayou@users.noreply.github.com>
Co-authored-by: achamayou <4016369+achamayou@users.noreply.github.com>
|
@copilot check previous CI runs, find out what failed and fix it. |
Co-authored-by: achamayou <4016369+achamayou@users.noreply.github.com>
Investigated the failing runs (ACI SNP Genoa/Milan + VMSS Virtual C): all hung in Root cause: in the new memory-BIO design, Fix (3a8... commit): drain outbound with |
Now that MbedTLS is gone from the tree, the TLS layer no longer needs to emulate its style. This reworks
ccf::tls::ContextandTLSSessionto use OpenSSL idioms directly, removing the callback machinery and the "negative return" overloading. Net ~330 fewer lines.Changes
context.h):set_bio()attaches plainBIO_s_memread/write BIOs to theSSL.TLSSessionfeeds inbound ciphertext viarecv()and drains outbound ciphertext viapending_write()/send()after each SSL operation. No moresend_callback/recv_callbackindirection.context.h,tls.h):handshake/read/write/closereturn0on success or an OpenSSLSSL_ERROR_*code (fromSSL_get_error); bytes transferred are reported through an out-param. Removes the negative-return hack. OnlyTLS_ERR_X509_VERIFYremains as a CCF-specific sentinel to distinguish a handshake cert-verification failure (reported by OpenSSL as a genericSSL_ERROR_SSL) so callers can treat it as an auth failure.tls_session.h):read/flush/do_handshake/closeswitch onSSL_ERROR_WANT_READ/WANT_WRITE/ZERO_RETURN/etc. instead of inspecting negated/overloaded returns. Outbound ciphertext is drained intopending_outand retried on ring-buffer backpressure. Deletes the obsoletehandle_send/handle_recv/*_callback_openssl/write_somepaths.PlaintextServer(plaintext_server.h): updated to the new virtual signatures over its own pair of memory BIOs.tls/test/main.cpp): rewritten from a threaded socketpair + callback BIOs to a single-threaded BIO pump; all existing cases preserved.tls/README.md,architecture/tls_internals.rst): describe the memory-BIO design and status-code contract; drop the MbedTLS-emulation and "future pure-OpenSSL" sections.Read path, before/after
Notes for reviewers
peer_cert_ok()onSSL_ERROR_SSL) is the one place where OpenSSL's status is reinterpreted.tls_testrun was not possible in the sandbox (requires OpenSSL 3.3 + Rust toolchain); logic was validated via syntax checks and standalone OpenSSL programs covering handshake, >16k records, close, and verify-failure. Please confirmtls_testis green in CI.