arch/arm/stm32: hardware TX timestamping via SO_TIMESTAMPING - #20148
daniel-p-carvalho wants to merge 8 commits into
Conversation
df6136c to
70cf835
Compare
|
|
||
| /* PTP Timestamping *********************************************************/ | ||
|
|
||
| #define SIOCG_TX_HW_TIMESTAMP _SIOC(0x0046) /* Get hardware TX timestamp */ |
There was a problem hiding this comment.
does Linux define the similar ioctl?
There was a problem hiding this comment.
No, Linux doesn't have a direct equivalent for any of the three - it exposes each NIC's PTP hardware clock as its own /dev/ptpN character device with a clockid_t, disciplined via the standard POSIX clock_adjtime() syscall rather than a dedicated network ioctl. TX timestamp retrieval similarly goes through SO_TIMESTAMPING + the socket error queue (MSG_ERRQUEUE/SCM_TIMESTAMPING), which NuttX's socket layer doesn't implement (MSG_ERRQUEUE is defined for API compat but unused, and there's a comment in inet_sockif.c noting SO_TIMESTAMPING as future work).
While looking into this I found drivers/timers/ptp_clock.c already implements a generic upper-half PTP hardware clock framework matching Linux's /dev/ptpN model closely - PTP_CLOCK_ADJTIME handling ADJ_FREQUENCY/ADJ_OFFSET via struct timex, the dynamic-clockid-to-fd conversion, etc. It has no registered lower-half driver anywhere in the tree yet, and its adjfine/adjphase ops map almost directly onto what this PR already implements in stm32_eth_ptp_adjust()/stm32_eth_ptp_adjphase(). I didn't know about it when I wrote this PR; migrating to it as a lower-half driver looks like the right direction for SIOCS_PTP_ADJFREQ/SIOCS_PTP_ADJPHASE, dropping both in favor of /dev/ptp0. SIOCG_TX_HW_TIMESTAMP is unrelated to that framework though - it's per-packet TX timestamp retrieval, not clock discipline, and there's no equivalent facility in ptp_clock.c for it; the real Linux equivalent there is SO_TIMESTAMPING/MSG_ERRQUEUE, which as noted isn't implemented in NuttX, so I don't see a smaller-scope alternative to keeping that one as a plain ioctl for now.
Would you rather I do the /dev/ptp0 migration for ADJFREQ/ADJPHASE in this PR, or land the current ioctls now and follow up separately?
There was a problem hiding this comment.
While looking into this I found drivers/timers/ptp_clock.c already implements a generic upper-half PTP hardware clock framework matching Linux's /dev/ptpN model closely - PTP_CLOCK_ADJTIME handling ADJ_FREQUENCY/ADJ_OFFSET via struct timex, the dynamic-clockid-to-fd conversion, etc. It has no registered lower-half driver anywhere in the tree yet, and its adjfine/adjphase ops map almost directly onto what this PR already implements in stm32_eth_ptp_adjust()/stm32_eth_ptp_adjphase(). I didn't know about it when I wrote this PR;
ptp driver framework is implemented by @Donny9 to let eth driver provide ptp clock in a portable way.
migrating to it as a lower-half driver looks like the right direction for SIOCS_PTP_ADJFREQ/SIOCS_PTP_ADJPHASE, dropping both in favor of /dev/ptp0.
yes, I think so.
SIOCG_TX_HW_TIMESTAMP is unrelated to that framework though - it's per-packet TX timestamp retrieval, not clock discipline, and there's no equivalent facility in ptp_clock.c for it; the real Linux equivalent there is SO_TIMESTAMPING/MSG_ERRQUEUE, which as noted isn't implemented in NuttX, so I don't see a smaller-scope alternative to keeping that one as a plain ioctl for now.
@wenquan2015 do some improvement in this area, which may simplify your code.
Would you rather I do the /dev/ptp0 migration for ADJFREQ/ADJPHASE in this PR, or land the current ioctls now and follow up separately?
it's better to algin your improvement inside both kernel and userspace to Linux design, so we can use other ptp library and utility in the future.
There was a problem hiding this comment.
@daniel-p-carvalho Hi, I've already implemented the SO_TIMESTAMPING + MSG_ERRQUEUE TX timestamp delivery and will submitt PRs to NuttX mainline this week after resolving the conflict. The full Linux-style flow is in place: sender tags iob->io_conn, driver captures the timestamp and loops the IOB back via RX path, userspace retrieves it with recvmsg(MSG_ERRQUEUE) + SCM_TIMESTAMPING cmsg.
@xiaoxiang781216 #20161
There was a problem hiding this comment.
Sounds good, I'll migrate SIOCS_PTP_ADJFREQ/SIOCS_PTP_ADJPHASE to /dev/ptp0 (registering a lower-half driver against drivers/timers/ptp_clock.c) in this PR.
For TX timestamp delivery, I'd rather hold off on migrating SIOCG_TX_HW_TIMESTAMP to SO_TIMESTAMPING until apache/nuttx#20161 merges. Should I keep this PR open as-is for that follow-up once apache/nuttx#20161 merges, mark it as draft in the meantime, or close it and submit a fresh PR later instead? Happy to do whichever fits your workflow better.
@wenquan2015 nice work, looking forward to building on top of it once it's in.
There was a problem hiding this comment.
it's better to split your pr into there new pr:
- the general network improvement or bugfix
- implement stm32 eth ptp driver
- implement stm32 eth hardware timestamp
it's fine to reuse this pr for item 2 or 3 if you want.
| /* PTP Timestamping *********************************************************/ | ||
|
|
||
| #define SIOCG_TX_HW_TIMESTAMP _SIOC(0x0046) /* Get hardware TX timestamp */ | ||
| #define SIOCS_PTP_ADJFREQ _SIOC(0x0047) /* Set PTP hardware frequency adjustment (ppb) */ |
|
|
||
| #define SIOCG_TX_HW_TIMESTAMP _SIOC(0x0046) /* Get hardware TX timestamp */ | ||
| #define SIOCS_PTP_ADJFREQ _SIOC(0x0047) /* Set PTP hardware frequency adjustment (ppb) */ | ||
| #define SIOCS_PTP_ADJPHASE _SIOC(0x0048) /* Set PTP hardware phase adjustment (ns) */ |
84e534c to
67293c8
Compare
Implement lower-half PTP hardware clock operations (struct ptp_lowerhalf_s and struct ptp_ops_s) in the STM32 Ethernet driver and register it with the generic PTP clock framework (drivers/timers/ptp_clock.c) to expose /dev/ptp0. Supported operations: - adjfine: adjust PTP clock frequency in parts per billion (ppb) - adjphase: adjust PTP clock phase via hardware TSSTU - adjtime: shift PTP clock time by signed delta in nanoseconds - gettime: atomic double-read of hardware timestamp registers - settime: initialize hardware timestamp counter via TSSTI - getres: return 1 ns clock resolution Also fix a sign bug in frequency trim where uint64_t addend promoted negative ppb adjustments to unsigned, corrupting frequency trim for crystals running faster than nominal. Follow-up to apache#20148 per review recommendation to use the standard POSIX /dev/ptp0 character driver instead of custom socket ioctls. Assisted-by: Gemini:gemini-3.8-pro Signed-off-by: Daniel P. Carvalho <danieloak@gmail.com>
Implement lower-half PTP hardware clock operations (struct ptp_lowerhalf_s and struct ptp_ops_s) in the STM32 Ethernet driver and register it with the generic PTP clock framework (drivers/timers/ptp_clock.c) to expose /dev/ptp0. Supported operations: - adjfine: adjust PTP clock frequency in parts per billion (ppb) - adjphase: adjust PTP clock phase via hardware TSSTU - adjtime: shift PTP clock time by signed delta in nanoseconds - gettime: atomic double-read of hardware timestamp registers - settime: initialize hardware timestamp counter via TSSTI - getres: return 1 ns clock resolution Also fix a sign bug in stm32_eth_ptp_adjust() where uint64_t addend promoted negative ppb adjustments to unsigned, corrupting frequency trim for crystals running faster than nominal. Follow-up to apache#20148 per review recommendation to use the standard POSIX /dev/ptp0 character driver instead of custom socket ioctls. Assisted-by: Gemini:gemini-3.8-pro Signed-off-by: Daniel P. Carvalho <danieloak@gmail.com>
Implement lower-half PTP hardware clock operations (struct ptp_lowerhalf_s and struct ptp_ops_s) in the STM32 Ethernet driver and register it with the generic PTP clock framework (drivers/timers/ptp_clock.c) to expose /dev/ptp0. Supported operations: - adjfine: adjust PTP clock frequency in parts per billion (ppb) - adjphase: adjust PTP clock phase via hardware TSSTU - adjtime: shift PTP clock time by signed delta in nanoseconds - gettime: atomic double-read of hardware timestamp registers - settime: initialize hardware timestamp counter via TSSTI - getres: return 1 ns clock resolution Also fix a sign bug in stm32_eth_ptp_adjust() where uint64_t addend promoted negative ppb adjustments to unsigned, corrupting frequency trim for crystals running faster than nominal. Follow-up to apache#20148 per review recommendation to use the standard POSIX /dev/ptp0 character driver instead of custom socket ioctls. Assisted-by: Gemini:gemini-3.8-pro Signed-off-by: Daniel P. Carvalho <danieloak@gmail.com>
Implement lower-half PTP hardware clock operations (struct ptp_lowerhalf_s and struct ptp_ops_s) in the STM32 Ethernet driver and register it with the generic PTP clock framework (drivers/timers/ptp_clock.c) to expose /dev/ptp0. Supported operations: - adjfine: adjust PTP clock frequency in parts per billion (ppb) - adjphase: adjust PTP clock phase via hardware TSSTU - adjtime: shift PTP clock time by signed delta in nanoseconds - gettime: atomic double-read of hardware timestamp registers - settime: initialize hardware timestamp counter via TSSTI - getres: return 1 ns clock resolution Also fix a sign bug in stm32_eth_ptp_adjust() where uint64_t addend promoted negative ppb adjustments to unsigned, corrupting frequency trim for crystals running faster than nominal. Follow-up to apache#20148 per review recommendation to use the standard POSIX /dev/ptp0 character driver instead of custom socket ioctls. Assisted-by: Gemini:gemini-3.8-pro Signed-off-by: Daniel P. Carvalho <danieloak@gmail.com>
|
Status update, and a heads-up on what will change in this PR. Following the review here (replace the custom socket ioctls with the generic PTP hardware clock, and use
What is left for this PR is the STM32 hardware TX timestamping itself, delivered through The comments above about the ioctl definitions and the |
Every other timer driver block in this Make.defs sets TMRDEPPATH and TMRVPATH so DEPPATH/VPATH include this directory. CONFIG_PTP_CLOCK and CONFIG_PTP_CLOCK_DUMMY were the only two missing it, leaving ptp_clock.c/ptp_clock_dummy.c unreachable via VPATH and without a generated dependency file when no other timer driver in this file is also selected. Assisted-by: Claude:claude-sonnet-5 Signed-off-by: Daniel P. Carvalho <danieloak@gmail.com> (cherry picked from commit 6d38122)
stm32_receive() called pkt_input() before stm32_eth_ptp_convert_rxtime(), so every packet handed to a packet socket carried the previous frame's RX timestamp instead of its own in dev->d_rxtime. Reorder so the timestamp is converted first. Assisted-by: Claude:claude-sonnet-5 Signed-off-by: Daniel P. Carvalho <danieloak@gmail.com> (cherry picked from commit 9bfa20d)
Set ETH_MACCR_ROD unconditionally when configuring the MAC. In half-duplex mode the MAC otherwise reflects every frame it transmits back to its own receiver, flooding the receive path with our own traffic right as a genuine reply arrives. The bit has no effect in full-duplex (confirmed on our hardware: fduplex=1), so setting it unconditionally is safe and changes nothing observable for boards already running full-duplex. The sibling stm32f7 driver has the same gap (ETH_MACCR_ROD cleared but never set) and stm32h7's equivalent ETH_MACCR_DO bit has the same issue; both are left out of scope here since only m3m4_v1 hardware was available to validate against. Assisted-by: Claude:claude-sonnet-5 Signed-off-by: Daniel P. Carvalho <danieloak@gmail.com> (cherry picked from commit 41536cb)
The MAC hardware counter is the PTP clock reference. Delivering its raw timestamp directly (instead of synthesizing one against CLOCK_REALTIME, which starts at an arbitrary boot-time phase) lets the PTP daemon close the feedback loop and phase-lock the MAC counter - and therefore the physical PPS output - to the master. Assisted-by: Gemini:gemini-3.8-flash-medium Signed-off-by: Daniel P. Carvalho <danieloak@gmail.com>
Implement lower-half PTP hardware clock operations (struct ptp_lowerhalf_s and struct ptp_ops_s) in the STM32 Ethernet driver and register it with the generic PTP clock framework (drivers/timers/ptp_clock.c) to expose /dev/ptp0. Supported operations: - adjfine: adjust PTP clock frequency in parts per billion (ppb) - adjphase: adjust PTP clock phase via hardware TSSTU - adjtime: shift PTP clock time by signed delta in nanoseconds - gettime: atomic double-read of hardware timestamp registers - settime: initialize hardware timestamp counter via TSSTI - getres: return 1 ns clock resolution Also fix a sign bug in stm32_eth_ptp_adjust() where uint64_t addend promoted negative ppb adjustments to unsigned, corrupting frequency trim for crystals running faster than nominal. Follow-up to apache#20148 per review recommendation to use the standard POSIX /dev/ptp0 character driver instead of custom socket ioctls. Assisted-by: Gemini:gemini-3.8-pro Signed-off-by: Daniel P. Carvalho <danieloak@gmail.com>
Variable conn is declared and initialized in append_timestamping() but never referenced, triggering -Wunused-variable compiler warning. Assisted-by: Gemini:gemini-3.8-pro Signed-off-by: Daniel P. Carvalho <danieloak@gmail.com>
…G loopback Implement hardware TX timestamping support for STM32 Ethernet MAC (stm32_eth_m3m4_v1.c) following the upstream SO_TIMESTAMPING loopback architecture (PR apache#20161). When an outgoing packet is flagged with SO_TIMESTAMPING (dev->d_iob->io_conn != NULL): - Clone the IOB and hold a reference in priv->tx_meta[txindex] - Set ETH_TDES0_TTSE on the transmit DMA descriptor - On transmission completion (stm32_freeframe), retrieve the hardware timestamp from TDES6/TDES7, convert to timespec via ptp_to_timespec(), and enqueue the clone onto priv->tx_tstampq - Deliver pending TX timestamp clones back to netdev RX path in stm32_receive using pkt_input(), where net/pkt intercepts the frame and delivers it to userspace via recvmsg(..., MSG_ERRQUEUE) - Properly drain pending queues and clones on interface down (stm32_ifdown) Assisted-by: Gemini:gemini-3.8-pro Signed-off-by: Daniel P. Carvalho <danieloak@gmail.com>
- Under CONFIG_NET_PROMISCUOUS, forward all control frames (ETH_MACFFR_PCF_ALL) instead of only non-PAUSE ones, so link-local PTP multicast reaches the DMA. - Move ptp_to_timespec() above its first user so the TX timestamp path can call it. - stm32_receive(): do not log frames already delivered to packet sockets (PTP, IPv6) as "Dropped, Unknown type". - stm32_tx_tstamp_flush(): clear io_conn before freeing the looped-back IOB. Signed-off-by: Daniel P. Carvalho <danieloak@gmail.com> Assisted-by: Claude:claude-sonnet-5
67293c8 to
ba8190d
Compare
|
I have updated this PR as announced above, so the diff and the older ioctl-related threads are now outdated. It now contains only the STM32 hardware TX timestamping, delivered through This PR must be merged after #20180. The first six commits are the same commits as #20180, which this PR needs because the TX and RX timestamps have to come from the same hardware counter, so they show up in this diff until #20180 is merged. Only the last two commits are new here. I will rebase it once #20180 is in, and only the last two commits need review here. |
Implement lower-half PTP hardware clock operations (struct ptp_lowerhalf_s and struct ptp_ops_s) in the STM32 Ethernet driver and register it with the generic PTP clock framework (drivers/timers/ptp_clock.c) to expose /dev/ptp0. Supported operations: - adjfine: adjust PTP clock frequency in parts per billion (ppb) - adjphase: adjust PTP clock phase via hardware TSSTU - adjtime: shift PTP clock time by signed delta in nanoseconds - gettime: atomic double-read of hardware timestamp registers - settime: initialize hardware timestamp counter via TSSTI - getres: return 1 ns clock resolution Also fix a sign bug in stm32_eth_ptp_adjust() where uint64_t addend promoted negative ppb adjustments to unsigned, corrupting frequency trim for crystals running faster than nominal. Follow-up to #20148 per review recommendation to use the standard POSIX /dev/ptp0 character driver instead of custom socket ioctls. Assisted-by: Gemini:gemini-3.8-pro Signed-off-by: Daniel P. Carvalho <danieloak@gmail.com>
Summary
The STM32 Ethernet MAC can capture a timestamp for every transmitted frame, but the driver only timestamped received frames. A PTP daemon that measures the path delay with the peer-to-peer mechanism needs the transmit time of its
Pdelay_Reqfrom the hardware. Taking it in software, aftersendmsg()returns, gave a negative peer delay on the board I tested with, which the daemon rejects.This PR adds hardware TX timestamping to the legacy STM32 Ethernet driver, delivered through the generic
SO_TIMESTAMPINGmechanism that #20161 added to the network stack: the driver clones the transmitted frame, attaches the hardware timestamp and loops it back to theAF_PACKETsocket, whererecvmsg(..., MSG_ERRQUEUE)returns it. No new socket option, ioctl or change in the network stack is needed. The TX timestamp is the raw MAC counter, the same time base as the RX timestamp and/dev/ptp0from #20180.This PR was reworked after review of its first version. The custom socket ioctls (
SIOCG_TX_HW_TIMESTAMP,SIOCS_PTP_ADJFREQ,SIOCS_PTP_ADJPHASE) are gone, replaced bySO_TIMESTAMPINGfor TX and by the PTP hardware clock in #20180. The other parts of the original 13 commits were merged separately (#20172, #20173, #20195) or moved to #20180.Commits
The first six are the commits of #20180, unchanged, and are reviewed there:
drivers/timers: Add TMRDEPPATH/TMRVPATH for PTP clock drivers.arch/arm/stm32: Convert RX hardware timestamp before pkt_input().arch/arm/stm32: Disable reception of self-transmitted frames.arch/arm/stm32: deliver direct hardware counter timestamps for PTP.arch/arm/stm32: implement PTP hardware clock driver (/dev/ptp0)net/pkt: remove unused variable conn in append_timestampingNew in this PR:
arch/arm/stm32: implement hardware TX timestamping via SO_TIMESTAMPING loopback.AddsCONFIG_STM32_ETH_TIMESTAMP_TX. When an outgoing frame is flagged for timestamping, the driver keeps a clone of it, setsETH_TDES0_TTSEin the transmit descriptor and, when the transmission completes, reads the timestamp fromTDES6/TDES7, converts it withptp_to_timespec()and queues the clone. The queued clones are handed back to the network stack fromstm32_receive()throughpkt_input(), wherenet/pktdelivers them to the error queue of the socket. Pending clones are released when the interface goes down.arch/arm/stm32: fix PTP multicast filter and RX/TX frame routing.Fixes found while running PTP with the change above. WithCONFIG_NET_PROMISCUOUSall control frames are forwarded (ETH_MACFFR_PCF_ALL), so the link-local PTP multicast reaches the DMA. Frames already delivered to packet sockets are no longer logged asDropped, Unknown type. Theio_connof a looped-back clone is cleared before its IOB is freed.ptp_to_timespec()is moved above its first user.Impact
CONFIG_STM32_ETH_TIMESTAMP_TX(default n, needsSTM32_ETH_PTP,NET_TIMESTAMPand enhanced descriptors), which only exists for the legacy STM32 MAC driver. Boards that do not enable it are not affected by commit 7. Commit 8 touches the receive path of the same driver, but only the promiscuous-mode filter and a log message./dev/ptp0as its clock.apps/netutils/ptpddoes that with-H -p /dev/ptp0.appsside,ptpdreading the TX timestamp from the error queue, is a follow-up to netutils/ptpd: IEEE 1588 P2P delay mechanism, hardware clock phase-lock and outlier rejection nuttx-apps#3782.Testing
Built for
stm32f4discovery:ethraw, a custom board configuration with the STM32F407 MAC and a DP83848 PHY, withCONFIG_STM32_ETH_TIMESTAMP_TX=y, without errors or new warnings../tools/checkpatch.sh -g upstream/master..HEADpasses.On hardware, against a GNSS-referenced
ptp4lGrandmaster using the IEC/IEEE 61850-9-3 profile, overAF_PACKETand through a Fast Ethernet switch, withptpd -H -p /dev/ptp0 -B -Pfrom apache/nuttx-apps#3782 plus the follow-up that reads the TX timestamps:path_delay9265.3 ns with σ = 2.5 ns over a 20 minute run (159 samples, no failed queries), phase offset about -2.2 µs, drift about -79.95 ppm.path_delaybetween 9.26 and 9.33 µs.ptpdtaking the transmit time in software, the peer delay was negative (about -18 µs) and was rejected, so the path delay stayed at 0.These runs were made on the same commits before the rebase onto the current
master. The rebase dropped one commit that had already been merged (#20195) and the files touched by this series are identical to the ones that were tested, but the rebased branch was only built, not run again. Other STM32 boards that use this driver were not tested.