feat: Add new thread_pool component built on top of Task#673
Open
guo-max wants to merge 19 commits into
Open
Conversation
|
✅Static analysis result - no issues found! ✅ |
Contributor
There was a problem hiding this comment.
Pull request overview
Adds a new thread_pool component to ESPP, providing an espp::ThreadPool API backed by espp::Task workers, plus an ESP-IDF example project demonstrating job submission and basic stats reporting.
Changes:
- Introduces
espp::ThreadPoolimplementation with a configurable worker count, optional bounded queue, and submission/statistics APIs. - Adds an ESP-IDF example (
components/thread_pool/example) showing how to run multiple jobs and wait for completion. - Adds component packaging metadata (
idf_component.yml) and CMake integration for the new component.
Reviewed changes
Copilot reviewed 10 out of 10 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
| components/thread_pool/src/thread_pool.cpp | Implements worker lifecycle, queueing, and worker callback behavior. |
| components/thread_pool/include/thread_pool.hpp | Public API, configuration, and documentation/comments for ThreadPool. |
| components/thread_pool/README.md | Component-level README and feature list. |
| components/thread_pool/idf_component.yml | Component Manager manifest (metadata, docs URL, dependencies). |
| components/thread_pool/CMakeLists.txt | Registers the component with ESP-IDF build system. |
| components/thread_pool/example/CMakeLists.txt | Example project CMake configuration and component selection. |
| components/thread_pool/example/main/CMakeLists.txt | Registers the example’s main component. |
| components/thread_pool/example/main/thread_pool_example.cpp | Example app demonstrating pool usage and completion waiting. |
| components/thread_pool/example/README.md | Example build/run instructions. |
| components/thread_pool/example/sdkconfig.defaults | Enables C++ exceptions for the example build. |
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 10 out of 10 changed files in this pull request and generated 4 comments.
Comments suppressed due to low confidence (2)
components/thread_pool/README.md:12
- README claims stop "drains queued jobs", but ThreadPool::stop() explicitly warns that queued jobs may not be executed (and the current implementation can leave queued jobs unprocessed). Please update this feature bullet to match the actual stop semantics so users aren’t misled.
- Graceful stop (stops workers; queued jobs may not be executed)
components/thread_pool/idf_component.yml:8
documentationpoints tocore/thread_pool.html, but there are no correspondingdoc/en/**pages and nodoc/DoxyfileINPUT/EXAMPLE_PATH entries for this new component. This will produce a broken documentation link and omit the API/example from the published docs; please add the required doc pages and Doxyfile entries (kept in alphabetical order) so the link resolves.
documentation: "https://esp-cpp.github.io/espp/core/thread_pool.html"
Comment on lines
+20
to
+22
| project(thread_pool_example) | ||
|
|
||
| set(CMAKE_CXX_STANDARD 20) |
Comment on lines
+1
to
+4
| idf_component_register( | ||
| INCLUDE_DIRS "include" | ||
| SRC_DIRS "src" | ||
| REQUIRES base_component task) |
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 15 out of 15 changed files in this pull request and generated 7 comments.
Comments suppressed due to low confidence (1)
doc/en/core/thread_pool_example.md:3
- The include path looks incorrect relative to
doc/en/core/thread_pool_example.md.../../components/...resolves underdoc/components/..., not<repo_root>/components/.... Update the include path (likely to../../../components/thread_pool/example/README.md) so docs builds can actually find the example README.
finger563
changed the title
thread_pool component built on top of Task
finger563
reviewed
Jul 23, 2026
finger563
reviewed
Jul 23, 2026
Contributor
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 17 out of 17 changed files in this pull request and generated 2 comments.
Comments suppressed due to low confidence (1)
components/thread_pool/src/thread_pool.cpp:42
start()setsrunning_ = truebefore attempting to start any workers. If all workers fail to start, there is a window wheresubmit()can return true (and incrementsubmitted_) even though the pool will immediately roll back to a non-running state and clear the queue. This can lead to accepted jobs being dropped and stats becoming inconsistent (accepted jobs later counted as rejected during rollback). Consider only settingrunning_to true after at least one worker successfully starts, and set it atomically based onstarted_countto avoid any acceptance window.
stopping_ = false;
running_.store(true);
}
std::size_t started_count = 0;
Comment on lines
+59
to
+61
| /// @brief Start all worker threads. | ||
| /// @note No-op if the pool is already running. | ||
| void start(); |
Contributor
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 16 out of 16 changed files in this pull request and generated 5 comments.
Comments suppressed due to low confidence (1)
doc/en/core/thread_pool_example.md:3
- The include directive is indented, which can prevent MyST/Sphinx from recognizing it as a directive (depending on configuration). Place the fenced directive at the start of the line (no leading spaces) to ensure it renders consistently.
Comment on lines
+6
to
+17
| template <> struct fmt::formatter<espp::ThreadPool::Stats> { | ||
| template <typename ParseContext> constexpr auto parse(ParseContext &ctx) const { | ||
| return ctx.begin(); | ||
| } | ||
|
|
||
| template <typename FormatContext> | ||
| auto format(espp::ThreadPool::Stats const &s, FormatContext &ctx) const { | ||
| return fmt::format_to(ctx.out(), | ||
| "ThreadPool::Stats{{submitted: {}, executed: {}, rejected: {}}}", | ||
| s.submitted, s.executed, s.rejected); | ||
| } | ||
| }; No newline at end of file |
Comment on lines
+31
to
+65
| void ThreadPool::start() { | ||
| std::lock_guard<std::mutex> lifecycle_lock(lifecycle_mutex_); | ||
| { | ||
| std::lock_guard<std::mutex> lock(queue_mutex_); | ||
| if (running_.load()) { | ||
| return; | ||
| } | ||
| stopping_ = false; | ||
| running_.store(true); | ||
| } | ||
|
|
||
| std::size_t started_count = 0; | ||
| for (std::size_t i = 0; i < workers_.size(); ++i) { | ||
| if (workers_[i]->start()) { | ||
| ++started_count; | ||
| } else { | ||
| logger_.warn("Failed to start worker {} (already started or insufficient memory)", i); | ||
| } | ||
| } | ||
|
|
||
| if (started_count == 0) { | ||
| logger_.error("No workers started; rolling back pool start"); | ||
| for (auto &worker : workers_) { | ||
| worker->stop(); | ||
| } | ||
| { | ||
| std::lock_guard<std::mutex> lock(queue_mutex_); | ||
| running_.store(false); | ||
| stopping_ = true; | ||
| rejected_ += static_cast<std::uint64_t>(queue_.size()); | ||
| queue_.clear(); | ||
| } | ||
| queue_has_space_cv_.notify_all(); | ||
| } | ||
| } |
Comment on lines
+67
to
+78
| void ThreadPool::stop() { | ||
| std::lock_guard<std::mutex> lifecycle_lock(lifecycle_mutex_); | ||
|
|
||
| { | ||
| std::lock_guard<std::mutex> lock(queue_mutex_); | ||
| if (!running_.exchange(false)) { | ||
| return; | ||
| } | ||
| stopping_ = true; | ||
| rejected_ += static_cast<std::uint64_t>(queue_.size()); | ||
| queue_.clear(); | ||
| } |
| license: "MIT" | ||
| description: "Generic thread pool component implemented with espp::Task and condition variables." | ||
| url: "https://github.com/esp-cpp/espp/tree/main/components/thread_pool" | ||
| repository: "git://github.com/esp-cpp/espp.git" |
Comment on lines
+134
to
+166
| // 1 slow worker, capacity 2: 1 executing + 2 queued = 3 slots before rejection | ||
| espp::ThreadPool pool({ | ||
| .worker_count = 1, | ||
| .max_queue_size = 2, | ||
| .auto_start = true, | ||
| .block_on_submit_when_full = false, | ||
| .worker_task_config = { | ||
| .name = "tp_worker", | ||
| .stack_size_bytes = 4096, | ||
| .priority = 5, | ||
| .core_id = -1, | ||
| }, | ||
| .log_level = espp::Logger::Verbosity::WARN, | ||
| }); | ||
|
|
||
| // Fill worker + queue | ||
| int fill_accepted = 0; | ||
| for (int i = 0; i < 3; ++i) { | ||
| if (pool.try_submit([&]() { std::this_thread::sleep_for(300ms); })) { | ||
| ++fill_accepted; | ||
| } | ||
| } | ||
| passed &= check(name, fill_accepted == 3, "first 3 try_submit calls should be accepted"); | ||
|
|
||
| // These should all be rejected immediately | ||
| int rejected_count = 0; | ||
| for (int i = 0; i < 3; ++i) { | ||
| if (!pool.try_submit([&]() {})) { | ||
| ++rejected_count; | ||
| } | ||
| } | ||
| passed &= check(name, rejected_count == 3, "try_submit when full should return false"); | ||
|
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
add Thread_pool component
Motivation and Context
A thread pool that dispatches submitted jobs to a fixed set of worker threads.
How has this been tested?
an example is added and tested.
The example was tested on an esp32 board. it use a thread_pool with 2 works and finished 8 jobs.
Screenshots (if appropriate, e.g. schematic, board, console logs, lab pictures):
Types of changes
Checklist:
Software
.github/workflows/build.ymlfile to add my new test to the automated cloud build github action.Hardware