[BREAKING] Move and update many component implementations to support litgen-based pybind11 generation to overhaul python library (#291)

* Update file system and cli to have guards around ESP-specific code and start fleshing out the espp library

* fleshing out library and updating how external libraries are copied; added color and event manager

* add simple cli test

* WIP trying to use litgen to automatically generate the python bindings 🚀

* WIP move implementation from header to source files so litgen can parse them effectively

* WIP update cmakelists

* WIP continuing to flesh out automatic binding support

* fix: move task auto to header

* update color, add missing pragma to fomatters files

* fixed unmatched suppression

* add missing task scope

* working on configuring binding generator

* update math components to better support autogenerating bindings

* WIP trying to get more of the bindings working...

* having to manually modify generated bindings... -.-

* generated python docs

* add requirements and gitignore

* wip trying to get binding to work better

* update gaussian so that alpha/beta/gamma class members are public instead of having getter/setter functions for them with the same name

* WIP removed autogenerated default constructors for bezier, gaussian, task, timer, socket, tcpsocket, udpsocket and fixed autogenerated bindings. trying to get autogeneration to work for template classes

* update python tests; ensure complete scoping for config members to facilitate proper binding generation; remove TcpTransmitConfig from detail namespace and update it to be TcpSocket::TransmitConfig with alternate workaround for gcc bug; updated ftp and rtsp accordingly

* WIP trying to flesh out the bindings some more. still have to do a lot of manual editing of the generated bindings file, but it is way better than having to write the whole thing from scratch

* revert back to implementation in main header even though it breaks litgen

* minor update

* move run_on_core out of espp::Task class and into espp::task namespace

* update bindings

* fix i2c

* remove accidentally added i2c files

* add missing param doc

* add more explicit scoping

* minor updates

* update cmakelists to install magic enum headers correctly

* minor update to support running examples on qtpy s3

* add some notes to the binding generator

* update to generate into espp instead of espp_lib

* add build library workflow

* move sys/stdio.h to stdio.h for linux compat

* update github action to support windows

* update build to use cmake completely for better x-plat compat

* WIP windows library

* add simple readme

* minor update

* starting work on porting socket to windows

* fix cmake

* WIP windows support

* comment out windows build for now

* update bindings

* feat(math): update gaussian to have getter/setter methods instead of public members

* update bindings

* update to use pybind11_add_module to simplify cmake lists, add windows build script, and update to build for release

* update ci library build

* ensure return code

* update python bindings

* use direct commands to get proper error status

* update file system to use std::filesystem when not on esp-idf

* clean up file system includes

* update ftp to remove use of posix functions and replace with only std::filesystem or espp::FileSystem

* explicit scoping

* some more explicit scoping

* more explicit

* try to undefine logger verbosity enums since windows is complaining about them...

* WIP trying to make socket a little more x-plat friendly

* fix sa

* include winsock in "C" linkage

* WIP msvc is kinda stupid about include headers so we have to add some shit to the beginning of every file which includes one of the socket files... -.-

* WIP getting windows build working

* WIP added support for tabulate on msvc

* updated to get closer to working compilation on windows

* add note

* test uplaoding output folder

* update to v4

* disable reuseaddr on windows

* update how msc_ver is checked

* update install and ensure that pc can work

* fix(socket): update to properly initialize socket library on windows before configuring socket. updated error logging to work across platforms

* add pc tests for udp client/server and update build scripts accordingly

* minor update

* minor update

* update readmes

Author: finger563

.github/workflows/build_libraries.yml

@@ -0,0 +1,103 @@
+name: Build Host C++ / Python Libraries
+
+on:
+  pull_request:
+    branches: [main]
+  push:
+    branches: [main]
+  release:
+    types: [published]
+  workflow_dispatch:
+
+jobs:
+  build_windows:
+
+    runs-on: windows-latest
+    continue-on-error: false
+
+    steps:
+    - name: Checkout repo
+      uses: actions/checkout@v4
+      with:
+        submodules: 'recursive'
+        fetch-depth: 0
+
+    - name: Set up Python
+      uses: actions/setup-python@v5
+      with:
+        python-version: '3.12'
+
+    - name: Build libraries
+      working-directory: lib/
+      run: |
+        mkdir build
+        cd build
+        cmake ..
+        cmake --build . --config Release --target install
+
+    - name: Upload output folder
+      uses: actions/upload-artifact@v4
+      with:
+        name: libespp_windows
+        path: lib/pc
+
+  build_linux:
+
+    runs-on: ubuntu-latest
+    continue-on-error: false
+
+    steps:
+    - name: Checkout repo
+      uses: actions/checkout@v4
+      with:
+        submodules: 'recursive'
+        fetch-depth: 0
+
+    - name: Set up Python
+      uses: actions/setup-python@v5
+      with:
+        python-version: '3.12'
+
+    - name: Build libraries
+      working-directory: lib/
+      run: |
+        ./build.sh
+
+    - name: Upload output folder
+      uses: actions/upload-artifact@v4
+      with:
+        name: libespp_linux
+        path: lib/pc
+
+  build_macos:
+
+    runs-on: macos-latest
+    continue-on-error: false
+
+    steps:
+    - name: Setup XCode
+      uses: maxim-lobanov/setup-xcode@v1
+      with:
+        xcode-version: latest-stable
+
+    - name: Set up Python
+      uses: actions/setup-python@v5
+      with:
+        python-version: '3.12'
+
+    - name: Checkout repo
+      uses: actions/checkout@v4
+      with:
+        submodules: 'recursive'
+        fetch-depth: 0
+
+    - name: Build libraries
+      working-directory: lib/
+      run: |
+        ./build.sh
+
+    - name: Upload output folder
+      uses: actions/upload-artifact@v4
+      with:
+        name: libespp_macos
+        path: lib/pc

components/base_component/include/base_component.hpp

@@ -29,15 +29,15 @@ class BaseComponent {
   /// \param level The verbosity level to use for the logger
   /// \sa Logger::Verbosity
   /// \sa Logger::set_verbosity
-  void set_log_level(Logger::Verbosity level) { logger_.set_verbosity(level); }
+  void set_log_level(espp::Logger::Verbosity level) { logger_.set_verbosity(level); }
 
   /// Set the log verbosity for the logger
   /// \param level The verbosity level to use for the logger
   /// \note This is a convenience method that calls set_log_level
   /// \sa set_log_level
   /// \sa Logger::Verbosity
   /// \sa Logger::set_verbosity
-  void set_log_verbosity(Logger::Verbosity level) { set_log_level(level); }
+  void set_log_verbosity(espp::Logger::Verbosity level) { set_log_level(level); }
 
   /// Get the log verbosity for the logger
   /// \return The verbosity level of the logger
@@ -58,13 +58,14 @@ class BaseComponent {
 protected:
   BaseComponent() = default;
 
-  explicit BaseComponent(std::string_view tag, Logger::Verbosity level = Logger::Verbosity::WARN)
+  explicit BaseComponent(std::string_view tag,
+                         espp::Logger::Verbosity level = espp::Logger::Verbosity::WARN)
       : logger_({.tag = tag, .level = level}) {}
 
-  explicit BaseComponent(const Logger::Config &logger_config)
+  explicit BaseComponent(const espp::Logger::Config &logger_config)
       : logger_(logger_config) {}
 
   /// The logger for this component
-  Logger logger_ = espp::Logger({.tag = "BaseComponent", .level = Logger::Verbosity::INFO});
+  Logger logger_ = espp::Logger({.tag = "BaseComponent", .level = espp::Logger::Verbosity::INFO});
 };
 } // namespace espp

components/cli/include/cli.hpp

@@ -1,5 +1,6 @@
 #pragma once
 
+#if defined(ESP_PLATFORM)
 #include <sdkconfig.h>
 
 #if CONFIG_COMPILER_CXX_EXCEPTIONS || defined(_DOXYGEN_)
@@ -13,8 +14,6 @@
 #include "esp_vfs_dev.h"
 #include "esp_vfs_usb_serial_jtag.h"
 
-#include <cli/cli.h>
-
 #include "line_input.hpp"
 
 #ifdef CONFIG_ESP_CONSOLE_USB_CDC
@@ -26,6 +25,8 @@
 #define STRINGIFY2(s) #s
 #endif // STRINGIFY
 
+#include <cli/cli.h>
+
 namespace espp {
 /**
  * @brief Class for implementing a basic Cli using the external cli library.
@@ -388,3 +389,5 @@ class Cli : private cli::CliSession {
 } // namespace espp
 
 #endif // CONFIG_COMPILER_CXX_EXCEPTIONS
+
+#endif // ESP_PLATFORM

components/cli/include/line_input.hpp

@@ -1,10 +1,15 @@
+#pragma once
+
+#if defined(ESP_PLATFORM)
+
 #include <sdkconfig.h>
 
 #if CONFIG_COMPILER_CXX_EXCEPTIONS || defined(_DOXYGEN_)
 
 #include <algorithm>
 #include <atomic>
 #include <deque>
+#include <functional>
 #include <iostream>
 #include <stdio.h>
 #include <string>
@@ -424,3 +429,5 @@ class LineInput {
 } // namespace espp
 
 #endif // CONFIG_COMPILER_CXX_EXCEPTIONS
+
+#endif // ESP_PLATFORM

components/color/include/color.hpp

@@ -84,6 +84,10 @@ class Rgb {
    */
   Rgb &operator+=(const Rgb &rhs);
 
+  bool operator==(const Rgb &rhs) const = default;
+
+  bool operator!=(const Rgb &rhs) const = default;
+
   /**
    * @brief Get a HSV representation of this RGB color.
    * @return An HSV object containing the HSV representation.
@@ -135,6 +139,10 @@ class Hsv {
    */
   Hsv &operator=(const Hsv &other) = default;
 
+  bool operator==(const Hsv &rhs) const = default;
+
+  bool operator!=(const Hsv &rhs) const = default;
+
   /**
    * @brief Assign the values of the provided Rgb object to this Hsv object.
    * @param rgb The Rgb object to convert and copy.
@@ -156,66 +164,6 @@ class Hsv {
   return fg(fmt::rgb(rgb.r * 255, rgb.g * 255, rgb.b * 255));
 }
 
-// equality operators
-[[maybe_unused]] static bool operator==(const Rgb &lhs, const Rgb &rhs) {
-  return lhs.r == rhs.r && lhs.g == rhs.g && lhs.b == rhs.b;
-}
-
-[[maybe_unused]] static bool operator==(const Hsv &lhs, const Hsv &rhs) {
-  return lhs.h == rhs.h && lhs.s == rhs.s && lhs.v == rhs.v;
-}
-
-// inequality operators
-[[maybe_unused]] static bool operator!=(const Rgb &lhs, const Rgb &rhs) { return !(lhs == rhs); }
-
-[[maybe_unused]] static bool operator!=(const Hsv &lhs, const Hsv &rhs) { return !(lhs == rhs); }
 } // namespace espp
 
-#include "format.hpp"
-
-// for allowing easy serialization/printing of the
-// Rgb
-template <> struct fmt::formatter<espp::Rgb> {
-  // Presentation format: 'f' - floating [0,1] (default), 'd' - integer [0,255], 'x' - hex integer.
-  char presentation = 'f';
-
-  template <typename ParseContext> constexpr auto parse(ParseContext &ctx) {
-    // Parse the presentation format and store it in the formatter:
-    auto it = ctx.begin(), end = ctx.end();
-    if (it != end && (*it == 'f' || *it == 'd' || *it == 'x'))
-      presentation = *it++;
-
-    // TODO: Check if reached the end of the range:
-    // if (it != end && *it != '}') throw format_error("invalid format");
-
-    // Return an iterator past the end of the parsed range:
-    return it;
-  }
-
-  template <typename FormatContext> auto format(espp::Rgb const &rgb, FormatContext &ctx) const {
-    switch (presentation) {
-    case 'f':
-      return fmt::format_to(ctx.out(), "({}, {}, {})", rgb.r, rgb.g, rgb.b);
-    case 'd':
-      return fmt::format_to(ctx.out(), "({}, {}, {})", static_cast<int>(rgb.r * 255),
-                            static_cast<int>(rgb.g * 255), static_cast<int>(rgb.b * 255));
-    case 'x':
-      return fmt::format_to(ctx.out(), "{:#08X}", rgb.hex());
-    default:
-      // shouldn't get here!
-      return fmt::format_to(ctx.out(), "({}, {}, {})", rgb.r, rgb.g, rgb.b);
-    }
-  }
-};
-
-// for allowing easy serialization/printing of the
-// Rgb
-template <> struct fmt::formatter<espp::Hsv> {
-  template <typename ParseContext> constexpr auto parse(ParseContext &ctx) const {
-    return ctx.begin();
-  }
-
-  template <typename FormatContext> auto format(espp::Hsv const &hsv, FormatContext &ctx) const {
-    return fmt::format_to(ctx.out(), "({}, {}, {})", hsv.h, hsv.s, hsv.v);
-  }
-};
+#include "color_formatters.hpp"

components/color/include/color_formatters.hpp

@@ -0,0 +1,50 @@
+#pragma once
+
+#include "format.hpp"
+
+// for allowing easy serialization/printing of the
+// Rgb
+template <> struct fmt::formatter<espp::Rgb> {
+  // Presentation format: 'f' - floating [0,1] (default), 'd' - integer [0,255], 'x' - hex integer.
+  char presentation = 'f';
+
+  template <typename ParseContext> constexpr auto parse(ParseContext &ctx) {
+    // Parse the presentation format and store it in the formatter:
+    auto it = ctx.begin(), end = ctx.end();
+    if (it != end && (*it == 'f' || *it == 'd' || *it == 'x'))
+      presentation = *it++;
+
+    // TODO: Check if reached the end of the range:
+    // if (it != end && *it != '}') throw format_error("invalid format");
+
+    // Return an iterator past the end of the parsed range:
+    return it;
+  }
+
+  template <typename FormatContext> auto format(espp::Rgb const &rgb, FormatContext &ctx) const {
+    switch (presentation) {
+    case 'f':
+      return fmt::format_to(ctx.out(), "({}, {}, {})", rgb.r, rgb.g, rgb.b);
+    case 'd':
+      return fmt::format_to(ctx.out(), "({}, {}, {})", static_cast<int>(rgb.r * 255),
+                            static_cast<int>(rgb.g * 255), static_cast<int>(rgb.b * 255));
+    case 'x':
+      return fmt::format_to(ctx.out(), "{:#08X}", rgb.hex());
+    default:
+      // shouldn't get here!
+      return fmt::format_to(ctx.out(), "({}, {}, {})", rgb.r, rgb.g, rgb.b);
+    }
+  }
+};
+
+// for allowing easy serialization/printing of the
+// Hsv
+template <> struct fmt::formatter<espp::Hsv> {
+  template <typename ParseContext> constexpr auto parse(ParseContext &ctx) const {
+    return ctx.begin();
+  }
+
+  template <typename FormatContext> auto format(espp::Hsv const &hsv, FormatContext &ctx) const {
+    return fmt::format_to(ctx.out(), "({}, {}, {})", hsv.h, hsv.s, hsv.v);
+  }
+};

components/event_manager/include/event_manager.hpp

@@ -34,7 +34,7 @@ namespace espp {
  * \section event_manager_ex1 Event Manager Example
  * \snippet event_manager_example.cpp event manager example
  */
-class EventManager : public BaseComponent {
+class EventManager : public espp::BaseComponent {
 public:
   /**
    * @brief Function definition for function prototypes to be called when
@@ -80,7 +80,8 @@ class EventManager : public BaseComponent {
    *         registered for that component.
    */
   bool add_subscriber(const std::string &topic, const std::string &component,
-                      const event_callback_fn &callback, const size_t stack_size_bytes = 8 * 1024);
+                      const espp::EventManager::event_callback_fn &callback,
+                      const size_t stack_size_bytes = 8192);
 
   /**
    * @brief Register a subscriber for \p component on \p topic.
@@ -96,7 +97,8 @@ class EventManager : public BaseComponent {
    *         registered for that component.
    */
   bool add_subscriber(const std::string &topic, const std::string &component,
-                      const event_callback_fn &callback, const Task::BaseConfig &task_config);
+                      const espp::EventManager::event_callback_fn &callback,
+                      const espp::Task::BaseConfig &task_config);
 
   /**
    * @brief Publish \p data on \p topic.
@@ -128,7 +130,7 @@ class EventManager : public BaseComponent {
 
 protected:
   EventManager()
-      : BaseComponent("Event Manager") {}
+      : espp::BaseComponent("Event Manager") {}
 
   struct SubscriberData {
     std::mutex m;