RobotoAI integration

.gitmodules

@@ -4,3 +4,6 @@
 [submodule "third_party/tomlplusplus"]
 	path = third_party/tomlplusplus
 	url = https://github.com/marzer/tomlplusplus.git
+[submodule "third_party/nlohmann"]
+	path = third_party/nlohmann
+	url = https://github.com/nlohmann/json.git

CLAUDE.md

@@ -0,0 +1,58 @@
+# Logloader
+
+## Architecture
+
+- `LogLoader` orchestrates download (MAVSDK) + upload (all backends)
+- `UploadBackend` base class with SQLite DB tracking
+- `FlightReviewBackend` extends UploadBackend - HTTP multipart upload to flight review servers
+- `RobotoBackend` extends UploadBackend - RobotoAI API + S3 upload with SigV4 signing
+- Config at `~/.local/share/logloader/config.toml`
+- Local server is always Flight Review (installed with ARK OS)
+
+## Build
+
+- `make` in this directory (cmake under the hood)
+- C++20, `-Wall -Wextra -Werror -Wpedantic`
+- MAVSDK is NOT installed on dev machines - it's an embedded target dependency
+
+## Pending Refactoring
+
+The following issues were identified during code review and should be addressed:
+
+### Bugs (fix first)
+
+1. **Data race on `_should_exit` in UploadBackend** (`UploadBackend.hpp:69`): `_should_exit` is a plain `bool` written from the main thread via `stop()` and read from the upload thread. Make it `std::atomic<bool>`.
+
+2. **Data race on `_loop_disabled` in LogLoader** (`LogLoader.hpp:65`): Same issue — plain `bool` accessed from multiple threads. Make it `std::atomic<bool>`.
+
+3. **`upload_pending_logs` returns early instead of continuing** (`LogLoader.cpp:367-375`): If one log entry has an empty UUID or missing filepath, the function `return`s, which kills the entire upload queue for that backend. Change `return` to `continue`.
+
+4. **Unsafe `getenv("HOME")`** (`main.cpp:21`): `getenv("HOME")` can return `nullptr` (e.g. systemd service). `std::string(nullptr)` is UB. Guard it.
+
+5. **Hardcoded `substr(8)` in RobotoBackend** (`RobotoBackend.cpp:118` and throughout): Assumes URL starts with exactly `https://`. Extract the host at construction time, similar to `FlightReviewBackend::sanitize_url_and_determine_protocol()`.
+
+6. **Missing space in log message** (`LogLoader.cpp:176`): `"Received " << size << "log entries"` — missing space before "log".
+
+### Restructure: Separate download tracking from upload backends
+
+**Problem:** `UploadBackend` currently owns download-related methods (`num_logs_to_download`, `get_next_log_to_download`, `update_download_status`, `add_log_entry`). Download is a single operation but is tracked redundantly in 3 separate databases. Every download-related call must be replicated across all backends:
+```cpp
+_local_server->add_log_entry(entry);
+_remote_server->add_log_entry(entry);
+if (_roboto_backend) _roboto_backend->add_log_entry(entry);
+```
+This pattern repeats in `request_log_entries` and `download_next_log`.
+
+**Solution:** Move download tracking to `LogLoader` with a single `logs.db`. Each `UploadBackend` only tracks its own upload state. Remove `add_log_entry`, `update_download_status`, `num_logs_to_download`, `get_next_log_to_download` from `UploadBackend`.
+
+### Restructure: Simplify `upload_log`
+
+**Problem:** `UploadBackend::upload_log()` (`UploadBackend.cpp:177-252`) reverse-engineers the UUID from the filename by parsing out the id/date, reading file size from disk, fabricating a `mavsdk::LogFiles::Entry`, and hashing. This is fragile — if `fs::file_size()` differs from the original MAVSDK-reported `size_bytes`, the UUID won't match and the upload silently fails.
+
+**Solution:** The caller (`upload_pending_logs`) already has the UUID. Pass it through, or restructure so `upload_log` receives the UUID directly. The filename-parsing logic can be removed entirely.
+
+### Nice-to-have
+
+- **No HTTP timeouts in FlightReviewBackend**: httplib clients have no timeout. A hanging server blocks the upload thread forever. Add `set_read_timeout()`.
+- **New SSL client per API call in RobotoBackend**: 5 TLS handshakes per upload. Consider reusing a single client across the upload flow.
+- **Redundant settings storage**: Both `_settings` and inherited `_base_settings` store `logs_directory`, `db_path`, `upload_enabled`. Pick one place.

CMakeLists.txt

@@ -28,11 +28,14 @@ find_package(MAVSDK REQUIRED)
 
 include_directories(third_party/cpp-httplib/)
 include_directories(third_party/tomlplusplus/)
+include_directories(third_party/nlohmann/include/)
 include_directories(${SQLite3_INCLUDE_DIRS})
 
 add_executable(${PROJECT_NAME}
     src/main.cpp
-    src/ServerInterface.cpp
+    src/UploadBackend.cpp
+    src/FlightReviewBackend.cpp
+    src/RobotoBackend.cpp
     src/LogLoader.cpp)
 
 target_link_libraries(${PROJECT_NAME}

config.toml

@@ -4,3 +4,9 @@ remote_server = "https://review.px4.io"
 email = ""
 upload_enabled = false
 public_logs = false
+
+[roboto]
+upload_enabled = false
+api_url = "https://api.roboto.ai"
+api_token = ""
+device_id = ""

src/FlightReviewBackend.cpp

@@ -0,0 +1,144 @@
+#include "FlightReviewBackend.hpp"
+#include "Log.hpp"
+
+#include <iostream>
+#include <filesystem>
+#include <fstream>
+#include <sstream>
+#define CPPHTTPLIB_OPENSSL_SUPPORT
+#include <httplib.h>
+
+namespace fs = std::filesystem;
+
+FlightReviewBackend::FlightReviewBackend(const FlightReviewBackend::Settings& settings)
+	: UploadBackend(settings.db_path, settings.upload_enabled)
+	, _settings(settings)
+{
+	sanitize_url_and_determine_protocol();
+}
+
+void FlightReviewBackend::sanitize_url_and_determine_protocol()
+{
+	std::string url = _settings.server_url;
+	std::string sanitized_url;
+	Protocol protocol;
+
+	std::string http_prefix = "http://";
+	std::string https_prefix = "https://";
+
+	size_t pos = std::string::npos;
+
+	if ((pos = url.find(https_prefix)) != std::string::npos) {
+		sanitized_url = url.substr(pos + https_prefix.length());
+		protocol = Protocol::Https;
+
+	} else if ((pos = url.find(http_prefix)) != std::string::npos) {
+		sanitized_url = url.substr(pos + http_prefix.length());
+		protocol = Protocol::Http;
+
+	} else {
+		sanitized_url = url;
+		protocol = Protocol::Https;
+	}
+
+	_settings.server_url = sanitized_url;
+	_protocol = protocol;
+}
+
+FlightReviewBackend::UploadResult FlightReviewBackend::upload(const std::string& filepath)
+{
+	// Skip files that are in progress (have a .lock file)
+	if (fs::exists(filepath + ".lock")) {
+		return {false, 0, "File is locked (currently being downloaded)"};
+	}
+
+	// Skip files that don't exist
+	if (!fs::exists(filepath)) {
+		return {false, 404, "Log file does not exist: " + filepath};
+	}
+
+	// Skip files with size zero
+	if (fs::file_size(filepath) == 0) {
+		return {false, 0, "Skipping zero-size log file: " + filepath};
+	}
+
+	if (!server_reachable()) {
+		return {false, 0, "Server unreachable: " + _settings.server_url};
+	}
+
+	std::ifstream file(filepath, std::ios::binary);
+
+	if (!file) {
+		return {false, 0, "Could not open file: " + filepath};
+	}
+
+	// Build multi-part form data
+	httplib::MultipartFormDataItems items = {
+		{"type", _settings.public_logs ? "flightreport" : "personal", "", ""}, // NOTE: backend logic is funky
+		{"description", "Uploaded by logloader", "", ""},
+		{"feedback", "", "", ""},
+		{"email", _settings.user_email, "", ""},
+		{"source", "auto", "", ""},
+		{"videoUrl", "", "", ""},
+		{"rating", "", "", ""},
+		{"windSpeed", "", "", ""},
+		{"public", _settings.public_logs ? "true" : "false", "", ""},
+	};
+
+	std::string content((std::istreambuf_iterator<char>(file)), std::istreambuf_iterator<char>());
+	items.push_back({"filearg", content, filepath, "application/octet-stream"});
+
+	LOG("Uploading " << fs::path(filepath).filename().string() << " to " << _settings.server_url);
+
+	// Post multi-part form
+	httplib::Result res;
+
+	if (_protocol == Protocol::Https) {
+		httplib::SSLClient cli(_settings.server_url);
+		cli.set_connection_timeout(10);
+		cli.set_read_timeout(30);
+		res = cli.Post("/upload", items);
+
+	} else {
+		httplib::Client cli(_settings.server_url);
+		cli.set_connection_timeout(10);
+		cli.set_read_timeout(30);
+		res = cli.Post("/upload", items);
+	}
+
+	if (res && res->status == 302) {
+		return {true, 302, "Success: " + _settings.server_url + res->get_header_value("Location")};
+
+	} else if (res && res->status == 400) {
+		return {false, 400, "Bad Request - Will not retry"};
+
+	} else {
+		return {false, res ? res->status : 0, "Will retry later"};
+	}
+}
+
+bool FlightReviewBackend::server_reachable()
+{
+	httplib::Result res;
+
+	if (_protocol == Protocol::Https) {
+		httplib::SSLClient cli(_settings.server_url);
+		cli.set_connection_timeout(10);
+		cli.set_read_timeout(30);
+		res = cli.Get("/");
+
+	} else {
+		httplib::Client cli(_settings.server_url);
+		cli.set_connection_timeout(10);
+		cli.set_read_timeout(30);
+		res = cli.Get("/");
+	}
+
+	bool success = res && res->status == 200;
+
+	if (!success) {
+		LOG("Connection to " << _settings.server_url << " failed: " << (res ? std::to_string(res->status) : "No response"));
+	}
+
+	return success;
+}

src/FlightReviewBackend.hpp

@@ -0,0 +1,32 @@
+#pragma once
+
+#include "UploadBackend.hpp"
+
+class FlightReviewBackend : public UploadBackend
+{
+public:
+	struct Settings {
+		std::string server_url;
+		std::string user_email;
+		std::string db_path;
+		bool upload_enabled {};
+		bool public_logs {};
+	};
+
+	FlightReviewBackend(const Settings& settings);
+
+protected:
+	UploadResult upload(const std::string& filepath) override;
+
+private:
+	enum class Protocol {
+		Http,
+		Https
+	};
+
+	void sanitize_url_and_determine_protocol();
+	bool server_reachable();
+
+	Settings _settings;
+	Protocol _protocol {Protocol::Https};
+};