feat: Improve logging & guidelines

blues-expert/Dockerfile

@@ -43,4 +43,4 @@ USER appuser
 EXPOSE 8080
 
 # Run the binary
-CMD ["./main"]
+CMD ["./main", "-log-level", "debug"]

blues-expert/lib/docs/arduino/best_practices.md

@@ -6,7 +6,6 @@ When creating a new Arduino project, there are a few best practices to follow to
 
 - The project or sketch should be in a directory of the same name as the project, e.g. 'app/app.ino' (this is required for the Arduino CLI to work correctly)
 - Create a 'README.md' in the same directory as the sketch, e.g. 'app/README.md'. This should contain a description of the project, along with instruction for how to connect any sensors to the Notecard.
-- Create a 'WORKFLOW.mmd' in the same directory as the sketch, e.g. 'app/WORKFLOW.mmd'. This should contain a diagram of the flow of code in the project.
 - Always assume the user is using the Blues Feather MCU (e.g. Swan) and Notecarrier-F. Where sensors are concerned, always default to using the I2C interface, if possible.
 
 ## Requirements

blues-expert/lib/docs/arduino/index.md

@@ -7,18 +7,62 @@ This library is recommended for Arduino-based devices, in particular those that
 
 ## Installing the note-arduino library
 
+If required, the user may wish to install the note-arduino library using the Arduino CLI.
+
 ### Arduino CLI
 
 ```bash
 arduino-cli lib install "Blues Wireless Notecard"
 ```
 
-### PlatformIO
+## Code Layout
 
-```bash
-pio lib install "Blues Wireless Notecard"
+When either creating a new project or retrofitting existing projects to use the Notecard, ensure that the following structures are present:
+
+- ALL Notecard code should be in a separate library file from the main sketch (VERY IMPORTANT).
+- Limit modifications to the user's Arduino code; prefer to add new functions to the library file.
+- A 'init' function that initializes the Notecard, along with `hub.set` commands to configure the Notecard.
+  - Before optimising the code, set the `mode` to `continuous` for easy debugging. Use a comment to indicate that the user should change this to `periodic` to fit their application.
+- The first `sendRequest()` call should instead be `sendRequestWithRetry()` to ensure that the request is retried if it fails. This is to address a potential race condition on cold boot.
+- Once the initial pass has been made, go to [power management](#power-management) to ensure that the Notecard is power efficient.
+
+## Design Patterns
+
+Work through the following design patterns to ensure that target projects are easy to maintain and extend.
+
+ALWAYS:
+
+- use templates for notes and ensure that the data types are correct. ALWAYS use the `firmware_best_practices` tool with the `templates` document type for more information.
+- use the I2C interface for Notecard communication, unless instructed otherwise by the user.
+- use Blues Expert tools to check and validate Notecard requests and responses.
+- generate a header comment in bother the `.c` and the `.h` files that contains the following information:
+
+```c
+/***************************************************************************
+  <LIBRARY_NAME> - Library for <DESCRIPTION>
+
+  This library encapsulates all Notecard functionality for the <PROJECT_NAME>.
+  This is specific to your project and is NOT A GENERAL PURPOSE LIBRARY.
+
+  THIS FILE SHOULD BE EDITED AFTER GENERATION.
+  IT IS PROVIDED AS A STARTING POINT FOR THE USER TO EDIT AND EXTEND.
+***************************************************************************/
 ```
 
+TRY TO:
+
+- use the Serial interface for debugging, if possible. This should be easily disabled by the user, if not needed.
+
+NEVER:
+
+- layer note-c calls within the user's application code. If the Notecard is used, it should be handled by a function in the newly created library file.
+
+## Power Management
+
+Implement the initial changes following this guide and then ask the user if they would like to optimise the code for power management.
+
+- use the `firmware_best_practices` tool with the `best_practices` document type.
+
 ## Further Reading
 
 Queries about note-arduino specifics should be made to the `docs_search` or `docs_search_expert` tool.

blues-expert/lib/docs/arduino/templates.md

@@ -4,17 +4,8 @@ When building an application that is expected to operate over a long period of t
 
 ## Working with Note Templates
 
-By default, the Notecard allows for maximum developer flexibility in the structure and content of Notes. As such, individual Notes in a Notefile do not share structure or schema. You can add JSON structures and payloads of any type and format to a Notefile, adding and removing fields as required by your application.
-
-In order to provide this simplicity to developers, the design of the Notefile system is primarily memory based and designed to support no more than 100 Notes per Notefile. As long as your data needs and sync periods ensure regular uploads of data to Notehub, this limit is adequate for most applications.
-
-Some applications, however, will need to track and stage bursts of data that may eclipse the 100 Note limit in a short period of time, and before a sync can occur. For these types of use cases, the Notecard supports using a flash-based storage system based on Note templates.
-
 Using the `note.template` request with any `.qo` or `.qos` Notefile, developers can provide the Notecard with a schema of sorts to apply to future Notes added to the Notefile. This template acts as a hint to the Notecard that allows it to internally store data as fixed-length records rather than as flexible JSON objects, which tend to be much larger.
 
-> Note:
-> Note Templates are required for both inbound and outbound Notefiles when using Notecard LoRa or NTN mode with Starnote.
-
 ## Creating a Template
 
 To create a template, use the file argument to specify the Notefile to which the template should be applied. Then, use the body argument to specify a template body, similar to the way you'd make a note.add request. That body must contain the name of each field expected in each note.add request, and a value that serves as the hint indicating the data type to the Notecard. Each field can be a boolean, integer, float, or string. The port argument is required on Notecard LoRa and Starnote, and is a unique integer in the range 1-100.

blues-expert/lib/docs/python/best_practices.md

@@ -0,0 +1,194 @@
+# Python Notecard Best Practices
+
+When creating a new Python project with the Notecard, there are a few best practices to follow to ensure that the project is easy to maintain and extend.
+
+## Project Structure
+
+- Organize your project with a clear directory structure, typically with a main application file (e.g. 'app.py' or 'main.py')
+- Create a 'README.md' in the project directory. This should contain a description of the project, setup instructions, and details for how to connect any sensors to the Notecard.
+- Create a 'requirements.txt' file to specify Python dependencies, including the note-python library
+- Common hardware configurations include Raspberry Pi, PC/Mac development machines, or Python-capable embedded systems. Where sensors are concerned, always default to using the I2C interface, if possible.
+
+## Requirements
+
+- Always use templates for notes when sending data to minimize bandwidth usage.
+- Use the note-python library for all Notecard interactions.
+
+## Suggestions
+
+- Do not introduce power management features until the user has confirmed that the application is working. Offer this as a follow up change.
+- Start with console debugging output to demonstrate that the application is working. After the user has confirmed that the application is working, logging can be adjusted.
+- If the user asks for their data to be uploaded at a specific interval, ensure to set the `mode` to `periodic` in the `hub.set` request and the `outbound` to their desired interval.
+- Use proper error handling and exception management when communicating with the Notecard.
+
+## Example Basic Project
+
+Use this example to get started with building a Python project that uses the Notecard.
+
+You will need to know the following before starting:
+
+- REQUIRED: The Product Unique Identifier for your application. This is a unique identifier for your application that is used to identify your Notehub project in the Notecard.
+- REQUIRED (if not using I2C): The Notecard's serial port. This is the serial port that the Notecard is connected to (e.g., '/dev/ttyUSB0' on Linux or 'COM3' on Windows).
+- REQUIRED (if not using serial): The I2C bus. On Raspberry Pi, this is typically bus 1.
+
+```python
+#!/usr/bin/env python3
+"""
+This example demonstrates the basic usage of the note-python library
+to communicate with the Blues Notecard, configure it, and send sensor data.
+"""
+
+import notecard
+import time
+import sys
+
+# Product UID for your Notehub project
+# This should be in the format "com.company.username:productname"
+# You must register at notehub.io to claim a product UID
+PRODUCT_UID = "com.my-company.my-name:my-project"
+
+def main():
+    """Main application loop"""
+    print(f"Starting Python application for {PRODUCT_UID}...")
+
+    # Initialize Notecard
+    # For I2C connection (default on Raspberry Pi):
+    card = notecard.OpenI2C(0, 0, 0, debug=True)
+
+    # For serial connection, use this instead:
+    # card = notecard.OpenSerial("/dev/ttyUSB0", debug=True)
+
+    # Configure the Notecard to connect to Notehub
+    req = {"req": "hub.set"}
+    req["product"] = PRODUCT_UID
+    req["mode"] = "continuous"  # Use "periodic" for battery-powered devices
+
+    try:
+        rsp = card.Transaction(req)
+        print("Notecard configured successfully")
+    except Exception as e:
+        print(f"Error configuring Notecard: {e}")
+        sys.exit(1)
+
+    # Main loop: send data every 15 seconds
+    event_counter = 0
+    max_events = 25
+
+    while event_counter < max_events:
+        event_counter += 1
+
+        # Read temperature from Notecard's built-in sensor
+        try:
+            temp_req = {"req": "card.temp"}
+            temp_rsp = card.Transaction(temp_req)
+            temperature = temp_rsp.get("value", 0)
+        except Exception as e:
+            print(f"Error reading temperature: {e}")
+            temperature = 0
+
+        # Read voltage from Notecard
+        try:
+            volt_req = {"req": "card.voltage"}
+            volt_rsp = card.Transaction(volt_req)
+            voltage = volt_rsp.get("value", 0)
+        except Exception as e:
+            print(f"Error reading voltage: {e}")
+            voltage = 0
+
+        # Send note to Notehub
+        try:
+            note_req = {"req": "note.add"}
+            note_req["sync"] = True  # Upload immediately for demonstration
+            note_req["body"] = {
+                "temp": temperature,
+                "voltage": voltage,
+                "count": event_counter
+            }
+            card.Transaction(note_req)
+            print(f"Sent event {event_counter}: temp={temperature:.2f}°C, voltage={voltage:.2f}V")
+        except Exception as e:
+            print(f"Error sending note: {e}")
+
+        # Wait before next reading
+        time.sleep(15)
+
+    print("Demo cycle complete. Program finished.")
+
+if __name__ == "__main__":
+    main()
+```
+
+This basic example demonstrates how to use the note-python library to send and receive JSON commands and responses.
+It does not make use of the Notecard's templated note feature, see below for more information.
+
+## Installation
+
+Install the note-python library:
+
+```bash
+pip install note-python
+```
+
+Or add it to your `requirements.txt`:
+
+```txt
+note-python>=1.2.0
+```
+
+## Use Templates
+
+When sending data repeatedly with the same structure, always use note templates to reduce bandwidth usage. Templates are defined once and then referenced in subsequent notes, sending only the data values without field names.
+
+Example of setting up a template:
+
+```python
+# Define the template structure
+template_req = {"req": "note.template"}
+template_req["file"] = "sensors.qo"
+template_req["body"] = {
+    "temp": 14.1,
+    "humidity": 12.1,
+    "pressure": 14.1
+}
+
+try:
+    card.Transaction(template_req)
+    print("Template created successfully")
+except Exception as e:
+    print(f"Error creating template: {e}")
+```
+
+After defining the template, send notes with just the values:
+
+```python
+# Send note using the template
+note_req = {"req": "note.add"}
+note_req["file"] = "sensors.qo"
+note_req["body"] = {
+    "temp": 23.5,
+    "humidity": 65.2,
+    "pressure": 1013.25
+}
+
+card.Transaction(note_req)
+```
+
+## Connection Options
+
+The note-python library supports both I2C and serial connections:
+
+### I2C Connection (Recommended for Raspberry Pi)
+
+```python
+card = notecard.OpenI2C(0, 0, 0, debug=True)
+```
+
+### Serial Connection
+
+```python
+# Linux/Mac
+card = notecard.OpenSerial("/dev/ttyUSB0", debug=True)
+
+# Windows
+card = notecard.OpenSerial("COM3", debug=True)
+```

blues-expert/lib/docs/python/index.md

@@ -8,10 +8,67 @@ Support for CircuitPython and MicroPython is also available.
 
 ## Installing the note-python SDK
 
+If required, the user may wish to install the note-python library using pip.
+
 ```bash
 pip install note-python
 ```
 
+Or add it to `requirements.txt`:
+
+```txt
+note-python>=1.2.0
+```
+
+## Code Layout
+
+When either creating a new project or retrofitting existing projects to use the Notecard, ensure that the following structures are present:
+
+- ALL Notecard code should be in a separate module/file from the main application (VERY IMPORTANT).
+- Limit modifications to the user's main application code; prefer to add new functions to the Notecard module.
+- An 'init' or 'setup' function that initializes the Notecard connection, along with `hub.set` commands to configure the Notecard.
+  - Before optimizing the code, set the `mode` to `continuous` for easy debugging. Use a comment to indicate that the user should change this to `periodic` to fit their application.
+- Include proper error handling with try/except blocks for all Notecard transactions.
+- Once the initial pass has been made, go to [power management](#power-management) to ensure that the Notecard is power efficient.
+
+## Design Patterns
+
+Work through the following design patterns to ensure that target projects are easy to maintain and extend.
+
+ALWAYS:
+
+- use templates for notes and ensure that the data types are correct. ALWAYS use the `firmware_best_practices` tool with the `templates` document type for more information.
+- use the I2C interface for Notecard communication (default on Raspberry Pi), unless instructed otherwise by the user.
+- use Blues Expert tools to check and validate Notecard requests and responses.
+- generate a header docstring in the Notecard module that contains the following information:
+
+```python
+"""
+<MODULE_NAME> - Module for <DESCRIPTION>
+
+This module encapsulates all Notecard functionality for the <PROJECT_NAME>.
+This is specific to your project and is NOT A GENERAL PURPOSE LIBRARY.
+
+THIS FILE SHOULD BE EDITED AFTER GENERATION.
+IT IS PROVIDED AS A STARTING POINT FOR THE USER TO EDIT AND EXTEND.
+"""
+```
+
+TRY TO:
+
+- use Python's logging module for debugging output. This should be easily configurable by the user.
+- follow PEP 8 style guidelines for Python code.
+
+NEVER:
+
+- layer note-python calls within the user's main application code. If the Notecard is used, it should be handled by a function in the newly created Notecard module.
+
+## Power Management
+
+Implement the initial changes following this guide and then ask the user if they would like to optimize the code for power management.
+
+- use the `firmware_best_practices` tool with the `best_practices` document type.
+
 ## Further Reading
 
 Queries about note-python specifics should be made to the `docs_search` or `docs_search_expert` tool.