diff --git a/.gitignore b/.gitignore index cfe08f3b..acb7b642 100644 --- a/.gitignore +++ b/.gitignore @@ -1,3 +1,7 @@ *~ #* .DS_Store + +# Ignore Rust build artifacts +/micropolis-rs/target/ +.venv/ diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 00000000..f3ada379 --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,159 @@ +# General Rules + +* Use offscreen mode to test GUI elements whenever it makes sense to do so, avoid using xvfb-run as it often causes issues with GUI testing. + +* At the start of every session, please read plan_of_action.md and session_handover.md. Then continue to do the activity you've been asked to do. At each checkpoint, prior to issuing the git commit and pull request notifications, I need you to update the plan_of_action.md to check of items that are complete and add any suggested work to be done, and also update migration_log.md. + +* When the context window or cache is getting full and tokens are running out, summarize the work done so far in hand over documents (e.g. plan_of_action.md and migration_log.md. + +* At the start of every session, please read plan_of_action.md and session_handover.md. Then continue to do the activity you've been asked to do. At each checkpoint, prior to issuing the git commit and pull request notifications, I need you to update the plan_of_action.md to check of items that are complete and add any suggested work to be done, and also update migration_log.md. +en the context window or cache is getting full and tokens are running out, summarize the work done so far in a handover document. + +* You can us this codebase: https://github.com/pierreyoda/micropolis-rs to guide your development + +# Python Coding Standards and Best Practices + +## Core Principles +* Prioritize readability, maintainability, and correctness. +* Write code that is self-documenting and follows Pythonic idioms. +* Adhere to the Zen of Python (import this). +* Ensure code is robust with comprehensive error handling and testing. +* Use map and list comprehensions instead of for loops where possible. + +## Code Style and Formatting (PEP 8) +* **Indentation**: Use 4 spaces per indentation level. +* **Line Length**: Limit lines to 88 characters for code (like the Black code formatter). +* **Imports**: + * Import modules at the top of the file. + * Group imports in the order: standard library, third-party, local application. + * Use absolute imports. + * Use a tool like `isort` to automatically manage import ordering. +* **Whitespace**: Use whitespace to improve readability, but avoid extraneous whitespace. +* **Autoformatters**: Use tools like `black` and `ruff` to enforce consistent style. + +## Naming Conventions (PEP 8) +* **Variables, Functions, and Modules**: `snake_case`. +* **Classes**: `PascalCase`. +* **Constants**: `UPPER_SNAKE_CASE`. +* **Private Attributes**: Use a leading underscore `_private_attribute`. +* **Name Mangling**: Use two leading underscores `__name_mangled` for attributes you don't want subclasses to inherit. + +## Documentation (Docstrings & Type Hinting) +* **Docstrings (PEP 257)**: + * Write docstrings for all public modules, classes, functions, and methods. + * Use the Google Python Style for docstrings. + * Include a one-line summary, followed by a more detailed description, `Args`, `Returns`, and `Raises` sections. +* **Type Hinting (PEP 484)**: + * Use type hints for all function signatures and variables where the type may not be obvious. + * This improves static analysis and code completion. + +```python +from typing import List + +def calculate_average(numbers: List[float]) -> float: + """Calculates the average of a list of numbers. + + Args: + numbers: A list of floating-point numbers. + + Returns: + The average of the numbers in the list. + + Raises: + ValueError: If the list of numbers is empty. + """ + if not numbers: + raise ValueError("The list of numbers cannot be empty.") + return sum(numbers) / len(numbers) +``` + +## Error Handling +* **Use specific exceptions**: Catch specific exceptions rather than using a bare `except:` clause. +* **Create custom exceptions**: Define custom exception classes for your application to represent specific error conditions. +* **Use `try...except...else...finally`**: + * The `try` block contains code that might raise an exception. + * The `except` block handles the exception. + * The `else` block executes if no exceptions are raised. + * The `finally` block executes no matter what, and is used for cleanup. + +## Logging +* Use the `logging` module instead of `print()` for debugging and tracking events. +* Configure different logging levels (`DEBUG`, `INFO`, `WARNING`, `ERROR`, `CRITICAL`). +* Use a centralized logging configuration for your application. + +```python +import logging + +logging.basicConfig(level=logging.INFO, format='%(asctime)s - %(levelname)s - %(message)s') + +def my_function(): + logging.info("Starting my_function.") + try: + # ... function logic ... + logging.debug("Intermediate step successful.") + except Exception as e: + logging.error(f"An error occurred: {e}", exc_info=True) + logging.info("Finished my_function.") +``` + +## Virtual Environments and Dependency Management +* **`venv`**: Always use a virtual environment for each project to isolate dependencies. +* **`pip` and `requirements.txt`**: Use `pip` to install packages and `pip freeze > requirements.txt` to record the dependencies. +* **Poetry or Pipenv**: For more complex projects, consider using tools like Poetry or Pipenv for dependency management and packaging. + +## Testing +* **`unittest` or `pytest`**: Write unit tests for your code. `pytest` is generally recommended for its simplicity and powerful features. +* **Test Coverage**: Aim for high test coverage to ensure your code is reliable. +* **Fixtures**: Use fixtures (especially in `pytest`) to set up and tear down test state. + +## Script Structure +* Use the `if __name__ == "__main__":` block to make your scripts reusable as modules. + +```python +def main(): + # Main logic of the script + pass + +if __name__ == "__main__": + main() +``` + +# LLM Rule: Debug Rule 66 - Verbose Step-Trace Debugging for Python + +**Description:** +This rule defines a process for instrumenting specified Python code (functions, methods) with detailed logging statements to trace execution flow and data state at runtime. The goal is to produce verbose console output that helps pinpoint exactly where a process is failing or where data becomes problematic. + +**Invocation:** +Triggered when the user explicitly requests `` for specific functions/methods, or asks for a highly detailed, step-by-step debugging trace on specified code sections. + +**Procedure:** + +1. **Identify Targets:** Determine the primary function(s) or method(s) suspected of causing the issue, including any key helper functions they call internally. + +2. **Instrument Code:** Using the `edit_file` tool, modify the target Python code by adding logging statements at critical points: + * **Function Entry/Exit:** + * Add `logging.debug(f"--- Entering {__name__}.{function.__name__} ---")` at the start. + * Add `logging.debug(f"--- Exiting {__name__}.{function.__name__} ---")` just before the return statement. + * **Argument Inspection:** + * Immediately after entry, log key input arguments: `logging.debug(f" Arg: {arg_name} = {repr(arg_value)}")`. + * **Major Logic Steps:** + * Add `logging.debug(" Step: [Description of step about to happen]...")` *before* the step. + * Add `logging.debug(" Step: [Description of step completed].")` *after* the step. + * **Data State Inspection:** + * Before a crucial operation on a variable (e.g., a pandas DataFrame), log its state: + * `logging.debug(f" Data State ({variable_name}): Shape={df.shape}, Columns={df.columns.tolist()}")` + * `logging.debug(f" Data Head:\n{df.head().to_string()}")` + * **Loops:** + * Inside the loop, log the current iteration: `logging.debug(f" Looping: Processing item {item}")` + * **Conditional Logic (`if/else`):** + * Add `logging.debug(" Condition TRUE: [Brief reason]")` inside the `if` block. + * Add `logging.debug(" Condition FALSE: [Brief reason]")` inside the `else` block. + * **Function Call Results:** + * Immediately after calling an important internal or helper function, log the result: `logging.debug(f" Result from {called_function.__name__}: {repr(result_variable)}")` + +3. **Output Formatting:** + * Use the `logging` module with a `DEBUG` level. + * Use f-strings for easy formatting. + * Use indentation to indicate nesting levels. + +4. **Result:** The execution of the instrumented code will produce a detailed trace, allowing the user and the AI to follow the execution path and inspect data at intermediate steps to identify the source of errors or unexpected behavior. diff --git a/GEMINI.md b/GEMINI.md new file mode 100644 index 00000000..6ca776d3 --- /dev/null +++ b/GEMINI.md @@ -0,0 +1,740 @@ +# General Rules + +* When the plan changes, update the project plan document +* When the context window or cache is getting full and tokens are running out, summarise the work done so far in a hand over document. + +# Bash Coding Standards and Best Practices + +## Core Principles +* Prioritize readability, maintainability, and error resilience +* Design scripts as self-documenting, self-contained utilities +* Anticipate and handle failures at every stage +* Provide clear logging for both success paths and error conditions + +## Script Structure +* Begin with a descriptive header comment explaining purpose, inputs, and outputs +* Place all constants and configuration at the top of the script +* Implement a modular, function-based design with single-responsibility functions +* Create a `main()` function to manage overall execution flow +* Call `main "$@"` at the end of the script to enable testing of individual functions + +```bash +#!/bin/bash +# Purpose: Process genome scaffolds and analyze sequence lengths +# Usage: ./script.sh + +# Configuration +readonly MIN_LENGTH="${2:-10000}" +readonly INPUT_DIR="${1:-/default/path}" + +# Functions +process_file() { + local file="$1" + # Processing logic +} + +main() { + log_message "INFO" "Starting processing with min_length=${MIN_LENGTH}" + # Main processing logic +} + +# Execute main function +main "$@" +``` + +## Variable Naming and Usage +* Use UPPERCASE for global constants and environment variables +* Use lowercase for local variables and function names +* Always quote variable references: `"${variable}"` not `$variable` +* Use meaningful, descriptive names that indicate purpose or content +* Declare variables with appropriate scope (`local` within functions) +* Set `readonly` for constants to prevent accidental modification + +```bash +# Global constants +readonly OUTPUT_DIR="/path/to/output" + +process_file() { + # Local variables + local input_file="$1" + local file_name="$(basename "${input_file}")" +} +``` + +## Error Handling and Validation +* Start scripts with `set -euo pipefail` to enable strict error handling +* Validate all inputs before processing (existence, permissions, format) +* Include custom error handling with descriptive messages +* Create error exit function with appropriate exit codes +* Use trap to handle cleanup on script exit +* Check command success with explicit conditionals + +```bash +set -euo pipefail + +error_exit() { + local message="$1" + local code="${2:-1}" + log_message "ERROR" "${message}" + exit "${code}" +} + +# Validate input file +if [[ ! -f "${INPUT_FILE}" ]]; then + error_exit "Input file does not exist: ${INPUT_FILE}" 2 +fi + +# Handle cleanup on exit +trap cleanup EXIT +``` + +## Logging and Debugging +* Implement hierarchical log levels: DEBUG, INFO, WARNING, ERROR, SUCCESS +* Include timestamps, log level, and context in all log messages +* Log both to files and to console with appropriate formatting +* Create separate log and error files for each run +* Use unique identifiers in log filenames (timestamp, job ID) +* Include resource usage data at regular intervals + +```bash +log_message() { + local level="$1" + local message="$2" + local timestamp="$(date '+%Y-%m-%d %H:%M:%S')" + echo "[${timestamp}] ${level}: ${message}" | tee -a "${LOG_FILE}" >&2 +} + +report_resources() { + log_message "RESOURCE" "Memory: $(free -h | grep 'Mem:' | awk '{print $3 " of " $2}')" + log_message "RESOURCE" "Disk: $(df -h "${OUTPUT_DIR}" | tail -1 | awk '{print $5 " used"}')" +} +``` + +## File and Directory Management +* Verify directory existence before writing; create if necessary +* Use atomic file operations to prevent corrupted outputs +* Write to temporary files first, then move to final location +* Implement consistent naming conventions across all files +* Create hierarchical directory structures for complex data +* Use full paths for critical operations, relative paths for others +* Clean up temporary files and directories on completion + +```bash +# Create output directory safely +mkdir -p "${OUTPUT_DIR}" + +# Write output atomically +generate_data > "${TEMP_FILE}" +mv "${TEMP_FILE}" "${OUTPUT_FILE}" + +# Clean up temporary files +cleanup() { + rm -rf "${TEMP_DIR}" + log_message "INFO" "Temporary files removed" +} +``` + +## Control Flow and Loops +* Use `if/then/else/fi` for conditionals (not curly braces) +* Use `for/do/done` for iteration (not curly braces) +* Prefer `for` loops with explicit sequences rather than C-style counting +* Use `while` loops for processing stream input +* Validate loop conditions before entering loops +* Include loop progress indicators for long-running operations + +```bash +# Preferred loop format +for file in "${FILES[@]}"; do + process_file "${file}" +done + +# Process file content line by line +while IFS= read -r line; do + process_line "${line}" +done < "${INPUT_FILE}" +``` + +## SLURM Job Management +* Generate dynamic job names that include relevant parameters +* Include job name in all output file paths for traceability +* Use job dependencies to create processing pipelines +* Prioritize resource efficiency (CPU, memory, runtime) +* Implement job array submissions for parallel batch processing +* Verify output file existence before submitting dependent jobs +* Use consistent partition assignment for related jobs + +```bash +# Dynamic job naming +JOB_NAME="process_${SAMPLE_ID}_${TIMESTAMP}" + +# SLURM directives +#SBATCH --job-name="${JOB_NAME}" +#SBATCH --output="logs/${JOB_NAME}_%j.out" +#SBATCH --error="logs/${JOB_NAME}_%j.err" +#SBATCH --time=1:00:00 +#SBATCH --mem=4G +#SBATCH --cpus-per-task=2 + +# Job dependency example +if [[ -f "${PREVIOUS_OUTPUT}" ]]; then + sbatch --dependency=afterok:${PREVIOUS_JOB_ID} "${NEXT_SCRIPT}" "${PARAMS}" +fi +``` + +## Performance Considerations +* Pre-allocate storage for large operations +* Process data in appropriate batch sizes +* Monitor and log resource usage at regular intervals +* Use pipes instead of temporary files for sequential operations +* Validate processed data after each critical step +* Implement checkpointing for long-running operations + +```bash +# Process in batches with progress tracking +local total_items="${#ITEMS[@]}" +local batch_size=100 +local processed=0 + +for ((i=0; i/dev/null; then + error_exit "Required command 'seqkit' not found" +fi + +# Execute with error checking +if ! seqkit stats "${INPUT_FILE}" > "${OUTPUT_FILE}"; then + error_exit "seqkit command failed on ${INPUT_FILE}" +fi +``` + +## Documentation and Maintainability +* Include usage examples in script header comments +* Document function purpose, parameters, and return values +* Add comments explaining complex logic or non-obvious decisions +* Include version information and change history +* Document dependencies and required environment +* Add inline citations to relevant documentation or references + +```bash +#============================================================================== +# analyze_scaffolds.sh - Analyze genome scaffold lengths +# +# Usage: ./analyze_scaffolds.sh INPUT_DIR MIN_LENGTH +# +# Examples: +# ./analyze_scaffolds.sh /data/scaffolds 10000 +# ./analyze_scaffolds.sh /data/scaffolds 5000 > results.txt +# +# Dependencies: +# - seqkit (v2.0+) +# - GNU coreutils +# +# Version: 1.2 (2025-03-28) +#============================================================================== +``` + + +# Handover document header - Handover Document + +Use this document as a general tempalte for writing a handover document + +## Current State + +### Key Architectural Components + +1. **Data Model**: + - Uses NetworkX's MultiDiGraph to represent relationships between entities + - Core relationships: + - Proteins → Peptides → Modifications + - Peptides → Grid Cells (with intensities) + - Samples → Conditions/Replicates + - Complete graph persistence to disk for faster reloading + +2. **Grid Assignment Logic**: + - Multiple strategies for coordinate determination: + - Explicit grid coordinates (Grid_Row/Col columns) + - Source file name pattern extraction (e.g., "_A1_", "B5.wiff") + - Area Sample number mapping to grid positions + - Retention time (RT) and mass-based coordinate calculation + - Fallback hash-based assignment for peptides without coordinates + - Comprehensive validation and consistency checking + +3. **Performance Optimizations**: + - Area sample grid cache prevents redundant calculations (major performance gain) + - Parallel processing with ThreadPoolExecutor for batch operations + - Reduced logging verbosity with environment variable `PROTEOMICS_LOG_LEVEL` + - Strategic caching of grid data for visualization + - Batch processing of graph updates + +4. **Visualization System**: + - Interactive 8x10 grid visualization replicating actual 2D gel appearance + - Heatmap view for comparative analysis across conditions + - Color-coded intensity representation + - PTM-specific filtering with Ascore cutoff support + +## Recent Refactoring + +The codebase has undergone significant optimization work focused on grid assignment performance: + +1. **Grid Assignment Performance**: + - Added caching system for Area Sample → Grid position mappings + - Eliminated redundant calculations of the same mapping + - Reduced verbose logging that was slowing down processing + - Implemented environment variable control for logging verbosity + - Added summary statistics instead of detailed peptide lists + +2. **Data Processing Optimization**: + - Better thread management in parallel processing + - Batch processing of rows for more efficient NetworkX operations + - Fixed error handling and added comprehensive logging + - Added area sample grid cache with efficient lookup + +3. **Logging Improvements**: + - Fixed compatibility with loguru logging system + - Added conditional logging based on verbosity level + - Reduced log file size while maintaining critical information + - Added top-5 grid summary at INFO level + +4. **Grid Consistency Analysis**: + - Added automatic analysis of grid assignments after loading + - Detection of peptides mapped to multiple grid cells + - Summary statistics for grid assignment patterns + - Support for investigating grid assignment inconsistencies + +## Known Issues + +1. **Neo4j References**: + The UI code may still contain Neo4j database references, despite the backend using NetworkX exclusively. The error "Could not connect to Neo4j database" indicates that some UI components are still trying to use a Neo4j connection that doesn't exist. + +2. **Memory Usage**: + While performance is improved, processing very large datasets may still require optimization of the in-memory graph structure. Consider implementing more aggressive caching or database offloading for extremely large datasets. + +3. **Grid Inconsistencies**: + Some peptides and proteins are assigned to multiple grid cells, which may need further investigation. The current implementation detects these cases but doesn't yet resolve them automatically. + +4. **UI/Backend Synchronization**: + The UI may not be fully aware of all the optimizations made in the backend. Some components might be using older patterns or assumptions that need to be updated. + +## Next Steps + +### Immediate Priorities + +1. **UI Refactoring**: + - Update UI components to fully use the NetworkX implementation + - Remove any remaining Neo4j dependencies + - Ensure heatmap view properly connects to the NetworkX backend + - Update any direct database queries to use the graph API instead + +2. **Testing and Validation**: + - Create comprehensive test cases for grid assignment logic + - Validate correct grid assignments with known datasets + - Test performance with larger datasets + - Verify threading behavior with stress tests + +### Medium-Term Improvements + +1. **Additional Caching Strategies**: + - Consider implementing more aggressive caching for large datasets + - Add optional persistent caching to disk for frequently accessed data + - Implement LRU cache for peptide-grid relationships + +2. **Visualization Enhancements**: + - Improve the heatmap view to better highlight inconsistent grid assignments + - Add visual indicators for peptides with multiple grid assignments + - Enhance the UI for grid browsing and exploration + - Add export options for grid assignment analysis + +3. **Data Processing Refinements**: + - Fine-tune the grid assignment algorithm for edge cases + - Add ML-based grid prediction for ambiguous cases + - Implement smarter batch processing for very large files + +### Long-Term Vision + +1. **Machine Learning Integration**: + - Develop predictive models for grid assignment + - Implement clustering for improved PTM analysis + - Add anomaly detection for unusual peptide patterns + +2. **Advanced Analytics**: + - Enhance statistical analysis of PTM patterns + - Add time-series analysis for longitudinal studies + - Implement comparison tools across multiple experiments + +3. **Architecture Evolution**: + - Consider hybrid database approach for very large datasets + - Evaluate microservices architecture for processing pipeline + - Explore cloud deployment options for collaborative research + +## Configuration and Environment + +### Environment Variables + +- `PROTEOMICS_LOG_LEVEL`: Controls logging verbosity (INFO or DEBUG) + +### Important Files + +- `app/modules/data_loader.py`: Core implementation of the NetworkX data model and grid assignment logic +- `app/ui/dashboard.py`: Main UI controller that might need Neo4j reference cleanup +- `app/ui/heatmap_view.py`: Likely contains Neo4j references that need updating +- `app/ui/gel_grid.py`: 2D gel grid visualization component + +### Logging + +- Primary log file: `proteomics_data.log` with 10MB rotation +- Debug level logging includes detailed grid assignment information +- INFO level provides summary statistics only + +## Performance Metrics + +- **File Loading**: Substantial speed improvement over previous implementation +- **Grid Assignment**: No longer redundantly calculates the same grid positions +- **Memory Usage**: More efficient with large datasets due to better caching +- **Concurrency**: Thread-safe implementation with proper error handling + +## Contact Information + +For questions about the recent refactoring work or implementation details: + +- Developer: [Your Name/Contact Info] +- Last Updated: [Current Date] + +## Acknowledgements + +This handover document summarizes the work completed during performance optimization and refactoring of the 2D Gel Dashboard for Proteomics application. The improvements focus on enhancing the grid assignment process, optimizing memory usage, and implementing a more efficient data processing pipeline. +handover.md +16 KB + + +# R Programming Standards and Best Practices + +This guide outlines the standards and best practices for writing efficient, maintainable, and reproducible R code. Following these guidelines will ensure consistency and readability across projects. + +## Core Principles + +* Write code that is **readable**, **maintainable**, and **reproducible** +* Prioritize **functional programming** approaches over imperative programming +* Design code to be **modular** with clearly defined responsibilities +* Follow **consistent naming conventions** and code organization +* Ensure **comprehensive documentation** of all functions and complex operations +* Prefer **tidyverse** approaches for data manipulation and visualization + +## Code Design and Workflow + +### General Programming Style + +* Break complex problems into smaller, focused functions +* Prioritize composable functions that each do one thing well +* Place commas at the start of each line rather than the end in multi-line statements +* Limit line length to 80-100 characters for improved readability +* Use the native pipe operator (`|>`) rather than the magrittr pipe (`%>%`) +* Align assignments within code blocks for improved readability +* Use meaningful spacing and indentation to visually organize your code + +### Commenting and Documentation + +* Include comprehensive header comments for scripts and functions +* Document **why** code was written, not just what it does +* Use roxygen2 style documentation for all functions: + - `@param` for all parameters with types and expected formats + - `@return` describing the output shape and format + - `@examples` showing usage with sample data + - `@description` for high-level function purpose +* Include details on input/output data frames, including expected columns +* Comment complex or non-obvious code sections with explanations + +```r +#' Calculate weighted mean of values by group +#' +#' @description +#' Calculates the weighted mean of a numeric column, with weights applied +#' and results grouped by a categorical variable. +#' +#' @param data A data frame containing the values and weights +#' @param value_col Name of the numeric column containing values +#' @param weight_col Name of the numeric column containing weights +#' @param group_col Name of the categorical column for grouping +#' +#' @return A data frame with columns: +#' - group: The grouping variable values +#' - weighted_mean: The calculated weighted means +#' +#' @examples +#' df <- data.frame( +#' category = c("A", "A", "B", "B", "C"), +#' value = c(10, 20, 15, 25, 30), +#' weight = c(1, 2, 1, 3, 2) +#' ) +#' calculateWeightedMeanByGroup(df, value_col = value, weight_col = weight, group_col = category) +calculateWeightedMeanByGroup <- function(data, value_col, weight_col, group_col) { + data |> + group_by({{ group_col }}) |> + summarize( + weighted_mean = weighted.mean({{ value_col }}, {{ weight_col }}) + ) +} +``` + +### Variable and Function Naming + +* Use **descriptive**, **precise** names that clearly indicate purpose +* Follow consistent naming conventions: + - `camelCase` for function names (e.g., `calculateMean`, `filterData`) + - `snake_case` for variable names (e.g., `total_count`, `mean_value`) + - `SCREAMING_SNAKE_CASE` for constants (e.g., `MAX_ITERATIONS`, `DEFAULT_PATH`) +* Never reuse variable names within the same script or function +* Avoid abbreviations unless very common in the domain +* Include relevant units in names where appropriate (e.g., `time_seconds`, `distance_km`) + +### Error Handling + +* Validate inputs at the beginning of functions +* Use informative error messages that explain what went wrong and how to fix it +* Avoid using `tryCatch()` during interactive development as it masks useful errors +* Only add error handling once code is working correctly and well-tested + +## Functional Programming Practices + +### Functional Programming Fundamentals + +* Design **pure functions** whenever possible (same inputs always produce same outputs) +* Isolate side effects (I/O, plotting, randomization) in dedicated functions +* Minimize state changes and mutation of data +* Use functional composition to build complex operations from simple ones + +### Avoiding Loops + +* Prefer vectorized operations and functional approaches over explicit loops +* Use the **purrr** package functions as your first choice for iteration: + - `map()`, `map_dbl()`, `map_chr()`, etc. for single-input iteration + - `map2()`, `pmap()` for multi-input iteration + - `walk()` family for side effects +* If purrr isn't available, use base R's `lapply()`, `sapply()`, or `vapply()` +* Only use `for` loops when vectorized solutions are impractical or inefficient + +```r +# AVOID THIS: +result <- c() +for (i in 1:length(numbers)) { + result[i] <- numbers[i]^2 +} + +# DO THIS INSTEAD: +# Vectorized solution +result <- numbers^2 + +# Or with purrr +result <- map_dbl(numbers, ~ .x^2) +``` + +### Efficient Data Manipulation Patterns + +* Pre-allocate output containers for large operations +* Use `reduce()` and `accumulate()` for sequential operations on list elements +* Create function factories to generate specialized functions + +```r +# Using reduce to sequentially combine elements +accumulate(letters[1:5], paste, sep = ".") +# [1] "a" "a.b" "a.b.c" "a.b.c.d" "a.b.c.d.e" + +# Function factory example +createScalingFunction <- function(scaling_factor) { + function(x) { + x * scaling_factor + } +} +double <- createScalingFunction(2) +triple <- createScalingFunction(3) +``` + +### Using Expand Grid for Combinations + +* Instead of nested loops or map calls, use `expand_grid()` with `pmap()` +* This provides explicit tabular structure for combinations and is easier to parallelize + +```r +# Process all combinations of two lists +params <- expand_grid( + x = 1:5, + y = c("a", "b", "c") +) + +results <- params |> + mutate(result = pmap_chr(list(x, y), ~ paste0(..1, ..2))) +``` + +## Tidyverse and Tidy Evaluation + +### Data Wrangling Standards + +* Use **tidyverse** functions for data manipulation (dplyr, tidyr, purrr) +* Chain operations with the pipe operator +* Use **janitor** for data cleaning and standardizing column names +* Keep transformations in logical groupings with clear intermediate object names + +### Using Tidy Evaluation + +* Use curly-curly (`{{}}`) for passing column names to tidyverse functions +* Use `.data[[var_name]]` pronoun for dynamic column selection with strings +* Use the `:=` operator when creating dynamic column names + +```r +# Using curly-curly for column name passing +summarizeByGroup <- function(data, group_var, summarize_var) { + data |> + group_by({{ group_var }}) |> + summarize(mean = mean({{ summarize_var }}, na.rm = TRUE)) +} + +# Using .data pronoun for string column names +summarizeByString <- function(data, group_name, value_name) { + data |> + group_by(.data[[group_name]]) |> + summarize(mean = mean(.data[[value_name]], na.rm = TRUE)) +} + +# Using := for dynamic column naming +createSummary <- function(data, var, stat_name = "mean") { + data |> + summarize("{{ var }}_{{ stat_name }}" := mean({{ var }}, na.rm = TRUE)) +} +``` + +## Package and Dependency Management + +### Loading Libraries + +* Use **pacman** for reproducible library management +* Load all packages at the beginning of scripts +* Provide fallback installation code for missing dependencies + +```r +# Standard package loading preamble +if (!require("pacman")) { + install.packages("pacman") + library(pacman) +} + +pacman::p_load( + tidyverse, # Data manipulation and visualization + lubridate, # Date handling + glue, # String interpolation + janitor, # Data cleaning + fs # File system operations +) + +# For Bioconductor packages +if (!requireNamespace("BiocManager", quietly = TRUE)) { + install.packages("BiocManager") +} +BiocManager::install("GenomicRanges") +``` + +### Managing Function Name Conflicts + +* Use the **conflicted** package to explicitly resolve function name conflicts +* Explicitly qualify ambiguous function calls with package names +* Set conflict preferences at the start of scripts + +```r +library(conflicted) + +# Set default preferences for common conflicts +conflicted::conflict_prefer("filter", "dplyr") +conflicted::conflict_prefer("select", "dplyr") +conflicted::conflict_prefer("rename", "dplyr") +``` + +## Project Organization and Workflow + +### Structure and Organization + +* Organize projects with consistent directory structure +* Create dedicated directories for data, scripts, outputs, and documentation +* Use relative paths with the **here** package for file references +* Maintain a clear separation between data acquisition, processing, and analysis + +### Reproducibility Best Practices + +* Use version control (Git) for tracking code changes +* Document dependencies with `sessionInfo()` or **renv** +* Include seed setting for random processes: `set.seed()` +* Create self-contained analysis with all data processing steps +* Write automated tests for critical functions + +## Editing and Refactoring Code + +* Always review code changes before submitting or finalizing +* Check that code deletions don't remove unrelated functionality +* When refactoring: + - Ensure all tests still pass + - Verify outputs remain unchanged + - Document significant design changes +* Maintain a change log for major modifications +* Focus changes on addressing specific requirements rather than wholesale rewrites + +--- + +Remember: Prioritize readability and maintainability over cleverness or terseness. Code is read far more often than it is written. + +# LLM Rule: Debug Rule 66 - Verbose Step-Trace Debugging + +**Description:** +This rule defines a process for instrumenting specified R code (functions, S4 methods) with detailed logging statements to trace execution flow and data state at runtime. The goal is to produce verbose console output that helps pinpoint exactly where a process is failing or where data becomes problematic. + +**Invocation:** +Triggered when the user explicitly requests `` for specific functions/methods, or asks for highly detailed, step-by-step debugging trace on specified code sections. + +**Procedure:** + +1. **Identify Targets:** Determine the primary function(s) or S4 method(s) suspected of causing the issue, including any key helper functions they call internally. + +2. **Instrument Code:** Using the `edit_file` tool, modify the target R code by adding logging statements at critical points: + * **Function Entry/Exit:** + * Add `message(sprintf("--- Entering [FunctionName] ---"))` at the start. + * Add `message(sprintf("--- Exiting [FunctionName] ---"))` just before the return statement. If possible, include the value being returned: `message(sprintf("--- Exiting [FunctionName]. Returning: %s ---", capture.output(str(returnValue))))`. + * **Argument Inspection:** + * Immediately after entry, log key input arguments using `message(sprintf(" [FunctionName] Arg: [ArgumentName] = %s", capture.output(str(argumentValue))))` or similar using `print()` or specific formatting depending on the argument type. + * **Major Logic Steps:** + * Add `message(sprintf(" [FunctionName] Step: [Description of step about to happen]..."))` *before* the step. + * Add `message(sprintf(" [FunctionName] Step: [Description of step completed]."))` *after* the step. (e.g., "Filtering data", "Pivoting wider", "Calling helper X"). + * **Data State Inspection:** + * *Before* a data frame/tibble/matrix is used in a crucial operation (passed to another function, used in a join, pivoted, filtered, passed to `stats::cor`, etc.), log its state: + * `message(sprintf(" Data State ([VariableName]): Dims = %d rows, %d cols", nrow(variableName), ncol(variableName)))` + * `message(" Data State ([VariableName]) Structure:")` + * `utils::str(variableName)` + * `message(" Data State ([VariableName]) Head:")` + * `print(head(variableName))` + * **Loops/Mapping Functions (e.g., `purrr::map`, `lapply`, `for`):** + * Inside the loop/map function, log the current iteration identifier(s): `message(sprintf(" [map/loop] Processing item: %s", itemIdentifier))` + * Log the result obtained for that iteration *before* it's returned or stored. + * **Conditional Logic (`if/else`):** + * Add `message(sprintf(" [FunctionName] Condition TRUE: [Brief reason]"))` inside the `if` block. + * Add `message(sprintf(" [FunctionName] Condition FALSE: [Brief reason]"))` inside the `else` block. + * **Function Call Results:** + * Immediately after calling an important internal or helper function, log the structure/value of the result: `message(sprintf(" [FunctionName] Result from [CalledFunction]: %s", capture.output(str(resultVariable))))` + +3. **Output Formatting:** + * Use `message()` for general flow and information. + * Use `print()` and `utils::str()` for detailed object inspection. + * Use `sprintf()` within `message()` for consistent formatting. + * Use indentation or prefixes (e.g., `>>`, ` `, ` `) to indicate nesting levels (e.g., inside helpers called by main functions, inside loops). + +4. **Result:** The execution of the instrumented code will produce a detailed trace in the R console, allowing the user and the AI to follow the execution path and inspect data at intermediate steps to identify the source of errors or unexpected behavior. + diff --git a/build.sh b/build.sh new file mode 100644 index 00000000..c7bbc661 --- /dev/null +++ b/build.sh @@ -0,0 +1,9 @@ +#!/bin/bash +set -e +# We are in micropolis-rs, so go up one level +cd .. +python3 -m venv .venv +source .venv/bin/activate +pip install arcade maturin +cd micropolis-rs +cargo test diff --git a/micropolis-arcade-ui/README.md b/micropolis-arcade-ui/README.md new file mode 100644 index 00000000..71abd434 --- /dev/null +++ b/micropolis-arcade-ui/README.md @@ -0,0 +1,3 @@ +# Micropolis Arcade UI + +This directory contains the Python UI for Micropolis, built using the Arcade library. diff --git a/micropolis-arcade-ui/main.py b/micropolis-arcade-ui/main.py new file mode 100644 index 00000000..889ff115 --- /dev/null +++ b/micropolis-arcade-ui/main.py @@ -0,0 +1,91 @@ +import os +os.environ["ARCADE_HEADLESS"] = "True" +import arcade +import micropolis_rs + +# --- Constants --- +SCREEN_WIDTH = 800 +SCREEN_HEIGHT = 600 +SCREEN_TITLE = "Micropolis" + +WORLD_WIDTH = 120 +WORLD_HEIGHT = 100 +TILE_SIZE = 8 + +COLOR_MAP = { + micropolis_rs.DIRT: arcade.color.BROWN, + micropolis_rs.RIVER: arcade.color.BLUE, + micropolis_rs.TREEBASE: arcade.color.DARK_GREEN, + micropolis_rs.WOODS: arcade.color.DARK_GREEN, + micropolis_rs.ROADBASE: arcade.color.GRAY, + micropolis_rs.RESBASE: arcade.color.GREEN, + micropolis_rs.COMBASE: arcade.color.LIGHT_BLUE, + micropolis_rs.INDBASE: arcade.color.YELLOW, +} + +class GameWindow(arcade.Window): + """ + Main application window. + """ + + def __init__(self, width, height, title): + super().__init__(width, height, title) + arcade.set_background_color(arcade.color.BLACK) + self.micropolis = micropolis_rs.Micropolis() + self.frame_count = 0 + + def setup(self): + """ Set up the game and initialize the variables. """ + pass + + def on_draw(self): + """ + Render the screen. + """ + self.clear() + + map_view = self.micropolis.get_map_view() + + for y in range(WORLD_HEIGHT): + for x in range(WORLD_WIDTH): + tile_type = map_view[y * WORLD_WIDTH + x] + color = COLOR_MAP.get(tile_type, arcade.color.BLACK) + + # For zone tiles, we need to check the base type + if micropolis_rs.RESBASE <= tile_type < micropolis_rs.COMBASE: + color = COLOR_MAP.get(micropolis_rs.RESBASE, arcade.color.BLACK) + elif micropolis_rs.COMBASE <= tile_type < micropolis_rs.INDBASE: + color = COLOR_MAP.get(micropolis_rs.COMBASE, arcade.color.BLACK) + elif micropolis_rs.INDBASE <= tile_type < micropolis_rs.PORTBASE: + color = COLOR_MAP.get(micropolis_rs.INDBASE, arcade.color.BLACK) + + + arcade.draw_rectangle_filled( + x * TILE_SIZE + TILE_SIZE / 2, + y * TILE_SIZE + TILE_SIZE / 2, + TILE_SIZE, + TILE_SIZE, + color, + ) + + def on_update(self, delta_time): + """ Movement and game logic """ + self.micropolis.step_simulation() + self.frame_count += 1 + if self.frame_count > 5: + print("Taking screenshot...") + image = arcade.get_image() + image.save("screenshot.png", "PNG") + print("Screenshot saved to screenshot.png") + self.close() + + +def main(): + """ Main method """ + window = GameWindow(SCREEN_WIDTH, SCREEN_HEIGHT, SCREEN_TITLE) + window.setup() + arcade.run() + + +if __name__ == "__main__": + main() diff --git a/micropolis-rs/Cargo.lock b/micropolis-rs/Cargo.lock new file mode 100644 index 00000000..44596189 --- /dev/null +++ b/micropolis-rs/Cargo.lock @@ -0,0 +1,403 @@ +# This file is automatically @generated by Cargo. +# It is not intended for manual editing. +version = 4 + +[[package]] +name = "autocfg" +version = "1.5.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c08606f8c3cbf4ce6ec8e28fb0014a2c086708fe954eaa885384a6165172e7e8" + +[[package]] +name = "bincode" +version = "1.3.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b1f45e9417d87227c7a56d22e471c6206462cba514c7590c09aff4cf6d1ddcad" +dependencies = [ + "serde", +] + +[[package]] +name = "bitflags" +version = "2.9.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "34efbcccd345379ca2868b2b2c9d3782e9cc58ba87bc7d79d5b53d9c9ae6f25d" + +[[package]] +name = "cfg-if" +version = "1.0.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2fd1289c04a9ea8cb22300a459a72a385d7c73d3259e2ed7dcb2af674838cfa9" + +[[package]] +name = "getrandom" +version = "0.2.16" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "335ff9f135e4384c8150d6f27c6daed433577f86b4750418338c01a1a2528592" +dependencies = [ + "cfg-if", + "libc", + "wasi", +] + +[[package]] +name = "heck" +version = "0.4.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "95505c38b4572b2d910cecb0281560f54b440a19336cbbcb27bf6ce6adc6f5a8" + +[[package]] +name = "indoc" +version = "2.0.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f4c7245a08504955605670dbf141fceab975f15ca21570696aebe9d2e71576bd" + +[[package]] +name = "libc" +version = "0.2.175" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "6a82ae493e598baaea5209805c49bbf2ea7de956d50d7da0da1164f9c6d28543" + +[[package]] +name = "lock_api" +version = "0.4.13" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "96936507f153605bddfcda068dd804796c84324ed2510809e5b2a624c81da765" +dependencies = [ + "autocfg", + "scopeguard", +] + +[[package]] +name = "memoffset" +version = "0.9.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "488016bfae457b036d996092f6cb448677611ce4449e970ceaf42695203f218a" +dependencies = [ + "autocfg", +] + +[[package]] +name = "micropolis-rs" +version = "0.1.0" +dependencies = [ + "bincode", + "pyo3", + "rand", + "serde", +] + +[[package]] +name = "once_cell" +version = "1.21.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "42f5e15c9953c5e4ccceeb2e7382a716482c34515315f7b03532b8b4e8393d2d" + +[[package]] +name = "parking_lot" +version = "0.12.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "70d58bf43669b5795d1576d0641cfb6fbb2057bf629506267a92807158584a13" +dependencies = [ + "lock_api", + "parking_lot_core", +] + +[[package]] +name = "parking_lot_core" +version = "0.9.11" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "bc838d2a56b5b1a6c25f55575dfc605fabb63bb2365f6c2353ef9159aa69e4a5" +dependencies = [ + "cfg-if", + "libc", + "redox_syscall", + "smallvec", + "windows-targets", +] + +[[package]] +name = "portable-atomic" +version = "1.11.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f84267b20a16ea918e43c6a88433c2d54fa145c92a811b5b047ccbe153674483" + +[[package]] +name = "ppv-lite86" +version = "0.2.21" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "85eae3c4ed2f50dcfe72643da4befc30deadb458a9b590d720cde2f2b1e97da9" +dependencies = [ + "zerocopy", +] + +[[package]] +name = "proc-macro2" +version = "1.0.101" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "89ae43fd86e4158d6db51ad8e2b80f313af9cc74f5c0e03ccb87de09998732de" +dependencies = [ + "unicode-ident", +] + +[[package]] +name = "pyo3" +version = "0.21.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a5e00b96a521718e08e03b1a622f01c8a8deb50719335de3f60b3b3950f069d8" +dependencies = [ + "cfg-if", + "indoc", + "libc", + "memoffset", + "parking_lot", + "portable-atomic", + "pyo3-build-config", + "pyo3-ffi", + "pyo3-macros", + "unindent", +] + +[[package]] +name = "pyo3-build-config" +version = "0.21.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7883df5835fafdad87c0d888b266c8ec0f4c9ca48a5bed6bbb592e8dedee1b50" +dependencies = [ + "once_cell", + "target-lexicon", +] + +[[package]] +name = "pyo3-ffi" +version = "0.21.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "01be5843dc60b916ab4dad1dca6d20b9b4e6ddc8e15f50c47fe6d85f1fb97403" +dependencies = [ + "libc", + "pyo3-build-config", +] + +[[package]] +name = "pyo3-macros" +version = "0.21.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "77b34069fc0682e11b31dbd10321cbf94808394c56fd996796ce45217dfac53c" +dependencies = [ + "proc-macro2", + "pyo3-macros-backend", + "quote", + "syn", +] + +[[package]] +name = "pyo3-macros-backend" +version = "0.21.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "08260721f32db5e1a5beae69a55553f56b99bd0e1c3e6e0a5e8851a9d0f5a85c" +dependencies = [ + "heck", + "proc-macro2", + "pyo3-build-config", + "quote", + "syn", +] + +[[package]] +name = "quote" +version = "1.0.40" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1885c039570dc00dcb4ff087a89e185fd56bae234ddc7f056a945bf36467248d" +dependencies = [ + "proc-macro2", +] + +[[package]] +name = "rand" +version = "0.8.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "34af8d1a0e25924bc5b7c43c079c942339d8f0a8b57c39049bef581b46327404" +dependencies = [ + "libc", + "rand_chacha", + "rand_core", +] + +[[package]] +name = "rand_chacha" +version = "0.3.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e6c10a63a0fa32252be49d21e7709d4d4baf8d231c2dbce1eaa8141b9b127d88" +dependencies = [ + "ppv-lite86", + "rand_core", +] + +[[package]] +name = "rand_core" +version = "0.6.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ec0be4795e2f6a28069bec0b5ff3e2ac9bafc99e6a9a7dc3547996c5c816922c" +dependencies = [ + "getrandom", +] + +[[package]] +name = "redox_syscall" +version = "0.5.17" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5407465600fb0548f1442edf71dd20683c6ed326200ace4b1ef0763521bb3b77" +dependencies = [ + "bitflags", +] + +[[package]] +name = "scopeguard" +version = "1.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "94143f37725109f92c262ed2cf5e59bce7498c01bcc1502d7b9afe439a4e9f49" + +[[package]] +name = "serde" +version = "1.0.219" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5f0e2c6ed6606019b4e29e69dbaba95b11854410e5347d525002456dbbb786b6" +dependencies = [ + "serde_derive", +] + +[[package]] +name = "serde_derive" +version = "1.0.219" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5b0276cf7f2c73365f7157c8123c21cd9a50fbbd844757af28ca1f5925fc2a00" +dependencies = [ + "proc-macro2", + "quote", + "syn", +] + +[[package]] +name = "smallvec" +version = "1.15.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "67b1b7a3b5fe4f1376887184045fcf45c69e92af734b7aaddc05fb777b6fbd03" + +[[package]] +name = "syn" +version = "2.0.106" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ede7c438028d4436d71104916910f5bb611972c5cfd7f89b8300a8186e6fada6" +dependencies = [ + "proc-macro2", + "quote", + "unicode-ident", +] + +[[package]] +name = "target-lexicon" +version = "0.12.16" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "61c41af27dd6d1e27b1b16b489db798443478cef1f06a660c96db617ba5de3b1" + +[[package]] +name = "unicode-ident" +version = "1.0.18" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5a5f39404a5da50712a4c1eecf25e90dd62b613502b7e925fd4e4d19b5c96512" + +[[package]] +name = "unindent" +version = "0.2.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7264e107f553ccae879d21fbea1d6724ac785e8c3bfc762137959b5802826ef3" + +[[package]] +name = "wasi" +version = "0.11.1+wasi-snapshot-preview1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ccf3ec651a847eb01de73ccad15eb7d99f80485de043efb2f370cd654f4ea44b" + +[[package]] +name = "windows-targets" +version = "0.52.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9b724f72796e036ab90c1021d4780d4d3d648aca59e491e6b98e725b84e99973" +dependencies = [ + "windows_aarch64_gnullvm", + "windows_aarch64_msvc", + "windows_i686_gnu", + "windows_i686_gnullvm", + "windows_i686_msvc", + "windows_x86_64_gnu", + "windows_x86_64_gnullvm", + "windows_x86_64_msvc", +] + +[[package]] +name = "windows_aarch64_gnullvm" +version = "0.52.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "32a4622180e7a0ec044bb555404c800bc9fd9ec262ec147edd5989ccd0c02cd3" + +[[package]] +name = "windows_aarch64_msvc" +version = "0.52.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "09ec2a7bb152e2252b53fa7803150007879548bc709c039df7627cabbd05d469" + +[[package]] +name = "windows_i686_gnu" +version = "0.52.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8e9b5ad5ab802e97eb8e295ac6720e509ee4c243f69d781394014ebfe8bbfa0b" + +[[package]] +name = "windows_i686_gnullvm" +version = "0.52.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0eee52d38c090b3caa76c563b86c3a4bd71ef1a819287c19d586d7334ae8ed66" + +[[package]] +name = "windows_i686_msvc" +version = "0.52.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "240948bc05c5e7c6dabba28bf89d89ffce3e303022809e73deaefe4f6ec56c66" + +[[package]] +name = "windows_x86_64_gnu" +version = "0.52.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "147a5c80aabfbf0c7d901cb5895d1de30ef2907eb21fbbab29ca94c5b08b1a78" + +[[package]] +name = "windows_x86_64_gnullvm" +version = "0.52.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "24d5b23dc417412679681396f2b49f3de8c1473deb516bd34410872eff51ed0d" + +[[package]] +name = "windows_x86_64_msvc" +version = "0.52.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "589f6da84c646204747d1270a2a5661ea66ed1cced2631d546fdfb155959f9ec" + +[[package]] +name = "zerocopy" +version = "0.8.26" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1039dd0d3c310cf05de012d8a39ff557cb0d23087fd44cad61df08fc31907a2f" +dependencies = [ + "zerocopy-derive", +] + +[[package]] +name = "zerocopy-derive" +version = "0.8.26" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9ecf5b4cc5364572d7f4c329661bcc82724222973f2cab6f050a4e5c22f75181" +dependencies = [ + "proc-macro2", + "quote", + "syn", +] diff --git a/micropolis-rs/Cargo.toml b/micropolis-rs/Cargo.toml new file mode 100644 index 00000000..efd01661 --- /dev/null +++ b/micropolis-rs/Cargo.toml @@ -0,0 +1,14 @@ +[package] +name = "micropolis-rs" +version = "0.1.0" +edition = "2021" + +[lib] +name = "micropolis_engine" +crate-type = ["cdylib", "rlib"] + +[dependencies] +pyo3 = { version = "0.21", features = ["extension-module"] } +serde = { version = "1.0", features = ["derive"] } +bincode = "1.3.3" +rand = "0.8.5" diff --git a/micropolis-rs/build.sh b/micropolis-rs/build.sh new file mode 100755 index 00000000..f801f481 --- /dev/null +++ b/micropolis-rs/build.sh @@ -0,0 +1,7 @@ +#!/bin/bash +set -e +python3 -m venv .venv +source .venv/bin/activate +pip install arcade maturin +cd micropolis-rs +cargo test diff --git a/micropolis-rs/city.dat b/micropolis-rs/city.dat new file mode 100644 index 00000000..ac57cf98 Binary files /dev/null and b/micropolis-rs/city.dat differ diff --git a/micropolis-rs/src/lib.rs b/micropolis-rs/src/lib.rs new file mode 100644 index 00000000..bf9170c1 --- /dev/null +++ b/micropolis-rs/src/lib.rs @@ -0,0 +1,482 @@ +use pyo3::prelude::*; +use serde::{Serialize, Deserialize}; + +// +// Micropolis - a city simulation engine in Rust +// + +mod sim; +mod zone; + +// Constants from sim.h +pub const PWRBIT: u16 = 32768; +pub const CONDBIT: u16 = 16384; +pub const BURNBIT: u16 = 8192; +pub const BULLBIT: u16 = 4096; +pub const ANIMBIT: u16 = 2048; +pub const ZONEBIT: u16 = 1024; +pub const ALLBITS: u16 = 64512; +pub const LOMASK: u16 = 1023; +pub const BNCNBIT: u16 = BURNBIT | CONDBIT; +pub const BLBNCNBIT: u16 = BULLBIT | BURNBIT | CONDBIT; + +const WORLD_X: usize = 120; +const WORLD_Y: usize = 100; +const HWLDX: usize = WORLD_X / 2; +const HWLDY: usize = WORLD_Y / 2; +const QWX: usize = WORLD_X / 4; +const QWY: usize = WORLD_Y / 4; +const SMX: usize = WORLD_X / 8; +const SMY: usize = (WORLD_Y + 7) / 8; + +const PROBNUM: usize = 10; + +const PWRMAPROW: usize = (WORLD_X + 15) / 16; +const PWRMAPSIZE: usize = PWRMAPROW * WORLD_Y; + +// Tile definitions +pub const DIRT: u16 = 0; +pub const RIVER: u16 = 2; +pub const REDGE: u16 = 3; +pub const CHANNEL: u16 = 4; +pub const FIRSTRIVEDGE: u16 = 5; +pub const LASTRIVEDGE: u16 = 20; +pub const TREEBASE: u16 = 21; +pub const LASTTREE: u16 = 36; +pub const WOODS: u16 = 37; +pub const UNUSED_TRASH1: u16 = 38; +pub const UNUSED_TRASH2: u16 = 39; +pub const WOODS2: u16 = 40; +pub const WOODS3: u16 = 41; +pub const WOODS4: u16 = 42; +pub const WOODS5: u16 = 43; +pub const RUBBLE: u16 = 44; +pub const LASTRUBBLE: u16 = 47; +pub const FLOOD: u16 = 48; +pub const LASTFLOOD: u16 = 51; +pub const RADTILE: u16 = 52; +pub const UNUSED_TRASH3: u16 = 53; +pub const UNUSED_TRASH4: u16 = 54; +pub const UNUSED_TRASH5: u16 = 55; +pub const FIRE: u16 = 56; +pub const FIREBASE: u16 = 56; +pub const LASTFIRE: u16 = 63; +pub const ROADBASE: u16 = 64; +pub const HBRIDGE: u16 = 64; +pub const VBRIDGE: u16 = 65; +pub const ROADS: u16 = 66; +pub const INTERSECTION: u16 = 76; +pub const HROADPOWER: u16 = 77; +pub const VROADPOWER: u16 = 78; +pub const BRWH: u16 = 79; +pub const LTRFBASE: u16 = 80; +pub const BRWV: u16 = 95; +pub const BRWXXX1: u16 = 111; +pub const BRWXXX2: u16 = 127; +pub const BRWXXX3: u16 = 143; +pub const HTRFBASE: u16 = 144; +pub const BRWXXX4: u16 = 159; +pub const BRWXXX5: u16 = 175; +pub const BRWXXX6: u16 = 191; +pub const LASTROAD: u16 = 206; +pub const BRWXXX7: u16 = 207; +pub const POWERBASE: u16 = 208; +pub const HPOWER: u16 = 208; +pub const VPOWER: u16 = 209; +pub const LHPOWER: u16 = 210; +pub const LVPOWER: u16 = 211; +pub const RAILHPOWERV: u16 = 221; +pub const RAILVPOWERH: u16 = 222; +pub const LASTPOWER: u16 = 222; +pub const UNUSED_TRASH6: u16 = 223; +pub const RAILBASE: u16 = 224; +pub const HRAIL: u16 = 224; +pub const VRAIL: u16 = 225; +pub const LHRAIL: u16 = 226; +pub const LVRAIL: u16 = 227; +pub const HRAILROAD: u16 = 237; +pub const VRAILROAD: u16 = 238; +pub const LASTRAIL: u16 = 238; +pub const ROADVPOWERH: u16 = 239; /* bogus? */ +pub const RESBASE: u16 = 240; +pub const FREEZ: u16 = 244; +pub const HOUSE: u16 = 249; +pub const LHTHR: u16 = 249; +pub const HHTHR: u16 = 260; +pub const RZB: u16 = 265; +pub const HOSPITAL: u16 = 409; +pub const CHURCH: u16 = 418; +pub const COMBASE: u16 = 423; +pub const COMCLR: u16 = 427; +pub const CZB: u16 = 436; +pub const INDBASE: u16 = 612; +pub const INDCLR: u16 = 616; +pub const LASTIND: u16 = 620; +pub const IND1: u16 = 621; +pub const IZB: u16 = 625; +pub const IND2: u16 = 641; +pub const IND3: u16 = 644; +pub const IND4: u16 = 649; +pub const IND5: u16 = 650; +pub const IND6: u16 = 676; +pub const IND7: u16 = 677; +pub const IND8: u16 = 686; +pub const IND9: u16 = 689; +pub const PORTBASE: u16 = 693; +pub const PORT: u16 = 698; +pub const LASTPORT: u16 = 708; +pub const AIRPORTBASE: u16 = 709; +pub const RADAR: u16 = 711; +pub const AIRPORT: u16 = 716; +pub const COALBASE: u16 = 745; +pub const POWERPLANT: u16 = 750; +pub const LASTPOWERPLANT: u16 = 760; +pub const FIRESTBASE: u16 = 761; +pub const FIRESTATION: u16 = 765; +pub const POLICESTBASE: u16 = 770; +pub const POLICESTATION: u16 = 774; +pub const STADIUMBASE: u16 = 779; +pub const STADIUM: u16 = 784; +pub const FULLSTADIUM: u16 = 800; +pub const NUCLEARBASE: u16 = 811; +pub const NUCLEAR: u16 = 816; +pub const LASTZONE: u16 = 826; +pub const LIGHTNINGBOLT: u16 = 827; +pub const HBRDG0: u16 = 828; +pub const HBRDG1: u16 = 829; +pub const HBRDG2: u16 = 830; +pub const HBRDG3: u16 = 831; +pub const RADAR0: u16 = 832; +pub const RADAR1: u16 = 833; +pub const RADAR2: u16 = 834; +pub const RADAR3: u16 = 835; +pub const RADAR4: u16 = 836; +pub const RADAR5: u16 = 837; +pub const RADAR6: u16 = 838; +pub const RADAR7: u16 = 839; +pub const FOUNTAIN: u16 = 840; +pub const INDBASE2: u16 = 844; +pub const TELEBASE: u16 = 844; +pub const TELELAST: u16 = 851; +pub const SMOKEBASE: u16 = 852; +pub const TINYEXP: u16 = 860; +pub const SOMETINYEXP: u16 = 864; +pub const LASTTINYEXP: u16 = 867; +pub const COALSMOKE1: u16 = 916; +pub const COALSMOKE2: u16 = 920; +pub const COALSMOKE3: u16 = 924; +pub const COALSMOKE4: u16 = 928; +pub const FOOTBALLGAME1: u16 = 932; +pub const FOOTBALLGAME2: u16 = 940; +pub const VBRDG0: u16 = 948; +pub const VBRDG1: u16 = 949; +pub const VBRDG2: u16 = 950; +pub const VBRDG3: u16 = 951; + +pub const TILE_COUNT: u16 = 960; + +#[pyclass] +#[derive(Clone, Serialize, Deserialize)] +pub struct CityStats { + #[pyo3(get)] + pub city_time: i64, + #[pyo3(get)] + pub total_funds: i64, + #[pyo3(get)] + pub total_pop: i32, + #[pyo3(get)] + pub res_pop: i32, + #[pyo3(get)] + pub com_pop: i32, + #[pyo3(get)] + pub ind_pop: i32, +} + +#[pyclass] +#[derive(Clone, Serialize, Deserialize)] +pub struct Micropolis { + // The main map + map: Vec>, + + // 2x2 averaged maps + pop_density: Vec>, + trf_density: Vec>, + pollution_mem: Vec>, + land_value_mem: Vec>, + crime_mem: Vec>, + + // 4x4 averaged maps + terrain_mem: Vec>, + + // 8x8 averaged maps + rate_og_mem: Vec>, + fire_st_map: Vec>, + police_map: Vec>, + + // Power map + power_map: Vec, + + // Simulation state variables + s_map_x: i16, + s_map_y: i16, + c_chr: u16, + c_chr9: u16, + new_power: bool, + pwrd_z_cnt: i32, + un_pwrd_z_cnt: i32, + res_z_pop: i32, + com_z_pop: i32, + ind_z_pop: i32, + fire_pop: i32, + hosp_pop: i32, + church_pop: i32, + need_hosp: i16, + need_church: i16, + road_total: i32, + rail_total: i32, + police_pop: i32, + fire_st_pop: i32, + stadium_pop: i32, + port_pop: i32, + aport_pop: i32, + coal_pop: i32, + nuclear_pop: i32, + power_stack_num: i32, + + #[pyo3(get, set)] + pub city_time: i64, + #[pyo3(get, set)] + pub total_funds: i64, + #[pyo3(get, set)] + pub city_tax: i16, + pub game_level: i16, + pub sim_speed: i16, + spd_cycle: i16, + f_cycle: i16, + s_cycle: i16, + do_initial_eval: bool, + av_city_tax: i16, + + res_pop: i32, + com_pop: i32, + ind_pop: i32, + total_pop: i32, + last_total_pop: i32, + res_valve: i16, + com_valve: i16, + ind_valve: i16, + res_cap: bool, + com_cap: bool, + ind_cap: bool, + + eval_valid: bool, + city_yes: i16, + city_no: i16, + problem_table: [i16; 10], + problem_taken: [i16; 10], + problem_votes: [i16; 10], + problem_order: [i16; 4], + city_pop: i64, + delta_city_pop: i64, + city_ass_value: i64, + city_class: i16, + city_score: i16, + delta_city_score: i16, + average_city_score: i16, + traffic_average: i16, + lv_average: i16, + pollute_average: i16, + crime_average: i16, + + // History arrays + res_his: Vec, + com_his: Vec, + ind_his: Vec, + money_his: Vec, + crime_his: Vec, + pollution_his: Vec, + misc_his: Vec, +} + +impl Micropolis { + pub fn test_bounds(&self, x: i16, y: i16) -> bool { + x >= 0 && x < WORLD_X as i16 && y >= 0 && y < WORLD_Y as i16 + } + + fn power_word(&self, x: i16, y: i16) -> usize { + (x as usize >> 4) + ((y as usize) << 3) + } +} + +#[pymethods] +impl Micropolis { + #[new] + pub fn new() -> Self { + Self { + map: vec![vec![DIRT; WORLD_Y]; WORLD_X], + pop_density: vec![vec![0; HWLDY]; HWLDX], + trf_density: vec![vec![0; HWLDY]; HWLDX], + pollution_mem: vec![vec![0; HWLDY]; HWLDX], + land_value_mem: vec![vec![0; HWLDY]; HWLDX], + crime_mem: vec![vec![0; HWLDY]; HWLDX], + terrain_mem: vec![vec![0; QWY]; QWX], + rate_og_mem: vec![vec![0; SMY]; SMX], + fire_st_map: vec![vec![0; SMY]; SMX], + police_map: vec![vec![0; SMY]; SMX], + power_map: vec![0; PWRMAPSIZE], + + s_map_x: 0, + s_map_y: 0, + c_chr: 0, + c_chr9: 0, + new_power: false, + pwrd_z_cnt: 0, + un_pwrd_z_cnt: 0, + res_z_pop: 0, + com_z_pop: 0, + ind_z_pop: 0, + fire_pop: 0, + hosp_pop: 0, + church_pop: 0, + need_hosp: 0, + need_church: 0, + road_total: 0, + rail_total: 0, + police_pop: 0, + fire_st_pop: 0, + stadium_pop: 0, + coal_pop: 0, + nuclear_pop: 0, + port_pop: 0, + aport_pop: 0, + power_stack_num: 0, + + city_time: 0, + total_funds: 20000, + city_tax: 7, + game_level: 0, + sim_speed: 0, + spd_cycle: 0, + f_cycle: 0, + s_cycle: 0, + do_initial_eval: false, + av_city_tax: 0, + res_pop: 0, + com_pop: 0, + ind_pop: 0, + total_pop: 0, + last_total_pop: 0, + res_valve: 0, + com_valve: 0, + ind_valve: 0, + res_cap: false, + com_cap: false, + ind_cap: false, + + eval_valid: false, + city_yes: 0, + city_no: 0, + problem_table: [0; 10], + problem_taken: [0; 10], + problem_votes: [0; 10], + problem_order: [0; 4], + city_pop: 0, + delta_city_pop: 0, + city_ass_value: 0, + city_class: 0, + city_score: 500, + delta_city_score: 0, + average_city_score: 0, + traffic_average: 0, + lv_average: 0, + pollute_average: 0, + crime_average: 0, + + res_his: vec![0; 480], + com_his: vec![0; 480], + ind_his: vec![0; 480], + money_his: vec![0; 480], + crime_his: vec![0; 480], + pollution_his: vec![0; 480], + misc_his: vec![0; 240], + } + } + + pub fn step_simulation(&mut self) { + self.sim_frame(); + } + + pub fn get_map_view(&self, x: usize, y: usize, w: usize, h: usize) -> PyResult> { + if x + w > WORLD_X || y + h > WORLD_Y { + return Err(PyErr::new::( + "View dimensions are out of bounds", + )); + } + let mut view = Vec::with_capacity(w * h); + for i in y..y + h { + for j in x..x + w { + view.push(self.map[j][i]); + } + } + Ok(view) + } + + #[staticmethod] + pub fn create_city(_width: usize, _height: usize) -> PyResult { + // The core engine currently only supports a fixed size. + // We accept width and height for API compatibility. + Ok(Micropolis::new()) + } + + pub fn get_city_stats(&self) -> CityStats { + CityStats { + city_time: self.city_time, + total_funds: self.total_funds, + total_pop: self.total_pop, + res_pop: self.res_pop, + com_pop: self.com_pop, + ind_pop: self.ind_pop, + } + } + + pub fn save_city(&self, path: String) -> PyResult<()> { + let encoded = bincode::serialize(self).map_err(|e| PyErr::new::(format!("Failed to serialize city: {}", e)))?; + std::fs::write(&path, encoded).map_err(|e| PyErr::new::(format!("Failed to write to file {}: {}", path, e)))?; + Ok(()) + } + + #[staticmethod] + pub fn load_city(path: String) -> PyResult { + let data = std::fs::read(&path).map_err(|e| PyErr::new::(format!("Failed to read from file {}: {}", path, e)))?; + let decoded: Micropolis = bincode::deserialize(&data).map_err(|e| PyErr::new::(format!("Failed to deserialize city: {}", e)))?; + Ok(decoded) + } +} + +/// A Python module implemented in Rust. +#[pymodule] +fn micropolis_engine(_py: Python<'_>, m: &Bound<'_, PyModule>) -> PyResult<()> { + m.add_class::()?; + m.add_class::()?; + m.add("DIRT", DIRT)?; + m.add("RIVER", RIVER)?; + m.add("TREEBASE", TREEBASE)?; + m.add("WOODS", WOODS)?; + m.add("ROADBASE", ROADBASE)?; + m.add("RESBASE", RESBASE)?; + m.add("COMBASE", COMBASE)?; + m.add("INDBASE", INDBASE)?; + m.add("PORTBASE", PORTBASE)?; + Ok(()) +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn it_works() { + let micro = Micropolis::new(); + assert_eq!(micro.city_time, 0); + } +} diff --git a/micropolis-rs/src/sim.rs b/micropolis-rs/src/sim.rs new file mode 100644 index 00000000..4324b509 --- /dev/null +++ b/micropolis-rs/src/sim.rs @@ -0,0 +1,448 @@ +use crate::Micropolis; + +impl Micropolis { + pub fn sim_frame(&mut self) { + if self.sim_speed == 0 { + return; + } + + self.spd_cycle = (self.spd_cycle + 1) % 1024; + + if self.sim_speed == 1 && self.spd_cycle % 5 != 0 { + return; + } + + if self.sim_speed == 2 && self.spd_cycle % 3 != 0 { + return; + } + + self.f_cycle = (self.f_cycle + 1) % 1024; + self.simulate(self.f_cycle & 15); + } + + fn simulate(&mut self, mod16: i16) { + let spd_pwr = [1, 2, 4, 5]; + let spd_ptl = [1, 2, 7, 17]; + let spd_cri = [1, 1, 8, 18]; + let spd_pop = [1, 1, 9, 19]; + let spd_fir = [1, 1, 10, 20]; + let mut x = self.sim_speed; + if x > 3 { + x = 3; + } + + match mod16 { + 0 => { + self.s_cycle = (self.s_cycle + 1) % 1024; + if self.do_initial_eval { + self.do_initial_eval = false; + self.city_evaluation(); + } + self.city_time += 1; + self.av_city_tax += self.city_tax; + if self.s_cycle & 1 == 0 { + self.set_valves(); + } + self.clear_census(); + } + 1 => self.map_scan(0, (1 * crate::WORLD_X / 8) as i16), + 2 => self.map_scan((1 * crate::WORLD_X / 8) as i16, (2 * crate::WORLD_X / 8) as i16), + 3 => self.map_scan((2 * crate::WORLD_X / 8) as i16, (3 * crate::WORLD_X / 8) as i16), + 4 => self.map_scan((3 * crate::WORLD_X / 8) as i16, (4 * crate::WORLD_X / 8) as i16), + 5 => self.map_scan((4 * crate::WORLD_X / 8) as i16, (5 * crate::WORLD_X / 8) as i16), + 6 => self.map_scan((5 * crate::WORLD_X / 8) as i16, (6 * crate::WORLD_X / 8) as i16), + 7 => self.map_scan((6 * crate::WORLD_X / 8) as i16, (7 * crate::WORLD_X / 8) as i16), + 8 => self.map_scan((7 * crate::WORLD_X / 8) as i16, crate::WORLD_X as i16), + 9 => { + // ... + } + 10 => { + // ... + } + 11 => { + if self.s_cycle % spd_pwr[x as usize] == 0 { + self.do_power_scan(); + } + } + 12 => { + if self.s_cycle % spd_ptl[x as usize] == 0 { + self.ptl_scan(); + } + } + 13 => { + if self.s_cycle % spd_cri[x as usize] == 0 { + self.crime_scan(); + } + } + 14 => { + if self.s_cycle % spd_pop[x as usize] == 0 { + self.pop_den_scan(); + } + } + 15 => { + if self.s_cycle % spd_fir[x as usize] == 0 { + self.fire_analysis(); + } + self.do_disasters(); + } + _ => (), + } + } + + fn city_evaluation(&mut self) { + self.eval_valid = false; + if self.total_pop > 0 { + self.get_ass_value(); + self.do_pop_num(); + self.do_problems(); + self.get_score(); + self.do_votes(); + self.change_eval(); + } else { + self.eval_init(); + self.change_eval(); + } + self.eval_valid = true; + } + fn get_ass_value(&mut self) { + let mut z = (self.road_total * 5) as i64; + z += (self.rail_total * 10) as i64; + z += (self.police_pop * 1000) as i64; + z += (self.fire_st_pop * 1000) as i64; + z += (self.hosp_pop * 400) as i64; + z += (self.stadium_pop * 3000) as i64; + z += (self.port_pop * 5000) as i64; + z += (self.aport_pop * 10000) as i64; + z += (self.coal_pop * 3000) as i64; + z += (self.nuclear_pop * 6000) as i64; + self.city_ass_value = z * 1000; + } + fn do_pop_num(&mut self) { + let old_city_pop = self.city_pop; + self.city_pop = ((self.res_pop + (self.com_pop * 8) + (self.ind_pop * 8)) * 20) as i64; + if old_city_pop == -1 { + self.delta_city_pop = 0; + } else { + self.delta_city_pop = self.city_pop - old_city_pop; + } + + self.city_class = 0; + if self.city_pop > 2000 { + self.city_class += 1; + } + if self.city_pop > 10000 { + self.city_class += 1; + } + if self.city_pop > 50000 { + self.city_class += 1; + } + if self.city_pop > 100000 { + self.city_class += 1; + } + if self.city_pop > 500000 { + self.city_class += 1; + } + } + fn do_problems(&mut self) { + for z in 0..crate::PROBNUM { + self.problem_table[z] = 0; + } + self.problem_table[0] = self.crime_average; + self.problem_table[1] = self.pollute_average; + self.problem_table[2] = (self.lv_average as f32 * 0.7) as i16; + self.problem_table[3] = self.city_tax * 10; + self.problem_table[4] = self.average_trf(); + self.problem_table[5] = self.get_unemployment(); + self.problem_table[6] = self.get_fire(); + self.vote_problems(); + for z in 0..crate::PROBNUM { + self.problem_taken[z] = 0; + } + for z in 0..4 { + let mut max = 0; + let mut this_prob = 0; + for x in 0..7 { + if self.problem_votes[x] > max && self.problem_taken[x] == 0 { + this_prob = x; + max = self.problem_votes[x]; + } + } + if max > 0 { + self.problem_taken[this_prob] = 1; + self.problem_order[z] = this_prob as i16; + } else { + self.problem_order[z] = 7; + self.problem_table[7] = 0; + } + } + } + fn vote_problems(&mut self) {} + fn average_trf(&mut self) -> i16 { + 0 + } + fn get_unemployment(&mut self) -> i16 { + 0 + } + fn get_fire(&mut self) -> i16 { + 0 + } + fn get_score(&mut self) {} + fn do_votes(&mut self) {} + fn change_eval(&mut self) {} + fn eval_init(&mut self) {} + + fn set_valves(&mut self) {} + fn clear_census(&mut self) { + self.pwrd_z_cnt = 0; + self.un_pwrd_z_cnt = 0; + self.fire_pop = 0; + self.road_total = 0; + self.rail_total = 0; + self.res_pop = 0; + self.com_pop = 0; + self.ind_pop = 0; + self.res_z_pop = 0; + self.com_z_pop = 0; + self.ind_z_pop = 0; + self.hosp_pop = 0; + self.church_pop = 0; + self.police_pop = 0; + self.fire_st_pop = 0; + self.stadium_pop = 0; + self.coal_pop = 0; + self.nuclear_pop = 0; + self.port_pop = 0; + self.aport_pop = 0; + self.power_stack_num = 0; + for x in 0..crate::SMX { + for y in 0..crate::SMY { + self.fire_st_map[x][y] = 0; + self.police_map[x][y] = 0; + } + } + } + fn map_scan(&mut self, x1: i16, x2: i16) { + for x in x1..x2 { + for y in 0..crate::WORLD_Y as i16 { + self.s_map_x = x; + self.s_map_y = y; + if let Some(chr) = self.get_tile(x, y) { + self.c_chr = chr; + if chr == 0 { + continue; + } + self.c_chr9 = chr & crate::LOMASK; + if self.c_chr9 >= crate::FLOOD { + if self.c_chr9 < crate::ROADBASE { + if self.c_chr9 >= crate::FIREBASE { + self.fire_pop += 1; + if self.rand(4) == 0 { + self.do_fire(); + } + continue; + } + if self.c_chr9 < crate::RADTILE { + self.do_flood(); + } else { + self.do_rad_tile(); + } + continue; + } + + if self.new_power && (self.c_chr & crate::CONDBIT) != 0 { + self.set_z_power(); + } + + if self.c_chr9 >= crate::ROADBASE && self.c_chr9 < crate::POWERBASE { + self.do_road(); + continue; + } + + if (self.c_chr & crate::ZONEBIT) != 0 { + self.do_zone(); + continue; + } + + if self.c_chr9 >= crate::RAILBASE && self.c_chr9 < crate::RESBASE { + self.do_rail(); + continue; + } + + if self.c_chr9 >= crate::SOMETINYEXP && self.c_chr9 <= crate::LASTTINYEXP { + let rand_val = self.rand(4); + self.set_tile(x, y, crate::RUBBLE + rand_val | crate::BULLBIT); + } + } + } + } + } + } + + fn get_tile(&self, x: i16, y: i16) -> Option { + if x < 0 || x >= crate::WORLD_X as i16 || y < 0 || y >= crate::WORLD_Y as i16 { + None + } else { + Some(self.map[x as usize][y as usize]) + } + } + + fn set_tile(&mut self, x: i16, y: i16, tile: u16) { + if x >= 0 && x < crate::WORLD_X as i16 && y >= 0 && y < crate::WORLD_Y as i16 { + self.map[x as usize][y as usize] = tile; + } + } + + fn rand(&mut self, _range: u16) -> u16 { + // A proper random number generator will be needed here. + // For now, returning a constant to allow compilation. + 0 + } + + fn do_fire(&mut self) { + // TODO: Port fire logic + } + + fn do_flood(&mut self) { + // TODO: Port flood logic + } + + fn do_rad_tile(&mut self) { + // TODO: Port radiation logic + } + + fn do_road(&mut self) { + // TODO: Port road logic + } + + fn do_rail(&mut self) { + // TODO: Port rail logic + } + + fn do_power_scan(&mut self) {} + fn ptl_scan(&mut self) {} + fn crime_scan(&mut self) {} + fn pop_den_scan(&mut self) {} + fn fire_analysis(&mut self) {} + fn do_disasters(&mut self) {} +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn test_do_problems() { + let mut micropolis = Micropolis::new(); + micropolis.crime_average = 100; + micropolis.pollute_average = 100; + micropolis.lv_average = 100; + micropolis.city_tax = 7; + + micropolis.do_problems(); + + assert_eq!(micropolis.problem_table[0], 100); + assert_eq!(micropolis.problem_table[1], 100); + assert_eq!(micropolis.problem_table[2], 70); + assert_eq!(micropolis.problem_table[3], 70); + } + + #[test] + fn test_do_pop_num() { + let mut micropolis = Micropolis::new(); + micropolis.res_pop = 100; + micropolis.com_pop = 100; + micropolis.ind_pop = 100; + + micropolis.do_pop_num(); + + let expected_city_pop = (100 + (100 * 8) + (100 * 8)) * 20; + assert_eq!(micropolis.city_pop, expected_city_pop); + assert_eq!(micropolis.delta_city_pop, expected_city_pop); + assert_eq!(micropolis.city_class, 2); + } + + #[test] + fn test_get_ass_value() { + let mut micropolis = Micropolis::new(); + micropolis.road_total = 10; + micropolis.rail_total = 10; + micropolis.police_pop = 10; + micropolis.fire_st_pop = 10; + micropolis.hosp_pop = 10; + micropolis.stadium_pop = 1; + micropolis.port_pop = 1; + micropolis.aport_pop = 1; + micropolis.coal_pop = 1; + micropolis.nuclear_pop = 1; + + micropolis.get_ass_value(); + + let mut expected_value = (10 * 5) as i64; + expected_value += (10 * 10) as i64; + expected_value += (10 * 1000) as i64; + expected_value += (10 * 1000) as i64; + expected_value += (10 * 400) as i64; + expected_value += (1 * 3000) as i64; + expected_value += (1 * 5000) as i64; + expected_value += (1 * 10000) as i64; + expected_value += (1 * 3000) as i64; + expected_value += (1 * 6000) as i64; + expected_value *= 1000; + + assert_eq!(micropolis.city_ass_value, expected_value); + } + + #[test] + fn test_clear_census() { + let mut micropolis = Micropolis::new(); + micropolis.pwrd_z_cnt = 1; + micropolis.un_pwrd_z_cnt = 1; + micropolis.fire_pop = 1; + micropolis.road_total = 1; + micropolis.rail_total = 1; + micropolis.res_pop = 1; + micropolis.com_pop = 1; + micropolis.ind_pop = 1; + micropolis.res_z_pop = 1; + micropolis.com_z_pop = 1; + micropolis.ind_z_pop = 1; + micropolis.hosp_pop = 1; + micropolis.church_pop = 1; + micropolis.police_pop = 1; + micropolis.fire_st_pop = 1; + micropolis.stadium_pop = 1; + micropolis.coal_pop = 1; + micropolis.nuclear_pop = 1; + micropolis.port_pop = 1; + micropolis.aport_pop = 1; + micropolis.power_stack_num = 1; + micropolis.fire_st_map[0][0] = 1; + micropolis.police_map[0][0] = 1; + + micropolis.clear_census(); + + assert_eq!(micropolis.pwrd_z_cnt, 0); + assert_eq!(micropolis.un_pwrd_z_cnt, 0); + assert_eq!(micropolis.fire_pop, 0); + assert_eq!(micropolis.road_total, 0); + assert_eq!(micropolis.rail_total, 0); + assert_eq!(micropolis.res_pop, 0); + assert_eq!(micropolis.com_pop, 0); + assert_eq!(micropolis.ind_pop, 0); + assert_eq!(micropolis.res_z_pop, 0); + assert_eq!(micropolis.com_z_pop, 0); + assert_eq!(micropolis.ind_z_pop, 0); + assert_eq!(micropolis.hosp_pop, 0); + assert_eq!(micropolis.church_pop, 0); + assert_eq!(micropolis.police_pop, 0); + assert_eq!(micropolis.fire_st_pop, 0); + assert_eq!(micropolis.stadium_pop, 0); + assert_eq!(micropolis.coal_pop, 0); + assert_eq!(micropolis.nuclear_pop, 0); + assert_eq!(micropolis.port_pop, 0); + assert_eq!(micropolis.aport_pop, 0); + assert_eq!(micropolis.power_stack_num, 0); + assert_eq!(micropolis.fire_st_map[0][0], 0); + assert_eq!(micropolis.police_map[0][0], 0); + } +} diff --git a/micropolis-rs/src/zone.rs b/micropolis-rs/src/zone.rs new file mode 100644 index 00000000..27253b71 --- /dev/null +++ b/micropolis-rs/src/zone.rs @@ -0,0 +1,225 @@ +// This file will contain the zone-related logic ported from s_zone.c. + +use crate::Micropolis; +use crate::{ + ANIMBIT, BNCNBIT, BULLBIT, BURNBIT, CHURCH, COMBASE, CONDBIT, FLOOD, HOSPITAL, INDBASE, + LASTRUBBLE, LOMASK, NUCLEAR, PORTBASE, POWERPLANT, PWRBIT, PWRMAPSIZE, ROADBASE, RESBASE, + RUBBLE, ZONEBIT, +}; +impl Micropolis { + pub(crate) fn do_zone(&mut self) { + let zone_pwr_flag = self.set_z_power(); + if zone_pwr_flag { + self.pwrd_z_cnt += 1; + } else { + self.un_pwrd_z_cnt += 1; + } + + if self.c_chr9 > PORTBASE { + // self.do_sp_zone(zone_pwr_flag); + return; + } + if self.c_chr9 < HOSPITAL { + // self.do_residential(zone_pwr_flag); + return; + } + if self.c_chr9 < COMBASE { + self.do_hosp_chur(); + return; + } + if self.c_chr9 < INDBASE { + // self.do_commercial(zone_pwr_flag); + return; + } + // self.do_industrial(zone_pwr_flag); + } + + fn do_hosp_chur(&mut self) { + if self.c_chr9 == HOSPITAL as u16 { + self.hosp_pop += 1; + if (self.city_time & 15) == 0 { + self.repair_zone(HOSPITAL, 3); + } + if self.need_hosp == -1 { + self.zone_plop(RESBASE); + } + } + if self.c_chr9 == CHURCH as u16 { + self.church_pop += 1; + if (self.city_time & 15) == 0 { + self.repair_zone(CHURCH, 3); + } + if self.need_church == -1 { + self.zone_plop(RESBASE); + } + } + } + + fn repair_zone(&mut self, zone_center: u16, mut zone_size: i16) { + zone_size -= 1; + let mut cnt = 0; + for y in -1..zone_size { + for x in -1..zone_size { + cnt += 1; + let xx = self.s_map_x + x; + let yy = self.s_map_y + y; + + if self.test_bounds(xx, yy) { + let mut th_ch = self.map[xx as usize][yy as usize]; + if (th_ch & ZONEBIT) != 0 { + continue; + } + if (th_ch & ANIMBIT) != 0 { + continue; + } + th_ch &= LOMASK; + if (th_ch >= RUBBLE && th_ch <= LASTRUBBLE) || th_ch < ROADBASE { + self.map[xx as usize][yy as usize] = + zone_center - 3 - zone_size as u16 + cnt + CONDBIT + BURNBIT; + } + } + } + } + } + + fn zone_plop(&mut self, mut base: u16) { + let zx = [-1, 0, 1, -1, 0, 1, -1, 0, 1]; + let zy = [-1, -1, -1, 0, 0, 0, 1, 1, 1]; + + for z in 0..9 { + let xx = self.s_map_x + zx[z]; + let yy = self.s_map_y + zy[z]; + if self.test_bounds(xx, yy) { + let tile = self.map[xx as usize][yy as usize] & LOMASK; + if tile >= FLOOD && tile < ROADBASE { + return; + } + } + } + + for z in 0..9 { + let xx = self.s_map_x + zx[z]; + let yy = self.s_map_y + zy[z]; + if self.test_bounds(xx, yy) { + self.map[xx as usize][yy as usize] = base + BNCNBIT; + } + base += 1; + } + + self.c_chr = self.map[self.s_map_x as usize][self.s_map_y as usize]; + self.set_z_power(); + self.map[self.s_map_x as usize][self.s_map_y as usize] |= ZONEBIT | BULLBIT; + } + + pub(crate) fn set_z_power(&mut self) -> bool { + if self.c_chr9 == NUCLEAR || self.c_chr9 == POWERPLANT { + self.map[self.s_map_x as usize][self.s_map_y as usize] |= PWRBIT; + return true; + } + + let power_word = self.power_word(self.s_map_x, self.s_map_y); + if power_word < PWRMAPSIZE { + if self.power_map[power_word] & (1 << (self.s_map_x & 15)) != 0 { + self.map[self.s_map_x as usize][self.s_map_y as usize] |= PWRBIT; + return true; + } + } + + self.map[self.s_map_x as usize][self.s_map_y as usize] &= !PWRBIT; + false + } +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::{FIRE, NUCLEAR, POWERPLANT, PWRBIT}; + + #[test] + fn test_do_hosp_chur() { + let mut micropolis = Micropolis::new(); + micropolis.c_chr9 = HOSPITAL as u16; + micropolis.do_hosp_chur(); + assert_eq!(micropolis.hosp_pop, 1); + + micropolis.need_hosp = -1; + micropolis.do_hosp_chur(); + assert_eq!(micropolis.hosp_pop, 2); + } + + #[test] + fn test_zone_plop() { + let mut micropolis = Micropolis::new(); + micropolis.s_map_x = 10; + micropolis.s_map_y = 10; + + micropolis.zone_plop(RESBASE); + + // Assert that the tiles have been plopped + for y in -1..=1 { + for x in -1..=1 { + let xx = (micropolis.s_map_x + x) as usize; + let yy = (micropolis.s_map_y + y) as usize; + assert_ne!(micropolis.map[xx][yy], 0); + } + } + + // Assert that fire prevents plopping + micropolis.map[10][10] = FIRE; + micropolis.zone_plop(RESBASE); + assert_eq!(micropolis.map[10][10], FIRE); + } + + #[test] + fn test_set_z_power() { + let mut micropolis = Micropolis::new(); + micropolis.s_map_x = 10; + micropolis.s_map_y = 10; + + // Test with power map + let power_word = micropolis.power_word(micropolis.s_map_x, micropolis.s_map_y); + micropolis.power_map[power_word] |= 1 << (micropolis.s_map_x & 15); + assert!(micropolis.set_z_power()); + assert_ne!( + micropolis.map[micropolis.s_map_x as usize][micropolis.s_map_y as usize] & PWRBIT, + 0 + ); + + micropolis.power_map[power_word] &= !(1 << (micropolis.s_map_x & 15)); + assert!(!micropolis.set_z_power()); + assert_eq!( + micropolis.map[micropolis.s_map_x as usize][micropolis.s_map_y as usize] & PWRBIT, + 0 + ); + + // Test with nuclear power plant + micropolis.c_chr9 = NUCLEAR; + assert!(micropolis.set_z_power()); + + // Test with coal power plant + micropolis.c_chr9 = POWERPLANT; + assert!(micropolis.set_z_power()); + } + + #[test] + fn test_repair_zone() { + let mut micropolis = Micropolis::new(); + micropolis.s_map_x = 10; + micropolis.s_map_y = 10; + + // Set some tiles to rubble + micropolis.map[10][10] = RUBBLE; + micropolis.map[11][11] = RUBBLE; + + micropolis.repair_zone(HOSPITAL, 3); + + // Assert that the rubble has been repaired + assert_ne!(micropolis.map[10][10], RUBBLE); + assert_ne!(micropolis.map[11][11], RUBBLE); + + // Also assert that a non-rubble tile was not changed + micropolis.map[12][12] = ROADBASE; + micropolis.repair_zone(HOSPITAL, 3); + assert_eq!(micropolis.map[12][12], ROADBASE); + } +} diff --git a/micropolis-rs/test_wheel.py b/micropolis-rs/test_wheel.py new file mode 100644 index 00000000..432ed180 --- /dev/null +++ b/micropolis-rs/test_wheel.py @@ -0,0 +1,70 @@ +import micropolis_engine + +def main(): + print("Testing micropolis_engine Python wheel...") + + # Create a new Micropolis instance using the new static method + print("Creating Micropolis instance via create_city(120, 100)...") + micro = micropolis_engine.Micropolis.create_city(120, 100) + print("Instance created.") + + # Get initial city stats + print("Getting initial city stats...") + stats = micro.get_city_stats() + print(f"Initial city time: {stats.city_time}") + print(f"Initial total funds: {stats.total_funds}") + print(f"Initial total population: {stats.total_pop}") + assert stats.city_time == 0 + assert stats.total_funds == 20000 + + # Step the simulation + print("Stepping simulation...") + micro.step_simulation() + print("Simulation stepped.") + + # Get map view (full) + print("Getting full map view (0, 0, 120, 100)...") + map_view = micro.get_map_view(0, 0, 120, 100) + print(f"Full map view length: {len(map_view)}") + print(f"Expected length: {120 * 100}") + assert len(map_view) == 120 * 100 + + # Get map view (partial) + print("Getting partial map view (10, 10, 20, 20)...") + partial_map_view = micro.get_map_view(10, 10, 20, 20) + print(f"Partial map view length: {len(partial_map_view)}") + print(f"Expected length: {20 * 20}") + assert len(partial_map_view) == 20 * 20 + + print("All tests passed!") + +def test_save_load(): + print("\nTesting save/load functionality...") + + # Create a new Micropolis instance + micro = micropolis_engine.Micropolis.create_city(120, 100) + micro.city_time = 123 + micro.total_funds = 456 + + # Save the city + file_path = "city.dat" + print(f"Saving city to {file_path}...") + micro.save_city(file_path) + print("City saved.") + + # Load the city + print(f"Loading city from {file_path}...") + loaded_micro = micropolis_engine.Micropolis.load_city(file_path) + print("City loaded.") + + # Compare stats + print("Comparing stats...") + assert loaded_micro.city_time == micro.city_time + assert loaded_micro.total_funds == micro.total_funds + print("Stats match.") + + print("Save/load test passed!") + +if __name__ == "__main__": + main() + test_save_load() diff --git a/micropolis-rs/tests/simulation_tests.rs b/micropolis-rs/tests/simulation_tests.rs new file mode 100644 index 00000000..668f11cb --- /dev/null +++ b/micropolis-rs/tests/simulation_tests.rs @@ -0,0 +1,21 @@ +// It's necessary to import the crate as a library +use micropolis_engine::Micropolis; + +#[test] +fn test_initial_state() { + let micro = Micropolis::new(); + assert_eq!(micro.city_time, 0); + assert_eq!(micro.total_funds, 20000); + assert_eq!(micro.city_tax, 7); + assert_eq!(micro.game_level, 0); +} + +#[test] +fn test_step_simulation() { + let mut micro = Micropolis::new(); + micro.sim_speed = 1; + for _ in 0..80 { + micro.step_simulation(); + } + assert_eq!(micro.city_time, 1); +} diff --git a/migration_log.md b/migration_log.md new file mode 100644 index 00000000..32273fb3 --- /dev/null +++ b/migration_log.md @@ -0,0 +1,14 @@ +# Migration Log + +## Session 1: 2025-08-27 + +* **Completed Tasks**: + * Fixed the `cargo.toml` file to allow the project to be built. + * Implemented the `do_hosp_chur` and `repair_zone` functions in `zone.rs`. + * Implemented the `get_ass_value` and `do_pop_num` functions in `sim.rs`. + * Implemented the `do_problems` function in `sim.rs` with stubs for helper functions. + * Added unit tests for all implemented functions. + * Added an integration test for the `step_simulation` function. +* **Notes**: + * The simulation logic is partially implemented. More functions need to be ported from the C code. + * The test coverage is still low. More tests need to be added. diff --git a/plan_of_action.md b/plan_of_action.md new file mode 100644 index 00000000..5d48df01 --- /dev/null +++ b/plan_of_action.md @@ -0,0 +1,54 @@ +# Phased Implementation Plan + +A phased implementation plan is recommended to manage complexity and provide clear milestones. + +--- + +## Phase 1: Core Simulation in Rust +This phase focuses on building the foundational simulation logic in Rust. + +* **Data Modeling**: Define the core simulation data structures, such as `City`, `Tile`, and `ZoneData`, within a new Rust library crate. +* **Logic Porting**: Systematically port the existing C simulation logic from files like `s_sim.c`, `s_zone.c`, `s_scan.c`, and `s_traf.c` into idiomatic Rust functions or systems. + * **Migration Targets**: + * `micropolis-activity/src/sim/s_sim.c`: Contains the main simulation loop and coordination logic. + * `micropolis-activity/src/sim/s_zone.c`: Handles logic related to zone development (residential, commercial, industrial). + * `micropolis-activity/src/sim/s_scan.c`: Contains scanning functions that iterate over the city map. + * `micropolis-activity/src/sim/s_traf.c`: Manages the traffic simulation logic. + * `micropolis-activity/src/sim/s_eval.c`: Contains city evaluation logic. + * **Completed**: + * Ported `DoHospChur` and `RepairZone` from `s_zone.c` to `zone.rs`. + * Ported `GetAssValue` and `DoPopNum` from `s_eval.c` to `sim.rs`. + * Ported `DoProblems` from `s_eval.c` to `sim.rs` with stubs for helper functions. +* **Unit Testing**: Develop a comprehensive suite of unit and integration tests in Rust to validate the correctness of the ported logic. This may involve comparing outputs against the original C implementation. This phase is complete when a headless Rust library can successfully load a city state, run a simulation for N cycles, and produce a verifiably correct new city state. + * **Completed**: + * Added unit tests for `do_hosp_chur` and `repair_zone` in `zone.rs`. + * Added unit tests for `get_ass_value` and `do_pop_num` in `sim.rs`. + * Added an integration test for `step_simulation`. + +--- + +## Phase 2: FFI API Definition and Scaffolding +This phase establishes the bridge between the Rust backend and the Python frontend. + +* **API Design**: Design a "chunky" API layer using **PyO3**. Define a minimal set of high-level functions required by the Python front-end, such as `create_city(width, height)`, `step_simulation(inputs)`, `get_map_view(x, y, w, h) -> Vec`, and `get_city_stats() -> StatsStruct`. +* **Build and Package**: Use **Maturin** to configure the project to build a Python wheel. +* **Verification**: Create a minimal "scaffolding" Python script to import the compiled module, call each API function, and print the returned data to the console. This phase is complete when the FFI bridge is proven to be functional. + +--- + +## Phase 3: Python UI Development with Arcade +This phase focuses on building the user interface using Python. + +* **Application Shell**: Set up the main application window, the core game loop structure, and the asset loading pipeline in Python using the **Arcade** library. +* **Rendering Logic**: Implement the rendering logic by creating a `TileMap` or `SpriteList` in Arcade and updating it in the `on_draw` method based on the data retrieved from the `get_map_view` FFI call. +* **Input Handling**: Implement user input handling for mouse and keyboard. Translate user actions, such as clicking a tile to zone it, into the simple data structures defined by the FFI API and pass them to the `step_simulation` function. This phase is complete when a user can view the city and perform basic interactions. + +--- + +## Phase 4: Integration and Polish +This final phase involves connecting all components and refining the application. + +* **Full UI Implementation**: Connect all remaining UI elements, such as the budget window, graphs, and menus, to the Rust backend using new FFI functions as needed. +* **Performance Profiling**: Conduct thorough performance profiling of the fully integrated application, with a focus on time spent at the FFI boundary. +* **Optimization and Refinement**: Based on the profiling results, refine the FFI API to eliminate any identified bottlenecks, likely by consolidating calls or optimizing data transfer formats. +* **Final Touches**: Add audio, save/load functionality (which will involve serializing the Rust simulation state), and other final polish features.