[Hosted MCP Server] CICD Robustness improvements

[gmechali]

I ran into flakiness due to propagation delay from testpypi where the version was found in the index, but it was not yet downloadable. Changes to wait_for_pypi worked to ensure we waited for downloadability.

Modified the server.py file to ensure the version is the DC MCP version, not the FastMCP version.
Verified against Autopush.

[gemini-code-assist]

Summary of Changes

Hello @gmechali, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly enhances the robustness and reliability of the Hosted MCP Server's CI/CD and deployment processes. It addresses issues with flaky package availability checks, ensures accurate version reporting for better observability, and introduces automation scripts for setting up necessary IAM permissions and Apigee integration. Additionally, a new verification script allows for quick post-deployment health checks, contributing to a more stable and manageable deployment pipeline.

Highlights

• CI/CD Robustness: Improved the PyPI package availability check in the CI/CD pipeline to ensure packages are not just listed but also downloadable, preventing flakiness due to propagation delays on TestPyPI.
• Server Versioning and Health Check: The Hosted MCP Server now explicitly reports its version (from datacommons_mcp.version) in its health check endpoint and during FastMCP initialization, providing clearer operational status.
• Deployment and Access Control Scripts: Added new shell scripts (setup_apigee_sa.sh, setup_iam.sh) to automate the setup of IAM roles and Apigee service accounts, streamlining the deployment and access configuration for Cloud Run services.
• Server Verification Utility: Introduced a Python script (verify_server.py) to perform a post-deployment functional verification of the MCP server, ensuring it responds correctly to the MCP handshake (initialize, initialized, tools/list).
• Version String Formatting: Adjusted the get_next_version.py script to correctly format release candidate versions with a dot (e.g., 1.0.rc1 instead of 1.0rc1).

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request improves the robustness of the CI/CD pipeline by enhancing the PyPI package verification process and adding version information to the server's health endpoint. Several new shell and Python scripts are introduced for setup and verification. While the core changes are beneficial, the new scripts could be improved for better robustness, reusability, and adherence to best practices. My review includes suggestions for improving argument handling in shell scripts, removing hardcoded values, using more specific exception handling in Python, and cleaning up unused code.

I am having trouble creating individual review comments. Click here to see my feedback.

scripts/setup_iam.sh (27)

The CI service account email is hardcoded. This reduces the script's reusability and is generally not a good practice. It's better to pass this value as a command-line argument.

packages/datacommons-mcp/datacommons_mcp/server.py (60)

According to PEP 8, imports should be grouped at the top of the file. This import should be moved to the top with other application-specific imports (e.g., after line 28) to improve code readability and maintainability.

scripts/setup_apigee_sa.sh (30-34)

The current argument check only validates if the third argument is present. A more robust approach is to check the total number of arguments passed to the script to ensure all required parameters are provided.

if [ "$#" -ne 3 ]; then
    echo "Usage: $0 <APIGEE_PROJECT_ID> <TARGET_RUN_PROJECT_ID> <CLOUD_RUN_SERVICE_NAME>"
    echo "Example: $0 datcom-apigee-dev datcom-mixer-autopush mcp-server-autopush"
    exit 1
fi

scripts/setup_apigee_sa.sh (52-60)

The current wait loop for IAM propagation does not handle the timeout case. If the service account doesn't become available within the timeout period, the script will proceed and likely fail on the next gcloud command with a less informative error. It's better to explicitly handle the timeout and exit with a clear error message.

echo "Waiting for Service Account propagation..."
i=0
until gcloud iam service-accounts describe "$SA_EMAIL" --project "$APIGEE_PROJECT" > /dev/null 2>&1; do
    i=$((i+1))
    if [ "$i" -gt 12 ]; then
        echo "Error: Timed out waiting for Service Account '$SA_EMAIL' to propagate." >&2
        exit 1
    fi
    echo "Waiting for SA to be globally visible... ($i/12)"
    sleep 5
done
echo "Service Account verified."

scripts/setup_iam.sh (31-34)

The argument check only verifies if the first argument is present. It's more robust to check for the exact number of required arguments.

if [ "$#" -ne 1 ]; then
    echo "Usage: $0 <TARGET_PROJECT_ID>"
    exit 1
fi

scripts/setup_iam.sh (68-79)

This block of code is commented out. To improve code clarity and maintainability, it should be removed if it's no longer needed.

scripts/verify_server.py (31-33)

Catching a generic Exception is too broad and can hide bugs. It's better to catch more specific exceptions that you expect subprocess.run to raise, such as subprocess.CalledProcessError for command failures or FileNotFoundError if gcloud is not installed.

    except (subprocess.CalledProcessError, FileNotFoundError) as e:
        print(f"Error getting identity token: {e}")
        return None

scripts/verify_server.py (73-75)

Catching a generic Exception is too broad. It can mask underlying issues. It's better to catch a more specific exception from the requests library, such as requests.exceptions.RequestException, to handle network or HTTP errors gracefully.

    except requests.exceptions.RequestException as e:
        print(f"Error: {e}")
        return False, None

scripts/wait_for_pypi.py (35-41)

The variables index_args and normalized_version are defined but never used in the new implementation. The if block is also empty. This unused code should be removed to improve clarity.

scripts/wait_for_pypi.py (66-67)

The else: pass statement is redundant and can be removed to simplify the code.

[clincoln8]

Thanks! Just need some lint fixes