fix: downmerge main to dev

[Kanchan-Microsoft]

Purpose

This pull request updates several dependencies in the project to newer versions and reorganizes requirements related to the agent framework and Azure SDKs. The most important changes are grouped below by theme.

Dependency upgrades:

• Upgraded azure-ai-projects to version 2.0.0b3 in both content-gen/infra/vscode_web/requirements.txt and content-gen/src/backend/requirements.txt for consistency across environments. [1] [2]

Agent framework and telemetry requirements:

• Replaced agent-framework with agent-framework-azure-ai and agent-framework-core, both pinned to 1.0.0b260114, and added opentelemetry-semantic-conventions-ai as a new dependency in content-gen/src/backend/requirements.txt.

Azure SDKs reorganization:

• Added azure-search-documents (version >=11.4.0) to content-gen/src/backend/requirements.txt and removed it from content-gen/scripts/requirements-post-deploy.txt, consolidating its usage in backend requirements. [1] [2]

Does this introduce a breaking change?

• Yes
• No

Golden Path Validation

• I have tested the primary workflows (the "golden path") to ensure they function correctly without errors.

Deployment Validation

• I have validated the deployment process successfully and all services are running as expected with this change.

What to Check

Verify that the following are valid

• ...

Other Information