fix: Remove manual type validation from SparkClient.connect()

[VHemanth45]

Removes redundant manual type validation from KubernetesBackend._create_session()
Fixes #272

[google-oss-prow]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign kramaranya for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[github-actions]

🎉 Welcome to the Kubeflow SDK! 🎉

Thanks for opening your first PR! We're happy to have you as part of our community 🚀

Here's what happens next:

• If you haven't already, please check out our Contributing Guide for repo-specific guidelines and the Kubeflow Contributor Guide for general community standards
• Our team will review your PR soon! cc @kubeflow/kubeflow-sdk-team

Join the community:

• Slack: Join our #kubeflow-ml-experience and #kubeflow-trainer Slack channels
• Meetings: Attend the Kubeflow SDK and ML Experience bi-weekly meetings

Feel free to ask questions in the comments if you need any help or clarification!
Thanks again for contributing to Kubeflow! 🙏

[Copilot]

Pull request overview

This PR removes redundant manual type validation from the KubernetesBackend._create_session() method. The isinstance() checks for resources_per_executor, spark_conf, num_executors, driver, and executor parameters are no longer necessary since:

1. The method already has complete type annotations
2. It's an internal method (prefix _) only called from within the module
3. All callers in the call chain also have proper type hints

This cleanup aligns with the codebase philosophy of avoiding overly defensive code when type hints already provide the necessary documentation and validation expectations.

Changes:

• Removed 14 lines of manual isinstance() type validation from _create_session()

[tariq-hasan]

Thanks for putting this together @VHemanth45!

There was some earlier discussion about moving validation to the Spark Operator side (similar to SparkApplication/TrainJob) before removing it from the SDK - #225 (comment)

Do we want to align on that first to avoid a validation gap? Tagging @andreyvelich and @ChenYi015 for guidance.

[andreyvelich]

Yes, we should add the webhook validation to the Spark Operator side as well.
I noticed that @tariq-hasan is assigned for this issue, so we can close this PR for now: #272
/close

[google-oss-prow]

@andreyvelich: Closed this PR.

Details

In response to this:

Yes, we should add the webhook validation to the Spark Operator side as well.
I noticed that @tariq-hasan is assigned for this issue, so we can close this PR for now: #272
/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.