YTsaurus operations often require deploying web services. These can be debugging UIs (such as Spark UI in SPYT), ML model inference servers, or APIs inside jobs.
Operation jobs run on cluster exec nodes, so services bind to network ports on these nodes — to receive incoming traffic. However, when attempting to directly access services inside a job, difficulties arise:
- Network isolation: the user may not have direct network access to exec node IP addresses (they may be in a closed perimeter).
- Dynamic addressing: even with network access, jobs can move between nodes, so the host and port of services constantly change.
- Security: direct access to a port on a node bypasses YTsaurus authentication mechanisms. Access control to the operation is not enforced.
Task proxy solves these problems by providing a single entry point. It allocates stable domains (FQDN) for each service and verifies user access rights before redirecting the request inside the job.
For more information, refer to:
- User docs for usage examples,
- Spark UI to learn how to open UI of SPYT clusters and jobs,
- Admin docs for installation instructions.
Install chart to cluster from local directory using:
helm install task-proxy \
-n ${NAMESPACE} \
-f values.yaml \
./chart