Proxy the transcode request?

[FreekingDean]

TL;DR: Im wondering if theres a reason why the UnicornLoadBalancer does not reverse proxy the transcode requests to the transcoders.

Context:
I am setting up UnicornLoadbalancer on Kubernetes. I am also setting up autoscaling. These two items combined mean I can not automatically create an ingress from the external world that will always hit the correct transcoder.

I was thinking about using a complicated kubernetes system that watches for new transcoder pods to start & create routes that way, but was thinking the UnicornLoadbalancer may be better suited to handle these requests since it already knows about the transcoders.

Thanks for an amazing project! 😄

[Maxou44]

If the load balancer proxy transcoder data, the load balancer will be a bottleneck for network, it's why we redirect :)

[FreekingDean]

That makes 100% sense!

So a few paths forward from what I can see:

1 - Use an external proxy (I am using traefik, but nginx could also be used). This sill has the same issue as you listed though, making whatever proxy you use the bottleneck for traffic.

2 - Expose each transcoder on a different port & setup port forwarding for each. This means theres almost no way to autoscale, and adding/removing transcode instances gets more tedious.

(Does that look right?)

[Maxou44]

With the first solution you'll have a bottleneck on your proxy network. The best solution is the second one and create/assign new transcoders "on the fly". K8s isn't the best solution to scale this project because you have to run a transcoder on each VM. And you can't use autoscaler based on CPU of your pods (K8s can't remove only "empty" transcoders)
The best solution is to ping /stats endpoint on LB, analyze score value and start new VM when needed (and downscale when you can by checking score and amount of current streams)

[FreekingDean]

The "When" to autoscale is a solvable problem! I can use custom metrics like number of max vs used sessions :)

Why does the transcoder care about what VM it runs on?

[Maxou44]

I just want to explain it's useless to put multiple "pods" on a same "node", because transcoder/ffmpeg use 100% of the available CPU. You have to scale your cluster at the same time as your transcoder pool

[FreekingDean]

AH, that makes 100% sense! :)

[brujoand]

I just want to explain it's useless to put multiple "pods" on a same "node", because transcoder/ffmpeg use 100% of the available CPU. You have to scale your cluster at the same time as your transcoder pool

This is only correct if you are running your pods without resource restrictions, which will cause other problems since now kubernetes can't balance the load intelligently.

[Maxou44]

But why use a limit? Start 2 transcoders with 50% of CPU and 1 with 100% of CPU is exactly the same for raw ffmpeg performance 🤔

[brujoand]

Ah, so one transcoder can handle multiple streams at the same time? If that's the case then I agree that it doesn't make as much sense to run multiple pods on the same node.

But if you limit the amount of concurrent streams pr pod, and limit the resources each pod can utilize I would think you'd be able to utilize the cluster much better, the trade off then would be single stream performance.

[Maxou44]

UnicornTranscoder can handle multiple streams, ffmpeg share the CPU power. It's useless to start 2 transcoders on a same CPU/server

[brujoand]

I understand what you're saying. If you want to maximize the utilization of a single node, one UnicornTranscoder can do that just fine, by spinning off x number of streams until the CPU is fully utilized. I have no problem with this logic. But I think you're missing the point of autoscaling on a kubernetes cluster. The scenario you are describing doesn't require kubernetes at all. You basically have dedicated nodes that can handle transcoding when needed. But for someone with a kubernetes cluster, they have other concerns. There might be a bunch of other things running on their clusters. So maybe they have 2 available CPU cores on one machine, and another has 16. So if you limit each pod to handle one transcoding job at the time and require 2 CPU cores pr pod they have well defined cpu requirements and kubernetes can balance the load. So the node with lot's of CPU available can spin up 8 pods, and the other 1. You could in theory have one pod on each node, and not limit CPU usage but in that case kubernetes has no way of knowing the requirements of each pod, and doesn't know how to distribute pods. When one pod starts using all the available CPU other pods get's killed off and the problem just cascades around the cluster