Ray over internet is also possible as well as K8 https://docs.ray.io/en/latest/cluster/kubernetes/index.html

HPC is very difficult domain to address all settings. With the current diverse hardware and compositional learning needs. Ray is now becoming de-facto framework for compositional learning infrastructure.

One thing valuable is "Fault Tolerance", https://docs.ray.io/en/latest/ray-core/fault-tolerance.html