Tensor Parallel?
#3
by Chumenta - opened
Look like this model not support Tensor Parallel or Pipeline Parallel, pretty bad, does we have some way to make it?
It does support it on VLLM. You just have to follow the instructions on the github repo for Hunyuan-MT. You have to install a special version of transformers from a commit and update VLLM.