ByteDanceBean bag large modelTeams andThe University of Hong KongPublicizing the results of joint research -- HybridFlow.
The official claim that HybridFlow (open source project name: veRL) is a flexible and efficient large model RL training framework , compatible with a variety of training and inference frameworks , support for flexible model deployment and a variety of RL algorithm implementation .
The framework adopts a hybrid programming model that combines the flexibility of Single-Controller and the efficiency of Multi-Controller to better implement and execute multiple RL algorithms, significantly improve training throughput, and reduce development and maintenance complexity.
▲ Flow of one iteration of 3D-HybridEngine (Hybrid Technology for Training Reasoning)
Experimental results show that HybridFlow, under various model sizes and RL algorithms, the1.5x to 20x increase in training throughput compared to other frameworks.
The paper has now been accepted by EuroSys 2025 and the code repository has been made available to the public with relevant links below:
- Link to paper: https://arxiv.org/abs/2409.19256
- Link to code: https://github.com/volcengine/veRL