mirror of
https://github.com/saymrwulf/onnxruntime.git
synced 2026-06-26 03:00:54 +00:00
d3f859fe30
* vectorized kernel * fix build * re-calibrate expected loss * fix build * re-calibrate convergence results * more re-calibrate on loss * divide kernels * adress comments * more calibration * calibration * per comments * enable sync Co-authored-by: Ethan Tao <ettao@OrtTrainingDev4.af05slrtruoetgaxwwjv5nsq5e.px.internal.cloudapp.net>
303 B
303 B
| 1 | step | total_loss | mlm_loss | nsp_loss |
|---|---|---|---|---|
| 2 | 0 | 11.2032 | 10.501 | 0.702181 |
| 3 | 5 | 9.53939 | 7.52411 | 2.01528 |
| 4 | 10 | 8.2614 | 7.564 | 0.697406 |
| 5 | 15 | 8.28412 | 7.55601 | 0.728112 |
| 6 | 20 | 8.17273 | 7.45947 | 0.71326 |
| 7 | 25 | 8.228 | 7.53251 | 0.695496 |
| 8 | 30 | 8.07991 | 7.38456 | 0.695344 |
| 9 | 35 | 7.96173 | 7.25046 | 0.711262 |
| 10 | 40 | 7.9463 | 7.25667 | 0.689625 |
| 11 | 45 | 7.92987 | 7.26442 | 0.665449 |