Training in progress, step 1000

README.md

@@ -4,8 +4,8 @@ library_name: transformers
 model_name: bartpho-cpotrl-summarization-word1
 tags:
 - generated_from_trainer
-- cpo
 - trl
+- cpo
 licence: license
 ---
 
@@ -27,7 +27,7 @@ print(output["generated_text"])
 
 ## Training procedure
 
-[<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="150" height="24"/>](https://wandb.ai/sonspeed-hanoi-university-of-science-and-technology/bartpho-summarization-cpotrl/runs/qeatwcuw) 
+[<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="150" height="24"/>](https://wandb.ai/sonspeed-hanoi-university-of-science-and-technology/bartpho-summarization-cpotrl/runs/r8xksjdz) 
 
 
 This model was trained with CPO, a method introduced in [Contrastive Preference Optimization: Pushing the Boundaries of LLM Performance in Machine Translation](https://huggingface.co/papers/2401.08417).

config.json

@@ -12,7 +12,7 @@
   "decoder_ffn_dim": 4096,
   "decoder_layerdrop": 0.0,
   "decoder_layers": 12,
-  "decoder_start_token_id": 0,
+  "decoder_start_token_id": 2,
   "dropout": 0.1,
   "encoder_attention_heads": 16,
   "encoder_ffn_dim": 4096,

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:c7c76dbb309b86d451025726119e397e9a2c6051a2fb948ee8212f87b6fb3d6e
+oid sha256:9d2d1aba2b5f3b031e74a830531224c4509632996119be5142675fa2c172fb78
 size 1681761380