doubi-killer commited on
Commit
e622081
Β·
verified Β·
1 Parent(s): 24780c0

Add PC-Talk checkpoint download instructions and usage terms

Browse files
Files changed (1) hide show
  1. README.md +91 -18
README.md CHANGED
@@ -1,24 +1,97 @@
1
- # Checkpoints
 
 
 
 
 
 
 
 
 
 
 
 
2
 
3
- The standalone source repository expects the following runtime weights. These
4
- files are distributed separately and are intentionally excluded from Git:
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
5
 
6
  ```text
7
- checkpoints/
8
- β”œβ”€β”€ lac.pth
9
- β”œβ”€β”€ emc.pth
10
- β”œβ”€β”€ style_encoder.pth
11
- β”œβ”€β”€ audio_encoder.pth
12
- β”œβ”€β”€ liveportrait/
13
- └── insightface/
 
 
 
 
 
 
 
 
 
14
  ```
15
 
16
- `lac.pth` contains both the LAC `model` and the refinement `model_ft`, so it is
17
- used for both stages by default. EMC is loaded only when emotion control is
18
- requested, and the style encoder is loaded only for a reference style.
19
 
20
- These files were selected from research experiment outputs, not from a
21
- documented public release. Their SHA-256 hashes are recorded in `SHA256SUMS`.
22
- Before publishing them, document training configuration and provenance, and
23
- confirm redistribution rights for the underlying datasets and every derived
24
- checkpoint.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: other
3
+ license_name: checkpoint-specific-upstream-terms
4
+ license_link: https://github.com/BQ-Wang0511/PC-Talk/blob/main/THIRD_PARTY_NOTICES.md
5
+ library_name: pytorch
6
+ pipeline_tag: image-to-video
7
+ tags:
8
+ - talking-head-generation
9
+ - audio-driven-face-animation
10
+ - liveportrait
11
+ - cvpr-2026
12
+ - arxiv:2503.14295
13
+ ---
14
 
15
+ # PC-Talk β€” CVPR 2026 Checkpoints
16
+
17
+ Runtime checkpoints for **PC-Talk: Precise Facial Animation Control for
18
+ Audio-Driven Talking Face Generation**.
19
+
20
+ - [Paper](https://arxiv.org/abs/2503.14295)
21
+ - [Project page](https://bq-wang0511.github.io/PC-Talk/)
22
+ - [Source code and full inference instructions](https://github.com/BQ-Wang0511/PC-Talk)
23
+
24
+ ## Download
25
+
26
+ Clone the source code and download the checkpoints directly into their runtime
27
+ location:
28
+
29
+ ```bash
30
+ git clone https://github.com/BQ-Wang0511/PC-Talk.git
31
+ cd PC-Talk
32
+ pip install -U huggingface_hub
33
+ hf download doubi-killer/PC-Talk --local-dir pctalk/checkpoints
34
+ ```
35
+
36
+ The repository contains 12 model files (about 920 MB), preserving the layout
37
+ expected by PC-Talk:
38
 
39
  ```text
40
+ audio_encoder.pth
41
+ lac.pth
42
+ emc.pth
43
+ style_encoder.pth
44
+ liveportrait/
45
+ β”œβ”€β”€ landmark.onnx
46
+ β”œβ”€β”€ base_models/
47
+ β”‚ β”œβ”€β”€ appearance_feature_extractor.pth
48
+ β”‚ β”œβ”€β”€ motion_extractor.pth
49
+ β”‚ β”œβ”€β”€ spade_generator.pth
50
+ β”‚ └── warping_module.pth
51
+ └── retargeting_models/
52
+ └── stitching_retargeting_module.pth
53
+ insightface/models/buffalo_l/
54
+ β”œβ”€β”€ 2d106det.onnx
55
+ └── det_10g.onnx
56
  ```
57
 
58
+ `lac.pth` includes both the LAC model and its refinement model. EMC is used for
59
+ emotion control; the style encoder is used for reference-style input. The
60
+ LivePortrait and face-analysis weights are included for the inference backend.
61
 
62
+ The original `SHA256SUMS` manifest uses paths relative to the **source-code
63
+ repository root**. After downloading, verify the files from that root:
64
+
65
+ ```bash
66
+ sha256sum -c pctalk/checkpoints/SHA256SUMS
67
+ ```
68
+
69
+ These are downloadable inference assets, not a Transformers-format model. Use
70
+ the PC-Talk source code to run inference. See its README for installation,
71
+ image/video examples, emotion controls and articulation settings.
72
+
73
+ ## Usage terms and safety
74
+
75
+ The source-code license does **not** automatically license every model weight.
76
+ Third-party components retain their upstream terms:
77
+
78
+ - [LivePortrait](https://github.com/KwaiVGI/LivePortrait)
79
+ - [InsightFace](https://github.com/deepinsight/insightface)
80
+ - [Wav2Lip](https://github.com/Rudrabha/Wav2Lip)
81
+
82
+ In particular, the InsightFace model assets are restricted to non-commercial
83
+ research use and must be replaced for commercial deployment. Consult the
84
+ upstream licenses and the source repository's third-party notices before use.
85
+ Only load PyTorch checkpoints from trusted sources.
86
+
87
+ ## Citation
88
+
89
+ ```bibtex
90
+ @inproceedings{wang2026pc,
91
+ title={PC-Talk: Precise Facial Animation Control for Audio-Driven Talking Face Generation},
92
+ author={Wang, Baiqin and Zhu, Xiangyu and Shen, Fan and Xu, Hao and Lei, Zhen},
93
+ booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
94
+ pages={25153--25162},
95
+ year={2026}
96
+ }
97
+ ```