請用此 Handle URI 來引用此文件：
http://tdr.lib.ntu.edu.tw/jspui/handle/123456789/79586完整後設資料紀錄
| DC 欄位 | 值 | 語言 |
|---|---|---|
| dc.contributor.advisor | 曹承礎(Seng-Cho Chou) | |
| dc.contributor.author | Wei-Yin Hsu | en |
| dc.contributor.author | 徐薇尹 | zh_TW |
| dc.date.accessioned | 2022-11-23T09:04:27Z | - |
| dc.date.available | 2021-11-08 | |
| dc.date.available | 2022-11-23T09:04:27Z | - |
| dc.date.copyright | 2021-11-08 | |
| dc.date.issued | 2021 | |
| dc.date.submitted | 2021-09-16 | |
| dc.identifier.citation | Westerlund, M. (2019). The Emergence of Deepfake Technology: A Review. Technology Innovation Management Review, 9(11): 40-53. http://doi.org/10.22215/timreview/1282 BBC News (2021). Deepfake pornography could become an 'epidemic', expert warns. Available at: https://www.bbc.com/news/uk-scotland-57254636 (Accessed: 30 May 2021). Li, Y., Lyu, S. (2018). Exposing deepfake videos by detecting face warping artifacts. arXiv preprint arXiv:1811.00656. CNN News (2021). South Korea has used AI to bring a dead superstar's voice back to the stage, but ethical concerns abound. Available at: https://edition.cnn.com/2021/01/25/asia/south-korea-kim-kwang-seok-ai-dst-hnk-intl/index.html (Accessed: 30 May 2021). CNN News (2021). Pennsylvania woman allegedly created deepfake images of cheerleading gym members to cyberbully them, according to criminal complaint. Available at: https://edition.cnn.com/2021/03/15/us/pennsylvania-deepfake-cheerleading/index.html (Accessed: 30 May 2021). CNN News (2019). 'Deepfake' Queen delivers alternative Christmas speech, in warning about misinformation. Available at:https://edition.cnn.com/2020/12/25/uk/deepfake-queen-speech-christmas-intl-gbr/index.html (Accessed: 30 May 2021). Li, Y., Yang, X., Sun, P., Qi, H., Lyu, S. (2020). Celeb-df: A large-scale challenging dataset for deepfake forensics. In Proceedings of the IEEE/CVFConference on Computer Vision and Pattern Recognition (pp. 3207-3216). Rossler, A., Cozzolino, D., Verdoliva, L., Riess, C., Thies, J., Nießner, M. (2019). Faceforensics++: Learning to detect manipulated facial images. In Proceedings of the IEEE/CVF International Conference on Computer Vision (pp. 1-11). Nicholas Dufour, Andrew Gully, Per Karlsson, Alexey Vic- tor Vorbyov, Thomas Leung, Jeremiah Childs, and Christoph Bregler. Deepfakes detection dataset by google jigsaw. Dolhansky, B., Howes, R., Pflaum, B., Baram, N., Ferrer, C. C. (2019). The deepfake detection challenge (dfdc) preview dataset. arXiv preprint arXiv:1910.08854. Li, Y., Yang, X., Sun, P., Qi, H., Lyu, S. (2020). Celeb-DF: A Large-scale Challenging Dataset for DeepFake Forensics. GitHub repository. Available at: https://github.com/yuezunli/celeb-deepfakeforensics (Accessed: 30 May 2021). K. He, X. Zhang, S. Ren and J. Sun, 'Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition,' in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 37, no. 9, pp. 1904-1916, 1 Sept. 2015, doi:10.1109/TPAMI.2015.2389824. Liu, M. Y., Breuel, T., Kautz, J. (2017). Unsupervised image-to-image translation networks. arXiv preprint arXiv:1703.00848. Petrov, I., Gao, D., Chervoniy, N., Liu, K., Marangonda, S., Umé, C., ... Zhang, W. (2020). Deepfacelab: A simple, flexible and extensible face swapping framework. arXiv preprint arXiv:2005.05535. Petrov, I., Gao, D., Chervoniy, N., Liu, K., Marangonda, S., Umé, C., ... Zhang,W. (2020). Deepfacelab: A simple, flexible and extensible face swapping framework. Github repository. Available at: https://github.com/iperov/DeepFaceLab (Accessed: 31 May 2021). faceswap. Github repository. Available at: https://github.com/deepfakes/faceswap (Accessed: 31 May 2021). faceswap-GAN. Github repository. Available at:https://github.com/shaoanlu/faceswap-GAN (Accessed: 31 May 2021). DFaker . Github repository. Available at: https://github.com/dfaker/df (Accessed:31 May 2021). FakeApp. https://www.malavida.com/en/soft/ (Accessed: 31 May 2021). Li, Y., Chang, M. C., Lyu, S. (2018). In ictu oculi: Exposing ai generated fake face videos by detecting eye blinking. arXiv preprint arXiv:1806.02877. Matern, F., Riess, C., Stamminger, M. (2019, January). Exploiting visual artifacts to expose deepfakes and face manipulations. In 2019 IEEE Winter Applications of Computer Vision Workshops (WACVW) (pp. 83-92). IEEE. Simonyan, K., Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556. He, K., Zhang, X., Ren, S., Sun, J. (2016). Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 770-778). Karras, T., Aila, T., Laine, S., Lehtinen, J. (2017). Progressive growing of gans for improved quality, stability, and variation. arXiv preprint arXiv:1710.10196. Kingma, D. P., Dhariwal, P. (2018). Glow: Generative flow with invertible 1x1 convolutions. arXiv preprint arXiv:1807.03039. Wang, T. C., Liu, M. Y., Zhu, J. Y., Tao, A., Kautz, J., Catanzaro, B. (2018). High-resolution image synthesis and semantic manipulation with conditional gans. In Proceedings of the IEEE conference on computer vision and pattern recognition(pp. 8798-8807). Afchar, D., Nozick, V., Yamagishi, J., Echizen, I. (2018, December). Mesonet: a compact facial video forgery detection network. In 2018 IEEE International Workshop on Information Forensics and Security (WIFS) (pp. 1-7). IEEE. Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., ... Rabinovich, A. (2015). Going deeper with convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 1-9). Thies, J., Zollhofer, M., Stamminger, M., Theobalt, C., Nießner, M. (2016). Face2face: Real-time face capture and reenactment of rgb videos. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 2387-2395). Güera, D., Delp, E. J. (2018, November). Deepfake video detection using recurrent neural networks. In 2018 15th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS) (pp. 1-6). IEEE. Nguyen, H. H., Yamagishi, J., Echizen, I. (2019, May). Capsule-forensics: Using capsule networks to detect forged images and videos. In ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)(pp. 2307-2311). IEEE. Hinton, G. E., Krizhevsky, A., Wang, S. D. (2011, June). Transforming auto-encoders. In International conference on artificial neural networks (pp. 44-51). Springer, Berlin, Heidelberg. Nguyen, H. H., Yamagishi, J., Echizen, I. (2019). Use of a capsule network to detect fake images and videos. arXiv preprint arXiv:1910.12467. Chollet, F. (2017). Xception: Deep learning with depthwise separable convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 1251-1258). Yang, X., Li, Y., Lyu, S. (2019, May). Exposing deep fakes using inconsistent head poses. In ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (pp. 8261-8265). IEEE. Korshunov, P., Marcel, S. (2018). Deepfakes: a new threat to face recognition? assessment and detection. arXiv preprint arXiv:1812.08685. ISO 690 VidTIMIT Audio-Video Dataset. Available at: https://conradsanderson.id.au/vidtimit/ (Accessed: 31 May 2021). Sanderson, C., Lovell, B. C. (2009, June). Multi-region probabilistic histograms for robust and scalable identity inference. In International conference on biometrics (pp. 199-208). Springer, Berlin, Heidelberg. Reinhard, E., Adhikhmin, M., Gooch, B., Shirley, P. (2001). Color transfer between images. IEEE Computer graphics and applications, 21(5), 34-41. Deng, J., Dong, W., Socher, R., Li, L. J., Li, K., Fei-Fei, L. (2009, June). Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition (pp. 248-255). Ieee. dlib toolkit Available at: https://pypi.org/project/dlib/ (Accessed: 24 July 2021). dlib pretrained models. Available at: http://dlib.net/files/ (Accessed: 24 July 2021). Nearest-neighbor interpolation. Available at Wikipedia: https://en.wikipedia.org/wiki/Nearest-neighbor_interpolation (Accessed: 24 July 2021). Bilinear interpolation. Available at Wikipedia: https://en.wikipedia.org/wiki/Bilinear_interpolation (Accessed: 24 July 2021). Ma, L., Jia, X., Sun, Q., Schiele, B., Tuytelaars, T., Van Gool, L. (2017). Pose guided person image generation. arXiv preprint arXiv:1705.09368. Wang, Z., Bovik, A. C., Sheikh, H. R., Simoncelli, E. P. (2004). Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing, 13(4), 600-612. FaceSwap by Kowalski. Github repository. Available at: https://github.com/MarekKowalski/FaceSwap/ (Accessed: 26 July 2021) Thies, J., Zollhöfer, M., Nießner, M. (2019). Deferred neural rendering: Image synthesis using neural textures. ACM Transactions on Graphics (TOG), 38(4), 1-12. Blanz, V., Vetter, T. (1999, July). A morphable model for the synthesis of 3D faces. In Proceedings of the 26th annual conference on Computer graphics and interactive techniques (pp. 187-194). Alexander, O., Rogers, M., Lambeth, W., Chiang, M., Debevec, P. (2009). The digital emily project: photoreal facial modeling and animation. In Acm siggraph 2009 courses (pp. 1-15). Cao, C., Weng, Y., Zhou, S., Tong, Y., Zhou, K. (2013). Facewarehouse: A 3d facial expression database for visual computing. IEEE Transactions on Visualization and Computer Graphics, 20(3), 413-425. Microsoft UVAtlas. Available at: https://github.com/Microsoft/UVAtlas (Accessed: 14 Aug 2021) Xception pre-trained model checkpoints. Available at: https://github.com/ondyari/FaceForensics/tree/master/classification (Accessed: 15 Aug 2021) DSP-FWA pre-trained model checkpoints. Available at Github: DSP-FWA: Dual Spatial Pyramid for Exposing Face Warp Artifacts in DeepFake Videos. https://github.com/yuezunli/DSP-FWA (Accessed: 15 Aug 2021) Pytorch ResNet Document Page. Available at pytorch.org: https://pytorch.org/vision/0.8/_modules/torchvision/models/resnet.html (Accessed: 16 Aug 2021) Pan, S. J., Yang, Q. (2009). A survey on transfer learning. IEEE Transactions on knowledge and data engineering, 22(10), 1345-1359. Cross-validation (statistics). Available at Wikipedia: https://en.wikipedia.org/wiki/Cross-validation_(statistics) (Accessed: 20 Aug 2021) | |
| dc.identifier.uri | http://tdr.lib.ntu.edu.tw/jspui/handle/123456789/79586 | - |
| dc.description.abstract | 本研究使用機器學習方法為基礎，實作偽造影片（DeepFake）辨識模型。我們的目標是訓練出可以在具有辨識難易度的Celeb-DF V2資料集上，辨識能力良好的模型，並期許此模型的辨識力能夠勝過既有的模型。本研究採取幀AUC分數（Frame-level AUC Score）作為評量模型辨識力的基準。 我們使用了在圖像分類領域享有盛名的Xception模型以及DSP-FWA模型作為骨架，使用這兩個模型的預訓練權重在部分的Celeb-DF V2資料集上進行轉移學習（ Transfer Learning ）。與之前的研究不同的是，我們在模型訓練的驗證階段採取了交叉驗證，並且挑選了不同的訓練資料。我們挑選在驗證資料集上擁有最高準確率的模型，作為最終模型。 我們使用Celeb-DF V2資料集官方釋出的測試資料作為測試資料集，並且比較我們的模型與既有的分類模型的幀AUC分數 （ Frame-level AUC Score ）。最後，根據我們得到的實驗結果，比起既有的模型，我們的模型擁有更高的幀AUC分數 （ Frame-level AUC Score ）。我們也探討了這些模型在不同的影片時間點的累積辨識準確率，並記錄了我們對於即時辨識偽造影片（DeepFake）情境中使用模型辨識的見解。 | zh_TW |
| dc.description.provenance | Made available in DSpace on 2022-11-23T09:04:27Z (GMT). No. of bitstreams: 1 U0001-1509202114355400.pdf: 7012357 bytes, checksum: 39b065133c58e5aacb6a21e28602f2f5 (MD5) Previous issue date: 2021 | en |
| dc.description.tableofcontents | APPROVAL i ACKNOWLEDGEMENT ii CHINESE ABSTRACT iii ABSTRACT iv CONTENTS v LIST OF FIGURES vii LIST OF TABLES viii Chapter 1 Introduction 1 1.1 Background 1 1.2 Objectives 2 Chapter 2 Related Work 4 2.1 Video Manipulation Methods 4 2.2 Manipulated Video Detection Models 5 2.3 DeepFake Video Datasets 6 2.4 The Celeb-DF Dataset (V2) 7 2.5 FaceForensics++ Dataset 13 2.6 The Xception Network 19 2.7 The DSP-FWA Network 26 Chapter 3 Methodology 31 3.1 Transfer Learning 31 3.2 Training Data 32 3.3 Pre-trained Models 34 3.4 Cross Validation 34 3.5 Experiment Setting 35 3.6 Model Performance Evaluation 35 Chapter 4 Result 37 4.1 Frame-level AUC Score 37 4.2 Accumulated Accuracy 39 4.3 Evaluation on other datasets 42 4.4 Error Analysis 43 Chapter 5 Conclusion 46 Reference 47 | |
| dc.language.iso | en | |
| dc.title | 以機器學習方法實作偽造影片分類系統 | zh_TW |
| dc.title | DeepFake Video Classification with Machine Learning | en |
| dc.date.schoolyear | 109-2 | |
| dc.description.degree | 碩士 | |
| dc.contributor.author-orcid | 0000-0001-5468-5402 | |
| dc.contributor.oralexamcommittee | 陳建錦(Hsin-Tsai Liu),盧信銘(Chih-Yang Tseng) | |
| dc.subject.keyword | 機器學習,圖像分類,影片分類,偽造影片辨識,深度偽造,轉移學習, | zh_TW |
| dc.subject.keyword | Machine learning,Image classification,Video classification,DeepFake video detection,DeepFake,Transfer learning, | en |
| dc.relation.page | 54 | |
| dc.identifier.doi | 10.6342/NTU202103192 | |
| dc.rights.note | 同意授權(全球公開) | |
| dc.date.accepted | 2021-09-16 | |
| dc.contributor.author-college | 管理學院 | zh_TW |
| dc.contributor.author-dept | 資訊管理學研究所 | zh_TW |
| 顯示於系所單位： | 資訊管理學系 | |
文件中的檔案：
| 檔案 | 大小 | 格式 | |
|---|---|---|---|
| U0001-1509202114355400.pdf | 6.85 MB | Adobe PDF | 檢視/開啟 |
系統中的文件，除了特別指名其著作權條款之外，均受到著作權保護，並且保留所有的權利。