請用此 Handle URI 來引用此文件：
http://tdr.lib.ntu.edu.tw/jspui/handle/123456789/81100完整後設資料紀錄
| DC 欄位 | 值 | 語言 |
|---|---|---|
| dc.contributor.advisor | 丁建均(Jian-Jiun Ding) | |
| dc.contributor.author | Hsuan-Wei Hsu | en |
| dc.contributor.author | 許軒瑋 | zh_TW |
| dc.date.accessioned | 2022-11-24T03:30:33Z | - |
| dc.date.available | 2021-09-02 | |
| dc.date.available | 2022-11-24T03:30:33Z | - |
| dc.date.copyright | 2021-09-02 | |
| dc.date.issued | 2021 | |
| dc.date.submitted | 2021-08-23 | |
| dc.identifier.citation | [1]D. Afchar, V. Nozick, J. Yamagishi, and I. Echizen. Mesonet: a compact facialvideo forgery detection network.2018IEEEInternationalWorkshoponInformationForensicsandSecurity(WIFS), Dec 2018. [2]F. Chollet. Xception: Deep learning with depthwise separable convolutions. In2017IEEEConferenceonComputerVisionandPatternRecognition(CVPR), pages1800–1807, 2017. [3]deepfakes. Deepfakes.https://github.com/deepfakes/faceswap, 2021. [4]deepfakes. Faceswap.https://github.com/MarekKowalski/FaceSwap, 2021. [5]T. DeVries and G. W. Taylor. Improved regularization of convolutional neural networks with cutout.arXivpreprintarXiv:1708.04552, 2017. [6]B. Dolhansky, J. Bitton, B. Pflaum, J. Lu, R. Howes, M. Wang, and C. Canton Ferrer.The deepfake detection challenge dataset.arXiveprints, pages arXiv–2006, 2020. [7]I. Goodfellow, J. PougetAbadie, M. Mirza, B. Xu, D. WardeFarley, S. Ozair,A. Courville, and Y. Bengio. Generative adversarial nets.Advancesinneuralinformationprocessingsystems, 27, 2014 [8]K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition,2015. [9]iperov. Deepfacelab the leading software for creating deepfakes.https://github.com/iperov/DeepFaceLab, 2021. [10]L. Jiang, R. Li, W. Wu, C. Qian, and C. C. Loy. Deeperforensics1.0: A largescale dataset for realworld face forgery detection. InProceedingsoftheIEEE/CVFConferenceonComputerVisionandPatternRecognition, pages 2889–2898, 2020. [11]D.K. Kim, D. Kim, and K. Kim. Facial manipulation detection based on the colordistribution analysis in edge region.arXivpreprintarXiv:2102.01381, 2021. [12]D. P. Kingma and M. Welling. Autoencoding variational bayes.arXivpreprintarXiv:1312.6114, 2013. [13]L. Li, J. Bao, T. Zhang, H. Yang, D. Chen, F. Wen, and B. Guo. Face xray formore general face forgery detection. InProceedingsoftheIEEE/CVFConferenceonComputerVisionandPatternRecognition, pages 5001–5010, 2020. [14]X. Li, Y. Lang, Y. Chen, X. Mao, Y. He, S. Wang, H. Xue, and Q. Lu. Sharp multiple instance learning for deepfake video detection.Proceedingsofthe28thACMInternationalConferenceonMultimedia, Oct 2020. [15]Y. Li, M.C. Chang, and S. Lyu. In ictu oculi: Exposing ai generated fake face videosby detecting eye blinking, 2018. [16]Y. Li and S. Lyu. Exposing deepfake videos by detecting face warping artifacts,2019. [17]Y. Li, X. Yang, P. Sun, H. Qi, and S. Lyu. Celebdf: A largescale challenging datasetfor deepfake forensics. InProceedingsoftheIEEE/CVFConferenceonComputerVisionandPatternRecognition, pages 3207–3216, 2020. [18]H. Liu, X. Li, W. Zhou, Y. Chen, Y. He, H. Xue, W. Zhang, and N. Yu. Spatialphaseshallow learning: Rethinking face forgery detection in frequency domain, 2021. [19]J. Lukas, J. Fridrich, and M. Goljan. Digital camera identification from sensor patternnoise.IEEETransactionsonInformationForensicsandSecurity, 1(2):205–214,2006. [20]I. Masi, A. Killekar, R. M. Mascarenhas, S. P. Gurudatt, and W. AbdAlmageed. Twobranch recurrent network for isolating deepfakes in videos. InEuropeanConferenceonComputerVision, pages 667–684. Springer, 2020. [21]F. Matern, C. Riess, and M. Stamminger. Exploiting visual artifacts to expose deepfakes and face manipulations. In2019IEEEWinterApplicationsofComputerVisionWorkshops(WACVW), pages 83–92, 2019. [22]H. H. Nguyen, F. Fang, J. Yamagishi, and I. Echizen. Multitask learning for detecting and segmenting manipulated facial images and videos, 2019. [23]H. H. Nguyen, J. Yamagishi, and I. Echizen. Capsuleforensics: Using capsule networks to detect forged images and videos, 2018. [24]G. R. Nick Dufour and J. Andrew Gully. Contributing data to deepfake detection research.https://ai.googleblog.com/2019/09/contributing-data-to-deepfake-detection.html, 2019. [25]A. Parkin and O. Grinchuk. Creating artificial modalities to solve rgb liveness.arXivpreprintarXiv:2006.16028, 2020. [26]A. Rössler, D. Cozzolino, L. Verdoliva, C. Riess, J. Thies, and M. Nießner. Faceforensics: A largescale video dataset for forgery detection in human faces.arXivpreprintarXiv:1803.09179, 2018. [27]A. Rossler, D. Cozzolino, L. Verdoliva, C. Riess, J. Thies, and M. Nießner. Faceforensics++: Learning to detect manipulated facial images. InProceedingsoftheIEEE/CVFInternationalConferenceonComputerVision, pages 1–11, 2019. [28]selimsef. dfdc_deepfake_challenge.https://github.com/selimsef/dfdc_deepfake_challenge, 2020. [29]L. N. Smith and N. Topin. Superconvergence: Very fast training of neural networksusing large learning rates, 2018. [30]J. Thies, M. Zollhöfer, and M. Nießner. Deferred neural rendering: Image synthesisusing neural textures.ACMTransactionsonGraphics(TOG), 38(4):1–12, 2019. [31]J. Thies, M. Zollhofer, M. Stamminger, C. Theobalt, and M. Nießner. Face2face:Realtime face capture and reenactment of rgb videos. InProceedingsoftheIEEEconferenceoncomputervisionandpatternrecognition, pages 2387–2395, 2016. [32]R. Tolosana, S. RomeroTapiador, J. Fierrez, and R. VeraRodriguez. Deepfakesevolution: Analysis of facial regions and fake detection performance, 2020. [33]V. Ume. Deeptomcruise tiktok breakdown.https://www.youtube.com/watch?v=wq-kmFCrF5Q ab_channel=VFXChrisUme, 2021. [34]R. Wang, F. JuefeiXu, L. Ma, X. Xie, Y. Huang, J. Wang, and Y. Liu. Fakespotter:A simple yet robust baseline for spotting aisynthesized fake faces, 2020. [35]X. Wang, Y. Yan, P. Tang, X. Bai, and W. Liu. Revisiting multiple instance neuralnetworks.PatternRecognition, 74:15–24, Feb 2018. [36]X. Xuan, B. Peng, W. Wang, and J. Dong. On the generalization of gan imageforensics. InChineseconferenceonbiometricrecognition, pages 134–141. Springer,2019. [37]X. Yang, Y. Li, and S. Lyu. Exposing deep fakes using inconsistent head poses, 2018. [38]K. Zhang, Z. Zhang, Z. Li, and Y. Qiao. Joint face detection and alignment using multitask cascaded convolutional networks.IEEESignalProcessingLetters,23(10):1499–1503, Oct 2016. [39]T. Zhao, X. Xu, M. Xu, H. Ding, Y. Xiong, and W. Xia. Learning to recognize patchwise consistency for deepfake detection.arXivpreprintarXiv:2012.09311, 2020. [40]Z.Zhong, L.Zheng, G.Kang, S.Li, andY.Yang. Randomerasingdataaugmentation.InProceedingsoftheAAAIConferenceonArtificialIntelligence, volume 34, pages13001–13008, 2020. [41]P. Zhou, X. Han, V. I. Morariu, and L. S. Davis. Twostream neural networks fortampered face detection, 2018. | |
| dc.identifier.uri | http://tdr.lib.ntu.edu.tw/jspui/handle/123456789/81100 | - |
| dc.description.abstract | 深度偽造技術(Deepfake)，用來指稱透過深度學習來做到人臉圖像合成目的的技術，近來這種技術被廣為濫用製造假新聞、偽造名人情色影片等等，造成社會中許多危害與信任危機，使得對應的檢測技術日益受到人們的重視。在檢測臉部偽造技術中，偽造生成方法隨著深度學習的發展品質不斷提升，使得偽造檢測這項議題難以得到一個通用解，為了不讓偵測Deepfake的技術落後於偽造生成的進步，許多研究人員與企業都相繼投入對抗深度人臉偽造，好比如Facebook、Microsoft等企業於2019年底聯合舉辦了Deepfake Detection Challenge，除此之外相關的資料集也陸續在提出，來提供開發者來建構更好的Deepfake檢測工具。在本篇論文中，我們綜合多種圖片噪音模態（high pass filter DCT、Error-Level-Analysis、Photo Response Non-Uniformity）的分析作為訓練輸入，目的是為了能得到更穩健的訓練模型以獲得更高的檢測精度，並且搭配兩分支的預測網路，來分離不同組成成份的偽造偽影（manipulation artifact、blending artifacts），最後透過多種loss的結合，讓特徵向量在高維空間中的分佈能符合我們的預期；總結而言，我們的檢測方法相較於過去許多的作法，除了在檢測圖像真假上有著更好的表現，還能夠去預測偽造區域（manipulation region、blending boundary）的所在，使得訓練模型的檢測結果更具有解釋性。 | zh_TW |
| dc.description.provenance | Made available in DSpace on 2022-11-24T03:30:33Z (GMT). No. of bitstreams: 1 U0001-1808202114311100.pdf: 7614505 bytes, checksum: e096ac9de35b8cc603d87fa0baa8cbd6 (MD5) Previous issue date: 2021 | en |
| dc.description.tableofcontents | Verification Letter from the Oral Examination Committee ---------------i Acknowledgements -----------------------------------------------------ii 摘要 ---------------------------------------------------------------- iii Abstract -------------------------------------------------------------iv Contents ------------------------------------------------------------vii List of Figures ------------------------------------------------------xi List of Tables -----------------------------------------------------xiii I. Topic Background ---------------------------------------------------1 Chapter 1. Introduction -----------------------------------------------2 Chapter 2. About Deepfake ---------------------------------------------5 II. Related works -----------------------------------------------------7 Chapter 3. Related Datasets -------------------------------------------8 3.1 Faceforensics++ -------------------------------------------------8 3.2 Celeb-DF(v2) ----------------------------------------------------9 Chapter 4. Review on Existing Forgery Detection Research -------------11 4.1 Relying on Artifacts -------------------------------------------11 4.1.1 Exposing DeepFake Videos By Detecting Face Warping Artifacts --11 4.1.2 In ictu oculi: Exposing ai generated fake face videos by detecting eye blinking --12 4.2 Binary Classifiers ---------------------------------------------13 4.2.1 Mesonet: a compact facial video forgery detection network --13 4.2.2 Capsule-Forensics: Using Capsule Networks to Detect Forged Images and Videos --14 4.2.3 Two-branch recurrent network for isolating deepfakes in videos --14 4.3 Generalization performance -------------------------------------15 4.3.1 On the generalization of GAN image forensics ---------------15 4.3.2 Face X-ray for more general face forgery detection ---------15 4.3.3 Learning to Recognize Patch-Wise Consistency for Deepfake Detection --17 III. Proposed Method and Discussion ----------------------------------18 Chapter 5. Proposed Methods ------------------------------------------19 5.1 Ideas ----------------------------------------------------------19 5.2 Proposed Architecture ------------------------------------------19 5.2.1 Use kinds of image noise analysis as training input --------21 I. High-Pass Filtering performed by Discrete Cosine Transform --21 II. Error Level Analysis ---------------------------------------23 III. Photo Response Non-Uniformity -----------------------------23 5.2.2 Summary ----------------------------------------------------24 5.2.3 Two branches of multi-task learning predict manipulation artifacts and blending artifacts --25 5.2.4 Multi-task learning details --------------------------------28 Chapter 6. Experimental Evaluation -----------------------------------32 6.1 Implementations Details ----------------------------------------32 6.2 Experiment Settings --------------------------------------------33 6.2.1 Pre-processing ---------------------------------------------33 6.2.2 Data Augmentation ------------------------------------------33 6.2.3 Hyper-parameters -------------------------------------------34 6.2.4 Evaluation Metrics -----------------------------------------34 6.3 In-Dataset Evaluation ------------------------------------------34 6.4 Cross-Dataset Evaluation ---------------------------------------36 6.5 Ablation Study -------------------------------------------------36 Chapter 7. Discussion ------------------------------------------------38 Chapter 8. Conclusion ------------------------------------------------41 Reference ------------------------------------------------------------43 Appendix A - Cutout-like Augmentation --------------------------------48 | |
| dc.language.iso | zh-TW | |
| dc.subject | 多任務學習 | zh_TW |
| dc.subject | 深度學習 | zh_TW |
| dc.subject | 深度偽造檢測 | zh_TW |
| dc.subject | 圖片雜訊分析 | zh_TW |
| dc.subject | 偽造偽影分離 | zh_TW |
| dc.subject | Forgery Artifacts Separation | en |
| dc.subject | Deep Learning | en |
| dc.subject | Deepfake Detection | en |
| dc.subject | Image Noise Analysis | en |
| dc.subject | Multi-task learning | en |
| dc.title | 基於多噪音模態與兩分支預測網路之深度偽造影片檢測 | zh_TW |
| dc.title | Using Multiple Noise Modalities with Two-Branch Prediction Network for Deepfake Detection | en |
| dc.date.schoolyear | 109-2 | |
| dc.description.degree | 碩士 | |
| dc.contributor.oralexamcommittee | 余執彰(Hsin-Tsai Liu),張榮吉(Chih-Yang Tseng),歐陽良昱 | |
| dc.subject.keyword | 深度學習,深度偽造檢測,圖片雜訊分析,偽造偽影分離,多任務學習, | zh_TW |
| dc.subject.keyword | Deep Learning,Deepfake Detection,Image Noise Analysis,Forgery Artifacts Separation,Multi-task learning, | en |
| dc.relation.page | 49 | |
| dc.identifier.doi | 10.6342/NTU202102467 | |
| dc.rights.note | 同意授權(限校園內公開) | |
| dc.date.accepted | 2021-08-23 | |
| dc.contributor.author-college | 電機資訊學院 | zh_TW |
| dc.contributor.author-dept | 電信工程學研究所 | zh_TW |
| 顯示於系所單位： | 電信工程學研究所 | |
文件中的檔案：
| 檔案 | 大小 | 格式 | |
|---|---|---|---|
| U0001-1808202114311100.pdf 授權僅限NTU校內IP使用（校園外請利用VPN校外連線服務） | 7.44 MB | Adobe PDF |
系統中的文件，除了特別指名其著作權條款之外，均受到著作權保護，並且保留所有的權利。