請用此 Handle URI 來引用此文件:
http://tdr.lib.ntu.edu.tw/jspui/handle/123456789/38357
完整後設資料紀錄
DC 欄位 | 值 | 語言 |
---|---|---|
dc.contributor.advisor | 歐陽彥正(Yen-Jen Oyang) | |
dc.contributor.author | Yu-Yen Ou | en |
dc.contributor.author | 歐昱言 | zh_TW |
dc.date.accessioned | 2021-06-13T16:31:18Z | - |
dc.date.available | 2008-07-13 | |
dc.date.copyright | 2005-07-13 | |
dc.date.issued | 2005 | |
dc.date.submitted | 2005-07-11 | |
dc.identifier.citation | [1] E. Artin. The Gamma Function. New York: Holt, Rinehart, and Winston,
1964. [2] F. Belloir, A. Fache, and A. Billat. A general approach to construct rbf netbased classifier. In Proceedings of the 7th European Symposium on Artificial Neural Network, pages 399—404, 1999. [3] J. Bentley. Multidimensional binary search trees used for associative searching. In Communications of the ACM, pages 18(9):509—517, 1975. [4] M. Bianchini, P. Frasconi, and M. Gori. Learning without local minima in radial basis function networks. IEEE Transactions on Neural Networks, 6(3):749—756, May 1995. [5] C. M. Bishop. Improving the generalization properties of radial basis function neural networks. Neural Computation, 3(4):579—588, 1991. [6] C. L. Blake and C. J. Merz. UCI repository of machine learning databases. Technical report, University of California, Department of Information and Computer Science, Irvine, CA, 1998. Available at http://www.ics.uci.edu/~mlearn/MLRepository.html. [7] L. Breiman,W.Meisel, and E. Purcell. Variable kernel es-timates of multivariate densities. Technometrics, 19:135—144, 1977. [8] D. S. Broomhead and D. Lowe. Multivariable functional interpolation and adaptive networks. Complex Systems, 2:321—355, 1988. [9] C. J. C. Burges and B. Sch¨olkopf. Improving the accuracy and speed of support vector machines. In M. C. Mozer, M. I. Jordan, and T. Petsche, editors, Advances in Neural Information Processing Systems, volume 9, page 375. The MIT Press, 1997. [10] C.-C. Chang and C.-J. Lin. LIBSVM: a library for support vector machines, 2001. Software available at http://www.csie.ntu.edu.tw/~cjlin/libsvm. 77 78 [11] E. I. Chang and R. P. Lippmann. A boundary hunting radial basis function classifier which allocates centers constructively. In Advances in Neural Information Processing Systems, volume 5, pages 131—138. Morgan Kaufmann, San Mateo, CA, 1993. [12] O. Chapelle, V. Vapnik, O. Bousquet, and S. Mukherjee. Choosing multiple parameters for support vector machines. Machine Learning, 46:131—159, 2002. [13] C.-Y. Chen, S.-C. Hwang, and Y.-J. Oyang. An incremental hierarchical data clustering algorithm based on gravity theory. In Proc. of PAKDD-2002, pages 237—250, 2002. [14] S. Chen, C. F. N. Cowan, and P. M. Grant. Orthogonal least squares learning algorithm for radial basis function networks. IEEE Transactions on Neural Networks, 2(2):302—309, Mar. 1991. [15] K.-M. Chung, W.-C. Kao, T. Sun, and C.-J. Lin. Decomposition methods for linear support vector machines. Neural Computation, 16:1689—1704, 2004. [16] C. Cortes and V. Vapnik. Support-vector network. Machine Learning, 20:273— 297, 1995. [17] J. A. Cuff and G. J. Barton. Evaluation and improvement of multiple sequence methods for protein secondary structure prediction. Proteins: Struct. Funct. Genet., 34:508—519, 1999. [18] D. DeCoste and K. Wagstaff. Alpha seeding for support vector machines. In Proceedings of International Conference on Knowledge Discovery and Data Mining (KDD-2000), 2000. [19] P. A. Devijver and J. Kittler. Pattern recognition : a statistical approach. Prentice Hall, 1982. [20] C. Ding and I. Dubchak. Multi-class protein fold recognition using support vector machines and neural networks, 2001. [21] K. Duan, S. S. Keerthi, and A. N. Poo. Evaluation of simple performance measures for tuning SVM hyperparameters. Neurocomputing, 51:41—59, 2003. [22] S. T. Dumais, J. Platt, D. Heckerman, and M. Sahami. Inductive learning algorithms and representations for text categorization. In Proc. ACM-Conf. Information and Knowledge Management (CIKM98), pages 148V—155, 1998. [23] T. S. Furey, N. Christianini, N. Duffy, D. W. Bednarski, M. Schummer, and D. Hauessler. Support vector machine classification and validation of cancer tissue samples using microarray expression data. Bioinformatics, 16(10):906— 914, 2000. 79 [24] J. Ghosh and A. Nag. An overview of radial basis function networks. Radial Basis Function Neural Network Theory and Applications, R. J. Howlerr and L. C. Jain (Eds), 2000. [25] T. R. Golub, D. K. Slonim, P. Tamayo, C. Huard, M. Gaasenbeek, J. P. Mesirov, H. Coller, M. L. Loh, J. R. Downing, M. A. Caligiuri, C. D. Bloomfield, and E. S. Lander. Molecular Classification of Cancer: Class Discovery and Class Prediction by Gene Expression Monitoring. Science, 286(5439):531—537, 1999. [26] J. Guo, H. Chen, Z. Sun, and Y. Lin. A novel method for protein secondary structure prediction using dual-layer svm and profiles. Proteins, 54(4):738—743, 2004. [27] E. Hartman, J. D. Keeler, and J. M. Kowalski. Layered neural networks with Gaussian hidden units as universal approximations. Neural Computation, 2(2):210—215, 1990. [28] C.-W. Hsu and C.-J. Lin. A comparison of methods for multi-class support vector machines. IEEE Transactions on Neural Networks, 13(2):415—425, 2002. [29] S. Hua and Z. Sun. A novel method of protein secondary structure prediction with high segment overlap measure: Svm approach. J. Mol. Biol., 308:397 — 407, 2001. [30] Y. Hwang and S. Bang. An efficient method to construct a radial basis function neural network classifier. Neural Networks, 10(8):1495—1503, 1997. [31] A. K. Jain and R. C. Dubes. Algorithms for Clustering Data. Prentice Hall International, 1988. [32] A. K. Jain, M. N. Murty, and P. J. Flynn. Data clustering: a review. ACM Computing Surveys, 31(3):264—323, Sept. 1999. [33] T. Joachims. Text categorization with support vector machines: learning with many relevant features. In Proceedings of ECML-98, 10th European Conference on Machine Learning, pages 137—142, 1998. [34] V. Kecman. Learning and Soft Computing: Support Vector Machines, Neural Networks, and Fuzzy Logic Models. The MIT Press, 2001. [35] S. S. Keerthi. Efficient tuning of SVM hyperparameters using radius/margin bound and iterative algorithms. IEEE Transactions on Neural Networks, 13:1225—1229, 2002. [36] S. S. Keerthi and C.-J. Lin. Asymptotic behaviors of support vector machines with Gaussian kernel. Neural Computation, 15:1667—1689, 2003. [37] H. Kim and H. Park. Protein secondary structure prediction based on an improved support vector machines approach. Protein Engineering, 16(8):553—560, 2003. 80 [38] R. D. King and M. J. E. Sternberg. Identification and application of the concepts important for accurate and reliable protein secondary structure prediction. Protein Sci., 5(11):2298—2310, 1996. [39] S. Lee and R. M. Kil. Multilayer feedforward potential function network. In IEEE Second International Conference on Neural Networks, volume I, pages 161—172, 1988. [40] Y. Liu, J. Carbonell, J. Klein-Seetharaman, and V. Gopalakrishnan. Comparison of probabilistic combination methods for protein secondary structure prediction. Bioinformatics, 20(17):3099—3107, 2004. [41] D. G. Lowe. Similarity metric learning for a variable-kernel classifier. Neural Computation, 7:72—85, 1995. [42] A. Lyhyaoui, M. Martinez, I. Mora, M. Vazquez, J.-L. Sancho, and A. R. Figueiras-Vidal. Sample selection via clustering to construct support vectorlike classifiers. IEEE Transactions on Neural Networks, 10(6):1474, Nov 1999. [43] P. Megdassy. Decomposition of superposition of distributed functions. Hungarian Academy of Sciences, Budapest, 1961. [44] C. Micchelli. Interpolation of scattered data: distance matrices and conditionally positive definite functions. Constructive Approximation, 2:11—22, 1986. [45] D. Michie, D. J. Spiegelhalter, and C. C. Taylor. Machine Learning, Neural and Statistical Classification. Prentice Hall, Englewood Cliffs, N.J., 1994. [46] T. M. Mitchell. Machine Learning. McGraw-Hill, 1997. [47] J. Moody and C. J. Darken. Fast learning in networks of locally-tuned processing units. Neural Computation, 1(2):281—294, 1989. [48] P. Niyogi and F. Girosi. On the relationship between generalization error, hypothesis complexity, and sample complexity for radial basis functions. Neural Computation, 8(4):819—842, 1996. [49] M. J. L. Orr. Regularisation in the selection of radial basis function centres, 1995. [50] M. J. L. Orr. Introduction to radial basis function networks. Technical report, Center for Cognitive Science, University of Edinburgh, UK, 1996. [51] Y.-Y. Ou, C.-Y. Chen, S.-C. Hwang, and Y.-J. Oyang. Expediting model selection for support vector machines based on data reduction. In Proc. IEEE International Conference on Systems, Man and Cybernetics (SMC2003), pages 786—791, 2003. 81 [52] Y.-J. Oyang, S.-C. Hwang, Y.-Y. Ou, C.-Y. Chen, and Z.-W. Chen. A novel learing algorithm for data classfication with radial basis function networks. In Proc. of 9th International Conference on Neural Information Processing, Singapore, 2002. [53] Y.-J. Oyang, S.-C. Hwang, Y.-Y. Ou, C.-Y. Chen, and Z.-W. Chen. Data classification with radial basis function networks based on a novel kernel density estimation algorithm. IEEE Transactions on Neural Networks, 16:225 — 236, Jan 2005. [54] Y.-J. Oyang, Y.-Y. Ou, S.-C. Hwang, C.-Y. Chen, and D. T.-H. Chang. Data classification with a relaxed model of variable kernel density estimation. In to appear in Proc. of IJCNN 2005, 2005. [55] A. Papoulis. Probability, Random Variables, and Stochastic Processes. McGraw- Hill, New York, NY, USA, 1991. [56] J. Park and I.W. Sandberg. Universal approximation using radial-basis-function networks. Neural Computation, 3(2):246—257, 1991. [57] J. Park and I. W. Sandberg. Approximation and radial-basis-function networks. Neural Computation, 5(3):305—316, 1993. [58] T. Poggio and F. Girosi. A theory of networks for approximation and learning. Technical Report A.I. Memo 1140, Massachusetts Institute of Technology, Artificial Intelligence Laboratory and Center for Biological Information Processing, Whitaker College, Jul 1989. [59] T. Poggio and F. Girosi. Networks for approximation and learning. In Proceedings of the IEEE (special issue: Neural Networks I: Theory and Modeling), volume 78, pages 1481—1497, 1990. [60] M. J. D. Powell. Radial basis functions for multivariable interpolation: A review. In J. C. Mason and M. G. Cox, editors, Algorithms for Approximation. Clarendon Press, London, 1987. [61] W. H. Press. Numerical Recipes in C. Cambridge University Press, Cambridge, second edition, 1992. [62] S. K. Riis and A. Krogh. Improving prediction of protein secondary structure using structured neural networks and multiple sequence alignments. J. Comput. Biol., 3:163 — 183, 1996. [63] B. Rost and C. Sander. Prediction of protein secondary structure structure at better than 70% accuracy. J. Mol. Biol., pages 584 — 599, 1993. [64] S. R. Sain and D. W. Scott. Zero-bias locally adaptive density estimators. Scandinavian Journal of Statistics, 29(3). 82 [65] I. Schoenberg. Metric spaces and positive definite functions. Ann. of Math., 44:522—536, 1938. [66] B. Sch¨olkopf, C. Burges, and V. Vapnik. Incorporating invariances in support vector learning machines, 1996. [67] B. W. Silverman. Density Estimation for Statistics and Data Analysis. Chapman & Hall, London, 1986. [68] I. Tarassenko and S. Roberts. Supervised and unsupervised learning in radial basis function classifiers. In IEE Proceedings-Vision, Image and Signal Processing, volume 141, pages 210—216, 1994. [69] A. N. Tikhonov and V. Y. Arsenin. Solutions of Ill-Posed Problems. V.H. Winston & Sons, John Wiley & Sons, Washington D.C., 1977. [70] J. J. Ward, L. J. McGuffin, B. F. Buxton, and D. T. Jones. Secondary structure prediction with support vector machines. Bioinformatics, 19(13):1650—1655, 2003. [71] D. R. Wilson and T. R. Martinez. Reduction techniques for instance-based learning algorithms. Machine Learning, 38(3):257—286, 2000. [72] I. H. Witten and E. Frank. Data mining. Morgan Kaufmann, Los Altos, US, 2000. [73] L. Xu, A. Krzyzak, and A. Yuille. On radial basis function nets and kernel regression: Statistical consistency, convergence rates, and receptive field size. Neural Networks, 7(4), 1994. [74] A. Zien, G. R¨atsch, S. Mika, B. Sch¨olkopf, T. Lengauer, and K.-R. M¨uller. Engineering support vector machine kernels that recognize translation initiation sites. Bioinformatics, 16, 2000. | |
dc.identifier.uri | http://tdr.lib.ntu.edu.tw/jspui/handle/123456789/38357 | - |
dc.description.abstract | 本論文主要是討論一系列使用RBF類神經網路 (Radial Basis Function Networks) 在機器學習 (Machine Learning) 領域的研究。
論文的第一個部分討論到如何用正規化程序 (regularization procedure) 有效率的建立一個RBF類神經網路。其中有兩個重要的問題,第一個問題是如何決定核心函數 (kernel function) 的個數與位置。第二個問題是如何決定核心函數組合成RBF類神經網路的權重。在這個論文,我用隨機與遞增式學習兩種方法來決定核心函數的位置,然後用正規化方法與 Cholesky 分解來決定各個核心函數的權重。實驗結果顯示這篇論文所使用的方法可以使用在機器學習與生物資訊領域,而且獲得不錯的結果。 第二個部分討論到一個新的核心密度預測 (kernel density estimation) 的演算法跟他的應用。這個演算法稱為 RVKDE (relaxed variable kernel density estimation) 是由我們實驗室團隊在近期所提出,並應用在許多方面,而我把他拿來應用在機器學習領域的兩個題目上,構成了這個論文的第二部分。 | zh_TW |
dc.description.abstract | This thesis reports a series of studies on machine learning with the radial basis function network (RBFN).
The first part of this thesis discusses how to construct an RBFN efficiently with the regularization procedure. In fact, construction of an RBFN with the regularization procedure involves two main issues. The first issue concerns the number of hidden nodes to be incorporated and where the centers of the associated kernel functions should be located. The second issue concerns how the links between the hidden layer and the output layer should be weighted. For the first issue, this thesis discusses the effects with a random samples based approach and an incremental clustering based approach. For the second issue, this thesis elaborates the effects with the Cholesky decomposition employed. Experimental results show that an RBFN constructed with the approaches proposed in this thesis is able to deliver the same level of classification accuracy as the SVM and offers several important advantages. Finally, this thesis reports the experimental results with the QuickRBF package, which has been developed based on the approaches proposed in this thesis, applied to bioinformatics problems. The second study reported in this thesis concerns how the novel relaxed variable kernel density estimation (RVKDE) algorithm that our research team has recently proposed performs in data classification applications. The experimental results reveal that the classifier configured with the RVKDE algorithm is capable of delivering the same level of accuracy as the SVM, while enjoying some advantages in comparison with the SVM. In particular, the time complexity for construction of a classifier with the RVKDE algorithm is O(nlogn), where n is the number of samples in the training data set. This means that it is highly efficient to construct a classifier with the RVKDE algorithm, in comparison with the SVM algorithm. Furthermore, the RVKDE based classifier is able to carry out data classification with more than two classes of samples in one single run. In other words, it does not need to invoke mechanisms such as one-against-one or one-against-all for handling data sets with more than two classes of samples. The successful experiences with the RVKDE algorithm in data classification applications then motivate the study presented next in this thesis. In Section 4.3, a RVKDE based data reduction approach for expediting the model selection process of the SVM is described. Experimental results show that, in comparison with the existing approaches, the data reduction based approach proposed in this thesis is able to expedite the model selection process by a larger degree and cause a smaller degradation of prediction accuracy. | en |
dc.description.provenance | Made available in DSpace on 2021-06-13T16:31:18Z (GMT). No. of bitstreams: 1 ntu-94-F89922043-1.pdf: 680989 bytes, checksum: e02d1c12cd6b98d4cb6971df524695f7 (MD5) Previous issue date: 2005 | en |
dc.description.tableofcontents | ABSTRACT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . i
LIST OF FIGURES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iv LIST OF TABLES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . v CHAPTER I. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.2 Machine Learning Algorithms for RBFN . . . . . . . . . . . . 3 1.3 The Organization of This Thesis . . . . . . . . . . . . . . . . 4 II. An Overview of Radial Basis Function Networks . . . . . . . 7 2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 2.2 Exact Interpolation . . . . . . . . . . . . . . . . . . . . . . . 9 2.3 Function Approximation . . . . . . . . . . . . . . . . . . . . . 10 2.4 Two-stage Training . . . . . . . . . . . . . . . . . . . . . . . 11 III. QuickRBF and Related Works . . . . . . . . . . . . . . . . . . . 12 3.1 An Efficient Regularization Mechanism for Radial Basis Function Networks . . . . . . . . . . . . . . . . . . . . . . . . . . 12 3.1.1 Traditional Least Mean Square Error Method . . . . 14 3.1.2 Proposed Least Mean Square Error Method with Statistics Techniques . . . . . . . . . . . . . . . . . 15 3.2 Construction of Radial Basis Function Networks with an Incremental Clustering Algorithm . . . . . . . . . . . . . . . . . 18 3.2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . 18 3.2.2 Determining the Centers . . . . . . . . . . . . . . . 21 3.2.3 Calculation of the Bandwidths . . . . . . . . . . . . 24 3.2.4 Calculation of theWeights . . . . . . . . . . . . . . 24 3.2.5 Experiments on Data Classification Data Sets . . . 25 3.2.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . 28 3.3 QuickRBF : A Radial Basis Function Network based Data Classification Package . . . . . . . . . . . . . . . . . . . . . . 29 3.3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . 30 3.3.2 Design and Implementation of the Learning Algorithm 30 3.3.3 Experiments on Data Classification Benchmark Data Sets . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 3.3.4 Experiments on Protein Secondary Structure Prediction Data Sets . . . . . . . . . . . . . . . . . . . 35 IV. The Relaxed Variable Kernel Density Estimation Algorithm and Its Applications . . . . . . . . . . . . . . . . . . . . . . . . . 38 4.1 The Relaxed Variable Kernel Density Estimation Algorithm . 38 4.1.1 Development of the Relaxed Variable Kernel Density Estimation Algorithm . . . . . . . . . . . . . . . . . 38 4.1.2 Properties of the Relaxed Variable Kernel Density Estimation . . . . . . . . . . . . . . . . . . . . . . . 44 4.2 A RVKDE based Data Classifier . . . . . . . . . . . . . . . . 46 4.2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . 46 4.2.2 RelatedWorks . . . . . . . . . . . . . . . . . . . . . 49 4.2.3 Overview of Data Classification with the RVKDE based Algorithm . . . . . . . . . . . . . . . . . . . . 51 4.2.4 Implementation Issues and Analysis of Time Complexity . . . . . . . . . . . . . . . . . . . . . . . . . 54 4.2.5 Experimental Results and Discussions . . . . . . . . 57 4.3 Expediting SVM Model Selection Process with the RVKDE Alogorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64 4.3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . 64 4.3.2 Model Selection for Support Vector Machines . . . . 67 4.3.3 The RVKDE Based Data Reduction Mechanism . . 68 4.3.4 Experimental Results . . . . . . . . . . . . . . . . . 71 V. Discussions and Conclusions . . . . . . . . . . . . . . . . . . . . 74 BIBLIOGRAPHY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77 | |
dc.language.iso | en | |
dc.title | 以RBF類神經網路為基礎之機器學習演算法研究 | zh_TW |
dc.title | A Study on Machine Learning with Radial Basis Function Networks | en |
dc.type | Thesis | |
dc.date.schoolyear | 93-2 | |
dc.description.degree | 博士 | |
dc.contributor.oralexamcommittee | 高成炎,趙坤茂,黃鎮剛,黃明經,洪炯宗 | |
dc.subject.keyword | 機器學習,分類演算法,生物資訊, | zh_TW |
dc.subject.keyword | Machine Learning,Data Classification,RBF,RBFN,bioinformatics, | en |
dc.relation.page | 82 | |
dc.rights.note | 有償授權 | |
dc.date.accepted | 2005-07-12 | |
dc.contributor.author-college | 電機資訊學院 | zh_TW |
dc.contributor.author-dept | 資訊工程學研究所 | zh_TW |
顯示於系所單位: | 資訊工程學系 |
文件中的檔案:
檔案 | 大小 | 格式 | |
---|---|---|---|
ntu-94-1.pdf 目前未授權公開取用 | 665.03 kB | Adobe PDF |
系統中的文件,除了特別指名其著作權條款之外,均受到著作權保護,並且保留所有的權利。