Question classification in Persian language based on conditional random fields
Published 1 October 2012
Ali Mollaei, Saeed Rahati, Azam Estaji
Citations7
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
This study is designed to present the architecture of question classification (QC) in Persian based on the Conditional Random Fields (CRF) machine learning model and evaluate effects of various features on its accuracy.
Abstract
The question classification system is one of the important subsystems in the Question Answering Systems (QAS). In such systems through retrieval methods and information extraction the texts are retrieved in order to get to a correct answer. The current study is designed to present the architecture of question classification (QC) in Persian based on the Conditional Random Fields (CRF) machine learning model and evaluate effects of various features on its accuracy. In this study, sentences were classified into two levels of coarse and fine classes based on the type of the answer to each question. After extracting features and setting sliding window on the CRF model, CRF question classifier (QC) is train. Then, the QC predicts labels for every token in question. Next, a majority voting on the question classification output, is used to extract a unique label for each question. Further, the effects of different features on the ultimate accuracy of the system were evaluated. Finally results of this question classifier, illustrate a satisfactory accuracy.
Keywords
Computer Science
ScholarlyCommons (University of Pennsylvania)Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data
12,978 Citations2001John Lafferty, Andrew McCallum +1 more
This work presents iterative parameter estimation algorithms for conditional random fields and compares the performance of the resulting models to HMMs and MEMMs on synthetic and natural-language data.
Maximum Entropy Markov Models for Information Extraction and Segmentation
1,333 Citations2000Andrew McCallum, Dayne Freitag +1 more
A new Markovian sequence model is presented that allows observations to be represented as arbitrary overlapping features (such as word, capitalization, formatting, part-of-speech), and defines the conditional probability of state sequences given observation sequences.
Learning question classifiers
1,276 Citations2002Xin Li, Dan Roth
A hierarchical classifier is learned that is guided by a layered semantic hierarchy of answer types, and eventually classifies questions into fine-grained classes.
Shallow parsing with conditional random fields
1,251 Citations2003Fei Sha, Fernando Pereira
This work shows how to train a conditional random field to achieve performance as good as any reported base noun-phrase chunking method on the CoNLL task, and better than any reported single model.
Semi-Markov Conditional Random Fields for Information Extraction
617 Citations2004Sunita Sarawagi, William W. Cohen
Intuitively, a semi-CRF on an input sequence x outputs a "segmentation" of x, in which labels are assigned to segments rather than to individual elements of xi, and transitions within a segment can be non-Markovian.
Information RetrievalAnalysis of Statistical Question Classification for Fact-Based Questions
134 Citations2005Donald Metzler, W. Bruce Croft
It is empirically show that a statistical approach to question classification is robust and achieves good performance on three diverse data sets with little or no hand tuning and it is found that semantic features tend to increase performance more than purely syntactic features.
Question classification for E-learning by artificial neural network
45 Citations2004Ting Fei, Heng Wei +2 more
This paper proposes a text categorization model using an artificial neural network trained by the backpropagation learning algorithm as the text classifier and shows that the system achieved the performance in terms of F/sub 1/ value of nearly 78%.
Classifying What-Type Questions by Head Noun Tagging
36 Citations2008Fangtao Li, Xian Zhang +2 more
2010 Seventh International Conference on Fuzzy Systems and Knowledge DiscoveryA Chinese question-answering system with question classification and answer clustering
18 Citations2010Kepei Zhang, Jieyu Zhao
An approach for Chinese question analysis and answer extraction that contains keyword extraction and question classification is proposed and the answer extraction is converted into a clustering problem.
An Integrated Approach for Question Classification in Chinese Cuisine Question Answering System
10 Citations2008Ling Xia, Zhi Teng +1 more
An integrated method on question classification of Chinese Sichuan cuisine QA system is presented and SVM classifier is used for secondary classification to the questions which cannot be matched with rules.
An Effective Feature-Weighting Model for Question Classification
9 Citations2007Peng Huang, Jiajun Bu +2 more
A feature-weighting model which assigns different weights to features instead of simple binary feature vectors is proposed, which shows that with this new feature- Weighting model the SVM-based classifier outperforms the one with- out it to some extent.
NTCIRChinese-Chinese and English-Chinese Question Answering with ASQA at NTCIR-6 CLQA
8 Citations2007Cheng‐Wei Lee, Min-Yuh Day +7 more
For NTCIR-6 CLQA, the question answering system ASQA (Academia Sinica Question Answering System) was improved so that it could deal with the Chinese-Chinese (C-C) subtask and the English- Chinese (E-C), and an English question classifier was built that adopts Question Informer as a key classification feature.
Improving the Accuracy of Question Classification with Machine Learning
7 Citations2007Tri Nguyên, Le-Minh Nguyen +1 more
This paper proposes to use semi-supervised learning to consume unlabeled questions with expectation to improve the performance of classifiers in the hierarchy, and proposes to automatically group question classes by clustering in order to expand a node which has a large number of classes in the question taxonomy.
Chinese Question Classification Based on Semantic Gram and SVM
6 Citations2009Liang Wang, Hui Zhang +2 more
This paper uses SVM as the classifier and proposes a new feature extraction method of Chinese questions which is called semantic gram extraction which is proposed based on the word semantics and N-gram.
Chinese Question Classification with Support Vector Machine
3 Citations2006Bo Liu, Zhifeng Hao +2 more
This paper performs the One-against-All Algorithm with support vector machine for Chinese question classification problems and shows the excellent general performance of the Algorithm.
Research on Chinese Question Classification Based on Hownet and Dependency Parsing
2 Citations2011Wei Zhang, Junjie Chen +1 more
A new method on feature extraction for question classification using HowNet and dependency parsing is presented, which shows that the model proposed in the paper is effective in improving question classification accuracy.
International Conference on Information Science and TechnologyChinese question answering system based on question classification and similarity
1 Citations2011Dongwei Guo, Jia Chenhui +2 more
The overall structure of Chinese question answering system is introduced, the question classification algorithm based on improved rules and answer extraction algorithm using semantic matching similarity comparison based on dependency trees are presented.
Journal of Anyang Normal UniversityPart-of Speech Tagging Based on Hidden Markov Model
1 Citations2010Xijie Wang
The representation of the Hidden Markov Model and the needs to solve the problem are introduced, and the parameter estimation method of the HMM model is discussed, and research on basic principles of Part-of Speech Tagging using Viterbi algorithm is discussed.