Subword Encoding in Lattice
Published 1 January 2019Open access
Jie Yang, Yue Zhang, Shuailong Liang
Citations59
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
This work investigates subword information for Chinese word segmentation, by integrating sub word embeddings trained using byte-pair encoding into a Lattice LSTM (LaLSTM) network over a character sequence.
Abstract
Jie Yang, Yue Zhang, Shuailong Liang. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). 2019.
Keywords
Computer Science
Dropout: a simple way to prevent neural networks from overfitting
34,279 Citations2014Nitish Srivastava, Geoffrey E. Hinton +3 more
It is shown that dropout improves the performance of neural networks on supervised learning tasks in vision, speech recognition, document classification and computational biology, obtaining state-of-the-art results on many benchmark data sets.
ScholarlyCommons (University of Pennsylvania)Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data
12,978 Citations2001John Lafferty, Andrew McCallum +1 more
This work presents iterative parameter estimation algorithms for conditional random fields and compares the performance of the resulting models to HMMs and MEMMs on synthetic and natural-language data.
arXiv (Cornell University)Efficient Estimation of Word Representations in Vector Space
11,710 Citations2013Tomáš Mikolov, Kai Chen +2 more
IEEE Transactions on Information TheoryError bounds for convolutional codes and an asymptotically optimum decoding algorithm
6,705 Citations1967Andrew J. Viterbi
The upper bound is obtained for a specific probabilistic nonsequential decoding algorithm which is shown to be asymptotically optimum for rates above R_{0} and whose performance bears certain similarities to that of sequential decoding algorithms.
Communications of the ACMTrie memory
1,081 Citations1960Edward Fredkin
In this paper several paradigms of trie memory are described and compared with other memory paradigm, their advantages and disadvantages are examined in detail, and applications are discussed.
Chinese NER Using Lattice LSTM
849 Citations2018Yue Zhang, Jie Yang
A lattice-structured LSTM model for Chinese NER, which encodes a sequence of input characters as well as all potential words that match a lexicon, is investigated, which outperforms both word-based and character-based L STM baselines.
Chinese segmentation and new word detection using conditional random fields
468 Citations2004Fuchun Peng, Fangfang Feng +1 more
The ability of linear-chain conditional random fields (CRFs) to perform robust and accurate Chinese word segmentation by providing a principled framework that easily supports the integration of domain knowledge in the form of multiple lexicons of characters and words is demonstrated.
Chinese Word Segmentation as Character Tagging
376 Citations2003Nianwen Xue
Preliminary results show that this approach is competitive against other supervised machine-learning segmenters reported in previous studies, achieving precision and recall rates of 95.01% and 94.94% respectively, trained on a 237K-word training set.
The Second International Chinese Word Segmentation Bakeoff.
308 Citations2005Thomas E. Emerson
The J-splint is a reliable, simple, and rapidly applied splint that prevents many of the complications and downfalls of other described temporizing measures and helps to provide excellent pain management in the acute setting.
Deep Learning for Chinese Word Segmentation and POS Tagging
295 Citations2013Xiaoqing Zheng, Hanyang Chen +1 more
This study explores the feasibility of performing Chinese word segmentation and POS tagging by deep learning, and describes a perceptron-style algorithm for training the neural networks, as an alternative to maximum-likelihood method to speed up the training process and make the learning algorithm easier to be implemented.
Long Short-Term Memory Neural Networks for Chinese Word Segmentation
295 Citations2015Xinchi Chen, Xipeng Qiu +3 more
A novel neural network model for Chinese word segmentation is proposed, which adopts the long short-term memory (LSTM) neural network to keep the previous important information in memory cell and avoids the limit of window size of local context.
NCRF++: An Open-source Neural Sequence Labeling Toolkit
186 Citations2018Jie Yang, Yue Zhang
NCRF++ is designed for quick implementation of different neural sequence labeling models with a CRF inference layer, and provides users with an inference for building the custom model structure through configuration file with flexible neural feature design and utilization.
Max-Margin Tensor Neural Network for Chinese Word Segmentation
176 Citations2014Wenzhe Pei, Tao Ge +1 more
A novel neural network model for Chinese word segmentation called Max-Margin Tensor Neural Network (MMTNN), which has the ability to model complicated interactions between tags and context characters and a new tensor factorization approach is proposed to speed up the model and avoid overfitting.
Neural Word Segmentation Learning for Chinese
150 Citations2016Deng Cai, Hai Zhao
A novel neural framework is proposed which thoroughly eliminates context windows and can utilize complete segmentation history and employs a gated combination neural network over characters to produce distributed representations of word candidates, which are then given to a long short-term memory (LSTM) language scoring model.
Chinese Segmentation with a Word-Based Perceptron Algorithm
132 Citations2007Yue Zhang, Stephen Clark
Open tests on the first and second SIGHAN bakeoffs show that the proposed word-based segmentor is competitive with the best in the literature, achieving the highest reported F-scores for a number of corpora.
Neural Word Segmentation with Rich Pretraining
126 Citations2017Jie Yang, Yue Zhang +1 more
This work investigates the effectiveness of a range of external training sources for neural word segmentation by building a modular segmentation model, pretraining the most important submodule using rich external sources and shows that such pretraining significantly improves the model, leading to accuracies competitive to the best methods on six benchmarks.
Gated Recursive Neural Network for Chinese Word Segmentation
122 Citations2015Xinchi Chen, Xipeng Qiu +2 more
A gated recursive neural network (GRNN) for Chinese word segmentation is proposed, which contains reset and update gates to incorporate the complicated combinations of the context characters.
Effective Tag Set Selection in Chinese Word Segmentation via Conditional Random Field Modeling
112 Citations2006Hai Zhao, Changning Huang +2 more
It is shown that there is a significant performance difference as different tag sets are selected, and the proposed method gives the state-of-the-art performance.
Fast and Accurate Neural Word Segmentation for Chinese
107 Citations2017Deng Cai, Hai Zhao +4 more
This paper proposes a greedy neural word segmenter with balanced word and character embedding inputs to alleviate the existing drawbacks of current neural models, capable of performing segmentation much faster and even more accurate than state-of-the-art neural models on Chinese benchmark datasets.
Dependency-based Gated Recursive Neural Network for Chinese Word Segmentation
62 Citations2016Jingjing Xu, Xu Sun
A dependency-based gated recursive neural network is proposed that is a competitive model for Chinese word segmentation and first collected by bi-directional long short term memory network, then combined and refined to long distance dependencies via gated recursion neural network.
Subword-based tagging by conditional random fields for Chinese word segmentation
58 Citations2006Ruiqiang Zhang, Genichiro Kikui +1 more
Two approaches to improve Chinese word segmentation are proposed: a subword-based tagging and a confidence measure approach that achieves higher F-scores in CITYU, PKU and MSR corpora than the best results from Sighan Bakeoff 2005.
Word-based and Character-based Word Segmentation Models: Comparison and Combination
54 Citations2010Weiwei Sun
A theoretical and empirical comparative analysis of the two dominant categories of approaches in Chinese word segmentation shows that, in spite of similar performance overall, the two models produce different distribution of segmentation errors, in a way that can be explained by theoretical properties of theTwo models.
Word-Context Character Embeddings for Chinese Word Segmentation
51 Citations2017Hao Zhou, Zhenting Yu +4 more
This work investigates training character embeddings on a word-based context in a similar way, showing that the simple method improves state-of-the-art neural word segmentation models significantly, beating tri-training baselines for leveraging auto-segmented data.
Accurate Linear-Time Chinese Word Segmentation via Embedding Matching
43 Citations2015Jianqiang Ma, Erhard Hinrichs
An embedding matching approach to Chinese word segmentation is proposed, which generalizes the traditional sequence labeling framework and takes advantage of distributed representations and achieves improved results over previous neural network-based word segmenters.
DAG-Structured Long Short-Term Memory for Semantic Compositionality
39 Citations2016Xiaodan Zhu, Parinaz Sobhani +1 more
This paper extends the chain-structured LSTM to directed acyclic graphs (DAGs), with the aim to endow linear-chain LSTMs with the capability of considering compositionality together with non-compositionality in the same semantic composition framework.
Lecture notes in computer scienceOverview of the NLPCC-ICCPOL 2016 Shared Task: Chinese Word Segmentation for Micro-Blog Texts
30 Citations2016Xipeng Qiu, Peng Qian +1 more
An overview for the shared task at the 5th CCF Conference on Natural Language Processing & Chinese Computing (NLPCC 2016): Chinese word segmentation for micro-blog texts and uses a new psychometric-inspired evaluation metric for ChineseWordSeg-Weibo.
arXiv (Cornell University)DAG-based Long Short-Term Memory for Neural Word Segmentation
8 Citations2017Chen Xin-chi, Zhan Shi +2 more
A new neural model to incorporate the word-level information for Chinese word segmentation is proposed, which still adopts the framework of character-based sequence labeling, which has advantages on both effectiveness and efficiency at the inference stage.