SOS4NLP

SOS4NLP: A survey list of surveys for natural language processing.

Mainly Contributed and Maintained by Yuan Zang.

Reading the surveys is an efficient way to learn about an academic field. This repository provides a paperlist of surveys for different areas of natural language processing.

Thanks for all great contributors! Everyone in Github is welcomed to make contribution to this repository.

Contents

• 0. Surveys of Natural Language Processing
• 1. Language Parsing
  • 1.1 Chinese Word Segmentation
  • 1.2 Syntactic Parsing
  • 1.3 Dependency Parsing
  • 1.4 Semantic Parsing
  • 1.5 Part of Speech Tagging
  • 1.6 Word Sense Disambiguation
  • 1.7 Named Entity Recognization
  • 1.8 Coreference Resolution
• 2. Natural Language Understanding and Generation
  • 2.1 Text Classification
  • 2.2 Sentiment Analysis
  • 2.3 Natural Language Inference
  • 2.4 Reading Comprehension
  • 2.5 Text Generation
  • 2.6 Machine Translation
  • 2.7 Text Summarization
• 3. Information Extraction
  • 3.1 Relation Extraction
  • 3.2 Event Extraction
  • 3.3 Open Information Extraction
• 4. Information Retrieval
• 5. Dialogue and Question Answering
  • 5.1 Dialogue
  • 5.2 Question Answering
• 6. Representation Learning
  • 6.1 Representation Learning
  • 6.2 Word Representation Learning
  • 6.3 Network Representation Learning
• 7. Knowledge Graph
  • 7.1 Knowledge Graph
  • 7.2 Commen Sense Knowledge Graph
• 8. Machine Learning for Natural Language Processing
  • 8.1 Deep Learning for Natural Language Processing
  • 8.2 Transformers and Pre-trained Language Models
  • 8.3 Graph Neural Networks
  • 8.4 Reinforcement Learning
  • 8.5 Data Augmentation
  • 8.6 Few and Zero Shot Learning
  • 8.7 Meta Learning
  • 8.8 Continual Learning
  • 8.9 Contrastive Learning
  • 8.10 Multi-Task Learning
  • 8.11 Intepretability and Analysis
  • 8.12 Security Threats and Defense
• 9. Natural language Processing Applications
  • 9.1 Legal Intelligence
  • 9.2 Bioinformatics
  • 9.3 Financial Intelligence
  • 9.4 Recommendation
  • 9.5 Computational Social Science
• Acknowledgements

0. Surveys of Natural Language Processing

1. Neural Network Methods for Natural Language Processing. Yoav Goldberg. SLHLT 2017. [paper]
2. Advances in natural language processing. Julia Hirschberg, Christopher D Manning. Science 2015. [paper]
3. Jumping NLP Curves: A Review of Natural Language Processing Research. Erik Cambria, Bebo White. CIM 2014. [paper]
4. Natural Language Processing: An Introduction. Prakash M Nadkarni, Lucila Ohno-Machado, Wendy W Chapman. JAMIA 2011. [paper]

1. Language Parsing

1.1 Chinese Word Segmentation

1. Chinese Word Segmentation: A Decade Review. Changning Huang, Hai Zhao. JCIP 2007. [paper]

1.2 Syntactic Parsing

1. Syntactic Parsing: A Survey.

   Alton F. Sanders and Ruth H. Sanders. Computers and the Humanities 1989. [paper]

1.3 Dependency Parsing

1. Dependency Parsing. Sandra Kubler, Ryan McDonald, Joakim Nivre. SLHLT 2009. [paper]

1.4 Semantic Parsing

1. A Survey on Semantic Parsing. Aishwarya Kamath, Rajarshi Das. AKBC 2018. [paper]

1.5 Part of Speech Tagging

1. Part‐of‐speech Tagging. Angel R Martinez. WIREs Comp Stats 2012. [paper]

1.6 Word Sense Disambiguation

1. Word Sense Disambiguation: A Survey. Alok Ranjan Pal. arXiv 2015. [paper]

2. Word Sense Disambiguation: A Survey. Roberto Navigli. CSUR 2009. [paper]

1.7 Named Entity Recognization

1. A Survey on Deep Learning for Named Entity Recognition. Jing Li, Aixin Sun, Jianglei Han, Chenliang Li. TKDE 2020. [paper]
2. A Survey of Named Entity Recognition and Classification. David Nadeau, Satoshi Sekine. Lingvisticae Investigationes 2007. [paper]

1.8 Coreference Resolution

1. Coreference Resolution: A Survey. Pradheep Elango. University of Wisconsin, Madison, WI 2005. [paper]

2. Natural Language Understanding and Generation

2.1 Text Classification

1. Text Classification Algorithms: A Survey. Kamran Kowsari, Kiana Jafari Meimandi, Mojtaba Heidarysafa, Sanjana Mendu, Laura Barnes, Donald Brown. Information 2019. [paper]

2. Semantic Text Classification: A Survey of Past and Recent Advances. Berna Altinel, Murat Can Ganiz. IP&M 2018. [paper]

2.2 Sentiment Analysis

1. A Survey of Sentiment Analysis in Social Media. Lin Yue, Weitong Chen, Xue Li, Wanli Zuo, Minghao Yin. KAIS 2019. [paper]

2. Sentiment Analysis Algorithms and Applications: A Survey. Walaa Medhat, Ahmed Hassan, Hoda Korashy. ASEJ 2014. [paper]

2.3 Natural Language Inference

1. Recent Advances in Natural Language Inference: A Survey of Benchmarks, Resources, and Approaches. Shane Storks, Qiaozi Gao, Joyce Y Chai. arXiv 2019. [paper]

2.4 Reading Comprehension

1. A Survey on Machine Reading Comprehension—Tasks, Evaluation Metrics and Benchmark Datasets. Changchang Zeng, Shaobo Li, Qin Li, Jie Hu, Jianjun Hu. AS 2020. [paper]

2. Neural Machine Reading Comprehension: Methods and Trends. Shanshan Liu, Xin Zhang, Sheng Zhang, Hui Wang, Weiming Zhang. AS 2019. [paper]

2.5 Text Generation

1. Pretrained Language Models for Text Generation: A Survey. Junyi Li, Tianyi Tang, Wayne Xin Zhao, Ji-Rong Wen. arXiv 2021. [paper]

2. Survey of the State of the Art in Natural Language Generation: Core Tasks, Applications and Evaluation. Albert Gatt, Emiel Krahmer. JAIR 2018. [paper]

2.6 Machine Translation

1. Neural Machine Translation: A Review of Methods, Resources, and Tools. Zhixing Tan, Shuo Wang, Zonghan Yang, Gang Chen, Xuancheng Huang, Maosong Sun, YangLiu. AI Open 2020. [paper]

2.7 Text Summarization

1. A Survey on Dialogue Summarization: Recent Advances and New Frontiers. Xiachong Feng, Xiaocheng Feng, Bing Qin. arXiv 2021. [paper]
2. The Factual Inconsistency Problem in Abstractive Text Summarization: A Survey. Yichong Huang, Xiachong Feng, Xiaocheng Feng, Bing Qin. arXiv 2021. [paper]
3. What Have We Achieved on Text Summarization? Dandan Huang, Leyang Cui, Sen Yang, Guangsheng Bao, Kun Wang, Jun Xie, Yue Zhang. EMNLP 2020. [paper]
4. Recent Automatic Text Summarization Techniques: A Survey. Mahak Gambhir, Vishal Gupta. AIR 2017. [paper]

3. Information Extraction

3.1 Relation Extraction

1. More Data, More Relations, More Context and More Openness: A Review and Outlook for Relation Extraction. Xu Han, Tianyu Gao, Yankai Lin, Hao Peng, Yaoliang Yang, Chaojun Xiao, Zhiyuan Liu, Peng Li, Maosong Sun, Jie Zhou. AACL 2020. [paper]

2. Relation Extraction: A Survey. Sachin Pawar, Girish K. Palshikara, Pushpak Bhattacharyyab. arXiv 2017. [paper]

3.2 Event Extraction

2. Extracting Events and Their Relations from Texts: A Survey on Recent Research Progress and Challenges. Kang Liu, Yubo Chen, Jian Liu, Xinyu Zuo, Jun Zhao. AI Open 2020. [paper]

3.3 Open Information Extraction

1. A Survey on Open Information Extraction. Christina Niklaus, Matthias Cetto, Andre Freitas, Siegfried Handschuh. COLING 2018. [paper]

4. Information Retrieval

1. Pretrained Transformers for Text Ranking: BERT and Beyond. Andrew Yates, Rodrigo Nogueira, Jimmy Lin. WSDM 2021. [paper]
2. Data Mining and Information Retrieval in the 21st Century: A Bibliographic Review. Jiaying Liu, Xiangjie, Kong, Xinyu Zhou, Lei Wang, Da Zhang, Ivan Lee, Bo Xu, Feng Xia. Science Review 2019. [paper]
3. Deep Learning for Matching in Search and Recommendation. Jun Xu, Xiangnan He, Hang Li. SIGIR tutorial 2018. [paper]
4. Neural Models for Information Retrieval. Bhaskar Mitra, Nick Craswell. arXiv 2017. [paper]

5. Dialogue and Question Answering

5.1 Dialogue

1. A Survey on Dialogue Systems: Recent Advances and New Frontiers. Hongshen Chen, Xiaorui Liu, Dawei Yin, Jiliang Tang. Acm SIGKDD Explorations Newsletter 2017. [paper]

5.2 Question Answering

1. Retrieving and Reading: A Comprehensive Survey on Open-domain Question Answering. Fengbin Zhu, Wenqiang Lei, Chao Wang, Jianming Zheng, Soujanya Poria, Tat-Seng Chua. arXiv 2021. [paper]
2. Core Techniques of Question Answering Systems over Knowledge Bases: A Survey. Dennis Diefenbach, Vanessa Lopez, Kamal Singh, Pierre Maret. KAIS 2018. [paper]
3. Question Answering Systems: Survey and Trends. Abdelghani Bouziane, Djelloul Bouchiha, Noureddine Doumi, Mimoun Malki. Procedia Computer Science 2015. [paper]

6. Representation Learning

6.1 Representation Learning

1. Representation Learning: A Review and New Perspectives. Yoshua Bengio, Aaron Courville, and Pascal Vincent. TPAMI 2013. [paper]

6.2 Word Representation Learning

1. From Word to Sense Embeddings: A Survey on Vector Representations of Meaning. Jose Camacho-Collados, Mohammad Taher Pilehvar. JAIR 2018. [paper]

6.3 Network Representation Learning

1. Network Representation Learning: A Macro and Micro View. Xueyi Liu, Jie Tang. AI Open 2021. [paper]
2. A Survey on Network Embedding. Peng Cui, Xiao Wang, Jian Pei, Wenwu Zhu. TKDE 2018. [paper]
3. Network Representation Learning: A Survey. Daokun Zhang, Jie Yin, Xingquan Zhu, Chengqi Zhang. TBD 2018. [paper]
4. Network Representation Learning: An Overview. Cunchao Tu, Cheng Yang, Zhiyuan Liu, Maosong Sun. SSI 2017. [paper]

7. Knowledge Graph

7.1 Knowledge Graph

1. Neural, Symbolic and Neural-symbolic Reasoning on Knowledge Graphs.

Jing Zhang, Bo Chen, Lingxi Zhang, Xirui Ke, Haipeng Ding. AI Open 2021. [paper]

1. Knowledge Graph Embedding: A Survey of Approaches and Applications. Quan Wang, Zhendong Mao, Bin Wang, Li Guo. TKDE 2017. [paper]

2. Knowledge Graph Refinement: A Survey of Approaches and Evaluation Methods. Heiko Paulheim. Semantic Web 2017. [paper]

3. Knowledge Representation Learning: A Review. Zhiyuan Liu, Maosong Sun, Yankai Lin, Ruobing Xie. JCRD 2016. [paper]

4. A Review of Relational Machine Learning for Knowledge Graphs. Maximilian Nickel, Kevin Murphy, Volker Tresp, Evgeniy Gabrilovich. Proceedings of the IEEE 2015. [paper]

7.2 Common Sense Knowledge Graph

1. Sememe Knowledge Computation: A Review of Recent Advances in Application and Expansion of Sememe Knowledge Bases. Fanchao Qi, Ruobing Xie, Yuan Zang, Zhiyuan Liu, Maosong Sun. FCS 2021. [paper]

8. Machine Learning for Natural Language Processing

8.1 Deep Learning for Natural Language Processing

1. A Survey of the Usages of Deep Learning for Natural Language Processing. Daniel W. Otter, Julian R. Medina, Jugal K. Kalita. TNNLS 2021. [paper]

2. Recent Trends in Deep Learning Based Natural Language Processing. Tom Young, Devamanyu Hazarika, Soujanya Poria, Erik Cambria. CIM 2018. [paper]

8.2 Transformers and Pre-trained Language Models

1. A Survey of Transformers. Tianyang Lin, Yuxin Wang, Xiangyang Liu, Xipeng Qiu. arXiv 2021. [paper]

2. Pre-Trained Models: Past, Present and Future. Xu Han, Zhengyan Zhang, Ning Ding, Yuxian Gu, Xiao Liu, Yuqi Huo, Jiezhong Qiu, Liang Zhang, Wentao Han, Minlie Huang, Qin Jin, Yanyan Lan, Yang Liu, Zhiyuan Liu, Zhiwu Lu, Xipeng Qiu, Ruihua Song, Jie Tang, Ji-Rong Wen, Jinhui Yuan, Wayne Xin Zhao, Jun Zhu. arXiv 2021. [paper]

3. Pre-trained models for natural language processing: A survey. XiPeng Qiu, TianXiang Sun, YiGe Xu, YunFan Shao, Ning Dai, XuanJing Huang. Science China Technological Sciences 2020. [paper]

4. Efficient Transformers: A Survey. Yi Tay, Mostafa Dehghani, Dara Bahri, Donald Metzler. arXiv 2020. [paper]

8.3 Graph Neural Networks

1. Graph Neural Networks for Natural Language Processing: A Survey. Lingfei Wu, Yu Chen, Kai Shen, Xiaojie Guo, Hanning Gao, Shucheng Li, Jian Pei, Bo Long. arXiv 2021. [paper]

2. Robustness of Deep Learning Models on Graphs: A Survey. Jiarong Xu, Junru Chen, Siqi You, Zhiqing Xiao, Yang Yang, Jiangang Lu. AI Open 2021. [paper]

3. Graph Neural Networks: A Review of Methods and Applications. Jie Zhou, Ganqu Cui, Shengding Hu, Zhengyan Zhang, Cheng Yang, Zhiyuan Liu, Lifeng Wang, Changcheng Li, Maosong Sun. AI Open 2020. [paper]

4. A Comprehensive Survey on Graph Neural Networks. Zonghan Wu, Shirui Pan, Fengwen Chen, Guodong Long, Chengqi Zhang, Philip S. Yu. TNNLS 2020. [paper]

8.4 Reinforcement Learning

1. A Survey of Reinforcement Learning Informed by Natural Language. Jelena Luketina, Nantas Nardelli, Gregory Farquhar, Jakob Foerster, Jacob Andreas, Edward Grefenstette, Shimon Whiteson, Tim Rocktäschel. IJCAI 2019. [paper]

8.5 Data Augmentation

2. A Survey of Data Augmentation Approaches for NLP. Steven Y. Feng, Varun Gangal, Jason Wei, Sarath Chandar, Soroush Vosoughi, Teruko Mitamura, Eduard Hovy. ACL Findings 2021. [paper]

3. An Empirical Survey of Data Augmentation for Limited Data Learning in NLP. Jiaao Chen, Derek Tam, Colin Raffel, Mohit Bansal, Diyi Yang. arXiv 2021. [paper]

8.6 Few and Zero Shot Learning

1. A Survey on Recent Approaches for Natural Language Processing in Low-Resource Scenarios. Michael A. Hedderich, Lukas Lange, Heike Adel, Jannik Strötgen, Dietrich Klakow. NAACL 2021. [paper]

2. A Survey of Zero-Shot Learning: Settings, Methods, and Applications. Wei Wang, Vincent W. Zheng, Han Yu, Chunyan Miao. ACM TIST 2019. [paper]

8.7 Meta Learning

1. Meta-learning in Neural Networks: A Survey. Timothy Hospedales, Antreas Antoniou, Paul Micaelli, Amos Storkey. PAMI 2020. [paper]
2. A Survey of Deep Meta‐learning. Mike Huisman, Jan N. van Rijn, Aske Plaat. AIR 2021. [paper]

8.8 Continual Learning

1. A Continual Learning Survey: Defying Forgetting in Classification Tasks. Matthias Delange, Rahaf Aljundi, Marc Masana, Sarah Parisot, Xu Jia, Ales Leonardis, Greg Slabaugh, Tinne Tuytelaars. PAMI 2021. [paper]

8.9 Contrastive Learning

1. A Survey on Contrastive Self-Supervised Learning. Ashish Jaiswal, Ashwin Ramesh Babu, Mohammad Zaki Zadeh, Debapriya Banerjee, Fillia Makedon. Technologies 2021. [paper]

8.10 Multi-Task Learning

1. Multi-task learning for natural language processing in the 2020s: Where are we going? Joseph Worsham, Jugal Kalita. Pattern Recognition Letters 2020. [paper]

2. An Overview of Multi-Task Learning in Deep Neural Networks. Sebastian Ruder. arXiv 2017. [paper]

8.11 Intepretability and Analysis

1. On Interpretability of Artificial Neural Networks: A Survey. Feng-Lei Fan, Jinjun Xiong, Mengzhou Li, Ge Wang. TRPMS 2021. [paper]

2. A Survey of the State of Explainable AI for Natural Language Processing. Marina Danilevsky, Kun Qian, Ranit Aharonov, Yannis Katsis, Ban Kawas, Prithviraj Sen. AACL 2020. [paper].

3. Machine Learning Interpretability: A Survey on Methods and Metrics. Diogo V. Carvalho, Eduardo M. Pereira, Jaime S. Cardoso. Electronics 2019. [paper]

4. Analysis Methods in Neural Language Processing: A Survey. Yonatan Belinkov, James Glass. TACL 2019. [paper]

5. Teach Me to Explain: A Review of Datasets for Explainable NLP. Sarah Wiegreffe, Ana Marasović. arXiv 2021. [paper]

8.12 Security Threats and Defense

1. Adversarial Attacks on Deep Learning Models in Natural Language Processing: A Survey. Wei Emma Zhang, Quan Z. Sheng, Ahoud Alhazmi, Chenliang Li. ACM TIST 2020. [paper]

2. Backdoor Learning: A Survey. Yiming Li, Baoyuan Wu, Yong Jiang, Zhifeng Li, Shu-Tao Xia. arXiv 2020. [paper].

3. A Survey of Privacy Attacks in Machine Learning. Maria Rigaki, Sebastian Garcia. arXiv 2020. [paper]

9. Natural language Processing Applications

9.1 Legal Intelligence

1. How Does NLP Benefit Legal System: A Summary of Legal Artificial Intelligence. Haoxi Zhong, Chaojun Xiao, Cunchao Tu, Tianyang Zhang, Zhiyuan Liu, Maosong Sun. ACL 2020. [paper]

9.2 Bioinformatics

1. Survey of Natural Language Processing Techniques in Bioinformatics. Zhiqiang Zeng, Hua Shi, Yun Wu, Zhiling Hong. CMMM 2015. [paper].

9.3 Financial Intelligence

1. Natural Language Based Financial Forecasting: A Survey. Frank Z. Xing, Erik Cambria, Roy E. Welsch. AIR 2018. [paper].

9.4 Recommendation

1. Knowledge Transfer via Pre-training for Recommendation: A Review and Prospect. Zheni Zeng, Chaojun Xiao, Yuan Yao, Ruobing Xie, Zhiyuan Liu, Fen Lin, Leyu Lin, Maosong Sun. Frontiers in Big Data 2021. [paper].

9.5 Computational Social Science

1. From Symbols to Embeddings: A Tale of Two Representations in Computational Social Science. Huimin Chen, Cheng Yang, Xuanming Zhang, Zhiyuan Liu, Maosong Sun, Jianbin Jin. arXiv 2021. [paper].

Acknowledgements

Great thanks to other contributor Shengding Hu, Chenglei Si and Han-Gil Kim! (names are not listed in particular order)

Please contact us if we miss your names in this list, we will add you back ASAP!