Recommended Reading on Temporal Point Process (TPP)

Mainly Contributed by Chenghao Yang.

Thanks to Hongyuan Mei and Jason Eisner for providing support and helpful comments.

Thanks for all great contributors on GitHub!

Contents

• 0. Toolkits
• 1. Survey Papers
• 2. Modeling Papers
  • 2.1 Temporal Point Process Modeling
  • 2.2 Structured Temporal Point Process Modeling
• 3. Algorithm Papers
  • 3.1 Training Algorithm
  • 3.2 Imputing
  • 3.3 Anomaly Detection
  • 3.4 Clustering
  • 3.5 Data Augmentation
  • 3.6 Denoising
  • 3.7 Querying
• 4. Application Papers
  • 4.1 Social Media
  • 4.2 Clinical Health
  • 4.3 E-commerce
  • 4.4 Causality Discovery
  • 4.5 Audio Processing
  • 4.6 Natural Language Processing
  • 4.7 Computer Vision
  • 4.8 Network Structure Discovery
  • 4.9 Science
  • 4.10 Reinforcement Learning
• 5. Benchmark
• 6. Research Opportunities
• Acknowledgements

0. Toolkits

1. EasyTPP: Towards Open Benchmarking the Temporal Point Processes. Siqiao Xue, Xiaoming Shi, Zhixuan Chu, Yan Wang, Fan Zhou, Hongyan Hao, Caigao Jiang, Chen Pan, Yi Xu, James Y. Zhang, Qingsong Wen, Jun Zhou, Hongyuan Mei. ICLR 2024. [pdf] [code]
2. TPPToolkits: Toolkits for Temporal Point Process. Hongyuan Mei, Chenghao Yang. [website]
3. PoPPy: A Point Process Toolbox Based on PyTorch. Hongteng Xu. arXiv 2018. [website] [pdf]
4. THAP: A Matlab Toolkit for Learning with Hawkes Processes. Hongteng Xu, Hongyuan Zha. arXiv 2017. [website] [pdf]
5. Tick: a Python library for statistical learning, with a particular emphasis on time-dependent modelling. Emmanuel Bacry, Martin Bompaire, Stéphane Gaïffas, Soren Poulsen. arXiv 2017. [website] [pdf]

1. Survey Papers

1. Transformers in Time Series: A Survey. Qingsong Wen, Tian Zhou, Chaoli Zhang, Weiqi Chen, Ziqing Ma, Junchi Yan, Liang Sun. IJCAI 2023. [pdf]
2. Neural Temporal Point Processes: A Review. Oleksandr Shchur, Ali Caner Türkmen, Tim Januschowski, Stephan Günnemann. IJCAI 2021. [pdf]
3. Recent Advance in Temporal Point Process: from Machine Learning Perspective. Junchi Yan. SJTU Technical Report 2019. [pdf]

2. Modeling Papers

2.1 Temporal Point Process Modeling

1. Neural Jump-Diffusion Temporal Point Processes. Shuai Zhang, Chuan Zhou, Yang Aron Liu, Peng Zhang, Xixun Lin, Zhi-Ming Ma. ICML 2024. [paper] [code]
2. Language Models Can Improve Event Prediction by Few-Shot Abductive Reasoning. Xiaoming Shi, Siqiao Xue, Kangrui Wang, Fan Zhou, James Y. Zhang, Jun Zhou, Chenhao Tan, Hongyuan Mei. NeurIPS 2023. [paper] [code]
3. Prompt-augmented Temporal Point Process for Streaming Event Sequence. Siqiao Xue, Yan Wang, Zhixuan Chu, Xiaoming Shi, Caigao Jiang, Hongyan Hao, Gangwei Jiang, Xiaoyun Feng, James Y. Zhang, Jun Zhou. NeurIPS 2023. [paper] [code]
4. Integration-free Training for Spatio-temporal Multimodal Covariate Deep Kernel Point Processes. Yixuan Zhang, Quyu Kong, Feng Zhou. NeurIPS 2023. [paper]
5. Sparse Transformer Hawkes Process for Long Event Sequences. Zhuoqun Li, Mingxuan Sun. ECML-PKDD 2023. [Springer]
6. Intensity-free Convolutional Temporal Point Process: Incorporating Local and Global Event Contexts. Wang-Tao Zhou, Zhao Kang, Ling Tian, Yi Su. Information Sciences 2023. [pdf]
7. Meta Temporal Point Processes. Wonho Bae, Mohamed Osama Ahmed, Frederick Tung, Gabriel L. Oliveira. ICLR 2023. [pdf]
8. HYPRO: A Hybridly Normalized Probabilistic Model for Long-Horizon Prediction of Event Sequences. Siqiao Xue, Xiaoming Shi, James Y Zhang, Hongyuan Mei. NeurIPS 2022. [pdf] [code (iLampard)] [code (alipay)]
9. Exploring Generative Neural Temporal Point Process. Haitao Lin, Lirong Wu, Guojiang Zhao, Pai Liu, Stan Z. Li. TMLR 2022. [pdf] [code]
10. Transformer Embeddings of Irregularly Spaced Events and Their Participants. Chenghao Yang, Hongyuan Mei, Jason Eisner. ICLR 2022. [pdf] [code]
11. Long Horizon Forecasting with Temporal Point Processes. Prathamesh Deshpande, Kamlesh Marathe, Abir De, Sunita Sarawagi. WSDM 2021. [pdf]
12. Deep Fourier Kernel for Self-Attentive Point Processes. Shixiang Zhu, Minghe Zhang, Ruyi Ding, Yao Xie. AISTATS 2021. [pdf]
13. Neural Spatio-Temporal Point Processes. Ricky T. Q. Chen, Brandon Amos, Maximilian Nickel. ICLR 2021. [pdf] [code]
14. Transformer Hawkes Process. Simiao Zuo, Haoming Jiang, Zichong Li, Tuo Zhao, Hongyuan Zha. ICML 2020. [pdf] [code]
15. Self-Attentive Hawkes Process. Qiang Zhang, Aldo Lipani, Omer Kirnap, Emine Yilmaz. ICML 2020. [pdf] [code]
16. Intensity-Free Learning of Temporal Point Processes. Oleksandr Shchur, Marin Biloš, Stephan Günnemann. ICLR 2020. [pdf] [code]
17. Fast and Flexible Temporal Point Processes with Triangular Maps. Oleksandr Shchur, Nicholas Gao, Marin Biloš, Stephan Günnemann. NeurIPS 2020. [pdf] [code + data]
18. Uncertainty on Asynchronous Time Event Prediction. Marin Biloš, Bertrand Charpentier, Stephan Günnemann. NeurIPS 2019. [pdf] [code]
19. Latent ODEs for Irregularly-Sampled Time Series. Yulia Rubanova, Ricky T. Q. Chen, David Duvenaud. NeurIPS 2019. [pdf] [code]
20. Neural Jump Stochastic Differential Equations. Junteng Jia, Austin R. Benson. NeurIPS 2019. [pdf] [code]
21. Fully Neural Network based Model for General Temporal Point Processes. Takahiro Omi, Naonori Ueda, Kazuyuki Aihara. NeurIPS 2019. [pdf] [code]
22. Deep Reinforcement Learning of Marked Temporal Point Processes. Utkarsh Upadhyay, Abir De, Manuel Gomez-Rodriguez. NeurIPS 2018. [pdf] [code]
23. Learning Conditional Generative Models for Temporal Point Processes. Shuai Xiao, Hongteng Xu, Junchi Yan, Mehrdad Farajtabar, Xiaokang Yang, Le Song, Hongyuan Zha. AAAI 2018. [pdf]
24. The Neural Hawkes Process: A Neurally Self-Modulating Multivariate Point Process. Hongyuan Mei, Jason Eisner. NeurIPS 2017. [pdf] [code] [spotlight]
25. Wasserstein Learning of Deep Generative Point Process Models. Shuai Xiao, Mehrdad Farajtabar, Xiaojing Ye, Junchi Yan, Le Song, Hongyuan Zha. NeurIPS 2017. [pdf] [code]
26. Cascade Dynamics Modeling with Attention-based Recurrent Neural Network. Yongqing Wang, Huawei Shen, Shenghua Liu, Jinhua Gao, Xueqi Cheng. IJCAI 2017. [pdf]
27. Modeling The Intensity Function Of Point Process Via Recurrent Neural Networks. Shuai Xiao, Junchi Yan, Stephen M. Chu, Xiaokang Yang, Hongyuan Zha. AAAI 2017. [pdf] [code]
28. Recurrent Marked Temporal Point Processes: Embedding Event History to Vector. Nan Du, Hanjun Dai, Rakshit Trivedi, Utkarsh Upadhyay, Manuel Gomez-Rodriguez, and Le Song. KDD 2016. [pdf] [code]
29. Isotonic Hawkes Processes. Yichen Wang, Bo Xie, Nan Du, Le Song. ICML 2016. [pdf]
30. Hawkes Processes with Stochastic Excitations. Young Lee, Kar Wai Lim, Cheng Soon Ong. ICML 2016. [pdf]
31. Learning Triggering Kernels for Multi-dimensional Hawkes Processes. Ke Zhou, Hongyuan Zha, and Le Song. ICML 2013. [pdf]

2.2 Structured Temporal Point Process Modeling

1. Neuro-Symbolic Temporal Point Processes. Yang Yang, Chao Yang, Boyang Li, Yinghao Fu, Shuang Li. ICML 2024. [pdf]
2. A Variational Autoencoder for Neural Temporal Point Processes with Dynamic Latent Graphs. Sikun Yang, Hongyuan Zha. AAAI 2024. [paper]
3. Transformer Embeddings of Irregularly Spaced Events and Their Participants. Chenghao Yang, Hongyuan Mei, Jason Eisner. ICLR 2022. [pdf] [code] (cross-listed as this paper covers both structured and unstructured TPP modeling. )
4. Learning Neural Point Processes with Latent Graphs. Qiang Zhang, Aldo Lipani, and Emine Yilmaz. WWW 2021. [pdf]
5. User-Dependent Neural Sequence Models for Continuous-Time Event Data. Alex Boyd, Robert Bamler, Stephan Mandt, Padhraic Smyth. NeurIPS 2020. [pdf] [code]
6. Neural Datalog Through Time: Informed Temporal Modeling via Logical Specification. Hongyuan Mei, Guanghui Qin, Minjie Xu, Jason Eisner. ICML 2020. [pdf] [code] [slides]
7. Temporal Logic Point Processes. Shuang Li, Lu Wang, Ruizhi Zhang, Xiaofu Chang, Xuqin Liu, Yao Xie, Yuan Qi, Le Song. ICML 2020. [pdf] [slideslive]
8. DyRep: Learning Representations over Dynamic Graphs. Rakshit Trivedi, Mehrdad Farajtabar, Prasenjeet Biswal, Hongyuan Zha. ICLR 2019. [pdf] [code]
9. Deep Mixture Point Processes: Spatio-temporal Event Prediction with Rich Contextual Information. Maya Okawa, Tomoharu Iwata, Takeshi Kurashima, Yusuke Tanaka, Hiroyuki Toda, Naonori Ueda. KDD 2019. [pdf]
10. Know-Evolve: Deep Temporal Reasoning for Dynamic Knowledge Graphs. Rakshit Trivedi, Hanjun Dai, Yichen Wang, Le Song. ICML 2017. [pdf]

3. Algorithm Papers

3.1 Training Algorithm

1. Learning Multivariate Temporal Point Processes via the Time-Change Theorem. Guilherme Augusto Zagatti, See Kiong Ng, Stéphane Bressan. AISTATS 2024. [pdf] [code]
2. SMURF-THP: Score Matching-based UnceRtainty quantiFication for Transformer Hawkes Process. Zichong Li, Yanbo Xu, Simiao Zuo, Haoming Jiang, Chao Zhang, Tuo Zhao, Hongyuan Zha. ICML 2023. [pdf] [code]
3. Noise-Contrastive Estimation for Multivariate Point Processes. Hongyuan Mei, Tom Wan, Jason Eisner. NeurIPS 2020. [pdf] [code] [slides]
4. Learning Temporal Point Processes via Reinforcement Learning. Shuang Li, Shuai Xiao, Shixiang Zhu, Nan Du, Yao Xie, Le Song. NeurIPS 2018. [pdf]
5. INITIATOR: Noise-contrastive Estimation for Marked Temporal Point Process. Ruocheng Guo, Jundong Li, and Huan Liu. IJCAI 2018. [pdf]
6. Improving Maximum Likelihood Estimation of Temporal Point Process via Discriminative and Adversarial Learning. Junchi Yan, Xin Liu, Liangliang Shi, Changsheng Li, Hongyuan Zha. IJCAI 2018. [pdf]

3.2 Imputing

1. Imputing Missing Events in Continuous-Time Event Streams. Hongyuan Mei, Guanghui Qin, Jason Eisner. ICML 2019. [pdf] [code] [slides]

3.3 Anomaly Detection

1. Detecting Anomalous Event Sequences with Temporal Point Processes. Oleksandr Shchur, Ali Caner Türkmen, Tim Januschowski, Jan Gasthaus, Stephan Günnemann. NeurIPS 2021. [pdf]
2. Sequential Adversarial Anomaly Detection for One-Class Event Data. Shixiang Zhu, Henry Shaowu Yuchi, Yao Xie. ICASSP 2020. [pdf]

3.4 Clustering

1. C-NTPP: Learning Cluster-Aware Neural Temporal Point Process. Fangyu Ding, Junchi Yan, Haiyang Wang. AAAI 2023. [pdf]
2. Learning Mixture of Neural Temporal Point Processes for Multi-dimensional Event Sequence Clustering. Yunhao Zhang, Junchi Yan, Xiaolu Zhang, Jun Zhou, Xiaokang Yang. IJCAI 2022. [pdf]
3. A Dirichlet Mixture Model of Hawkes Processes for Event Sequence Clustering. Hongteng Xu, Hongyuan Zha. NeurIPS 2017. [pdf] [code]
4. Definition of Distance for Nonlinear Time Series Analysis of Marked Point Process Data. Koji Iwayama, Yoshito Hirata, Kazuyuki Aihara. Physics Letters A 2017. [ScienceDirect]

3.5 Data Augmentation

1. Benefits from superposed hawkes processes. Hongteng Xu, Dixin Luo, Xu Chen, Lawrence Carin. AISTATS 2018. [pdf] [code]
2. Learning Hawkes processes from short doubly-censored event sequences. Hongteng Xu, Dixin Luo, Hongyuan Zha. ICML 2017. [pdf]
3. Transforming spatial point processes into Poisson processes using random superposition. Jesper Møller, Kasper K. Berthelsen. Advances in Applied Probability 2012. [pdf]

3.6 Denoising

1. Learning Hawkes processes under synchronization noise. William Trouleau, Jalal Etesami, Matthias Grossglauser, Negar Kiyavash, Patrick Thiran. ICML 2019. [pdf]
2. Learning registered point processes from idiosyncratic observations. Hongteng Xu, Lawrence Carin, Hongyuan Zha. ICML 2018. [pdf]

3.7 Querying

1. Probabilistic Querying of Continuous-Time Event Sequences. Alex Boyd, Yuxin Chang, Stephan Mandt, Padhraic Smyth. AISTATS 2023. [pdf] [code]

4. Application Papers

4.1 Social Media

1. Identifying Coordinated Accounts on Social Media through Hidden Influence and Group Behaviours. Karishma Sharma, Yizhou Zhang, Emilio Ferrara, Yan Liu. KDD 2021. [pdf] [code]
2. COEVOLVE: A Joint Point Process Model for Information Diffusion and Network Co-evolution. Mehrdad Farajtabar, Yichen Wang, Manuel Gomez Rodriguez, Shuang Li, Hongyuan Zha, Le Song. JMLR 2017. [pdf] [code]
3. DeepHawkes: Bridging the Gap between Prediction and Understanding of Information Cascades. Qi Cao, Huawei Shen, Keting Cen, Wentao Ouyang, Xueqi Cheng. CIKM 2017. [pdf] [code]
4. The Bayesian Echo Chamber: Modeling Social Influence via Linguistic Accommodation. Fangjian Guo, Charles Blundell, Hanna Wallach, Katherine Heller. AISTATS 2015. [pdf]
5. Learning Social Infectivity in Sparse Low-rank Networks Using Multi-dimensional Hawkes Processes. Ke Zhou, Hongyuan Zha, Le Song. AISTATS 2013. [pdf]

4.2 Clinical Health

1. Continuous-Time Decision Transformer for Healthcare Applications. Zhiyue Zhang, Hongyuan Mei, Yanxun Xu. AISTATS 2023. [pdf] [code]
2. Personalized Dynamic Treatment Regimes in Continuous Time: A Bayesian Joint Model for Optimizing Clinical Decisions with Timing. William Hua, Hongyuan Mei, Sarah Zohar, Magali Giral, Yanxun Xu. Bayesian Analysis (2021). [pdf] [code]
3. Neural Temporal Point Processes For Modelling Electronic Health Records. Joseph Enguehard, Dan Busbridge, Adam Bozson, Claire Woodcock, Nils Hammerla. Machine Learning for Health 2020. [pdf] [code]
4. Patient Flow Prediction via Discriminative Learning of Mutually-Correcting Processes. Hongteng Xu, Weichang Wu, Shamim Nemati, Hongyuan Zha. TKDE 2016. [pdf]
5. Constructing Disease Network and Temporal Progression Model via Context-sensitive Hawkes Process. Edward Choi, Nan Du, Robert Chen, Le Song, Jimeng Sun. ICDM 2015. [pdf]

4.3 E-commerce

1. Time is of the Essence: a Joint Hierarchical RNN and Point Process Model for Time and Item Predictions. Bjørnar Vassøy, Massimiliano Ruocco, Eliezer de Souza da Silva, Erlend Aune. WSDM 2019. [pdf] [code]
2. Intermittent Demand Forecasting with Deep Renewal Processes. Ali Caner Turkmen, Yuyang Wang, Tim Januschowski. TPP@NeurIPS 2019. [pdf]
3. Recurrent Spatio-Temporal Point Process for Check-in Time Prediction. Guolei Yang, Ying Cai, Chandan K. Reddy. CIKM 2018. [pdf]
4. Time-sensitive Recommendation from Recurrent User Activities. Nan Du, Yichen Wang, Niao He, Jimeng Sun, and Le Song. NeurIPS 2015. [pdf]

4.4 Causality Discovery

1. Structural Hawkes Processes for Learning Causal Structure from Discrete-Time Event Sequences. Jie Qiao, Ruichu Cai, Siyu Wu, Yu Xiang, Keli Zhang, Zhifeng Hao. IJCAI 2023. [pdf]
2. CAUSE: Learning Granger Causality from Event Sequences using Attribution Methods. Wei Zhang, Thomas Kobber Panum, Somesh Jha, Prasad Chalasani, David Page. ICML 2020. [pdf] [slideslive] [code & data]
3. Uncovering Causality from Multivariate Hawkes Integrated Cumulants. Massil Achab, Emmanuel Bacry, Stéphane Gaïffas, Iacopo Mastromatteo, Jean-Francois Muzy. ICML 2017. [pdf] [code]
4. Graphical Modeling for Multivariate Hawkes Processes with Nonparametric Link Functions. Michael Eichler, Rainer Dahlhaus, Johannes Dueck. Journal of Time Series Analysis 2017. [pdf]
5. Learning Granger Causality for Hawkes Processes. Hongteng Xu, Mehrdad Farajtabar, Hongyuan Zha. ICML 2016. [pdf]
6. Learning Network of Multivariate Hawkes Processes: A Time Series Approach. Jalal Etesami, Negar Kiyavash, Kun Zhang, Kushagra Singhal. UAI 2016. [pdf]

4.5 Audio Processing

1. Recurrent Poisson Process Unit for Speech Recognition. Hengguan Huang, Hao Wang, Brian Mak. AAAI 2019. [pdf]

4.6 Natural Language Processing

1. Early Rumor Detection Using Neural Hawkes Process with a New Benchmark Dataset. Fengzhu Zeng, Wei Gao. NAACL 2022. [pdf] [code & dataset]
2. Hawkes processes for Continuous Time Sequence Classification: An Application to Rumour Stance Classification in Twitter. Michal Lukasik, P. K. Srijith, Duy Vu, Kalina Bontcheva, Arkaitz Zubiaga, Trevor Cohn. ACL 2016. [pdf] [code]
3. Dirichlet-Hawkes Processes with Applications to Clustering Continuous-time Document Streams. Nan Du, Mehrdad Farajtabar, Amr Ahmed, Alexander J Smola, Le Song. KDD 2015. [pdf]
4. Hawkestopic: A Joint Model for Network Inference and Topic Modeling from Text-based Cascades. Xinran He, Theodoros Rekatsinas, James Foulds, Lise Getoor, Yan Liu. ICML 2015. [pdf]

4.7 Computer Vision

1. Egocentric Activity Prediction via Event Modulated Attention. Yang Shen, Bingbing Ni, Zefan Li, Ning Zhuang. ECCV 2018. [pdf]

4.8 Network Structure Discovery

1. Discovering Latent Network Structure in Point Process Data. Scott Linderman and Ryan Adams. ICML 2014. [pdf] [code]

4.9 Science

1. Weather Knows What Will Occur: Urban Public Nuisance Events Prediction and Control with Meteorological Assistance. Yi Xie, Tianyu Qiu, Yun Xiong, Xiuqi Huang, Xiaofeng Gao, Chao Chen, Qiang Wang, Haihong Li. KDD 2024. [acm page] [video]
2. Point process models for sequence detection in high-dimensional neural spike trains. Alex Williams, Anthony Degleris, Yixin Wang, Scott Linderman. NeurIPS 2021. [pdf] [code]
3. Point Process Latent Variable Models of Larval Zebrafish Behavior. Anuj Sharma, Robert Johnson, Florian Engert, Scott Linderman. NeurIPS 2018. [pdf]
4. Mixture of Mutually Exciting Processes for Viral Diffusion. Shuang-Hong Yang, Hongyuan Zha. ICML 2013. [pdf]

4.10 Reinforcement Learning

1. Bellman Meets Hawkes: Model-Based Reinforcement Learning via Temporal Point Processes. Chao Qu, Xiaoyu Tan, Siqiao Xue, Xiaoming Shi, James Zhang, Hongyuan Mei. AAAI 2023. [pdf] [code]

5. Benchmark

1. EasyTPP: Towards Open Benchmarking the Temporal Point Processes. Siqiao Xue, Xiaoming Shi, Zhixuan Chu, Yan Wang, Fan Zhou, Hongyan Hao, Caigao Jiang, Chen Pan, Yi Xu, James Y. Zhang, Qingsong Wen, Jun Zhou, Hongyuan Mei. Arxiv 2023. [pdf] [code]

6. Research Opportunities

Note: Papers listed in this section are loosely related to TPP (e.g., may not contain continuous-time modeling), but we find them insightful and open up new research opportunities.

1. Predictive Querying for Autoregressive Neural Sequence Models. Alex Boyd, Sam Showalter, Stephan Mandt, Padhraic Smyth. NeurIPS 2022. [pdf] [code]
   Why this is relevant? See here for compilers' comments (click to expand)

   (Comments: This is for general autoregressive models and the authors does not do experiments on TPP, but TPP practioners may find it interesting to think about how we can deal with queries beyond one-step-ahead, like How likely is event A to occur before event B? and How likely is event C to occur (once of more) within the next K steps of the sequence?. An example application in TPP is published in AISTATS 2023, which is listed in Section ``Querying'' of this paper list. )

Acknowledgements

We thank all the contributors to this list. And more contributions are very welcome.