Adaptive Conformal Risk Control for Payment Fraud Detection under Concept Drift and Delayed Feedback
DOI:
https://doi.org/10.54691/kzrea147Keywords:
Payment fraud detection, conformal risk control, concept drift, delayed feedback, chargebacks, inverse probability of censoring weighting, online calibrationAbstract
Payment fraud engines turn risk scores into approve-or-review decisions, and the quantity an operator must keep within budget is the fraud value released to auto-approval. Two features of production systems undermine static guarantees on that quantity: the fraud mix drifts, and the chargebacks that reveal fraud arrive weeks after the decision, so a transaction without a chargeback is not yet known to be legitimate. We present an adaptive conformal risk controller that keeps an exposure-capped, amount-stratified and value-ordered conformal calibration layer as a monotone family of policies and steers one scalar, the nominal level of that layer, by integral feedback on chargebacks. The feedback is progressive: every approved transaction contributes inverse-probability-of-censoring-weighted increments at several label ages, which telescope to its true monetary loss when the dispute window closes. This yields a distribution-free certificate on the long-run released fraud value that holds under arbitrary drift and under a misspecified delay distribution, whereas naive and single-stage early feedback converge to biased levels. On five 24-month Sparkov streams with a camouflaged-fraud drift scenario and on the 183-day ULB/Worldline handbook stream, static calibration released 2.63 times the budget and monthly recalibration 1.55 times, while the proposed controller held 1.01 ± 0.02 times, close to an oracle with instantly revealed labels. When chargeback delays grew by 50% after deployment, it stayed at 1.08 times, against 1.48 times for single-stage correction.
Downloads
References
[1] Dal Pozzolo, A., Boracchi, G., Caelen, O., Alippi, C., & Bontempi, G. (2018). Credit card fraud detection: A realistic modeling and a novel learning strategy. IEEE Transactions on Neural Networks and Learning Systems, 29(8), 3784–3797. https://doi.org/10.1109/TNNLS.2017.2736643 DOI: https://doi.org/10.1109/TNNLS.2017.2736643
[2] Dal Pozzolo, A., Caelen, O., Le Borgne, Y.-A., Waterschoot, S., & Bontempi, G. (2014). Learned lessons in credit card fraud detection from a practitioner perspective. Expert Systems with Applications, 41(10), 4915–4928. https://doi.org/10.1016/j.eswa.2014.02.026 DOI: https://doi.org/10.1016/j.eswa.2014.02.026
[3] Gama, J., Žliobaitė, I., Bifet, A., Pechenizkiy, M., & Bouchachia, A. (2014). A survey on concept drift adaptation. ACM Computing Surveys, 46(4), Article 44. https://doi.org/10.1145/2523813 DOI: https://doi.org/10.1145/2523813
[4] Lu, J., Liu, A., Dong, F., Gu, F., Gama, J., & Zhang, G. (2019). Learning under concept drift: A review. IEEE Transactions on Knowledge and Data Engineering, 31(12), 2346–2363. https://doi.org/10.1109/TKDE.2018.2876857 DOI: https://doi.org/10.1109/TKDE.2018.2876857
[5] Grzenda, M., Gomes, H. M., & Bifet, A. (2020). Delayed labelling evaluation for data streams. Data Mining and Knowledge Discovery, 34, 1237–1266. https://doi.org/10.1007/s10618-019-00654-y DOI: https://doi.org/10.1007/s10618-019-00654-y
[6] Žliobaitė, I. (2010). Change with delayed labeling: When is it detectable? In Proceedings of the IEEE International Conference on Data Mining Workshops (ICDMW) (pp. 843–850). https://doi.org/10.1109/ICDMW.2010.49 DOI: https://doi.org/10.1109/ICDMW.2010.49
[7] Correa Bahnsen, A., Aouada, D., Stojanovic, A., & Ottersten, B. (2016). Feature engineering strategies for credit card fraud detection. Expert Systems with Applications, 51, 134–142. https://doi.org/10.1016/j.eswa.2015.12.030 DOI: https://doi.org/10.1016/j.eswa.2015.12.030
[8] Elkan, C. (2001). The foundations of cost-sensitive learning. In Proceedings of the 17th International Joint Conference on Artificial Intelligence (IJCAI) (pp. 973–978).
[9] Carcillo, F., Dal Pozzolo, A., Le Borgne, Y.-A., Caelen, O., Mazzer, Y., & Bontempi, G. (2018). SCARFF: A scalable framework for streaming credit card fraud detection with Spark. Information Fusion, 41, 182–194. https://doi.org/10.1016/j.inffus.2017.09.001 DOI: https://doi.org/10.1016/j.inffus.2017.09.005
[10] Vovk, V., Gammerman, A., & Shafer, G. (2005). Algorithmic learning in a random world. Springer.
[11] Bates, S., Angelopoulos, A., Lei, L., Malik, J., & Jordan, M. I. (2021). Distribution-free, risk-controlling prediction sets. Journal of the ACM, 68(6), Article 43. https://doi.org/10.1145/3478535 DOI: https://doi.org/10.1145/3478535
[12] Angelopoulos, A. N., Bates, S., Fisch, A., Lei, L., & Schuster, T. (2024). Conformal risk control. In Proceedings of the 12th International Conference on Learning Representations (ICLR).
[13] Zi, B., Wang, J., & Zhang, Y. (2026). Latency-aware risk control for budget-constrained payment fraud decisioning. In Proceedings of ICCIA 2026 (pp. 257–269).
[14] Gibbs, I., & Candès, E. (2021). Adaptive conformal inference under distribution shift. Advances in Neural Information Processing Systems, 34, 1660–1672.
[15] Gibbs, I., & Candès, E. J. (2024). Conformal inference for online prediction with arbitrary distribution shifts. Journal of Machine Learning Research, 25(162), 1–36.
[16] Feldman, S., Ringel, L., Bates, S., & Romano, Y. (2023). Achieving risk control in online learning settings. Transactions on Machine Learning Research.
[17] Angelopoulos, A. N., Candès, E. J., & Tibshirani, R. J. (2023). Conformal PID control for time series prediction. Advances in Neural Information Processing Systems, 36. DOI: https://doi.org/10.52202/075280-1000
[18] Le Borgne, Y.-A., Siblini, W., Lebichot, B., & Bontempi, G. (2022). Reproducible machine learning for credit card fraud detection—Practical handbook. Université Libre de Bruxelles. https://github.com/Fraud-Detection-Handbook/fraud-detection-handbook
[19] Deng, J. (2026). When can fraud operations authorize automation? A decision-support framework for fresh audit evidence and review workload. arXiv:2608.08577.
[20] Zhu, R., Zhang, X., Wang, T., Liao, J., Chung, S. H., & Zhang, X. (2026). DISCO: Decoupling representation learning and risk control for reliable credit card fraud detection. Decision Support Systems, 208, Article 114717. https://doi.org/10.1016/j.dss.2026.114717 DOI: https://doi.org/10.1016/j.dss.2026.114717
[21] Tibshirani, R. J., Foygel Barber, R., Candès, E. J., & Ramdas, A. (2019). Conformal prediction under covariate shift. Advances in Neural Information Processing Systems, 32, 2530–2540.
[22] Foygel Barber, R., Candès, E. J., Ramdas, A., & Tibshirani, R. J. (2023). Conformal prediction beyond exchangeability. The Annals of Statistics, 51(2), 816–845. https://doi.org/10.1214/23-AOS2276 DOI: https://doi.org/10.1214/23-AOS2276
[23] Robins, J. M., Rotnitzky, A., & Zhao, L. P. (1994). Estimation of regression coefficients when some regressors are not always observed. Journal of the American Statistical Association, 89(427), 846–866. https://doi.org/10.1080/01621459.1994.10476818 DOI: https://doi.org/10.1080/01621459.1994.10476818
[24] Candès, E. J., Lei, L., & Ren, Z. (2023). Conformalized survival analysis. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 85(1), 24–45. https://doi.org/10.1093/jrsssb/qkac004 DOI: https://doi.org/10.1093/jrsssb/qkac004
[25] Joulani, P., György, A., & Szepesvári, C. (2013). Online learning under delayed feedback. In Proceedings of the 30th International Conference on Machine Learning (ICML), PMLR (Vol. 28, No. 3, pp. 1453–1461).
[26] El Halabi, L., & Brandt, A. (2026). Adaptive conformal inference under delayed feedback: Coverage guarantees and a delay-to-memory diagnostic. arXiv:2609.07251.
[27] Chapelle, O. (2014). Modeling delayed feedback in display advertising. In Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD) (pp. 1097–1105). https://doi.org/10.1145/2623330.2623634 DOI: https://doi.org/10.1145/2623330.2623634
[28] Yasui, S., Morishita, G., Fujita, K., & Shibata, M. (2020). A feedback shift correction in predicting conversion rates under delayed feedback. In Proceedings of the Web Conference (WWW) (pp. 2740–2746). https://doi.org/10.1145/3366423.3380032 DOI: https://doi.org/10.1145/3366423.3380032
[29] Sparkov data generation. (2022). GitHub repository (Commit b5eb45c). https://github.com/namebrandon/Sparkov_Data_Generation
[30] Ke, G., Meng, Q., Finley, T., Wang, T., Chen, W., Ma, W., Ye, Q., & Liu, T.-Y. (2017). LightGBM: A highly efficient gradient boosting decision tree. Advances in Neural Information Processing Systems, 30, 3146–3154.
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Academic Journal of Finance and Accounting
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
