When Does Hybrid Fraud Screening Earn Its Alerts? A Budget-Constrained Comparison of Rules, Anomaly Detection, and Supervised Learning
DOI:
https://doi.org/10.54691/r98yg421Keywords:
Financial fraud detection, anomaly detection, auditable rules, imbalanced learning, review capacity, temporal validation, synthetic dataAbstract
Financial fraud screening operates under fixed investigation capacity, so overall discrimination may not indicate how many fraudulent transactions reach reviewers. We evaluate a two-parameter, rule-gated ranking that combines supervised histogram gradient boosting, unsupervised Isolation Forest, and three auditable red flags. Coefficients are selected exclusively on a chronological validation split to optimize fraud recall within the highest-risk 1% of transactions. Twenty-five independently seeded, explicitly synthetic streams supply 2,250,000 transactions, each with 60/20/20 temporal train/validation/test partitions. On held-out blocks, the complete hybrid averages 14.53% recall at 1% workload versus 14.08% for boosted trees (mean paired difference +0.45 percentage points; 95% seed-level t interval −0.55 to +1.45; two-sided Monte Carlo sign-flip p≈0.376). The no-rule-bonus variant still gates anomaly evidence and reaches 14.96%; a truly ungated supervised-anomaly baseline reaches 14.05%. This separates anomaly gating from direct rule-score uplift. Budget and review-window analyses show that the marginal value of explicit rule gating depends on the allocation policy and validation-selected coefficients. The central contribution is a reproducible test of when an auditable hybrid changes a restricted review queue, including cases where added components provide no benefit. Results concern simulated, not observed financial activity.
Downloads
References
[1] Ngai, E. W. T., Hu, Y., Wong, Y. H., Chen, Y., & Sun, X. (2011). The application of data mining techniques in financial fraud detection: A classification framework and an academic review of literature. Decision Support Systems, 50(3), 559–569. https://doi.org/10.1016/j.dss.2010.08.006 DOI: https://doi.org/10.1016/j.dss.2010.08.006
[2] West, J., & Bhattacharya, M. (2016). Intelligent financial fraud detection: A comprehensive review. Computers & Security, 57, 47–66. https://doi.org/10.1016/j.cose.2015.09.005 DOI: https://doi.org/10.1016/j.cose.2015.09.005
[3] Bolton, R. J., & Hand, D. J. (2002). Statistical fraud detection: A review. Statistical Science, 17(3), 235–255. https://doi.org/10.1214/ss/1042727940 DOI: https://doi.org/10.1214/ss/1042727940
[4] Dal Pozzolo, A., Boracchi, G., Caelen, O., Alippi, C., & Bontempi, G. (2018). Credit card fraud detection: A realistic modeling and a novel learning strategy. IEEE Transactions on Neural Networks and Learning Systems, 29(8), 3784–3797. https://doi.org/10.1109/TNNLS.2017.2736643 DOI: https://doi.org/10.1109/TNNLS.2017.2736643
[5] Whitrow, C., Hand, D. J., Juszczak, P., Weston, D., & Adams, N. M. (2009). Transaction aggregation as a strategy for credit card fraud detection. Data Mining and Knowledge Discovery, 18, 30–55. https://doi.org/10.10618-008-0116-z DOI: https://doi.org/10.1007/s10618-008-0116-z
[6] Bahnsen, A. C., Aouada, D., Stojanovic, A., & Ottersten, B. (2016). Feature engineering strategies for credit card fraud detection. Expert Systems with Applications, 51, 134–142. https://doi.org/10.1016/j.eswa.2015.12.030 DOI: https://doi.org/10.1016/j.eswa.2015.12.030
[7] Chen, J., Han, X., Zhang, X., & Yang, Y. (2023). Exact concept-level explanations for early detection of financial misstatements. Eurasia Journal of Science and Technology, 5(2), 34–48. DOI: https://doi.org/10.61784/ejst4162
[8] Carcillo, F., Le Borgne, Y.-A., Caelen, O., Kessaci, Y., Oble, F., & Bontempi, G. (2021). Combining unsupervised and supervised learning in credit card fraud detection. Information Sciences, 557, 317–331. https://doi.org/10.1016/j.ins.2019.05.042 DOI: https://doi.org/10.1016/j.ins.2019.05.042
[9] Fiore, U., De Santis, A., Perla, F., Zanetti, P., & Palmieri, F. (2019). Using generative adversarial networks for improving classification effectiveness in credit card fraud detection. Information Sciences, 479, 448–455. https://doi.org/10.1016/j.ins.2017.12.030 DOI: https://doi.org/10.1016/j.ins.2017.12.030
[10] Ashtiani, M. N., & Raahemi, B. (2022). Intelligent fraud detection in financial statements using machine learning and data mining: A systematic literature review. IEEE Access, 10, 72504–72525. https://doi.org/10.1109/ACCESS.2021.3096799 DOI: https://doi.org/10.1109/ACCESS.2021.3096799
[11] He, H., & Garcia, E. A. (2009). Learning from imbalanced data. IEEE Transactions on Knowledge and Data Engineering, 21(9), 1263–1284. https://doi.org/10.1109/TKDE.2008.239 DOI: https://doi.org/10.1109/TKDE.2008.239
[12] Chawla, N. V., Bowyer, K. W., Hall, L. O., & Kegelmeyer, W. P. (2002). SMOTE: Synthetic minority over-sampling technique. Journal of Artificial Intelligence Research, 16, 321–357. https://doi.org/10.1613/jair.953 DOI: https://doi.org/10.1613/jair.953
[13] Breiman, L. (2001). Random forests. Machine Learning, 45, 5–32. https://doi.org/10.1023/A:1010933404324 DOI: https://doi.org/10.1023/A:1010933404324
[14] Zhou, F. (2026). Moment-constrained side-force allocation for wind-assisted merchant ships: Hydrodynamic performance of retractable anti-leeway fins and flap rudders. Innovation and Technology Studies, 3(1), 27–41. DOI: https://doi.org/10.61784/its4027
[15] Liu, F. T., Ting, K. M., & Zhou, Z.-H. (2008). Isolation forest. Proceedings of the 8th IEEE International Conference on Data Mining, 413–422. https://doi.org/10.1109/ICDM.2008.17 DOI: https://doi.org/10.1109/ICDM.2008.17
[16] Chen, J., Zhang, X., & Han, X. (2023). Explainable identification of tax-related financial reporting anomalies in U.S. public companies: Cross-fitted tax-consistency residuals and Owen-value Shapley propagation. World Journal of Economics and Business Research, 1(1), 33–45. DOI: https://doi.org/10.61784/wjebr4125
[17] Niculescu-Mizil, A., & Caruana, R. (2005). Predicting good probabilities with supervised learning. Proceedings of the 22nd International Conference on Machine Learning, 625–632. https://doi.org/10.1145/1102351.1102430 DOI: https://doi.org/10.1145/1102351.1102430
[18] Dal Pozzolo, A., Caelen, O., Johnson, R. A., & Bontempi, G. (2015). Calibrating probability with undersampling for unbalanced classification. Proceedings of the IEEE Symposium Series on Computational Intelligence, 159–166. https://doi.org/10.1109/SSCI.2015.33 DOI: https://doi.org/10.1109/SSCI.2015.33
[19] Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., & Duchesnay, É. (2011). Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12, 2825–2830.
[20] Davis, J., & Goadrich, M. (2006). The relationship between precision-recall and ROC curves. Proceedings of the 23rd International Conference on Machine Learning, 233–240. https://doi.org/10.1145/1143844.1143874 DOI: https://doi.org/10.1145/1143844.1143874
[21] Saito, T., & Rehmsmeier, M. (2015). The precision-recall plot is more informative than the ROC plot when evaluating binary classifiers on imbalanced datasets. PLOS ONE, 10(3), Article e0118432. https://doi.org/10.1371/journal.pone.0118432 DOI: https://doi.org/10.1371/journal.pone.0118432
[22] Hand, D. J. (2009). Measuring classifier performance: A coherent alternative to the area under the ROC curve. Machine Learning, 77, 103–123. https://doi.org/10.1007/s10994-009-5119-5 DOI: https://doi.org/10.1007/s10994-009-5119-5
[23] Ribeiro, M. T., Singh, S., & Guestrin, C. (2016). Why should I trust you?: Explaining the predictions of any classifier. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 1135–1144. https://doi.org/10.1145/2939672.2939778 DOI: https://doi.org/10.1145/2939672.2939778
[24] Wang, J., Ge, Y., Liu, J., & Jiang, B. (2024). Early warning of liquidity stress in U.S. banks with seasonally aligned frailty hazards and delayed-feedback conformal alarms. Journal of Trends in Finance and Economics, 1(1), 85–100. DOI: https://doi.org/10.61784/jtfe4078
[25] DeLong, E. R., DeLong, D. M., & Clarke-Pearson, D. L. (1988). Comparing the areas under two or more correlated receiver operating characteristic curves: A nonparametric approach. Biometrics, 44(3), 837–845. https://doi.org/10.2307/2531595 DOI: https://doi.org/10.2307/2531595
[26] McNemar, Q. (1947). Note on the sampling error of the difference between correlated proportions or percentages. Psychometrika, 12(2), 153–157. https://doi.org/10.1007/BF02295996 DOI: https://doi.org/10.1007/BF02295996
[27] Lin, Y., Zhang, H., & Li, M. (2026). Subgraph-Level Temporal Graph Transformer With Laundering-Motif Prototypes for Anti-Money Laundering on Blockchains. IEEE Access. https://doi.org/10.1109/ACCESS.2026.3731160 DOI: https://doi.org/10.1109/ACCESS.2026.3731160
[28] Liu, Y., Xu, K., & Chen, L. (2024). Temporal Logical Attention Network for Log-Based Anomaly Detection in Distributed Systems. Sensors, 24, Article 7949. https://doi.org/10.3390/s24247949 DOI: https://doi.org/10.3390/s24247949
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Academic Journal of Finance and Accounting
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
