Traditional auditing methods require collating massive amounts of financial indicators and related transaction data, which can be labor-intensive. Typical machine learning models are relatively weak for imbalanced data, and this work aims to focus on a novel approach to fraud detection. This paper presents a fraud detection framework via adopting a machine learning method integrated with a recently proposed meta-heuristics algorithm Egret Swarm Optimization Algorithm (ESOA). A cost-sensitive objective function and loss function were then constructed, and a non-linear model was used to map the predicted values into the labels of 0 (non-fraud) and 1 (fraud). In the experiment section, an AAER benchmark dataset collected by the UCB's Center for Financial Reporting and Management is utilized to verify the performance of the proposed approach. A detailed comparison with recently proposed state-of-the-art algorithms such as Logit (67.20%), SVM-FK (62.60%), RUSBoost (72.60%), as well as BAS (84.90%) indicates that ESOA (96.27%) outperforms the other algorithms in terms of Accuracy (ACC), Sensitivity (SEN), Precision (PREC), and Area Under the Curve (AUC) metrics. To our knowledge, this is the highest fraud detection accuracy reported in the existing literature.