Engineering audit protocol for trustworthy electricity-theft detection on non-stationary smart-meter data

Abstract

Electricity theft is the dominant non-technical loss in advanced metering infrastructure, threatening smart-grid reliability and utility revenue worldwide. Despite a proliferation of deep-learning detectors, the data on which they are evaluated is rarely audited for leakage; undetected leakage in the evaluation pipeline can inflate the performance these detectors report. This paper reframes electricity-theft detection as a critical-infrastructure integrity problem and proposes a model-agnostic engineering audit that exposes such inflation before deployment. The audit applies four validation gates: a data-provenance audit, a temporal integrity check, a leakage detection gate, and a realistic distribution-shift test; results are reported under a three-tier disclosure standard of full population, stable regime, and audited cohort, with a 12-item checklist. On the State Grid Corporation of China dataset, whose missingness exhibits a 41.44 percentage-point structural break in a single month at January 2016, six of eight architectures lose 28.5% to 32.6% of their reported precision-recall area under the curve (PR-AUC; absolute drop 0.109 to 0.134) once the shortcut is removed. Customer-grouped split leakage is operationally negligible (maximum |ΔPR-AUC| = 0.0035), and an audit reveals detection reliability varies by a factor of seven across 22 subgroups. A permutation-importance probe identifies consumption scale and variability computed over observed days, together with a direct contribution from the missingness rate, as the signals correlated with the regime and available to every detector, indicating the likely mechanism of the inflation. The gap between reported and audited performance gives utilities a verification step that current reporting practice does not provide.