Bayesian based stopping rules for behavioral VHDL verification

Abstract

Verification of complex behavioral models has become a critical and time-consuming process in hardware design. During behavioral model verification, it is important to determine the stopping point for the current test strategy and for moving to a different test strategy. It has been shown that the location of the stopping point is highly dependent on the statistical model one chooses to describe the coverage behavior during the verification process. This research presents Bayesian based statistical stopping rules for behavioral VHDL model verification. Unlike other existing approaches, the statistical assumptions of the proposed stopping rules are based on experimental evaluation of probability distribution functions and correlation functions extracted from simulation results of a suite of HDL models. The resulting assumptions are then employed with proper testing criteria in developing the proposed stopping rules. Fourteen behavioral VHDL models were experimented with to determine the efficiency of the proposed stopping rules over the existing stopping rules. Two metrics for measuring efficiency have been developed to objectively compare between different stopping rules. Results show that the efficiency of using the proposed stopping rules are up to 3 times better than that of using the best existing stopping rule and up to two orders of magnitude better than that without using stopping rules. We also propose a statistical forecasting model that predicts the future coverage during the verification process. The proposed model has high prediction accuracies in determining the probability of having coverage in the future as well as the expected waiting time to a new coverage item within the same prediction window. The estimated prediction error of having coverage was found to be 2% in the worst case when predicting the next 1000 simulation cycles. When extending the prediction window size to 10,000 simulation cycles, the maximum expected errors in predicting the interruption probability is 13%. We believe that the proposed stopping rules and the statistical forecasting model will transform the existing brute-force approaches in behavioral model verification to more intelligent and statistical approaches to meet the tight time-to-market requirement of complex electronic systems without sacrificing the quality of the design.