criterion performance measurements

overview

want to understand this report?

Word32/printf

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 1.6526703973042254e-6 1.673263279124281e-6 1.6993837062490973e-6
Standard deviation 5.738605119921833e-8 7.952605477201168e-8 1.0255036063460193e-7

Outlying measurements have severe (0.62761611359828%) effect on estimated standard deviation.

Word32/showHex

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 1.3199492416934907e-7 1.3302384275461326e-7 1.3457811985404517e-7
Standard deviation 3.555874861170519e-9 4.3399836596536445e-9 5.366242683169714e-9

Outlying measurements have moderate (0.49664033110308425%) effect on estimated standard deviation.

Word32/xbuild

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 5.286030893152601e-8 5.3161268363733895e-8 5.354426665169117e-8
Standard deviation 9.440320168498297e-10 1.1301527994119945e-9 1.395487111706175e-9

Outlying measurements have moderate (0.3078849892895587%) effect on estimated standard deviation.

Word32/xbuildu

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 1.228085975128646e-7 1.2366948438620195e-7 1.2476091370117593e-7
Standard deviation 2.649270581979967e-9 3.1180123405451734e-9 4.273265777534988e-9

Outlying measurements have moderate (0.3714208692473372%) effect on estimated standard deviation.

Word32/xshowl

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 2.109994522164871e-7 2.1226767159896135e-7 2.139467452975211e-7
Standard deviation 3.172886199208217e-9 4.9472904642137575e-9 7.407810675643109e-9

Outlying measurements have moderate (0.3239694314826444%) effect on estimated standard deviation.

Word32/xshowlp

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 2.2907272249674676e-7 2.3092534450312488e-7 2.3422112869612027e-7
Standard deviation 5.772475244253203e-9 8.07035016184256e-9 1.2024463772890042e-8

Outlying measurements have severe (0.5200604958432723%) effect on estimated standard deviation.

Word32/xshowlu

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 2.8924755228179816e-7 2.9092394681479877e-7 2.935736098784101e-7
Standard deviation 4.739837736429587e-9 7.037001342304216e-9 1.129677693657394e-8

Outlying measurements have moderate (0.3363907539621082%) effect on estimated standard deviation.

Word32/xshowlpu

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 3.1045298776710656e-7 3.1235079050577414e-7 3.154634154251918e-7
Standard deviation 6.536612804898373e-9 8.829469120506299e-9 1.2302007176698619e-8

Outlying measurements have moderate (0.40650923571498626%) effect on estimated standard deviation.

Word32/xshow

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 2.3340263744879649e-7 2.3469443267233177e-7 2.3619516301597985e-7
Standard deviation 4.06558370929341e-9 5.075029088666515e-9 6.797172769356892e-9

Outlying measurements have moderate (0.2901829237204042%) effect on estimated standard deviation.

Word32/xshowp

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 2.469398073235772e-7 2.483781449276144e-7 2.498949759188196e-7
Standard deviation 4.292063986102203e-9 5.621225528220052e-9 7.582800154059545e-9

Outlying measurements have moderate (0.3077495757261389%) effect on estimated standard deviation.

Word32/xshowu

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 3.113064705686556e-7 3.1342027378682637e-7 3.1585706037370135e-7
Standard deviation 6.240772715853902e-9 8.174098415906014e-9 1.1463148987895081e-8

Outlying measurements have moderate (0.36864114173323576%) effect on estimated standard deviation.

Word32/xshowpu

lower bound estimate upper bound
OLS regression xxx xxx xxx
R² goodness-of-fit xxx xxx xxx
Mean execution time 3.315797369967679e-7 3.335558971396557e-7 3.355818484921985e-7
Standard deviation 5.111611960176227e-9 6.885576660617828e-9 9.782183916628599e-9

Outlying measurements have moderate (0.26568695716465995%) effect on estimated standard deviation.

understanding this report

In this report, each function benchmarked by criterion is assigned a section of its own. The charts in each section are active; if you hover your mouse over data points and annotations, you will see more details.

Under the charts is a small table. The first two rows are the results of a linear regression run on the measurements displayed in the right-hand chart.

We use a statistical technique called the bootstrap to provide confidence intervals on our estimates. The bootstrap-derived upper and lower bounds on estimates let you see how accurate we believe those estimates to be. (Hover the mouse over the table headers to see the confidence levels.)

A noisy benchmarking environment can cause some or many measurements to fall far from the mean. These outlying measurements can have a significant inflationary effect on the estimate of the standard deviation. We calculate and display an estimate of the extent to which the standard deviation has been inflated by outliers.