Triangle tests

Introduction

A triangle test is a test beloved by sensory scientists for its simplicity and general use in detecting presence of product differences. The principle is simple. Test subjects get served three samples. One of these contains A, two of these contain B. The test subject has the task to indicate which is the odd sample. If enough odd samples are selected then the samples are declared different. The reverse may also happen. If few enough odd samples are selected the products are declared 'same'. This is a commercially relevant question as A may contain ingredients which are cheaper or from a competing supplier.

Statistical Testing 

As follows from the setup, the chance to select the odd sample is 1/3 under H0; products are the same. This allows calculation of the critical values. For 10 to 50 triangles these are plotted below.

The alpha is not quite 5%. It is guaranteed to be at least most 5%. The figure below shows that most of the time the tests are conservative. The jumps are because of the discrete nature of triangle tests. For 10 tests the critical value is the same as for 11 tests, resulting in a difference in alpha.

The same effect is seen in the error of the second kind. The figure below shows power for the alternative test of 60% correct. The figure also shows that in the neighborhood of 30 triangles must be done to have faith in the ability to detect a sample which gives 60% correct.

This leads to the question: what could be detected with 90% power? This is plotted below. Extreme cases, say 70% or 80% correct can be detected quite easy. Problem is, these are not relevant. In general creation or application staff would reject these before going to the sensory lab. Hence 30 to 40 tests is in general found the minimum to test when searching to declare samples close enough.

R script

triangle <- data.frame(n=10:50)

triangle$crit.val95 <- qbinom(.95,triangle$n,prob=1/3)

triangle$ph0 <- pbinom(triangle$crit.val,triangle$n,1/3)

triangle$pow.6 <- pbinom(triangle$crit.val95,triangle$n,.6,lower.tail=FALSE)

triangle$get.pow.90 <- sapply(1:nrow(triangle), function(row) {

      uniroot(function(x) 

          pbinom(triangle$crit.val95[row],

                 triangle$n[row],x,lower.tail=FALSE)

            -.90,

          interval=c(1/3,1))$root

    }

)

plot(crit.val95~n,data=triangle,xlab='Number of triangles',type='b',

    ylab='Critical value',main='Triangle test at 95%')

plot(ph0~n,data=triangle,xlab='Number of triangles',type='b',

    ylab='1-alpha',main='Triangle test at 95%')

plot(pow.6~n,data=triangle,xlab='Number of triangles',type='b',

    ylab='Power for H1 (60% correct)',main='Triangle test at 95%')

plot(get.pow.90~n,data=triangle,xlab='Number of triangles',type='b',

    ylab='Proportion correct for 90% power',main='Triangle test at 95%')