Logo Universität Bremen, zurück zur Startseite
Fachbereich 12 - Erziehungs- und Bildungswissenschaften

Quality assessment

Quality assessment

The classification of estimates into three quality classes follows standards of good scientific work developed in the CLANDESTINO methodological report, and pertains to the documentation, reliability and validity of an estimate. Details about the quality assessment, including examples, can be found in the CLANDESTINO classification report. Here, we only give a brief indication as to how estimates of absolute numbers and indicators of the composition are classified.

ESTIMATES OF ABSOLUTE NUMBERS

Quality classes

Explanation and examples

High quality estimate

Estimate fulfilling usual academic standards: full documentation, comprehensive and consistent, limitations clearly indicated:

  • Study with trust-based micro-data survey and adjustment for data bias (centre sampling in Italy)
  • Study with micro-apprehension data and adjustment for data bias (capture-recapture in the Netherlands)
  • Comprehensively and rigourously implemented and well documented multiplier or residual study
 

Medium quality estimate

Careful estimate: short explanation, largely consistent and comprehensive, limitations clearly indicated (at least explicit written statement):

  • Simple multiplier calculations
  • Simple residual estimates
  • Adjustment of older estimates with partly insufficient data
  • Aggregate estimates for different groups, partly relying on plausibility calculations
 

Low quality estimate

Unexplained or unreliable estimate:

  • No explanation
  • Indication of poor reliability and lacking empirical foundation for substantial aspects of calculation
  • Inadequate method or application of method (e.g. national level Delphi, or plausibility calculation from econometric estimate)
 

Low quality estimate with plausibility warning

Misleading low quality estimate:

  • Relevant in national discourse
  • Indications that it is much too high or too low
 

ESTIMATES OF COMPOSITIONAL INDICATORS (PERCENTAGES)

Quality classes

Explanation and example

High quality estimate

 
  • Indicator from micro-data study with a credible claim to have eliminated data bias
  • Large data set not likely to have a considerable bias with regard to the compositional criterion
 

Medium quality estimate

 
  • Double minmax: combination of two indicators using data with uni-directional data so that minimum and maximum assessment is possible (e.g min 10% and max 50% women)
  • Indicator from small data set that is not likely to have a considerable bias with regard to the compositional criterion
  • Indicator from large biased data set with careful reliability adjustment and discussion (e.g. Greek residual calculations)
 

Low quality estimate

 
  • Indicator from data with strong uni-directional bias with minimum or maximum assessment (but not both)
  • Indicator likely to be biased, but unclear or unknown direction
 

Low quality estimate with plausibility warning

 
  • Indicator relevant in national discourse, but with strong or unclear bias
  • No minimum maximum assessment possible
  • Other data or research indicate that the indicator may be seriously misleading
 
Aktualisiert von: Ulrich Achenbach
RSS
Seite drucken