Click here to see the meta data of this asset.

Appendix: Project methodology

From: Concurrent Substance Use and Mental Health Problems in Youth: Screening for Concurrent Substance Use and Mental Health Problems in Youth (© 2009, CAMH)

Project scope

The strategy for searching and reviewing the relevant literature was an iterative one that began by casting a wide net and instituting increasingly stringent criteria and rating processes to arrive at the most promising, well-validated and reliable screening tools. Within this general strategy, several important overriding criteria put some critical boundaries around the scope of the project and, by implication, the eventual end product. These criteria related to:

  • age of the target audience for the tools
  • tools for screening versus assessment purposes
  • type of disorder or substance use/mental health problem
  • types of tool and administration procedures
  • language and cultural issues
  • availability of research information in published versus "grey" literature
  • accessibility of the tool (i.e., tools that are free-of-charge versus tools that have costs associated with them).

Literature search and review process

The search and review process was divided into six broad steps:

  1. Gathering abstracts, titles and articles.
  2. Trimming the list.
  3. Validating the list of tools from the articles.
  4. Broadening the search based on the identified tools themselves.
  5. Rating the final subset of articles.
  6. Describing the tools represented in the best validation studies.

Figure 2 provides a high level overview of the search, retrieval and review process. It integrates the steps incorporated in the initial search covering 1986–2006 and for a subsequent update that covered 2006-2008.

 

Figure 2: Overview of Steps in the Search and Retrieval Process

A structured literature search of bibliographic databases based on defined algorithm (plus Internet search)

Yield: 2,734 abstracts/titles 

Grey arrow pointing down

A trimming process to begin removing extraneous material and to identify specific tools of potential interest

Yield: 244 tools identified 

Grey arrow pointing down

A second exhaustive bibliographic search based on the name of each tool and other search terms, and a scan of cited reference material

Yield: 3,393 abstracts/titles 

Grey arrow pointing down

Review of abstracts/titles, including approximately 600 papers, with stringent criteria consistent with study goals.

Yield: 33 articles focused on 20 distinct tools 

 

Table 3 lists the screening tools in each of the three categories ("Mental Health but not SUD," "Substance Use Disorder but no other MH" and "Both SUD and other MH") that survived the iterative and rigorous review process. These tools in Table 3 also underwent a rating of the quality of the article against a set of recognized "quality of reporting criteria" designed specifically for diagnostic screening tools4. These criteria are known as the STARD criteria and are described in the next section.

In terms of the number of separate screening tools, this final stage of the review and rating process included:

  • nine tools validated to screen for mental disorders but not substance use disorders
  • seven tools validated to screen for substance use disorders but not other mental disorders
  • four tools validated to screen for both mental and substance use disorders. 

 

Table 3: Tools and Studies Emerging from the Search, Retrieval and Review Process

Note: We are currently experiencing some issues with table display on Portico. As an alternative, links have been provided to images of the three groups. Click on the table title to see them.

A. Mental Health but not SUD 5 - click to view image

Tool (9 tools in total) 

Author/Reference6

STARD rating 

# of articles 

Pediatric Symptom Checklist (PSC) 

Gardner, Lucas, Kolko & Campo (2007)

Tzoumas, Tzoumas, Burlingame et al. (2007)

Anderson, Spratt, Macias et al. (1999) 

26

24

 

26 

Youth Outcome Questionnaire (Y-OQ-12) 

Tzoumas, Tzoumas, Burlingame et al. (2007) 

26 

1

Child Behavior Checklist (CBCL) 

Ferdinand (2008)

Gardner, Lucas, Kolko, & Campo (2007)

Krol, De Bruyn, Coolen, & van Aarle (2006)

Becker, Hagenberg, Roessner et al. (2004)

Jensen, Salzberg et al. (1993) 

24

26 

20

 

24

 

20  

Strengths and Difficulties Questionnaire (SDQ) 

Lukumar, Wijewardana et al. (2008)

Alyahri & Goodman (2006)

Samad, Hollis, Prine, & Goodman (2005)

Becker, Hagenberg, Roessner et al. (2004)

Becker, Woerner, Hasselhorn et al. (2004)

Hawes & Dadds (2004)  

15

 

24

17

 

25

 

22

 

19 

Youth Self-Report (YSR) 

Ferdinand (2008)

Doyle, Mick, & Biederman (2007)

Becker, Hagenberg, Roessner et al. (2004) 

24

15

 

24 

Reporting Questionnaire for Children (RQC) 

Tadesse, Kebede, Tegegne & Alem (1999) 

23

1

Child Symptom Inventory-4 (CSI-4) 

Sprafkin, Gadow, Salisbury et al. (2002) 

23

1

General Health Questionnaire (GHQ) 

Banks (1983) 

18

1

Early Childhood Inventory-4 (ECI-4)  

Sprafkin, Volpe, Gadow et al. (2002)  

18

1

B. Substance Use Disorder but no other MH 7 - click to view image

Tool (7 tools in total)

Author/Reference

STARD rating

# of articles

CRAFFT

Bernard, Bolognini, Plancherel et al. (2005)

Knight, Sherrit, Gates & Harris (2004)

Knight, Sherrit, Shrier et al. (2002)

18

24

 

29

3

RAFFT

Bastiaens, Francis & Lewis (2000)

25

1

Alcohol/Drug Acknowledgement Scale (ACK)

Stein & Graham (2001)

21

1

MacAndrew Alcoholism Scale-Revised (MAC-R)

Stein & Graham (2001)

21

1

DEP-ADO

Bernard, Bolognini, Plancherel et al. (2005)

18

1

Rutgers Alcohol Problem Index (RAPI)

Ginzler, Garrett, Baer & Peterson (2007)

9

1

C. Both SUD and other MH - click to view image

Tool (4 tools in total)

Author/Reference

STARD rating

# of articles

Problem Oriented Screening Tool for Teenagers (POSIT)

Latimer, O'Brien, McDouall et al. (2004)

Danesco & Marques (2002)

Latimer, Winters & Stinchfield (1997)

McLaney, Del Boca & Babor (1994)

26

 

8

16

 

15

4

DISC Predictive Scales (DPS)

Roberts, Stuart & Lam (2008)

McReynolds, Wasserman & Fisher (2007)

Lucas, Zhang, Fisher et al. (2001)

25

23 

26

3

GAIN Short Screener (GSS)

Dennis, Chan & Funk (2006)

22

1

Drug Use Screening Inventory (DUSI) and (DUSI-R)

Kirisci, Tarter, Mezzich & Reynolds (2008)

Kirisci, Mezzich & Tarter (1995)

19

 

20

2

 

Reference list for Table 3

Alyahri, A. & Goodman, R. (2006). Validation of the Arabic Strengths and Difficulties Questionnaire and the Development and Well-Being Assessment. Eastern Mediterranean Health Journal, 12 Suppl 2, S138-146.

Anderson, D. L., Spratt, E. G., Macias, M. M., Jellinek, M. S., Murphy, J. M., Pagano, M. et al. (1999). Use of the Pediatric Symptom Checklist in the pediatric neurology population. Pediatric Neurology, 20, 116-120.

Banks, M. H. (1983). Validation of the General Health Questionnaire in a young community sample. Psychological Medicine, 13, 349-353.

Bastiaens, L., Francis, G., & Lewis, K. (2000). The RAFFT as a screening tool for adolescent substance use disorders. American Journal on Addictions, 9, 10-16.

Becker, A., Hagenberg, N., Roessner, V., Woerner, W., & Rothenberger, A. (2004). Evaluation of the self-reported SDQ in a clinical setting: Do self-reports tell us more than ratings by adult informants? European Child & Adolescent Psychiatry, 13 Suppl 2, 17-24.

Becker, A., Woerner, W., Hasselhorn, M., Banaschewski, T., & Rothenberger, A. (2004). Validation of the parent and teacher SDQ in a clinical sample. European Child & Adolescent Psychiatry, 13 Suppl 2, II11-16.

Bernard, M., Bolognini, M., Plancherel, B., Chinet, L., Laget, J., Stephan, P. et al. (2005). French validity of two substance-use screening tests among adolescents: a comparison of the CRAFFT and DEP-ADO. Journal of Substance Use, 10, 385-395.

Danseco, E. R. & Marques, P. R. (2002). Development and validation of a POSIT-Short Form: screening for problem behaviors among adolescents at risk for substance use. Journal of Child & Adolescent Substance Abuse, 11(3), 17-36.

Dennis, M. L., Chan, Y. F., & Funk, R. R. (2006). Development and validation of the GAIN Short Screener (GSS) for internalizing, externalizing and substance use disorders and crime/violence problems among adolescents and adults. American Journal on Addictions, 15, 80-91

Doyle, R., Mick, E., Biederman, J. (2007). Convergence between the Achenbach Youth Self-Report and structured diagnostic interview diagnoses in ADHD and non-ADHD youth. The Journal of Nervous and Mental Disease, 195, 350-352.

Ferdinand, R. F. (2008). Validity of the CBCL/YSR DSM-IV scales anxiety problems and affective problems. Journal of Anxiety Disorders, 22, 126-134.

Gardner, W., Lucas, A., Kolko, D. J., & Campo, J. V. (2007). Comparison of the PSC-17 and alternative mental health screens in an at-risk primary care sample. Journal of the American Academy of Child & Adolescent Psychiatry, 46, 611-618.

Ginzler, J. A., Garrett, S. B., Baer, J. S., & Peterson, P. L. (2007). Measurement of negative consequences of substance use in street youth: An expanded use of the Rutgers Alcohol Problem Index. Addictive Behaviors, 32, 1519-1525.

Hawes, D. J., & Dadds, M. R. (2004). Australian data and psychometric properties of the Strengths and Difficulties Questionnaire. Australian and New Zealand Journal of Psychiatry, 38, 644–651.

Jensen P. S., Salzberg A. D., Richters J. E., Watanabe H. K. (1993). Scales, diagnoses, and child psychopathology. I. CBCL and DISC relationships. Journal of American Academy of Child & Adolescent Psychiatry, 32, 397-406.

Kirisci, L. Mezzich, A., & Tarter, R. (1995). Norms and sensitivity of the adolescent version of the drug use screening inventory. Addictive Behaviors, 20, 149-157.

Kirisci, L., Tarter, R., Mezzich, A., & Reynolds, M. (2008). Screening current and future diagnosis of psychiatric disorders using the Revised Drug Use Screening Inventory. American Journal of Drug & Alcohol Abuse, 34, 653-665.

Knight, J. R., Sherritt, L., Gates, E. A., & Harris, S. K. (2004). Should the CRAFFT substance abuse screening test be shortened? Journal of Clinical Outcomes Management, 11, 19-25.

Knight, J. R., Sherritt, L., Shrier, L. A., Harris, S. K., & Chang, G. (2002). Validity of the CRAFFT substance abuse screening test among adolescent clinic patients. Archives of Pediatrics & Adolescent Medicine, 156, 607-614.

Krol, N. P. C. M., De Bruyn, E. E. J., Coolen, J. C., & van Aarle, E. J. M. (2006). From CBCL to DSM: A Comparison of two methods to screen for DSM-IV diagnoses using CBCL data. Journal of Clinical Child and Adolescent Psychology, 35, 127-135.

Latimer, W. W., O'Brien, M. S., McDouall, J., Toussova, O., Floyd, L. J., & Vazquez, M. (2004). Screening for "substance abuse" among school-based youth in Mexico using the Problem Oriented Screening Instrument (POSIT) for Teenagers. Substance Use and Misuse, 39, 307-329.

Latimer, W. W., Winters, K. C., & Stinchfield, R. D. (1997). Screening for drug abuse among adolescents in clinical and correctional settings using the Problem-Oriented Screening Instrument for Teenagers. American Journal of Drug and Alcohol Abuse, 23, 79-98.

Lucas, C. P., Zhang, H., Fisher, P. W., Shaffer, D., Regier, D. A., Narrow, W. E. et al. (2001). The DISC Predictive Scales (DPS): Efficiently screening for diagnoses. Journal of the American Academy of Child & Adolescent Psychiatry, 40, 443-449.

Lukumar, P., Wijewardana, K., Hermansson, J., Lindmark, G. (2008). Validity and reliability of Tamil version of strengths and difficulties questionnaire self-report. Ceylon Medical Journal, 53, 48-52.

McLaney, M. A., Del Boca, F., & Babor, T. (1994). A validation study of the Problem-Oriented Screening Instrument for Teenagers (POSIT). Journal of Mental Health, 3, 363-376.

McReynolds, L. S., Wasserman, G. A., Fisher, P., & Lucas, C. P. (2007). Diagnostic screening with incarcerated youths: Comparing the DPS and Voice DISC. Criminal Justice and Behavior, 34, 830-845.

Roberts, N., Stuart, H., & Lam, M. (2008). High school mental health survey: Assessment of a mental health screen. The Canadian Journal of Psychiatry, 53, 314-322.

Roberts, N., Stuart, H., & Lam, M. (2008). High school mental health survey: Assessment of a mental health screen. The Canadian Journal of Psychiatry, 53, 314-322.

Samad L., Hollis C., Prince M., Goodman R. (2005) Child and adolescent psychopathology in a developing country: Testing the validity of the Strengths and Difficulties Questionnaire (Urdu version). International Journal of Methods in Psychiatric Research, 14, 158-166.

Sprafkin, J., Gadow, K. D., Salisbury, H., Schneider, J., & Loney, J. (2002). Further evidence of reliability and validity of the Child Symptom Inventory-4: parent checklist in clinically referred boys. Journal of Clinical Child and Adolescent Psychology, 31, 513-524.

Stein, L. A. & Graham, J. R. (2001). Use of the MMPI-A to detect substance abuse in a juvenile correctional setting. Journal of Personality Assessment, 77, 508-523.

Tadesse, B., Kebede, D., Tegegne, T., & Alem, A. (1999). Childhood behavioural disorders in Ambo district, western Ethiopia. II. Validation of the RQC. Acta Psychiatrica Scandinavica, Supplement 99, 98-101.

Tadesse, B., Kebede, D., Tegegne, T., & Alem, A. (1999). Childhood behavioural disorders in Ambo district, western Ethiopia. II. Validation of the RQC. Acta Psychiatrica Scandinavica, Supplement 99, 98-101.

Tzoumas, A. C., Tzoumas, J. L., Burlingame, G. M., Nelson, P. L., Wells, M. G., & Gray, D. D. (2007). The Y-OQ-12: Psychosocial screening of youth in primary care medicine using items from an outcome measure. Clinical Psychology & Psychotherapy, 14, 488-503.

4. Bossuyt, P.M., Reitsma, J.B., Bruns, D.E., Gatsonis, C.A., Glasziou, P.P., Irwig, L.M. et al. (2003). Towards complete and accurate reporting of studies of diagnostic accuracy: the STARD initiative. British Medical Journal, 326, 41–44.

5. Tools included may also include subscales for other problem areas related to psychosocial issues and problems areas, but NOT substance use disorders specifically.

6. Full reference is included in the reference list above.

7. Tools included may also include subscales for other problem areas related to psychosocial issues and problems areas, but NOT mental disorders specifically.