Specific mental disorder screening compilation may detect general mental disorders

Provenance and peer review: peer reviewed. Received: 01/30/2018. Accepted: 03/26/2018. Objective: To evaluate whether a short compilation of screening tools for specific disorders could identify Mental or Emotional Disorders (MEDs) in the general population. Methods: We selected validated screening tools for the most prevalent MEDs. In order to be selected, these tools should maintain the psychometric properties of the complete instrument with a reduced number of items. These instruments were: Patient Health Questionnaire-2 (PHQ-2), Generalized Anxiety Disorder Scale-2 (GAD-2), item 3 of the Alcohol Use Disorders Identification Test (AUDIT), and three items on the Adolescent Psychotic-Like Symptom Screener (APSS-3). We called this compilation of screening tools Mini Screening for Mental Disorders (Mini-SMD). The study was divided in two phases. Firstly, 545 subjects were interviewed with the Mini-SMD and COOP/WONCA-Feelings at their residences. Subsequently, subjects who had agreed to participate (230) were reinterviewed with Mini-SMD, COOP/ WONCA-Feelings and MINI interview. Test-retest reliability was calculated by Intraclass Correlation Coefficient (ICC). Receiver operating characteristic (ROC) curves were generated for the analysis of discriminative validity. Concurrent validity was calculated by analyzing the correlation between Mini-SMD and COOP/WONCA-Feelings. Results: The joint administration of screening tools for specific disorders showed sensitivities that ranged from 0.76 to 0.88 and specificities from 0.67 to 0.85. The ICC value for the total score of Mini-SMD was 0.78. The area under the curve was 0.84, with a sensitivity of 0.74 and specificity of 0.76 (for a cutoff ≥ 4). Conclusion: This study showed that a short compilation of screening tools for specific disorders can detect MEDs in general population. Abstract


Introduction
Mental or emotional disorders (MEDs) are common to all countries and affect all ages and socioeconomic groups.A World Health Organization (WHO) multicenter study conducted in 14 countries showed that 20% of people who were treated in primary health care (PHC) units had at least one diagnosis of a current MED. 1 However, these cases are often not detected.Fewer than 50% of patients with a detected MED are adequately treated by PHC. 2 Several factors contribute to these low rates, such as poor funding for mental health and inappropriate infrastructure for referral patients. 3nçalves et al. 4 reported that training Brazilian PHC staff in shared mental health care for a limited period of time was ineffective to improve MEDs recognition.One possibility for early detection would be training health care professionals to use screening tools and structured interviews, together with theoretical and practical mental health training.Screening questionnaires may help early detection and increase the likelihood that those who need treatment are diagnosed and properly treated. 5jetivo: Evaluar si un breve compilado de herramientas de detección para trastornos mentales específicos puede detectar los trastornos mentales y emocionales en la población general.Método: Herramientas de detección validadas para los trastornos emocionales y mentales más frecuentes han sido seleccionadas.Como criterios de selección, estas herramientas deberían mantener las propiedades psicométricas del instrumento completo con sólo uno o pocos elementos.Las herramientas seleccionadas fueron: el Patient Health Questionnaire-2 (PHQ-2), el Generalized Anxiety Disorder Scale-2 (GAD-2), el elemento 3 del Alcohol Use Disorders Identification Test (AUDIT) y tres elementos del Adolescent Psychotic-Like Symptom Screener (APSS-3).Este compilado de herramientas de detección ha sido denominado el Mini Detección para Trastornos Mentales (Mini-DTM).El estudio se dividió en dos etapas.En la primera etapa, 545 sujetos fueron entrevistados en sus residencias con la herramienta de detección Mini-DTM y COOP/WONCA-Sentimientos. En la segunda etapa, a los sujetos que aceptaron participar (230) se entrevistaron con el Mini-DTM, COOP/WONCA-Sentimientos y la entrevista diagnóstica MINI.La fiabilidad evaluar/revaluar fue calculada mediante el Coeficiente de Correlación Intraclase (ICC).La Curva ROC (Receiver Operating Characteristic) fue generada para el análisis de la validez discriminante.La validez concurrente se calculó mediante el análisis de la correlación entre el Mini-DTM y el COOP/WONCA-Sentimientos. Resultados: La administración conjunta de las herramientas de detección para trastornos específicos mostró sensibilidades que oscilaron de 0,76 a 0,88 y especificidades que oscilaron de 0,67 a 0,85.El valor del ICC para la puntuación total del Mini-DTM fue 0,78.El área bajo la curva para la detección de trastornos mentales fue 0,84, con una sensibilidad de 0,74 y especificidad de 0,76 (punto de corte ≥ 4).Conclusión: Este estudio demostró que un breve compilado de herramientas de detección para trastornos mentales específicos (Mini-DTM) puede detectar trastornos mentales y emocionales en la población general.
There is a broad literature on the use of screening instruments to identify specific mental disorders, such as the Patient Health Questionnaire-9 (PHQ-9) 6,7 for depression disorders, General Anxiety Disorder (GAD-7) 8,9 for anxiety disorders, and the Alcohol Use Disorder Test (AUDIT) 10,11 for alcohol abuse.
There are a few short screening tools for detecting multiple disorders or instruments that assess more than one MED. 3,6,7A systematic review of the literature calls "bundled screening" tools those that simultaneously assess multiple mental health disorders.There are two types of "bundled screening", which are: (1) administration of a single tool that collectively encompasses more than one health condition (individual tools that assess more than one mental or substance use disorder in a single instrument) and, (2) administration of several brief instruments at the same time for isolated health conditions (tools that assess only one mental disorder using five or fewer items). 3ong the individual tools that assess more than one mental disorder, attention should be paid to the Self Reporting Questionnaire (SRQ), 12 the WHO Well-Being Index-5 Version 1 13 and the Dartmouth Cooperative Information Project Functional Health Assessment Charts/WONCA (COOP/WONCA). 14These tools assess the general state of suffering/well-being but do not evaluate specific MEDs symptoms, except for a general reference to depression and anxiety. 15[18][19] Mental disorders show high rates of co-occurring symptoms. 3Therefore, we postulate that the simultaneous administration of a few single-disorder tools would identify not only the specific disorder of that instrument but it would also identify general MEDs.For a bundled screening would be useful that the single-disorder screening tools include the more prevalent disorders or whose early detection is important in primary health care.Anxiety and depression disorders are the most frequent MEDs in several countries, 20,21 including Brazil. 22Alcohol use and abuse is usually related to comorbidities, such as depression, suicide, use of violence, and use of tobacco and illicit drugs, 23 while psychotic disorders may be associated with significant cognitive and social impairment. 24e present study evaluated whether a short compilation of screening tools for specific disorders could identify MEDs in the general population.In that way, short screening tools for depression, anxiety, alcohol abuse, and psychotic symptoms were compiled, and the psychometric properties of the single-instruments and the aggregated scores of the four screening tools, as a multiple-disorder screen, were evaluated.

Study design
This was a clinical assay designed as a psychometric cross-sectional study.The study was divided into two phases.The first one was applying the screening tools.Second phase was to apply once again the screening tools (retest) and a diagnostic interview (Mini International Neuropsychiatric Interview -MINI).

Study location
The study was conducted from May to December 2013 in areas assisted by PHC services linked to the Ribeirão Preto Medical School, state of São Paulo, Brazil.

Participants
Sample size was calculated based on a sample calculation study for the accuracy of diagnostic tests. 25 considered that a sensitivity above 0.70 indicates a scale that could have satisfactory performance. 26In this sense, we estimated the sensitivity of 0.75 and the minimum acceptable lower confidence limit as 0.65.
With these values, the number of cases for the expected sensitivities with a 0.95 probability was 230 subjects.
A total of 545 interviews was performed in the first phase of the study to reach 230 double-assessments (first and second interviews).Data collection was concluded when 230 pairs of interviews were reached.
Participants were selected by a random sample of households in the study area, based on a digital map.All residents of the selected households who met the inclusion criteria and agreed to participate were interviewed.
The inclusion criteria included: ≥ 18 years of age, voluntary participation, and residence within the study area.Exclusion criteria were presence of severe cognitive impairment and those who did not complete both phases of the study.All participants signed a consent form.

Compilation of screening tools for specific mental disorders
We selected screening tools for the most prevalent MEDs in the general population and whose early detection would be important for PHC, such as depression, anxiety, alcohol and psychotic disorders.
The selection criteria for the adopted instruments were: i) it should screen anxiety, depression, alcohol abuse/dependency and psychotic disorder; ii) it should be well validated; and iii) it should have previously demonstrated that one or a few items maintain the psychometric properties of the complete instrument.
The instruments and their selected items are described below: a) The Patient Health Questionnaire-2 The PHQ-2 is the short version of the Patient Health Questionnaire-9 (PHQ-9), 6 composed of two items related to depressed mood and loss of interest.The PHQ-9 evaluates the presence of one of the Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition diagnostic criteria for major depressive disorder. 7The PHQ-2 is validated, including a Brazilian sample. 27

b) Generalized Anxiety Disorder Scale-2 (GAD-2)
The GAD-2 8 represents a short version of the Generalized Anxiety Disorder Scale-7 9 with two items focused on nervousness and worries.The GAD-2 is validated for the four most common anxiety disorders (generalized anxiety disorder, panic disorder, social anxiety disorder, and post-traumatic stress disorder). 8The translation of GAD-2 was validated in Portuguese. 28

c) Item 3 of the Alcohol Use Disorders Identification Test (AUDIT)
AUDIT is an instrument developed by the WHO that identifies alcohol use disorders and is widely validated, 10 including a Brazilian/Portuguese version. 11It has been demonstrated that item 3 (AUDIT-3) of this Portuguese version has psychometric properties similar to the full AUDIT. 29

d) Three items on the Adolescent Psychotic-Like Symptom Screener (APSS-3)
The APSS is a 7-item instrument that evaluates psychotic disorders.A study of its psychometric characteristics showed that only three items on this scale had adequate sensitivity, specificity, and positive and negative predictive values.These items evaluate paranoia and auditory/visual hallucinations. 30Two bilingual researchers translated the APSS-3 items independently, and the final version was decided by consensus.
Each item on the PHQ-2, GAD-2, and APSS-3 was assessed on a Likert-like scale ranging from 0 to 3. In order to keep the scale range uniform, we adapted the AUDIT-3 ranging from 0 to 3, by blending the degrees 0 (never) and 1 (less than monthly).Table 1 shows the compilation of screening tools for specific mental disorders.The sensitivity and specificity of the PHQ-2, GAD-2, AUDIT-3, and APSS-3 were calculated concerning depressive disorders (major depressive disorder and dysthymia); anxiety disorders (social phobia, obsessivecompulsive disorder, generalized anxiety disorder, and post-traumatic stress disorder); alcohol abuse or dependence; and psychotic disorders, respectively.To test whether this compilation of validated screening tools could be used as a general screening instrument for MEDs, we considered the sum of the items, ranging from 0 to 18.We called it Mini Screening for Mental Disorders (Mini-SMD).

Other Assessment Tools
The MINI has been selected as the gold standard instrument for discriminative validity analysis 31 (already validated in a Brazilian population similar to that of this study, with excellent psychometric indices). 32e Dartmouth Cooperative Information Project Functional Health Assessment Charts/WONCA (COOP/WONCA), developed by the World Association of Family Physicians is a short instrument that assesses several issues, such as physical fitness, feelings, daily and social activities, and overall health. 33t was validated in a Brazilian PHC population, and only one item on this scale (feelings) had satisfactory psychometric properties for general wellness assessment with good psychometric indices for MEDs screening. 14OP/WONCA-Feelings was used for the concurrent validity analysis and compared to the Mini-SMD.
A socioeconomic questionnaire designed to obtain gender, age, and educational and socioeconomic level was administered.Socioeconomic level was evaluated by the Brazilian Economic Classification Criteria, 34 which uses an operational criterion for classification based on existing products in households.
The original five levels were grouped into three categories.

Procedures
All interviewers were previously trained to apply the instruments (three workshops with a total of 6 hours for participants in the first phase, and five workshops with a total of 20 hours for participants in the second phase).In the first phase of the study, 20 medical students administered the Mini-SMD and the COOP/ WONCA-Feelings.Each interviewer randomly received a list of addresses that were visited.An electronic version of the Mini-SMD and a socioeconomic datasheet were developed, allowing them to use tablets or smartphones to record the data.The second phase of the study occurred at an average of 3 weeks after the first interview.After scheduling by phone, five mental health professionals (four psychologists and one occupational therapist) returned to the residences of the participants, and once again applied the Mini-SMD and COOP/WONCA-Feelings.They also applied the MINI interview (used as the outcome variable).The mental health professionals and participants did not know the results of the first phase.All participants signed an informed consent form.

Data Analysis
Mental disorders were diagnosed using the normative data of the MINI interview as the reference.
Receiver operating characteristic (ROC) curves were generated to evaluate the discriminative validity of the PHQ-2, GAD-2, AUDIT-3, and APSS-3 for screening specific diagnostic categories.The score for each scale was compared with the presence or absence of a specific diagnostic category.The diagnostic categories were: depressive disorders (major depressive disorder and dysthymia) for the PHQ-2; anxiety disorders (social phobia, obsessive-compulsive disorder, generalized anxiety disorder, and post-traumatic stress disorder) for the GAD-2; abuse or dependence of alcohol for the AUDIT-3; and psychotic disorders for the APSS.
We used the ROC curve to evaluate the discriminative validity of the Mini-SMD as a screen for all mental disorders.The sum of the scores of the six items on the Mini-SMD was compared with the presence or absence of any mental disorder diagnosed by the MINI.
Sensitivity and specificity were calculated based on the best cutoff points for each specific scale and for the total score on the Mini-SMD.
The concurrent validity of the Mini-SMD in relation to COOP/WONCA-Feelings was analyzed by Pearson's correlation analysis.
Test-retest reliability of the Mini-SMD was evaluated by comparing the scores of the first and second applications of the instrument using the intraclass correlation coefficient (ICC).Defined reference values were: small correlation (≤ 0.25), low correlation (0.26-0.49), moderate correlation (0.50-0.69), high correlation (0.70-0.89), and very high correlation (> 0.90). 35atistical analyses were performed using the SPSS version 17.0 (SPSS Inc., Chicago, IL, USA).A p-value < 0.05 was considered significant.

Ethical aspects
Study and consent terms were approved by the Research Ethics Committee of the University Hospital of the Ribeirão Preto Medical School-University of São Paulo (HCFMRP-USP) under process number 151906.

Results
Table 2 shows the characteristics for those subjects who completed both phases of the study: Participants were predominantly female, middle aged, of high socioeconomic status, and had 9-11 years of education.

Discriminative Validity
The area under the curve (AUC), the best cutoff score, and sensitivity and specificity of the PHQ-2, GAD-2, AUDIT-3, and APSS-3, are shown in Table 3.The AUC (~0.8), sensitivity and specificity values of the screening scales for specific diagnostic categories (depression, anxiety, psychotic disorder, and alcohol abuse/dependence) indicate good discrimination. 36igure 1 shows the ROC curve and sensitivity and specificity for the different total scores on the Mini-SMD.The AUC was 0.84, with a cutoff score of ≥ 4, sensitivity of 0.74 and specificity of 0.76.The cutoff score of ≥ 3 privileges, sensitivity (0.86) with specificity of 0.66 and the cutoff score of ≥ 5 privileges, specificity (0.84) with sensitivity of 0.68.

Concurrent Validity
Pearson's correlation analysis was used to assess the concurrent validity between the Mini-SMD and COOP/WONCA-Feelings.The value was r = 0.60 (p < 0.01), indicating satisfactory equivalence in the parameters evaluated between the instruments.

Reliability
The test-retest reliability of the Mini-SMD showed an ICC of 0.78 (95% confidence interval = 0.715-0.831),which was considered high.

Summary of the main findings of the study
This study showed that a short compilation of screening tools for specific disorders (Mini-SMD) can screen for MEDs in general.
The specific items on the Mini-SMD were taken from instruments already validated; 6,9,11,30 one or two items from each instrument that presented good psychometric qualities when compared with the complete instrument; that is, the PHQ-2, 37 GAD-2, 8 AUDIT-3, 29 and APSS-3. 30When these same items were compiled into a single assessment tool (Mini-SMD), they remained adequate to screen.The PHQ-2, GAD-2, AUDIT-3, and APSS-3 tracked depression disorders, anxiety disorders, alcohol abuse/dependency, and psychotic disorders, respectively, with sensitivities of 0.76-0.88 and specificities of 0.67-0.85.
In addition, this study confirms the hypothesis that a short compilation of screening tools for specific disorders can identify MEDs in general.The AUC of the total Mini-SMD score suggests adequate discriminative ability to assess multiple MEDs, 36,38 as it satisfactorily discriminated between the presence or absence of a MED.The cutoff point of 4 suggests a balance between sensitivity (0.74) and specificity (0.76).The cut-off point of 3 favors sensitivity (0.86) and the cut-off point of 5 favors specificity (0.84).
The choice between cutoff points should consider the purpose of the test.Higher sensitivity values indicate a greater power to detect positive cases.This is an important requirement for screening diseases in the PHC population, as it is desirable to detect the largest number of patients possible.Tests with high specificity values indicate a greater ability to detect truly negative cases.They are very useful to reduce the risk of detecting false-positives, at the cost of not detecting some patients.More specific tests prevent over-notification, reducing the need for diagnosis in many cases that would be negative, which could be useful in services with high demand and/or limited resources for diagnostic confirmation. 5

Comparison with existing literature
By comparing the psychometric qualities of the specific items on the Mini-SMD with the literature, it can be observed that the sensitivity and specificity of the item that evaluated alcohol use/dependence remained similar to the original instrument. 29The Mini-SMD, compiled of items that assess depression, anxiety and psychotic disorders, presented reduced psychometric properties when compared with the validation of the original studies. 8,27,30However, they were considered satisfactory, and we must consider that the psychometric qualities of each of these three screening instruments were evaluated in relation to a set of disorders in the present study.For example, the depression item indicates the possibility for major depressive disorder or dysthymia, and the item for anxiety disorders may indicate social phobia, obsessive-compulsive disorder, generalized anxiety disorder, and/or post-traumatic stress disorder.It must be highlighted the ability of the individual Mini-SMD items to assess MEDs.
The total scores on the Mini-SMD as a general screening for MEDs were consistent with those of other studies that analyzed the discriminative validity of screening instruments.A Brazilian validation study of the SRQ-20, WHO-5, and COOP-WONCA showed sensitivity and specificity values slightly higher than those obtained with the Mini-SMD. 14However, patient characteristics varied between studies, i.e., patients awaiting care at PHC services in the Azevedo-Marques 14 study and a general population sample in the Mini-SMD study.This difference was observed in the sensitivity and specificity values of the COOP-WONCA-Feelings, which was evaluated in the two studies and showed sensitivity of 0.84 and specificity of 0.86 in a previous study and 0.72 and 0.64, respectively, in the present study.
The advantage of the Mini-SMD when compared to the SRQ-20, WHO-5, and COOP/WONCA-Feelings is that the Mini-SMD is not limited to evaluating general welfare. 14The Mini-SMD is a short compilation of screening tools for specific disorders.Besides screening subjects with a MED, it also suggests one or more probable diagnostic categories.The indication of a specific symptomatology can suggest which disorder the administrator of the diagnostic interview should focus.
Reliability of the total score on the Mini-SMD is indicative of high agreement, according to Domholdt (2000). 38However, this value was lower (0.78) than that of another widely used screening tool, the SRQ-20 (0.93). 39Notably, the same raters applied tests and retests in the SRQ-20 study, which differs from the present study, in which evaluators were different (medical students in phase 1 and mental health professionals in phase 2).This might have mitigated the degree of agreement, although this condition may be closer to clinical practice, as a screening instrument will probably be applied by a variety of PHC professionals (e.g., community health workers and/or nursing technicians).A diagnostic confirmation will likely be made by other health professionals (e.g., family doctors or mental health experts).
Concurrent validity of the Mini-SMD was compared to that of COOP/WONCA-Feelings.The correlation coefficient between the instruments indicated a moderate correlation.This may be justified due to the fact that the two instruments have several specificities (e.g., Mini-SMD tackles symptoms of specific MEDs).
Administering a multiple disorder scale is useful to inform about undetected conditions in scales that evaluate only one symptomatology. 3The advantage of compiling multiple disorders in the Mini-SMD is that it brings together evaluation items that are common in several MEDs (e.g., depression and anxiety).
The Mini-SMD is a short screening measure, easy to apply, and requires little training for the administrators.It shows the characteristics that instruments of triage must offer, including speed, accessibility, good reliability, and ease of administration. 3,5

Strengths and limitations of the study
This study had some limitations.First, the screening tool was developed for an adult population (≥ 18 years) and was validated in a particular geographic region with a predominance of high educational and socioeconomic levels, which makes generalization more difficult.The second limitation is that the assessment was restricted to the evaluation of the interviewer.Reliability was measured by different evaluators and could be considered another limitation of the study, as it is necessary to consider questions related to their characteristics.Nevertheless, phase 1 interviewers underwent intense training before data collection and phase 2 interviewers were mental health professionals.

Implications for research in the area and/or the professional practice
In conclusion, the present study suggests that a short compilation of screening tools for specific mental disorders (Mini-SMD) is able to screen for MEDs in general.The Mini-SMD is a tool that potentially contributes to the early diagnosis of MEDs, evaluating the main psychiatric disorders most prevalent in the population.One of its main advantages is the fact that Mini-SMD is a short instrument of easy application.
This is an important factor in PHC, in which there is a great demand for cases and where MEDs are often not detected.Thus, diagnostic interviews would only be used in cases with a positive screening.Another advantage of this instrument is the application in scientific research, in order to identify patients in a population with a specific disorder of interest.Nevertheless, more studies are needed to evaluate different populations and its applicability in PHC.

Figure 1 .
Figure 1.Receiver operating characteristic curve from the Mini-SMD as a general screen for mental disorders.

Table 1 .
Composition of the Mini-SMD

Table 2 .
Socioeconomic characteristics of the sample

Table 3 .
Area under the receiver operating characteristic (ROC) curve (95% confidence interval), sensitivity and specificity for best cutoff scores