Detecting duplicate patient participation in clinical trials is a major challenge because repeated patients can undermine the credibility and accuracy of the trial’s findings and result in significant health and financial risks. Developing accurate automated speaker verification (ASV) models is crucial to verify the identity of enrolled individuals and remove duplicates, but the size and quality of data influence ASV performance. However, there has been limited investigation into the factors that can affect ASV capabilities in clinical environments. In this paper, we bridge the gap by conducting analysis of how participant demographic characteristics, audio quality criteria, and severity level of Alzheimer’s disease (AD) impact the performance of ASV utilizing a dataset of speech recordings from 659 participants with varying levels of AD, obtained through multiple speech tasks. Our results indicate that ASV performance: 1) is slightly better on male speakers than on female speakers; 2) degrades for individuals who are above 70 years old; 3) is comparatively better for nonnative English speakers than for native English speakers; 4) is negatively affected by clinician interference, noisy background, and unclear participant speech; 5) tends to decrease with an increase in the severity level of AD. Our study finds that voice biometrics raise fairness concerns as certain subgroups exhibit different ASV performances owing to their inherent voice characteristics. Moreover, the performance of ASV is influenced by the quality of speech recordings, which underscores the importance of improving the data collection settings in clinical trials.
Healthcare systems are increasingly relying on automatic speaker verification (ASV) models to ensure secure and accurate identification of patients and healthcare providers, with the aim of preventing fraud, safeguarding patient privacy, and ensuring the accuracy of medical records (Upadhyay et al., 2022; Arasteh et al., 2022). Conducting large-scale clinical trials, involving numerous patients, doctors, clinics, and even different countries can pose significant challenges in identifying instances of duplicate participation, which occurs when a single individual joins the same study more than once, either at different sites or time points, leading to skewed results and undermining the validity of study findings (Irum and Salman, 2019). Shiovitz et al. (2013) discovered that as much as 7.78% of patients involved in a clinical study were duplicated across different sites. In some cases, individuals participate in multiple clinical trials concurrently in order to earn more money. When a trial enrolls an adequate number of substandard participants, it risks not meeting the primary endpoints and ultimately causing a multimillion-dollar study to fail. Pinho et al. (2021) examined the financial effect of duplicate participants on the pharmaceutical companies conducting a set of short-term study programs across psychiatric disorders including Schizophrenia, Major Depressive Disorder, and Bipolar Depression. Based on their results, enrolling ineligible subjects in the selected studies results in a loss of around $29,680,000 for the sponsor pharmaceutical company. In addition, duplicate participation results in higher placebo rates and compromised data integrity. These findings highlight the importance of addressing the duplicate participant problem and underscore the need for reliable and accurate ASV methods in healthcare systems to verify whether an unknown voice belongs to a known enrolled individual (Upadhyay et al., 2022; Arasteh et al., 2022). Cognitive impairment has been linked to a decline in vocabulary richness, syntactic complexity, and speech fluency, according to previous research (Thomas et al., 2005; Roark et al., 2011; Guinn and Habash, 2012; Meilán et al., 2012). Therefore, arXiv:2306.12444v1 [eess.AS] 20 Jun 2023 it is important to investigate whether the abnormal speech patterns exhibited by individuals with cognitive impairment can affect ASV performance. Despite this concern, there is a paucity of research examining the relationship between cognitive impairment and ASV in the existing literature. This research gap motivated us to address this issue by examining the effect of Alzheimer’s disease (AD) severity level on ASV performance. Furthermore, external factors such as participants’ demographic information (Si et al., 2021), recording environment, or data collection procedure (Woo et al., 2006; Wan, 2017) may also have an impact on ASV performance, but their impact is not well-studied in the healthcare industry. An extensive analysis of these external factors could provide valuable insights into the accuracy and reliability of ASV models and identify potential sources of bias due to differences in inherent voice characteristics among subgroups (Si et al., 2021). The purpose of this study is to investigate the effectiveness of ASV models in identifying duplicate patient participation in large-scale clinical trials, and to explore the factors that influence ASV performance in such settings. To this end, we utilize a longitudinal clinical dataset of English speech recordings obtained through multiple speech tasks from 659 participants with varying levels of AD. We employ the TitaNet model, an end-to-end deep learning text-independent ASV model pre-trained on a large volume of speech recordings of English speakers. ASV models can be classified into two groups: text-dependent (TD) and text-independent (TI). TI ASV models allow for more flexibility in the enrollment and verification phases without constraints on the speech content. When pre-trained on extensive audio datasets, TI models demonstrate a comparable level of accuracy to TD models. We evaluate the performance of TitaNet on our dataset in a zero-shot setting, achieving a 3.1% equal error rate (EER). In addition, we analyze the impact of various external factors on ASV performance, including participant demographic characteristics (i.e., age, and gender), audio quality criteria (i.e., clinician interference, background noise, participant accent, and participant clarity), as well as AD severity level. This study aims to provide valuable insights into the factors that can affect the performance of ASV models in clinical trial environments, with the goal of improving the accuracy, fairness, and reliability. Our findings indicate that voice biometrics may present fairness issues, as certain subgroups demonstrate differing speaker verification performances due to their inherent voice characteristics. In addition, the quality of speech recordings can impact ASV performance, highlighting the importance of monitoring and enhancing data collection and recording settings during clinical trials.
Read the rest here.