Artificial Intelligence in Practice

Advantages, Limitations, and Applications

When asking potential technologies, vendors, or features about what AI-powered means, the following terms may be used. This section provides a basic context for neurology practices to understand advantages and limitations when a given application is described as AI-powered. Interested neurologists, researchers, trainees or administrators are encouraged to review other sections of the AAN’s AI Resources Center to dive deeper.

Machine Learning (ML)

Advantages

Can discover and utilize hidden patterns and insights not obvious to users with traditional data tools. Most useful with good quality and generalizable training data with specific problems to solve.

Limitations

Results are limited by the quality and source of examples used to train the model. The ML model may overfit (learn) biases and idiosyncrasies in training data and may perform less well with data that differs from the training data.

Applications

ML models may find patterns in clinical data to predict a neurologic diagnosis, disease course, response to treatment, hospitalized length of stay or other metric; they may also find patterns in operational data to identify patterns of no-shows, patient feedback, appointment lengths, or productivity that can help update office workflows.

Deep Learning (neural networks)

Advantages

Ability to handle large and complex datasets (e.g., images, speech, EEG waveforms, medical records). Can automatically learn features of relevance (e.g., identify ventricles vs cyst in a brain scan or detection of spikes from an EEG). Deep learning has some potential to generalize with related data with similar features.

Limitations

Deep learning has similar limitations to ML. Deep learning models can be harder to interpret or explain than traditional ML models and may learn spurious, irrelevant, or unwanted patterns (https://pubmed.ncbi.nlm.nih.gov/38198251/).

Applications

A few clinical applications approved by the FDA apply deep learning to narrow, specific use cases such as image analysis (e.g., detection of aneurysms from MRI scans or EEG analysis and seizure detection). Additionally, speech-to-text systems are commonly used for transcription and are starting to be tested for voice commands.

Natural language processing (NLP)

Advantages

NLP is a user friendly way to interact with than computers. It unlocks the ability to give speech commands and interpret free text in documentation. NLP’s performance increases with more specific use cases (e.g., phone tree menu of choices vs. free-flowing conversation).

Limitations

Performance is dependent on training datasets, and biases in training data and language may become part of the NLP model. Language accents, styles of speaking, idioms, abbreviations, misspellings, ambiguities remain frequent challenges. NLP models trained with neurology domains may do better than general medical models.

Applications

NLP is widely used for speech-to-text processing, chatbots, automated phone trees, and virtual translators. Emerging uses include voice commands to interact with an EHR and summarization of volumes of documents such as scientific papers or medical records. Research uses are leveraging NLP to extract key data from free text notes to support quality improvement, clinical decision making, and billing.

Generative AI

Advantages

Highly engaging and user friendly, leveraging natural speech or text inputs and responses that are highly readable, usually relevant, and can be easily conversational in nature.

Limitations

LLMs may generate and create false responses, known as hallucinations (e.g., bibliography references that do not exist). (Ge 2023). LLMs are also probabilistic, so reliability and consistency of responses add to concerns about accuracy of responses.

Applications

Not generally validated for practical use, though interest is very high and specific uses cases are increasingly seen in pilot and research stages. Utilization in the clinical space requires careful vetting, assessment, and validation given the risk of hallucinations.

Applications that generate medical notes based on a recorded transcript; create discharge/after-visit summaries or translations for patients; assist with prior authorization requests and appeals; and draft responses to patient questions are currently being tested.

Generative AI can also be utilized to review and summarize large medical records, which, if confirmed as accurate and reliable, has multiple benefits including reduced administrative burden, quicker access to care for patient, and increased clinical trial matching and recruitment.

Clinical algorithms

Advantages

Widely used in both simple forms and increasingly with machine learning based algorithms. When used with deep learning algorithms, large complex input data sets such as genetic information, abundant EHR data in a hospitalization, free text notes, or medical images can be processed.

Limitations

Simple algorithms are understandable and consistent but they are inflexible. More complex algorithms using deep learning and large language models are subject to limitations of the training dataset and are increasingly probabilistic (i.e., cannot predict output), requiring careful validation.

Applications

In the practice of medicine, these algorithms can be used to consider inputs like symptoms and medical history to help a physician decide which test to order or treatment to prescribe based on medical guidelines. Models can identify early warning signs of patient deterioration, predict medication compliance, and forecast hospital resource utilization. In use now, EHR systems and researchers are integrating a variety of predictive modeling algorithms to predict occurrence of sepsis infection and patient readmission.