With HealthGPT, a private LLM based on TII’s open-source Falcon 7B model, Samsung-owned HARMAN joined the generative AI competition in the healthcare industry back in October 2023. Based on Llama 2, HealthGPT Chat is the most recent version.
Given the volume of sensitive and unstructured data, the healthcare industry is one of the most difficult to use generative AI in.
“Enterprises’ worries about data security and privacy are the main driving force behind the creation of a private LLM like HealthGPT.
Ganesh also mentioned that another issue with public models is their structural limits, such as token restrictions and a lack of control over the entire processing chain. Due to the direct impact that problems with public models have on corporate performance, this lack of control may be harmful.
HARMAN entered the private model area around the time of the open-sourcing of numerous basic models in late February of last year, having recognized the seriousness of these problems. Around that same period, there was an online leak of Meta’s LLaMA 1. Models like the Llama 2 and the TII Falcon 7B were released after this. However, rather than developing an LLM internally, the business decided to take advantage of open-source models.
“We decided against developing a basic model in-house because of the enormous financial and resource commitment that would have been necessary—roughly $30 to $40 million. Rather, we concentrated on making use of already-existing open-source foundation structures,” he said.
Absence of Information
The inefficiencies in the healthcare industry, the volume of unstructured data, and the difficulties decision-makers have in concluding this data led us to conclude that the healthcare domain was the most pertinent one for our methodology, according to Ganesh.
In light of the dearth of training data, the team used clinical trial data from studies on cancer, immunological disorders, and cardiac disorders as a rich supply of publically available data, which they then used to train HealthGPT.
Still, the issue lies not just in the volume of data that is accessible but also in its low quality. Biased results may arise from models that were trained on skewed datasets. “We have systems in place to recognize and address data bias. This goes beyond only the data preparation phase; bias is carefully examined in the output as well,” he continued.
Extensive querying and rigorous testing procedures are essential to preserving the privacy and integrity of processed data.
Managing Hallucinations Cautionously
The tendency of LLMs to provide inaccurate information is nothing new. However, in vital fields like life science, where diagnosis accuracy and dependability are crucial, these kinds of hallucinations might put you in grave danger. HARMAN employs both human-in-the-loop interaction and automated procedures to mitigate these hazards.
HARMAN manages hallucinations in a variety of ways. First, HealthGPT uses barriers to control the degree of delusions. “The models’ accuracy was about 74% at first, but with constant improvement, it has increased to over 85–90%,” Ganesh stated.
Second, the model’s interface lets users adjust parameters like token count and temperature, enabling them to gauge the degree of hallucinations. Hallucinations increase in higher temperatures and decrease in lower temps. Thirdly, human oversight occurs when the team’s physician confirms AI-generated outcomes. A feedback mechanism for an ongoing loop of user input that aids in model improvement is added to this. Moreover, the RAG function enhances the reliability of the material by adding references to responses. The system also contains a benchmarking part that assesses how well the model performs in comparison to other research and models.
Nonetheless, the key to HealthGPT’s success is ethical AI. We opted for private LLM in part because it guarantees end-user control. The HealthGPT model operates within the user’s own Virtual Private Cloud (VPC) and cloud environment, as opposed to models hosted on unidentified cloud instances, Ganesh clarified, highlighting how the method enhances security and privacy as the model is tuned on the user’s data within their controlled environment.
Additionally, pre-fine-tuning tests are put in place to find abnormalities, with an emphasis on privacy via automated processes for managing Protected Health Information (PHI) and Personally Identifiable Information (PII).
Human-centric attitude is the hallmark of Harman’s approach in the healthcare industry as well as other domains. This strategy centers solutions around the individual, or decision-maker, and takes a holistic approach to problem-solving. This idea underpins all of Harman’s dealings with its diverse worldwide customer base, which ranges from businesses in the proof-of-concept (POC) phase to those with more sophisticated implementations.
What Comes Next
HealthGPT has discovered interesting proof-of-concept (POC) tales demonstrating its potential applications in various sectors, despite not having been deployed for live customers yet. A proof of concept showcases the application of HealthGPT for tailored data analysis in various industries, enabling customization for specific requirements like medication discovery in the pharmaceutical industry.
An additional narrative highlights the model’s ability to process vast amounts of structured data by utilizing it to draw insights from data from medical instruments. In the final user scenario, a pharmaceutical business integrates data from clinical trials and PubMed information to support drug discovery using the model. Among the significant clients of HARMAN services are IQVIA, Roche, and Aetrex.
For further versions, we are testing Mistral AI’s Mixtral 7B right now. Ganesh stated, “The goal is to continuously push the boundaries of auto foundation models.”
As part of its generative AI strategy, the organization is integrating a wider range of data sources while maintaining data quality and HIPAA compliance. This means that developers working with healthcare data need to receive intensive training. In addition, Ganesh and his group are aiming to include multimodal elements.
But regardless of how good the generative AI solution it develops is, HARMAN’s approach will always be defined by a human-centric worldview. This strategy centers solutions around the individual, or decision-maker, and takes a holistic approach to problem-solving.
Furthermore, in line with HARMAN’s strategic goals, there are plans to extend HealthGPT outside of the healthcare industry and into fields like manufacturing and IT management.
