This blog post contains contributions from Dale Bodian and Jason Warner.
The most expensive AI mistakes usually come from forcing complex methods onto datasets that aren’t ready for them.
In the first part of our ‘AI in the Wild’ series, we talked about these mistakes and the importance of AI-readiness. In the second part of our series, we’ll discuss what happens once you’ve adequately prepared, with a focus on machine learning.
As bioinformatics experts, we hear people using AI as almost a direct swap for “anything done on a computer.” But that’s not the case. AI refers to models trained on data, ranging from predictive systems that recognize patterns to generative tools. And there are several types of AI, from machine learning (ML) and natural language processing (NLP) to large language models (LLMs).
Machine Learning is a subset of AI that relies on algorithms to analyze patterns in data and make inferences about new data. The more data the model receives, the stronger it gets. These stories show how ML can expand human interpretation yet still require human validation.
A Large Pharmaceutical Company Needs a Better Off-Target Detection Tool: From Standard Pipeline Build to ML-Powered Models
A large pharmaceutical company came to us dissatisfied with their existing off-target detection tool. What started as a standard pipeline build transformed into an ML-powered tool headed for a conference presentation.
The client’s existing tool lacked customization and didn’t detect patterns well for certain enzymes. Beyond missing critical information, these limitations can have real-world implications downstream, including dangerous outcomes and unwanted side effects.
A novel idea came to Diamond Age’s Jason Warner while reviewing the data–an image-based ML model. The idea was inspired by an existing tool called Deep Variant, which uses image analysis for a different purpose.
Increasing Data Confidence and Expanding Human Capacity
First, we built a standard pipeline and then added the ML layer. We converted sequencing data into images and then fed them into a Convolutional Neural Network (CNN) for classification. CNN is an ML model designed to analyze images. It’s used for facial recognition technology. The model was trained on approximately 1,500 manually curated images and built with portability and scalability in mind.
Establishing ground truth and validating the data are critical in any AI project, so we had multiple team members independently review images and compare them. The classifications from the CNN were compared to the classifications from the manual review.
It’s also important to mention that we ran both the standard pipeline and the ML model in parallel. The results from the CNN and conventional method mostly overlapped, which only increased confidence in shared hits. The CNN results agreed with the manually reviewed data and found more hits overall than the original tool.
Both the accuracy, as shown by validation, and the increased hits (compared to the standard method) are important for this application. Missing or miscalling a site could have serious consequences.
The scientists shared the technology internally and continue to use it. They were so enthusiastic about the expanded capacity that the client team presented the work at the American Society of Gene and Cell Therapy (ASGCT).
AI in Bioinformatics Requires Humans to Validate
One of the key takeaways for this large pharmaceutical company’s project was to accept nothing at face value. The technology was rigorously tested and verified by scientists. For any company in the gene editing or gene therapy space, this type of AI technology expands the team’s capacity and the existing tools.
However, as Eleanor has said many times, “AI is going to help biologists do biology way better. But it still requires biologists.”
Human validation was also the theme of our other ML project. That project also demonstrated that AI is always evolving, and so should we.
Here’s how we helped a clinical-stage biopharma company make sense of complex data, turning overwhelming outputs into insights they can actually use.
A Clinical-Stage Biopharma’s Complex Data: Improving Workflows with AI and a Novel Idea
The client’s standard bioinformatics workflow produced thousands of biological themes per analysis, many of which overlapped. Overwhelmed with information, the client felt as though they might be missing the bigger picture.
They suspected AI could help improve workflows and cut through the noise and even had an idea to make it happen.
The idea was to evaluate theme clusters and use AI to generate descriptions for them. We worked with them to refine and implement the idea, with the ultimate goal of providing an automated resource for every experiment.
Evaluating Different Clustering Methods to Find a Balance
We evaluated various clustering methods, focusing on interpretability and size. Aiming for a good balance, we targeted clusters that were meaningful enough to interpret but broad enough to simplify. To effectively summarize the results, we tested against real experimental data that the client already understood.
Once the clusters were established, we tackled the next challenge: labeling each one in plain English. We used a text-based machine learning (ML) approach to label each cluster. It surfaces the most relevant words from biological theme descriptions.
The result was an improved workflow, just as the client suspected. But that’s not where the story ends.
From Machine Learning to Large Language Models
Four years later, the technology had progressed, and the client reached out for help again. We were tasked with testing newer AI models to see if they could improve the annotations. Goodbye to ML; hello LLMs.
Using the client’s defined inputs and outputs, we implemented the LLM approach and took the following actions:
- Reviewed for consistency by running multiple tests early on.
- Checked for accuracy, asking questions like: Did it make sense? Was it adding irrelevant information?
- Validated data by sharing outputs with internal domain experts.
- Analyzed the quality of the results by comparing old annotations to new.
Through this research, we confirmed for the client that using a new AI model would improve the annotations. The LLM generates phrases that are more user-friendly than word lists. It also produces more complete and accurate annotations, with fewer meaningless outputs.
Augmenting Human Interpretation in Data Analysis
As with the first iteration of this project, our client shared the results across the organization and with leadership. Today, the LLM approach is permanently integrated into their standard workflow.
Throughout the four years of this project, we led with a good mix of science and skepticism. It can be too easy to accept plausible-looking results without scrutiny. By tapping into our team’s ability to distill complex information using AI, the client expanded their capabilities while maintaining a measured approach to AI integration. We supported the client by updating the original results from an early model with a more advanced model. This was done successfully at a sensitive time when the change in annotations could have otherwise disrupted their research.
AI is not simply a time-saving tool. When applied with the right expertise and skepticism, it augments human interpretation and expands what’s possible. For this client, it can summarize a large body of biological knowledge into a small number of units that can be examined independently, making complex results easier to navigate without losing the signal.
A Bioinformatics Expert is Imperative
At Diamond Age, we believe that in most cases, you don’t need custom AI systems to stay competitive. Competitive advantage typically comes from strong biology, good data discipline, and thoughtful application of the right methods.
Finding a long-term, quality partner is just as valuable as, if not more valuable than any AI system. At Diamond Age, that’s where we shine. We are an integrated bioinformatics partner with deep computational expertise across diverse life science environments.
Our highly regarded team helps biotech and pharmaceutical teams make confident decisions about science and AI.
Reach out to learn how we can support your company in addressing complex biological questions with urgency, adaptability, and scientific rigor.