Finding the Information ‘Needle in a Haystack’

Digging through volumes of pharmaceutical data in any form, be that of lab reports, experimental results, clinical trial reports, scientific publications, patent filings, to even emails is a gargantuan task. The data may deal with diseases, genes, drugs, active agents and mechanisms of action and can be textual, structured data like molecule structures, formulae, SAS data sets from clinical trials, curves, diagrams, and more. When put together, all of this information can be retained in hundreds of millions of documents and billions of database records.

Compounding this volume of information are the billions of database records from internal and external trade sources that may be related to a life sciences project. World-renowned pharmaceutical and chemical companies, such as AstraZeneca and Biogen, rely on search and analytic technology to solve real world problems by providing a single point of access to information extracted from all these data sources. Search and Analytics solutions specialize in finding that data ‘needle in a haystack.’

To find that ‘needle,’ advanced organizations turn to the power of Search Based Applications (SBA). Imagine that your company has an idea, but the data required to obtain meaningful results is spread across multiple business units or even enterprises in different formats. What if you could quickly develop an application that could bring all the data together and allow you to create search queries to find the data that you require? And more importantly, what if the application could be built in a month’s time using highly advanced natural language processing that allows you to make sense of the complex information in scientific publications or clinical trial reports using artificial intelligence and machine learning from Spark; statistical analysis of structured data; and above all, combined statistical and linguistic/semantic analysis?

Advanced search and analytics platforms index all the structured and unstructured data sources and create a semantically enriched index, optimized for performance in dealing with user search queries. In fact, some search and analytics solutions even offer as many as 140 smart connectors, ‘out of the box,’ that can seamlessly connect multiple sources of data. , These companies integrate your company’s and industry specific dictionaries and ontologies allowing the information to be integrated and indexed, putting your specific knowledge ‘under the hood’ of one platform – making it an intelligent partner for anyone searching for relevant information for his/her subject.

The Pharma industry is starting to efficiently leverage SBAs in multiple ways. A major benefit of SBAs is that it allows companies to find subject experts. A company can quickly get a dynamically calculated list of people with their respective domains of expertise related to your question/subject. The results correspond to an ‘Expert Graph’ calculated from the ‘footprint’ experts leave in texts and data.

To make the most of your volumes of data, look for search and analytics solutions that will also allow you to build on your network of experts outside of your own internal resources. For example, you can extend your search for experts on a particular subject by ploughing through massive amounts of data, in particular scientific publications, publicly available trial reports, patent filings, and reports from previous collaboration projects, in order to identify the best available experts – “Key Opinion Leaders” – and the organizations they belong to.

It’s also valuable to use a search and analytics solution to access the latest scientific information in your field with automatic alerts. This is extremely valuable because it allows you to discover research trends in your field and potentially monitor the competition. Such SBAs may easily cover as many as 110 million documents: all accessible external data sources including publications, Embase, Medline, Scopus, clinical trial reports and your company’s internal data sources via SharePoint, Documentum, etc.

Clinical trials going over many years generate millions of SAS datasets and billions of rows per drugs and studies. Over time, Biostatisticians face tremendous challenges performing their analysis with the right datasets. It is difficult to get a comprehensive list of patients having certain diseases within trials on a drug; ensuring completeness of results is nearly impossible with traditional tools and processes.

With powerful search and analytics indexing technology, scientists are able to search complex content with very precise criteria. They can retrieve subjects that have shown certain diseases by specifying exact or fuzzy values on the AELLT variable for instance. The scientists can then filter based on additional criteria like age and can combine about 900 CDISC variables and add any specific variables you may have. They can search datasets based on the structure and metadata. Possibly even more important, the scientists can even search across many drugs and studies, merging current data silos. In total, pharmaceutical company scientists are transforming their growing clinical trials data to a valuable asset that can be searched in real time.

Taken all together, an advanced search and analytics platform is able to leverage data indexing and analytics technology and enable organizations to create their own Search Based Applications. In doing so, companies are able to:

1. Accelerate research and time-to-market of drugs
2. Quickly find experts on a particular subject
3. Find key opinion leaders and R&D; cooperation partners
4. Push latest news on a subject to partners
5. Monitor publications on particular subjects

The result? That ‘Needle in the Haystack’ just became much easier to find!