Comprehensive Software Reviews to make better IT decisions
Google’s AI System Spots Breast Cancer More Accurately Than Radiologists
The artificial intelligence (AI) system developed by Google Health more accurately identifies breast cancer than human experts, reports The Guardian (and others). The system has been tested in the UK and the US, and the results are published in Nature, one of the most prestigious scientific journals in the world.
The system produced 5.7% fewer false positives and 9.4% fewer false negatives in the US data. (False positives are cases where the system mistakenly labelled the image as cancerous while the patient in fact did not have cancer. False negatives are the opposite: these are the cases where the system missed the tumors.)
The UK results were more modest: there were 1.2% fewer false positives and 2.7% fewer false negatives compared to human experts.
The difference in performance seems to do with how breast cancer screening works: in the US, women are seen every one to two years and the mammograms are examined by one radiologist. In the UK, the screening period is every three years, and two radiologists review the results. (A third expert usually gets involved in cases of disagreement.)
It is also worth keeping in mind when considering the results that the AI system received less information than the human experts: it had no access to patient histories, for example, or prior mammograms, notes Google Blog on this topic.
The system was developed over two years in collaboration with DeepMind (a Google subsidiary), Cancer Research UK Imperial Centre, Northwestern University, and Royal Surrey County Hospital.
While the results are quite promising, two things are important:
First, the performance of the AI system slips when tested on images from different mammogram systems. The study reported in Nature used images from machines provided by a single manufacturer. This is known as data capture bias, and it is a common problem in image recognition systems.
Google and the University of Toronto researchers write in an unrelated paper that “the performance of a model can vary depending on what instruments were used to capture the input to the model. For example, a face detection model may perform differently depending on the camera’s hardware and software, including lens, image stabilization, high dynamic range techniques, and background blurring for portrait mode. Performance may also vary across real or simulated traditional camera settings such as aperture, shutter speed and ISO. Similarly, video and audio input will be dependent on the choice of recording instruments and their settings.” And since mammograms are a type of image, factors impacting the performance of an image recognition system also apply to those interpreting mammograms.
The second point that many people make, including the creators of the Google AI system, is that the breast cancer recognition system is not meant to replace radiologists. It will rather complement them, as a “second pair of eyes and a safety net,” says Dr. Caroline Rubin, Vice President for Clinical Radiology at the Royal College of Radiologists.
And some countries simply don’t have enough radiologists – in the UK, for example, 8% of hospital posts for breast radiology are unfilled, with many older radiologists retiring and not enough new ones joining in, perhaps scared away by the AI advances such as the one reported here.
Want to Know More?
Looking to learn where else AI and ML are used in healthcare? See our blueprint Get Started With AI.
To learn about data capture bias and other machine/algorithmic biases, including data and design biases, consult our blueprint Mitigate Machine Bias.
The EU plans to invest €6 billion to build a single European data space, reports EURACTIV. The envisioned space will house personal, business, and “high-quality industrial data” and create the infrastructure for data sharing and use across businesses and nations.
“Facebook quietly acquired another UK AI startup and almost no one noticed,” reported TechCrunch on February 10. We looked into why.
In a landmark ruling, a Dutch court has ordered an immediate halt to the government’s use of an automated system for detection of welfare fraud.
Databricks, a data processing and analytics platform with a strong focus on AI and ML, has partnered with Immuta to deliver automated end-to-end data governance for AI, data science, and ML projects.
CognitiveScale has been named one of the 50 Smartest Companies of the Year 2019 by The Silicon Review. The recognition is for “transforming customer engagement and lifetime value with Artificial Intelligence.”
Facebook agreed to pay $550 million to settle a class action lawsuit with a group of users in Illinois over its use of facial recognition technology (FRT) to tag individuals in photographs, reports the BBC.
AI has been making headlines in healthcare for some time, and the current outbreak of the coronavirus in Wuhan, China, (with cases now in other parts of the world) – or, more specifically, the early warning of the outbreak – is another example.
Google founders Larry Page and Sergey Brin are stepping down as CEO and President of Alphabet, respectively. Google CEO Sundar Pichai will take over as Alphabet’s CEO. Both Page and Brin will remain actively involved as board members, shareholders, and cofounders.
I recently had an opportunity to speak with a KPMG partner in the Canadian risk consulting practice and with the head of data science for Canada about several things, including KPMG Ignite. This is what I learned.