All Segments
Mathematical Approaches to Image Processing with Carola Schönlieb
23 segments available
Carola Schönlieb - http://www.damtp.cam.ac.uk/user/cbs31/Home.html - is an applied mathematician at the University of Cambridge. She’s also a Turing Fellow at the Alan Turing Institute - https://www.turing.ac.uk/ - and the head of the Image Analysis group - http://www.damtp.cam.ac.uk/research/cia/ - at Cambridge’s Department of Applied Mathematics and Theoretical Physics - http://www.damtp.cam.ac.uk/ In this episode we cover mathematical approaches to image processing. The YC podcast is hosted by Craig Cannon - https://twitter.com/craigcannon
Segments Timeline
From Differential Equations to Image Processing
Carola Schönlieb shares her journey from researching partial differential equations in Austria to discovering their applications in image processing. She explains how her initial work on the Cahn-Hilliard equation, which models phase separation in alloys, laid the groundwork for her later focus on image restoration techniques.
"we ought to start with a little bit of your background so what did you start researching and then what are you researching now okay so I started out my research in mathematics in Austria in Vienna whe..."
The Eye-Opening Moment in Image Restoration
Schönlieb recounts a pivotal moment during her research when she learned how the Cahn-Hilliard equation could be applied to image restoration. She draws parallels to content-aware fill techniques in Photoshop, emphasizing the mathematical foundations that predate modern image editing software.
"react to perturbations that are naturally occurring because we are in real life and things happen gosh okay yeah so it's more an understanding of the physical processes involved in you know mixture of..."
Inverse Imaging Problems Explained
In this segment, Schönlieb discusses her transition to inverse imaging problems, particularly in biomedical imaging. She explains how techniques like computer tomography (CT) involve reconstructing images from line integrals, highlighting the challenges of data limitations and noise in medical imaging.
"region so is this similar to like content-aware fill in photoshop exactly okay but this predates the Photoshop development I assume it actually does and I mean also to content-aware I feel is actually..."
Challenges in Image Reconstruction
Schönlieb elaborates on the complexities of reconstructing high-resolution images from limited data in CT scans. She discusses the balance between obtaining sufficient data and minimizing radiation exposure to patients, as well as the inherent noise in measurements that complicates the reconstruction process.
"the objects all right projections meaning in the CT sends a particular sense which is that you send x-rays through the body and what you're measuring so you're sending them through what you're measuri..."
Denoising Techniques in Image Processing
This segment focuses on the integration of denoising techniques within image reconstruction algorithms. Schönlieb explains the importance of preserving edges in images while reducing noise, discussing methods like total variation regularization and median filtering as effective strategies in image denoising.
"there is noise because these are measurements right and there is always noise and measurements and so were you doing denoising work as well at the same time it's it's it's it's integrated in the recon..."
The Art of Blurring Edges
Schönlieb contrasts her work in image processing with practical applications in Photoshop, particularly the manipulation of edges. She highlights the significance of softening edges to create a more natural appearance in edited images, showcasing the intersection of mathematics and artistic techniques in image editing.
"you want to keep and so you know their various techniques but one very successful one is total variation regularization for instance which is a technique that has been used a lot by people in image de..."
Handcrafted vs. Machine Learning Models
Schönlieb contrasts traditional handcrafted image denoising algorithms with modern machine learning approaches. She notes that while handcrafted methods are still relevant, deep neural networks are increasingly outperforming them in various scenarios. The discussion touches on the limitations of neural networks when faced with unfamiliar data, emphasizing the ongoing need for handcrafted models.
"blurring the edges oh to trick someone into thinking that it was in the same photo yeah so so in your context these these algorithms that will handle the edge sharpness are they hand coated or are usi..."
Challenges in Biomedical Imaging
This segment focuses on the complexities of image denoising in biomedical imaging, particularly in CT and MRI scans. Schönlieb explains how different scanners produce varying data and how this affects the performance of machine learning algorithms. She highlights the importance of understanding the acquisition process and the challenges posed by different imaging technologies.
"lot so while you know in certain scenarios if you know what you want to apply your image denoising approach well it's like the image net thing from like almost 10 years exactly yeah if you know that t..."
Exploring Neural Networks in Image Processing
Schönlieb discusses the exciting opportunities that neural networks present for image processing, particularly in contrast to handcrafted models. She emphasizes the need for mathematicians to explore the unknowns within these algorithms and to bring structure to neural networks, which could enhance their interpretability and stability.
"things that also people start you know more and more hopeful he started you know do some research and understanding this that even small perturbance s that are but that are consistent yeah in small di..."
Combining Handcrafted Models with Neural Networks
In this segment, Schönlieb shares her research on integrating handcrafted models with neural networks. She discusses the potential of parameter estimation to refine these models while maintaining their interpretability. The conversation highlights the balance between leveraging machine learning advancements and preserving the mathematical foundations of image processing.
"because we can prove properties about the denoising abilities of these methods of how stable they are for instance to perturbations in the images we know we know how that works so we can prove things ..."
Iterative Approaches in Image Reconstruction
Schönlieb explains the iterative methods being explored in image reconstruction, particularly in computer tomography. She discusses how feeding prior information into neural networks can enhance their performance, emphasizing the importance of understanding the underlying data structure in imaging processes.
"you have to change your model in a certain way okay but you understand why things are happening yeah if you have millions of parameters and then you know you train this algorithm to do something and t..."
Data Tagging in Machine Learning Systems
The segment concludes with a discussion on the complexities of tagging data for machine learning systems in imaging. Schönlieb addresses the challenges of providing neural networks with comprehensive data beyond the original source material, highlighting the importance of context in training effective models.
"still understand right and that you can still prove things about you still have guarantees on your solution you know you have guarantees that if you you you don't have these adversarial errors that if..."
The Challenge of Minimizing Loss in Neural Networks
This segment focuses on the challenges of minimizing loss functions in neural networks, particularly in image denoising tasks. Schönlieb explains the concept of training on finite datasets and the risks of overfitting, emphasizing the need for models to generalize to unseen data.
"computationally we're doing this in a sequential manner okay so we're not so you can do it in different ways but in a sequential manner means that you're not feeding it to 10,000 images at the same ti..."
Training Neural Networks for Image Denoising
Schönlieb discusses the process of training neural networks to denoise images, detailing how the model learns from both noisy and clean images. She highlights the importance of achieving a balance between fitting the training set and maintaining the ability to generalize to new images.
"for this training set okay and so there are different types of optimization methods that people are using but the main thing in machine learning is stochastic optimization so you don't minimize X you ..."
Collaborations in Medical Imaging
In this segment, Schönlieb shares her collaborative efforts with clinicians and medical physicists at the University Hospital in Cambridge. She describes the development of algorithms that maximize image quality from limited data, particularly in magnetic resonance tomography.
"a clean image okay and you want that to work over all the images on the training set gotcha okay okay so but let's say you have 10,000 of these images that you both know the clean and noisy image if y..."
Dynamic Imaging Challenges in Chemical Engineering
Schönlieb discusses the challenges of dynamic imaging in chemical engineering, particularly in tracking processes over time with limited data. She explains how the need for high-resolution images complicates data acquisition and reconstruction in real-time scenarios.
"really I mean there are some attempts to understand this but all of this is not really is I'm hand waving here because I can't really say anything mathematically about that but I have you pushed your ..."
Aerial Imaging and Forest Monitoring
In this segment, Schönlieb talks about her work in aerial imaging for forest health monitoring. She explains the use of various imaging techniques, including hyperspectral imaging and lidar, to analyze forest compositions and detect invasive species from airborne data.
"it means per timestamp you can't acquire as much data as if you would have you know if you just have one second for reconstructing your organ inside the body uh-huh at this particular time stamp and t..."
Lidar Technology for 3D Tree Modeling
Schönlieb introduces Lidar technology as a method for creating 3D models of trees, contrasting it with traditional imaging techniques. She shares insights from a documentary about Lidar's application in archaeology, emphasizing its efficiency in revealing hidden structures compared to conventional excavation methods.
"gotcha okay and then the other thing so this is one and then or two aerial photographs and hyperspectral imaging and then the third thing that they often acquiring are lidar measurements yeah where yo..."
Denoising Camera Footage with Advanced Techniques
This segment covers the application of mathematical approaches to denoise camera footage, particularly in security contexts. Schönlieb reflects on the prevalence of CCTV cameras and the challenges of enhancing pixelated images, drawing parallels to crime shows that depict unrealistic image enhancement.
"right like being tracked everywhere like in the UK in particular like I imagine people are looking to do this right you know it's quite funny because when you think about these crime TV shows CSI what..."
Spectral Photography and Historical Fingerprints
Schönlieb recounts a fascinating case involving spectral photography used to uncover historical fingerprints, discussing the controversy surrounding the authenticity of the findings. She highlights the intersection of technology and historical research, raising questions about legitimacy in scientific discoveries.
"know haha if you're right or wrong right just just by chance I was reading a New Yorker article from I think 2010 about this guy in Montreal allegedly finding five hundred year old fingerprints using ..."
Virtual Restoration of Historical Art
In this segment, Schönlieb shares her experience with virtual restoration during her PhD, focusing on wall frescoes in Vienna. She explains the importance of creating virtual templates for restoration, allowing conservators to visualize potential outcomes without physically altering the original artwork.
"people that it's fake yeah is it like yeah what direction are you going with with art so it kind of in Cambridge it's not well okay let me say bit more so when I again during my PhD in Vienna there wa..."
Illuminated Manuscripts and Virtual Exhibitions
Schönlieb discusses her collaboration with the Fitzwilliam Museum on illuminated manuscripts, emphasizing the fragility of these artifacts. She describes a recent exhibition that showcased both the original and virtually restored versions of a manuscript, illustrating the potential of digital restoration techniques.
"even with paintings you know if you do something if you do if you manually really you know physically restore them yeah you've done it I mean you can still maybe you know try to do I mean you're you a..."
Getting Started in Image Processing Research
In this final segment, Schönlieb offers advice for those interested in pursuing research in image processing. She recommends exploring foundational texts and online resources, particularly from UCLA, to gain insights into mathematical approaches and recent advancements in the field.
"last year there was an exhibition in the Fitzwilliam Museum which is which was called color and in this exhibition we had one piece which was in a page of an illuminated manuscript which had been alte..."