[] Internship at ASTRI - 3D reconstruction and Point Cloud processing
My ASTRI Internship: Pioneering a Diffusion-Based Algorithm for 3D Stomach Reconstruction
=================================================================
Hey everyone, it's Zhengyan here. As some of you know, I spent my summer interning at the Hong Kong Applied Science and Technology Research Institute (ASTRI) in their Advanced Electronics Components and Systems department. It was an incredible experience that combined cutting-edge tech, medical applications, and a bit of physics-inspired creativity. Today, I want to share what I worked on—a project aimed at improving diagnostic accuracy for stomach diseases using a magnetic capsule endoscope. We even submitted a paper to CVPR 2026 based on the algorithm I developed! Let's dive in.
The Problem: Scanning the Stomach with a Tiny Capsule
The project revolved around a swallowable capsule that scans the stomach for diseases. My mentor, LiYuan Qiu—a PhD from Tsinghua University—was leading the effort. The capsule is controlled externally via magnetic fields, capturing images as it moves around inside the stomach. Sounds cool, right? But there's a catch: the manipulation isn't precise, and parts of the stomach often get missed during scans. This leads to incomplete diagnoses, which is a big deal in medical imaging.
Traditional methods to detect unscanned regions rely on comparing capsule images to a database of "feature" images (like using CNNs or transformers for similarity matching). These approaches are biased and inaccurate because stomachs vary in shape, and the images have limited visual cues. LiYuan's innovative idea was to use state-of-the-art 3D reconstruction techniques (variants of TSDF and Gaussian splatting) to rebuild the stomach in 3D. Unscanned areas would appear as holes or missing surfaces in the reconstructed mesh—much more precise than approximations.
My role? Help build this end-to-end algorithm. It broke down into a few key parts.
Part 1: Data Preparation and Pipeline Design
First up was gathering high-quality data for training the reconstruction models. I extracted stomach meshes from CT scan files using mass density filtering and manual cropping in tools like Blender. Then, I cleaned them up—removing noise, fixing topologies—and augmented the dataset by creating variations of each mesh. It was tedious (think hours in Blender), but essential. I ended up with over a dozen polished stomach models, documented the whole workflow for future interns, and set up a data pipeline. These meshes served as ground truth for LiYuan's deep learning models.
Part 2: Tackling the Messy Reconstruction Output
The real challenge came next: processing the output from the 3D reconstruction. The models spat out double-layered, poor-quality meshes—think noisy point clouds with inner and outer shells connected at edges, full of holes and artifacts. This "double surface artifact" is common in TSDF fusion due to truncation issues, making the mesh unusable for analysis.
I started with conventional methods:
Poisson Surface Reconstruction: Sometimes worked okay but flipped normals, merged layers erratically, and created imaginary shapes.
Ball-Pivoting Algorithm (BPA): Failed to distinguish inner/outer surfaces, resulting in incomplete, unsmooth meshes with weird bridges.
DBScan, Alpha Shapes, Marching Cubes, etc.: All struggled with the double layers, holes, and non-uniform densities. They either bridged gaps incorrectly or produced artifacts.
After weeks of trial and error (including custom stats-based approaches like k-nearest neighbors and PCA for layer separation), nothing stuck. Existing algorithms just weren't designed for non-watertight, double-layered point clouds like ours.
That's when LiYuan and I brainstormed a wild idea: simulate particle diffusion, like gas filling a container. In physics, diffusion ensures particles collide with every inner surface in a closed space through random motion. We adapted this to "fill" the mesh with virtual particles, tracking collisions to extract only the inner layer points. Particles escaping through holes would avoid tagging outer surfaces.
I coded this "Diffusion-Scatter" algorithm from scratch. It starts with spawn points inside the point cloud, simulates random walks, and marks collided points as inner. For non-watertight models (with holes), we added an escape boundary to capture leaking particles. The result? A clean, single-layer point cloud in about 10 seconds for 40,000 points—robust across complex geometries, sizes, and even open topologies.
One key insight: "Inner" vs. "outer" is statistical. In hole-free meshes, it extracts perfectly; with holes, it takes longer but still nails it with minimal errors (easily fixed as outliers).
Results and Impact
The algorithm crushed the competition in tests. For example, on stomach point clouds:
- Our method: Clean, single-layer mesh preserving holes and smooth geometries.
- Poisson: Inconsistent normals, double layers, weird merges.
- BPA: Incomplete, bridged surfaces.
- Clustering + Poisson: Total mess.
We were so excited that we decided to publish. ASTRI funded the submission after a demo to the department manager. The paper, "Diffusion-Driven Inter-Outer Surface Separation for Non-Watertight Point Clouds," is now under review at CVPR 2026. It highlights applications in medical imaging, robotics, AR/VR, and more.
I presented my work at the internship exhibition, and saying goodbye was bittersweet. Huge thanks to LiYuan for her guidance—she helped me turn a tough problem into a potential publication.
Reflections
This internship taught me that innovation often comes from blending domains (like physics and computer vision). It was laborious at times, but seeing the algorithm turn chaos into clarity was rewarding. If you're into 3D reconstruction or medical tech, hit me up—happy to chat code or collabs!
Stay tuned for updates on the paper. Fingers crossed for CVPR!
- Zhengyan