Tu–Th 3:30–4:45pm · Iribe Center 1207
This graduate course explores machine learning techniques for physical sensing and perception in real‑world environments: beamforming and DoA; radar/SAR and spectral analysis; localization and sensor fusion (with IMUs); volumetric rendering (NeRF, Gaussian Splatting); room/underwater acoustics; and LLM‑grounded multimodal perception. Learning is through lectures, paper discussions, hands‑on assignments, and a team project.
| Component | Weight | Notes |
|---|---|---|
| Term Project (x1) | 50% | One semester-long project completed in groups of two students. Evaluation will be based on the project report (30%) and the end-semester project presentation (20%). |
| Paper presentation (x1) | 20% | One in-class presentation of a scientific article. |
| Programming Assignment (x1) | 15% | One programming assignment in a group of two students. |
| MCQ Quiz (x2) | 10% | Two in-class multiple choice questions quizzes. |
| Attendance | 5% | Attendance and participation in class discussions. |
See the full, detailed syllabus with readings and applications.
| Topics | Date | Materials |
|---|---|---|
| 0 Course overview | Sep 2 | Class Slides |
| 1_1 Signals and sepectral analysis basics | Sep 4 | Class Slides Working principles of sensors (refer slides) Signals - definition and types (refer slides) Complex model for signals (Quadrature signals) [Optional] (Multivariable Vector-Valued Functions) |
| 1_2 Frequency Domain View | Sep 9 | Class Slides Spectrogram - Time-Frequency view of signals (refer slides) The idea of sampling (refer slides) Signals as vectors and change in bases in vector spaces. (refer slides) [Reference] (3Blue1Brown Tutorial on Change of basis) |
| 1_3 Frequency Domain View | Sep 16 | Class Slides |
| 1_4 Frequency in learning | Sep 23 | Class Slides |
| 1_5 Frequency in learning | Sep 25 | Class Slides |
| 1_6 Frequency in learning | Sep 30 | Class Slides |
| 2_1 Inverse Problems in Sensing/Perception | Oct 2 | Class Slides |
| Submit project proposal. Due Oct 16th, 11:59 pm | ||
| Programming homework released. Due Oct 20th, 11:59 pm | ||
| 2_2 Inverse Problems: Spatial Perception | Oct 7 | Class Slides |
| 2_3 Inverse Problems: Spatial Perception | Oct 9 | Class Slides |
| Fall break | Oct 14 | |
| 2_4 Inverse Problems: Spatial Perception: DoA | Oct 16 | Class Slides |
| 2_5 Inverse Problems: Spatial Perception: DoA Subspace algorithm | Oct 21 | Class Slides |
| 2_6 Inverse Problems: Spatial Perception | Oct 23 | Class Slides |
| 3_1 Active Wave Imaging | Oct 28 | Class Slides |
| Recap | Oct 30 | In-class Quiz (Open books/notes/computer, but closed Internet/AI) |
| 3_2 Active Wave Imaging | Nov 4 | Class Slides Student presentation: Mahima Beltur & Sungjin Hwang Paper: VINet: Visual-Inertial Odometry as a Sequence-to-Sequence Learning Problem |
| 3_3 Active Wave Imaging | Nov 6 | Class Slides Student presentation: Danyil Butkovskyi & Daniel Pitzele Paper: SoundSpaces: Audio-Visual Navigation in 3D Environments |
| 4_1 Volumetric Sensing | Nov 11 | Class Slides Student presentation: Tingfang Cheng & Eileen Yuan Paper: TrOCR: Transformer-Based Optical Character Recognition with Pre-trained Models |
| 4_2 Volumetric Sensing | Nov 13 | Class Slides Student presentation: Carwyn Collinsworth & Kalonji Harrington Paper: Adversarial attacks on deep-learning-based radar range profile target recognition |
| 4_3 Volumetric Sensing | Nov 18 | Class Slides Student presentation: Aritrik Ghosh & Daniel Lian Paper: DeepSense: Fast Wideband Spectrum Sensing Through Real-Time In-the-Loop Deep Learning |
| 4_4 Volumetric Sensing | Nov 20 | Class Slides Student presentation: Max Li Paper: Deep Non-Line-Of-Sight Reconstruction |
| 4_5 Volumetric Sensing | Nov 25 | Class Slides Student presentation: Alec Luterman & Alexander Teacu Paper: PointPainting: Sequential Fusion for 3D Object Detection |
| Thanksgiving break | Nov 27 | |
| NO CLASS | Dec 2 | |
| Dec 4 | Student presentation-1: Aditya Shelke & Bruce Gao Paper 1: Through-Wall Human Pose Estimation Using Radio Signals Student presentation-2: Garner Thompson Paper 2: RadarSplat: Radar Gaussian Splatting for High-Fidelity Data Synthesis and 3D Reconstruction of Autonomous Driving Scenes | |
| Dec 9 | In-class Quiz (Open books/notes/computer, but closed Internet/AI) Student presentation-1: Jincheng Yang & Phuc Nguyen Paper 1: DROID-SLAM: Deep Visual SLAM for Monocular, Stereo, and RGB-D Cameras Student presentation-2: Yisheng Zhang & Yangsheng Xu Paper 2: Deep Learning for Size-Agnostic Inverse Design of Random-Network 3D Printed Mechanical Metamaterials | |
| Final Project Presentations | Dec 11 | |
| Submit Project Report on ELMS | Dec 12 | Project report should be 8 pages (US letter size) max. (excluding references). 10 point Times font. Should contain the names of the team members. Submit your project report in pdf format. |