原始笔记链接：https://mp.weixin.qq.com/s?__biz=Mzg4MjgxMjgyMg==&mid=2247486680&idx=1&sn=edf41d4f95395d7294bc958ea68d3a68&chksm=cf51be21f826373790bc6d79bcea6eb2cb3d09bb1860bba0af0fd5e60c448ca006976503e460#rd
$\uparrow$点击上述链接即可阅读全文

IEEE TPAMI 2023 | CoIR: Compressive Implicit Radar

毫米波雷达成像论文阅读笔记： IEEE TPAMI 2023 | CoIR: Compressive Implicit Radar

Abstract

• 背景

  • mmWave radars suffer from low angular resolution due to small apertures and conventional signal processing
  • 稀疏阵列雷达 can increase aperture size while minimizing power consumption and readout bandwidth

• 方法 ：提出 Compressive Implicit Radar (CoIR)

  • 目标： high accuracy sparse radar imaging using a single radar chip

  • Leverages : CNN decoder and compressed sensing

  • 贡献：

    ✅ 设计稀疏线阵： with 5.5x fewer antennas than conventional MIMO arrays

    ✅ 提出ComDecoder ：a fully convolutional implicit neural network architecture

    ✅ 证明了CoIR的有效性 ：in both simulation and real-world experiments，且 不需要 auxiliary sensors

• 实验结果

  • improved performance over standard mmWave radars and other untrained methods on simulated and real data
  • System does not require training data or auxiliary sensors
    

1 INTRODUCTION

基于光学的Depth imaging及其缺点

• Depth imaging
  • crucial for applications like SLAM, autonomous driving, security monitoring
• Typical sensors: cameras, LiDAR
  • Cameras: high-resolution near-field depth imaging
  • LiDAR: directly outputs dense point cloud with high range/angular resolution
• Limitation : degraded performance in visually degraded environments like fog, smoke

毫米波雷达成像的优点和挑战

• 优点
  • penetrate through fog/smoke without performance degradation
• 挑战
  • low angular resolution $\delta \approx \lambda /d$
  • Increasing $d$ increases cost, power consumption and readout bandwidth

已有提高角分辨率的工作和缺陷

• 已有思路
  • Large physical arrays
  • MIMO arrays
  • SAR
  • Sensor fusion
  • Optimization with handcrafted priors
  • Deep learning
• 不足
  • Slow acquisition
  • Increased hardware complexity
  • Require large datasets
  • Limited generalizability

The proposed CoIR:

• Key observation:

  • INR provides inductive bias towards natural solutions for imaging inverse problems

• 方法

  • Leverages implicit neural representations (INRs) + compressed sensing

• 贡献

  • Designed sparse linear array with 5.5x fewer antennas
  • Proposed convolutional decoder architecture ComDecoder
  • Demonstrated improved performance over standard mmWave radars and competitive untrained methods

2 RELATED WORK

2.1 mmWave Imaging Systems

• 提高角度分辨率的方法及其缺点

  • Large physical arrays： expensive, large data volumes
  • MIMO arrays： requires many radar chips to synthesize large aperture
  • SAR techniques：slow imaging, bulky scanners
  • Sensor fusion: fails if one modality fails
  • Deep learning: requires large labeled datasets, limited generalizability

• proposed CoIR 的不同:

  • 仅使用 single chip sparse MIMO array

  • 使用 未经训练 的 神经网络

    ✅ 无需训练数据

2.2 Sparse Radar Imaging

• 稀疏雷达成像技术：

  • 1 Sparse aperture array designs
  • 2 Sparse reconstruction methods

• 1 Sparse aperture array designs

  • 使用欠奈奎斯特采样 减少 天线数

  • 优化方法：

    ✅ Convex relaxations

    ✅ Prior knowledge of number of reflectors

• 2 Sparse reconstruction methods

  • Super-resolution algorithms

    ✅ MUSIC, ESPRIT

    ✅ Require incoherent signals, known number of targets

  • Compressed sensing (CS) optimization:

    ✅ 使用稀疏先验，如 spatial sparsity, TV norm

    ✅ Challenging to design priors, scene dependent

• proposed CoIR 的不同:

  • Sparse array design

    🚩 inspired by prior work but modified due to hardware constraints

  • Uses untrained neural network

    🚩 as complex prior instead of handcrafted prior

    ✅ Neural network prior shows affinity for natural features and noise robustness

2.3 Implicit Neural Representations

• 两类INR architectures:

  • 1 Convolutional methods
  • 2 Coordinate-based MLP methods

• 1 Convolutional methods ，适合：

  • Compressed sensing
  • Image super-resolution
  • Image denoising
  • Accelerated MRI

• 2 Coordinate-based MLP methods ，适合：

  • Novel view synthesis
  • Dynamic illumination
  • PDE solutions
  • Image deconvolution

• CoIR中的ComDecoder：

  • 属于 Convolutional methods

  • tailored for sparse radar imaging

  • Key properties：

    🚩 Convolutional operations capture local spatial information

    🚩 Upsampling induces notion of resolution per layer

    🚩 Residual blocks smooth optimization and propagate information between layers

    🚩 Together these inductive biases improve performance on sparse radar imaging

  • Differences from prior works:

    ✅ CoIR uses untrained INR as complex prior for sparse radar imaging

    ✅ Prior works use INR for natural images or other imaging modalities

3 RADAR IMAGING BACKGROUND

• 发射信号模型

  • $y_{tx}(t)=e^{j2\pi (f_{0}t+\frac{1}{2}B\tau t^2)},0\le t\le T$
  • $f_0$: carrier frequency
  • $B$: chirp bandwidth
  • $T$: pulse duration

• 场景模型 （离散反射体分布）

  • $\overline{x}[n_r,n_{\theta }]\in {\mathbb{C}}^{K\times L}$
  • $n_r$: range bin index
  • $n_{\theta }$: angle bin index

• 回波信号模型

  • $z[n,m]={\sum }_{n_r=0}^{K-1}{\sum }_{n_{\theta }=0}^{L-1}\overline{x}[n_r,n_{\theta }]e^{j2\pi {\psi }_{\theta }(n_{\theta })m}{e}^{j2\pi {\psi }_r(n_r)n}+w[n,m]$
  • ${\psi }_{\theta }(n_{\theta })=\frac{f_{0}d}{c}\sin (b_{\theta }[n_{\theta }])$: spatial frequency
  • ${\psi }_r(n_r)=\frac{B}{N}\frac{2b_r[n_r]}{c}$: normalized temporal frequency
  • $w[n,m]$: noise

• Compact matrix form

  • $z=F(\overline{x})+w$

  • $F$: 2D FFT

  • Goal: recover $\overline{x}$ from under-sampled measurements $z$

4 PROPOSED METHOD

• 目标 :

  • Recover scene reflectivity $\overline{x}$ from under-sampled measurements $z$

• Measurements :

  • $z=M\odot F(\overline{x})+w$

  • $M$: binary mask implementing under-sampling

  • $w$: noise

• 困难 :

  • under-sampling causes grating lobes in sparse array PSF leading to aliasing in image

• 解决方法

  • Optimize weights of untrained deep CNN $G(C;p)$ to solve inverse problem

    ✅ $G$: untrained CNN,

    ✅ $C$: fixed noise input,

    ✅ $p$: CNN parameters

  • Optimization objective:

    🚩 $\hat{p}={\mathrm{argmin}}_p||z-M\odot F(G(C;p))|{|}_2+{\lambda }_L||G(C;p)|{|}_1$

    🚩 ${\lambda }_L$: sparsity regularization strength

  • Key observation:

    🚩 INR provides inductive bias towards natural solutions for imaging inverse problems

• 优点 :

  • CNN architecture has high impedance to noise

  • Learned solution balances fitting salient features and suppressing artifacts

4.1 Sparse Aperture Design

• 目标
  • Design a sparse MIMO virtual array that improves imaging accuracy when used with ComDecoder
• 设计准测 (metrics)
  • PSF main lobe half-power beamwidth (HPBW)
  • Peak sidelobe level (SLL)
  • Presence of grating lobes
• 硬件约束
  • Max aperture 86λ/2
  • Limited to 4 TX and 4 RX due to commercial single radar chip
• 设计方法
  • Select 4-element minimum redundancy array for RX to avoid grating lobes
  • Grid search over TX positions to minimize SLL
• 比较对象（baselines）
  • Full: Ideal full Nyquist sampled array
  • Sub-apt: Largest Nyquist sampled MIMO array given constraints
  • Sub-samp: Largest sub-Nyquist array given constraints
• 设计结果
  • RX: [0, 1, 4, 6] λ/2
  • TX: [0, 46, 59, 79] λ/2
  • Gives 5.5x fewer antennas than conventional MIMO array

• 优点 :
  • Avoids grating lobes
  • Minimizes HPBW
  • Minimizes SLL
  • Satisfies hardware constraints

4.2 Neural Network Architecture

提出 ComDecoder：convolutional decoder architecture

• ComDecoder :

  • Maps latent variables C to image G(C;p)
  • 优化：Parameters p optimized to reconstruct image

• 网络结构 :

  • Series of upsampling and residual convolution blocks
  • Use SiLU activation instead of ReLU
  • No upsampling in last layer, uses 1x1 conv instead

• 超参数 :

  • 6 layers (including last layer)
  • 128 channels per layer
  • Fixed input C sampled from uniform distribution

• 优化过程 :

  • Update network weights p using backpropagation and Adam
  • Takes <50 s per 256x256 image using 2000 iterations

• 优点 :

  • SiLU increased expressivity over ReLU
  • Upsampling reinforces multi-resolution nature
  • Residual blocks enable information flow between layers
  • Inductive biases improve performance on sparse radar imaging

5 COMPETING UNTRAINED METHODS

7个baselines: Compared CoIR against several untrained methods

• 1 Delay-and-Sum (DAS)

  • Standard beamforming method

• 2 Sparse DAS

  • DAS with under-sampled measurements

• 3 Gradient Descent with L1 Regularization (GD+L1 Reg)

  • Directly optimizes reflectivity distribution with sparsity prior

• 4 Implicit Neural Representations:

  • 4.1 INR-ReLU

    ✅ MLP-based, uses Fourier feature encoding

  • 4.2 SIREN

    ✅ MLP-based, uses sinusoidal activation functions

• 5 Deep Image Prior (DIP)

  • U-Net style convolutional autoencoder

• 6 DeepDecoder

  • Decoder-only convolutional network

• 7 ConvDecoder

  • Variant of DeepDecoder with some modifications

6 SIMULATION RESULTS

在仿真数据上评估所提出的CoIR

• 仿真数据生成:

  • Synthesizes radar data cube from 2D reflectivity images
  • Uses LiDAR point clouds to generate realistic reflectivity distributions

• 评估标准:

  • PSNR, SSIM, MAE between reconstruction and ground truth image

• 实验:

  • 1 Vary SNR from 35dB to 11dB

    ✅ ComDecoder gave superior PSNR over all methods at all SNRs

    ✅ ComDecoder and DIP gave comparable SSIM

    ✅ ComDecoder and DIP gave lowest MAE
    

  • 2 Visualize reconstructions at 19dB SNR

    ✅ ComDecoder gave most accurate recovery of extended reflectors

    ✅ Other CNN methods also improved over Sparse DAS

    ✅ SIREN struggled to distinguish clutter and true reflectors
    

  • 3 Additional analyses:

    ✅ Compared different CNN decoder architectures

    ✅ Evaluated computational complexity (in supplementary)

• 总结：

  • ComDecoder 在 simulated data 上 SOTA

7 EXPERIMENTAL RESULTS

在真实采集的Coloradar dataset上评估所有方法

• Radar system:

  • 77 GHz FMCW with 1.282 GHz bandwidth

  • 86λ/2 uniform linear array

• Metrics :

  • 与 full array DAS reconstruction 进行对比

• Experiments :

  • 1 不同场景下的重建效果

    ✅ ComDecoder accurately recovered dominant features

    ✅ DIP also performed well but more artifacts than ComDecoder

    ✅ SIREN struggled in indoor scene due to noise
    

  • 2 Evaluate 鲁棒性 across multiple outdoor scenes

    ✅ ComDecoder gave high fidelity reconstructions closest to DAS

    ✅ SIREN fit strong reflectors but also artifacts

    ✅ GD+L1 located dominant reflectors but artifacts remained

    ✅ DIP performed well but more artifacts than ComDecoder

• 总结：
  • ComDecoder 在 real data 上 SOTA
  • 显著好于其他untrained methods

8 DISCUSSION & LIMITATIONS

Limitations

• 1 Assume static scene in forward model
  • Cannot handle moving objects
• 2 Single bounce scattering model may not match real-world
• 3 Slow optimization time (tens of seconds)
  • Explore different initialization strategies

Future work

• 1 Demonstrated 2D range-angle slices due to linear array
  • 2D array needed for full 3D, but increases complexity
• 2 CoIR could benefit other array-based imaging modalities:
  • SAR, ultrasound, etc.

9 CONCLUSION

Proposed CoIR

• 1 Designed sparse linear array with 5.5x fewer antennas

• 2 Proposed convolutional decoder architecture ComDecoder

• 3 Demonstrated superior performance on simulated and real mmWave radar data