內容大鋼
我們的物理世界是建立在三維空間之中的。為了創造能夠理解並與之交互的技術，我們的數據也必須是三維的。這本實用指南為數據科學家、工程師、研究人員提供了使用Python處理3D數據的實踐方法。從3D重建到3D深度學習技術，你將學習如何從海量數據集中提取有價值的洞察，包括點雲、體素、3D CAD模型、網格、圖像等。
Florent Poux博士將幫助你借助前沿演算法和空間AI模型的潛力，開發以自動化為核心的生產就緒系統（production-ready system）。通過本書，你將習得3D數據科學的知識與代碼，實現以下目標：
理解3D數據的核心概念和表示方法
使用強大的Python庫載入、操作、分析和可視化3D數據
應用先進的AI演算法進行3D模式識別（包括監督與非監督方法）
使用3D重建技術生成3D數據集
實現自動化3D建模與生成式AI工作流
探索在電腦視覺/圖形、地理空間情報、科學計算、機器人技術、自動駕駛等領域的實際應用
構建服務於空間AI解決方案的精準數字環境
目錄
Table of Contents
Foreword
Preface
1. Introduction to 3D Data Science
3D Data Science in Brief
Dimensions and 3D Data Science
Spatial AI: From Reality to Virtuality
3D Data: Fundamental Building Blocks
Geometry, Topology, and Semantics
Integrating Geometry, Topology, and Semantics
Introduction to 3D Point Clouds
The 3D Data Science Modular Workflow
Data Acquisition
Preprocessing
Registration
3D Data Classification (Semantic Injection)
Structuration/Modeling
3D Data Analysis
3D Data Visualization
Application (Software) Development
The Case for Automation
Workflow Challenges in 3D Data Science
3D Data Science in the Industry
Summary
2. Resources and Software Essentials
Fundamental Resources
Mathematics
Computer Science
3D Data Expertise
Artificial Intelligence for 3D
Hardware Recommendations for 3D
Local 3D Development
Cloud Computing
Essential Software and Tools for 3D
3D Reconstruction Software
3D Data Processing Software
3D Visualization Software
Summary
3. 3D Python and 3D Data Setup
3D Python Setup and Libraries
Choice of OS
Environment Setup
Base Python Libraries
3D Python Libraries
The Python IDE
Creating a 3D Python Program
Importing 3D Data in Python
Extracting Specific Attributes
Conducting Attribute-based Data Analysis
3D Data Visualization and Export
3D Reconstruction Methods
Real-World 3D Reconstruction (Sensor-Based)
Creative 3D Reconstruction
3D Dataset: Curation
3D Data from Image-based Reconstruction
Multimodal Web Scraping
Summary
4. 3D Data Representation and Structuration
3D Data Representations
3D Point Clouds
Image-based Representations
Volumetric (Voxel) Models
High-level 3D Data Representation
3D Surface Models
3D Data Canonical Link
Mesh to Point Cloud
Voxel to Point Cloud
Raster to Point Cloud
3D Data Structures: k-d Trees, Octrees, BVH
k-d Trees
Octrees
File Organization
Summary
5. Developing a Multimodal 3D Viewer with Python
3D Python and Code Setup
3D Data Curation
3D Data Preparation
Initial Profiling
3D Data Downsampling
Data Preprocessing
3D Data Visualization
Multimodal 3D Experience
Point of Interest Query
Manual Boundary Selection
Find High and Low Points
Point Cloud Voxelization
Built Coverage Extraction
Summary
6. Point Cloud Data Engineering
Fundamentals
Initial Preprocessing
Feature Extraction Fundamentals
Strategies for Point Cloud Feature Extraction
Global Feature Extraction
Local Feature Extraction
Principal Component Analysis
Python and Data Preparation
Cluster Identification with pandas
3D Data Normalization
Extracting the Principal Components
3D Visualization of PCA
3D Data Registration: Unifying Perspectives
3D Data Registration Fundamentals
Registration Initialization
Coarse Registration
Iterative Closest Point
Fine Registration: ICP
Summary
7. Building 3D Analytical Apps
3D Project Environment Preparation
Gathering Datasets
Python and Environment Setup
3D Data Fundamentals with PyVista
3D Data Structure Creation (KDTree)
Covariance Matrix, Eigenvalues, and Eigenvectors
Planarity, Linearity, Omnivariance, Verticality, Normals
Neighborhood Definition and Selection
Automation and Scaling
Interactive Thresholding
3D Data Results Export
Summary
8. 3D Data Analysis
Types of 3D Data Analysis
3D Descriptive Data Analysis
3D Exploratory Data Analysis
3D Predictive Data Analysis
3D Prescriptive Data Analysis
Additional Considerations
3D Data Analytical Tools
Environment and Data Preparation
Metadata Analysis and Data Profiling
Geometry and Shape Analysis
Statistical Analysis
Attribute Analysis
3D Diagnostic Tools
3D Deviation Analysis: Planar Case
3D Deviation Analysis: Mesh Case
Summary
9. 3D Shape Recognition
RANSAC from Scratch: 3D Planar Shape Recognition
RANSAC
Data and Environment Setup
Geometric Model Selection
3D Shape Fitting
Iteration and Function Definition
Application 1: RANSAC for Segmentation Tasks
Application 2: RANSAC for Analytical Tasks
Application 3: RANSAC for Modeling Tasks
Region Growing for 3D Shape Detection
Region Growing Principles
Region Growing: Real-World Setup
Region Growing: Implementation
A Hybrid Approach: RANSAC and Region Growing
Summary
10. 3D Modeling: Advanced Techniques
High-Fidelity Meshing
General Overview of High-Fidelity 3D Meshes
The Mission
Data Preparation
Choose a Meshing Strategy
Other 3D Meshing Strategies
3D Meshing with Python
Levels of Detail Creation
Visualization and Software
3D Voxels and Voxelization
Python Environment Initialization
Loading the Data
Creating the Voxel Grid
Generating the Voxel Cubes (3D Meshes)
Export the Mesh Object (.ply or.obj)
Parametric Modeling
CadQuery and Environment Setup
I/O for Parametric Models: 2D (DXF) and 3D (STL)
Parametric Modeling Techniques
The Boolean Operations
Modeling Various Pieces
Conclusion
Monocular Image-based 3D Modeling: Depth Estimation and Reconstruction
Setting Up the Environment and Installing the Libraries
Gathering a Dataset
Image Preprocessing and Model Setup
Depth Estimation Predictions from the Model
Point Cloud Generation
Defining the Camera Intrinsics
3D Modeling: 3D Point Cloud to Mesh
Summary
11. 3D Building Reconstruction from LiDAR Data
Phase 1: 3D Python Setup
Project Environment Setup
Project Notebook Setup
Phase 2: Data Preparation
Aerial LiDAR Data Curation
Aerial LiDAR Data Preprocessing
Phase 3: Experiments
Unsupervised Point Cloud Segmentation
3D House Segment Isolation
2D Building Footprint Extraction
Semantic and Attribute Extraction
2D to 3D Vectors
3D Model Creation: Mesh
Phase 4: Automation and Scaling
Summary
12. 3D Machine Learning: Clustering
Clustering for Unsupervised Segmentation
Clustering Fundamentals
Clustering Representativity
Types of Clustering Algorithms
k-Means Clustering
k-Means: Workflow Definition
3D Python Context Definition
LiDAR Data Preprocessing
k-Means Implementation
DBSCAN for Unsupervised Segmentation
DBSCAN Principles
The Strategy
Experimental Setup
3D Planar Shape Recognition with RANSAC
DBSCAN for 3D Point Cloud Segmentation
The Multi-RANSAC Framework
Multi-RANSAC Refinement with DBSCAN
DBSCAN Refinement
DBSCAN Versus k-Means
Summary
13. Graphs and Foundation Models for Unsupervised Segmentation
Connectivity-based Clustering
The Mission Brief
Core Principles
Step 1: Environment Setup
Step 2: Graph Theory for 3D Clustering
Step 3: Graph Analytics
Step 4: Plotting Graphs (Optional)
Step 5: Connected Components for Point Clouds
Step 6: Euclidean Clustering for 3D Point Clouds
Discussion and Perspectives
The Segment Anything Model
The Mission
3D Project Setup
Segment Anything Model Core Concepts
3D Point Cloud to Image Projections
Unsupervised Segmentation with SAM
Summary
14. Supervised 3D Machine Learning Fundamentals
From Unsupervised to Supervised Learning
Supervised Learning Concepts
Supervised Learning Classification
3D Semantic Segmentation Example
3D Point Cloud Semantic Segmentation
3D Python and Data Setup
Feature Selection and Preparation
Metrics and Models
Inference and Generalization
Specializing 3D Machine Learning with 3D Deep Learning
Summary
15. 3D Deep Learning with PyTorch
3D Deep Learning Backbone
Network Architecture
Data Preparation
AI Model Training
Serving a Trained Model
Implementation with PyTorch
Installing PyTorch (with CUDA)
Tensors: The Building Blocks
Neural Network Modules
Defining a 3D Neural Network
Hyperparameter Definition
Optimizer and Loss Functions
PyTorch DataLoader
PyTorch Training Loop
PyTorch Inference
3D Deep Learning: The Architectures
3D Convolutional Neural Networks: Voxels
3D Graph Neural Networks
Point-based Architectures: PointNet and Point Clouds
Multiview CNNs
3D Machine Learning Versus 3D Deep Learning
Fine-Tuning, Transfer Learning, and 3D Data Augmentation
Transfer Learning
Fine-Tuning
3D Data Augmentation: Expanding the Dataset
Summary
16. PointNet for 3D Object Classification
PointNet: A Point-based 3D Deep Learning Architecture
3D Object Classification
3D Object Classification Fundamentals
Environment Setup
Dataset Curation
PointNet: Dataset Preparation
PointNet Architecture Definition
PointNet Loss Definition
PointNet Training
PointNet Metrics and Evaluation
PointNet Real-World Inference
Large-Scale Semantic Segmentation Considerations
Summary
17. The 3D Data Science Workflow
3D Data Acquisition
3D Data Preparation and Engineering
Noise Removal
Subsampling
Feature Extraction
3D Data Modeling
3D Mesh Reconstruction
Voxelization of 3D Digital Environments
k-d Trees
Octrees
Semantic Extraction
Clustering and Unsupervised Segmentation
Semantic Segmentation
3D Object Classification
3D Data Visualization and Analysis
3D Shape Recognition
3D Data Analytical Tools
3D Multimodal Python Viewer
Summary
18. From 3D Generative AI to Spatial AI
Advanced 3D Projects
Generative AI for 3D Reconstruction
3D Deep Point Cloud Registration
3D Semantic Modeling
3D Semantic Extraction with Transformers
3D Gaussian Splatting for 3D Visualization
Spatial AI: The Future of 3D Experiences
3D Scene Understanding with Open Vocabularies
3D Spatial AI Reasoning
Conclusion
Index
[