非朗伯体材质重建的近景摄影测量几何图像可微渲染方法

doi:10.11947/j.AGCS.2025.20240257

测绘学报 ›› 2025, Vol. 54 ›› Issue (4): 736-748.doi: 10.11947/j.AGCS.2025.20240257

• 摄影测量学与遥感 • 上一篇

非朗伯体材质重建的近景摄影测量几何图像可微渲染方法

胡翰¹(), 郭晓林¹, 熊浪¹, 葛旭明¹(), 曾浩炜², 朱庆¹

^1.西南交通大学地球科学与工程学院，四川　成都　611756
^2.成都市勘察测绘研究院，四川　成都　610023

收稿日期:2024-06-25 发布日期:2025-05-30
通讯作者: 葛旭明 E-mail:han.hu@swjtu.edu.cn;xuming.ge@swjtu.edu.cn
作者简介:胡翰（1988—），男，博士，教授，博士生导师，研究方向为倾斜摄影测量与三维地理信息系统。　E-mail：han.hu@swjtu.edu.cn
基金资助:
国家重点研发计划(2022YFF0904400);国家自然科学基金(42071355);四川省杰出青年科技基金(22JCQN0110);四川省自然科学基金(2024NSFSC0785)

Close-range photogrammetric differential rendering using geometry buffer for the material reconstruction of non-lambertian surface model

Han HU¹(), Xiaolin GUO¹, Lang XIONG¹, Xuming GE¹(), Haowei ZENG², Qing ZHU¹

^1.Faculty of Geosciences and Engineering, Southwest Jiaotong University, Chengdu 611756, China
^2.Chengdu Institute of Survey & Investigation, Chengdu 610023, China

Received:2024-06-25 Published:2025-05-30
Contact: Xuming GE E-mail:han.hu@swjtu.edu.cn;xuming.ge@swjtu.edu.cn
About author:HU Han (1988—), male, PhD, professor, PhD supervisor, majors in oblique photogrammetry and 3D GIS.　E-mail: han.hu@swjtu.edu.cn
Supported by:
The National Key Research and Development Program of China(2022YFF0904400);The National Natural Science Foundation of China(42071355);The Sichuan Science and Technology Fund for Distinguished Young Scholars(22JCQN0110);The Natural Science Foundation of Sichuan Province(2024NSFSC0785)

摘要/Abstract

摘要：

瓷器等非朗伯体文物的光学纹理材质是三维模型真实感可视化渲染的关键基础。由于非朗伯体表面对不同视角影像存在高光等色彩差异，既有摄影测量纹理重建方法，仅通过匀色、羽化平滑等方式粗暴地缓解色彩差异；基于隐式空间表达的神经辐射场方法，仅利用多层感知机模糊地拟合空间对不同反射方向色彩，导致真实感不足。为此，本文提出非朗伯体材质重建的近景摄影测量几何图像可微渲染方法，通过手持式立体扫描仪和可控照明条件的近景摄影测量方法，获取影像准确的定向参数、模型几何及其初始彩色纹理，将反照率、粗糙度、法线扰动等材质看作待优化的平面张量，通过光栅化的方式映射至屏幕空间，得到具有坐标、法线、纹理坐标、材质等丰富属性的几何图像；借鉴延迟着色技术，利用几何图像在屏幕空间模拟像素着色器，通过顾及高光的Cook-Torrance微平面模型实现可微渲染和反向回馈优化未知的材质信息。通过典型非朗伯体目标的试验证明，本文方法与真实照片的影像相似度（SSIM）优于0.85，比基于隐空间表示的三维高斯点和多分辨率哈希编码渲染精度分别提升8.7%和7.2%，显著提升了模型的真实感。

关键词: 近景摄影测量, 非朗伯体材质, 几何图像, 延迟着色器, 可微渲染

Abstract:

The optical texture material of non-lambertian artifacts such as porcelain is a critical foundation for the realistic visualization and rendering of 3D models. Due to the color differences such as highlights caused by non-lambertian surfaces from different viewing angles, existing photogrammetric texture reconstruction methods crudely alleviate these color differences through color equalization and feathering; Neural radiance field methods based on implicit spatial representation rely solely on multi-layer perceptron to blur spatial fitting for different reflection directions, leading to insufficient realism. Therefore, this study proposes a differentiable rendering method for non-lambertian material reconstruction using close-range photogrammetry geometry images. By employing a handheld stereo scanner and controlled lighting conditions, the method accurately acquires oriented parameters, model geometry, and initial color texture of images. Material properties such as albedo, roughness, and normal perturbation are treated as planar tensors to be optimized, mapped to screen space via rasterization to obtain rich geometric images with coordinates, normal, texture coordinates, and material attributes; Drawing on deferred shading techniques, the geometric image simulates pixel shaders in screen space, achieving differentiable rendering and inverse feedback optimization of unknown material information using the Cook-Torrance microfacet model for highlights. Experimental results on typical non-lambertian targets demonstrate that the proposed method achieves a structural similarity index (SSIM) better than 0.85 compared to real images, improving rendering accuracy by 8.7% and 7.2% over 3D Gaussian points and multi-resolution hash encoding based on implicit space representation, respectively, significantly enhancing the realism of the model.

Key words: close-range photogrammetry, non-lambertian material, geometry image, deferred shading, differentiable r endering

中图分类号:

P234.1

胡翰, 郭晓林, 熊浪, 葛旭明, 曾浩炜, 朱庆. 非朗伯体材质重建的近景摄影测量几何图像可微渲染方法[J]. 测绘学报, 2025, 54(4): 736-748.

Han HU, Xiaolin GUO, Lang XIONG, Xuming GE, Haowei ZENG, Qing ZHU. Close-range photogrammetric differential rendering using geometry buffer for the material reconstruction of non-lambertian surface model[J]. Acta Geodaetica et Cartographica Sinica, 2025, 54(4): 736-748.

图/表 15

图1

图2

图3

图4

图5

图6

图7

图8

表1

图9

图10

图11

表2

图12

图13

参考文献 40

[1]	WAECHTER M, MOEHRLE N, GOESELE M. Let there be color! large-scale texturing of 3D reconstructions[C]//Proceedings of 2014 European Conference on Computer Vision. Cham: Springer International Publishing, 2014: 836-850.
[2]	ZHOU Qianyi, KOLTUN V. Color map optimization for 3D reconstruction with consumer depth cameras[J]. ACM Transactions on Graphics, 2014, 33(4): 1-10.
[3]	肖腾, 王鑫, 梅熙, 等. 摄影测量局部场景稳健合并的并行式运动恢复结构方法[J]. 测绘学报, 2024, 53(2): 332-343. DOI:. doi: 10.11947/j.AGCS.2024.20220321
	XIAO Teng, WANG Xin, MEI Xi, et al. Robust merging of subblock reconstructions for parallel structure from motion in photogrammetry[J]. Acta Geodaetica et Cartographica Sinica, 2024, 53(2): 332-343. DOI:. doi: 10.11947/j.AGCS.2024.20220321
[4]	姜三, 马一尘, 李清泉, 等. 无序无人机影像的并行化SfM三维重建方法[J]. 测绘学报, 2024, 53(5): 946-958. DOI:. doi: 10.11947/j.AGCS.2024.20230335
	JIANG San, MA Yichen, LI Qingquan, et al. Parallel SfM-based 3D reconstruction for unordered UAV images[J]. Acta Geodaetica et Cartographica Sinica, 2024, 53(5): 946-958. DOI:. doi: 10.11947/j.AGCS.2024.20230335
[5]	付琳, 洪海波, 王晰, 等. 基于逆向反射模型的非朗伯光度立体视觉[J]. 光学学报, 2020, 40(5): 151-161.
	FU Lin, HONG Haibo, WANG Xi, et al. Non-Lambertian photometric stereo vision based on inverse reflectance model[J]. Acta Optica Sinica, 2020, 40(5): 151-161.
[6]	MILDENHALL B, SRINIVASAN P P, TANCIK M, et al. NeRF: representing scenes as neural radiance fields for view synthesis[J]. Communications of the ACM, 2022, 65(1): 99-106.
[7]	MÜLLER T, EVANS A, SCHIED C, et al. Instant neural graphics primitives with a multiresolution hash encoding[J]. ACM Transactions on Graphics, 2022, 41(4): 1-15.
[8]	FRIDOVICH-KEIL S, YU A, TANCIK M, et al. Plenoxels: radiance fields without neural networks[C]//Proceedings of 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New Orleans: IEEE, 2022: 5501-5510.
[9]	XU Qiangeng, XU Zexiang, PHILIP J, et al. Point-NeRF: point-based neural radiance fields[C]//Proceedings of 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New Orleans: IEEE, 2022: 5438-5448.
[10]	KERBL B, KOPANAS G, LEIMKUEHLER T, et al. 3D Gaussian splatting for real-time radiance field rendering[J]. ACM Transactions on Graphics, 2023, 42(4): 1-14.
[11]	AZINOVIC D, MARTIN-BRUALLA R, GOLDMAN D B, et al. Neural RGB-D surface reconstruction[C]//Proceedings of 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New Orleans: IEEE, 2022: 6290-6301.
[12]	WANG Peng, LIU Lingjie, LIU Yuan, et al. NeuS: learning neural implicit surfaces by volume rendering for multi-view reconstruction[EB/OL]. [2024-04-20]. https://arxiv.org/abs/2106.10689v3.
[13]	YANG Bangbang, BAO Chong, ZENG Junyi, et al. NeuMesh: learning disentangled neural mesh-based implicit field for Geometry and Texture editing[C]//Proceedings of 2022 European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2022: 597-614.
[14]	WIZADWONGSA S, PHONGTHAWEE P, YENPHRAPHAI J, et al. NeX: real-time view synthesis with neural basis expansion[C]//Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville: IEEE, 2021: 8534-8543.
[15]	MILLER B, GEORGIEV I, JAROSZ W. A null-scattering path integral formulation of light transport[J]. ACM Transactions on Graphics, 2019, 38(4): 1-13.
[16]	HART J C. Sphere tracing: a geometric method for the antialiased ray tracing of implicit surfaces[J]. The Visual Computer, 1996, 12(10): 527-545.
[17]	TEWARI A, THIES J, MILDENHALL B, et al. Advances in neural rendering[J]. Computer Graphics Forum, 2022, 41(2): 703-735.
[18]	TUCKER R, SNAVELY N. Single-view view synthesis with multiplane images[C]//Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020: 551-560.
[19]	MÜLLER T, ROUSSELLE F, NOVÁK J, et al. Real-time neural radiance caching for path tracing[J]. ACM Transactions on Graphics, 2021, 40(4): 1-16.
[20]	CHEN Anpei, XU Zexiang, GEIGER A, et al. TensoRF: tensorial radiance fields[C]//Proceedings of 2022 European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2022: 333-350.
[21]	许威威, 周漾, 吴鸿智, 等. 可微绘制技术研究进展[J]. 中国图象图形学报, 2021, 26(6): 1521-1535.
	XU Weiwei, ZHOU Yang, WU Hongzhi, et al. Differential rendering: a survey[J]. Journal of Image and Graphics, 2021, 26(6): 1521-1535.
[22]	HU Han, DING Yulin, ZHU Qing, et al. Stable least-squares matching for oblique images using bound constrained optimization and a robust loss function[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2016, 118: 53-67.
[23]	ENGEL J, SCHÖPS T, CREMERS D. LSD-SLAM: large-scale direct monocular SLAM[C]//Proceedings of 2014 European Conference on Computer Vision. Cham: Springer International Publishing, 2014: 834-849.
[24]	VU H H, LABATUT P, PONS J P, et al. High accuracy and visibility-consistent dense multiview stereo[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2012, 34(5): 889-901.
[25]	FURUKAWA Y, PONCE J. Dense 3D motion capture from synchronized video streams[M]//RONFARD R, TAUBIN G, ed. Image and geometry processing for 3D cinematography. Berlin: Springer, 2010: 193-211.
[26]	GOURAUD H. Continuous shading of curved surfaces[M]//WOLFE R, ed. Seminal graphics: pioneering efforts that shaped the field. New York: Association for Computing Machinery, 1998: 87-93.
[27]	ZHU Qing, WANG Zhendong, HU Han, et al. Leveraging photogrammetric mesh models for aerial-ground feature point matching toward integrated 3D reconstruction[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2020, 166: 26-40.
[28]	KARIS B. Real shading in unreal Engine 4[J]. SIGGRAPH, 2013, 4(3): 1-21.
[29]	LAINE S, HELLSTEN J, KARRAS T, et al. Modular primitives for high-performance differentiable rendering[J]. ACM Transactions on Graphics, 2020, 39(6): 1-14.
[30]	HARGREAVES S, HARRIS M. Deferred shading[C]//Proceedings of 2004 Game Developers Conference. San Jose: [s. n.]. 2004, 2: 31.
[31]	中观自动化科技有限公司. AtlaScan多模式多功能量测3D扫描仪[EB/OL]. [2024-04-20]. https://www.zg-3d.com/AtlaScan.html.
	ZG Technology Co., Ltd.. AtlaScan multimodal multi-functional measurement 3D scanner[EB/OL]. [2024-04-20]. https://www.zg-3d.com/AtlaScan.html.
[32]	CHENG H K, SCHWING A G. XMem: long-term video object segmentation with an Atkinson-Shiffrin memory model[C]//Proceedings of 2022 European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2022: 640-658.
[33]	AGISOFT . Agisoft metashape[EB/OL]. [2024-04-20]. https://www.agisoft.com/.
[34]	KAJIYA J T. The rendering equation[C]//Proceedings of the 13th Annual Conference on Computer Graphics and Interactive Techniques. New York: ACM Press, 1986: 143-150.
[35]	WALTER B, MARSCHNER S R, LI H, et al. Microfacet models for refraction through rough surfaces[C]//Proceedings of the 18th Eurographics Conference on Rendering Techniques. Goslar: Eurographics Association, 2007: 195-206.
[36]	WANG Z, BOVIK A C, SHEIKH H R, et al. Image quality assessment: from error visibility to structural similarity[J]. IEEE Transactions on Image Processing, 2004, 13(4): 600-612.
[37]	ZHANG R, ISOLA P, EFROS A A, et al. The unreasonable effectiveness of deep features as a perceptual metric[C]//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 586-595.
[38]	BOSS M, JAMPANI V, BRAUN R, et al. Neural-pil: neural pre-integrated lighting for reflectance decomposition[J]. Advances in Neural Information Processing Systems, 2021, 34: 10691-10704.
[39]	VERBIN D, HEDMAN P, MILDENHALL B, et al. Ref-NeRF: structured view-dependent appearance for neural radiance fields[C]//Proceedings of 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New Orleans: IEEE, 2022: 5481-5490.
[40]	朱庆, 张利国, 丁雨淋, 等. 从实景三维建模到数字孪生建模[J]. 测绘学报, 2022, 51(6): 1040-1049. DOI:. doi: 10.11947/j.AGCS.2022.20210640
	ZHU Qing, ZHANG Liguo, DING Yulin, et al. From real 3D modeling to digital twin modeling[J]. Acta Geodaetica et Cartographica Sinica, 2022, 51(6): 1040-1049. DOI:. doi: 10.11947/j.AGCS.2022.20210640

名称	模型尺寸（高×宽）/cm	网格模型数量（顶点/面片）	空三精度/像素
花瓶	10×6	6.06×10⁵/12.11×10⁵	1.44
碗	7×13	11.55×10⁵/23.10×10⁵	1.11
锥形花盆	12.5×9	23.69×10⁵/47.29×10⁵	0.52
罐	12×10	22.54×10⁵/44.91×10⁵	0.58
桶形花盆	14×15	45.79×10⁵/91.57×10⁵	0.77

方法	PSNR↑	SSIM↑	LPIPS↓
摄影测量方法	14.61	0.81	0.13
3DGS	17.03	0.80	0.11
InstantNGP	18.26	0.79	0.15
Neural-PIL	17.67	0.61	0.22
Ref-NeRF	21.93	0.81	0.28
本文方法	24.19	0.89	0.09
摄影测量方法	15.44	0.89	0.08
3DGS	24.49	0.88	0.10
InstantNGP	22.78	0.92	0.14
Neural-PIL	18.69	0.56	0.33
Ref-NeRF	21.75	0.82	0.31
本文方法	27.56	0.91	0.09
摄影测量方法	20.24	0.85	0.14
3DGS	29.01	0.86	0.14
InstantNGP	21.99	0.87	0.24
Neural-PIL	17.64	0.55	0.26
Ref-NeRF	23.48	0.84	0.28
本文方法	33.23	0.95	0.04
摄影测量方法	17.62	0.86	0.13
3DGS	29.78	0.88	0.11
InstantNGP	21.92	0.88	0.28
Neural-PIL	19.06	0.60	0.20
Ref-NeRF	26.37	0.88	0.26
本文方法	28.93	0.93	0.08
摄影测量方法	19.43	0.85	0.12
3DGS	25.56	0.82	0.13
InstantNGP	23.04	0.84	0.31
Neural-PIL	18.85	0.47	0.36
Ref-NeRF	24.77	0.82	0.33
本文方法	30.82	0.93	0.11

非朗伯体材质重建的近景摄影测量几何图像可微渲染方法

Close-range photogrammetric differential rendering using geometry buffer for the material reconstruction of non-lambertian surface model

RichHTML

PDF

可视化

摘要/Abstract

引用本文

使用本文

图/表 15

参考文献 40

相关文章 2

编辑推荐

Metrics

本文评价

[1]	雷臻, 张帆, 向瀚宇, 杨冲, 黄先锋. 多视角翻转拍摄物体的完整三维重建[J]. 测绘学报, 2023, 52(8): 1305-1316.
[2]	盛庆红, 柳建锋, 虞梦昕, 王惠南. 对偶四元数近景影像空中三角测量法[J]. 测绘学报, 2015, 44(5): 503-509.