Journals / Balkan Journal of Electrical and Computer Engineering / 2026 / Volume: 14

Multi-View RGB-D 3D Point Cloud Reconstruction Using a Robotic Manipulator

Robotik Manipülatör Kullanılarak Çoklu Görüş Açılarından RGB-D Tabanlı 3B Nokta Bulutu Yeniden Oluşturma

Pages
—
Publication Date
2026-10-30
🏛️Bibliographic metadata and full text access for this article are provided via DergiPark.
DergiPark Record ↗
✨

AI Summary & Key Takeaways

AI Insight
⚡ TL;DR(Too Long; Didn't Read)

The comparison showed that increasing the number of waypoints reduced the median point spacing and improved cross-trajectory consistency approximately in proportion to the square root of the point density, while leaving the pose errors and geometric accuracy essentially unchanged.

📌 3 Key Takeaways

🎯
Objective & Hypothesis:In this study, a multi-view RGB-D-based three-dimensional point cloud acquisition system is developed using a six-degree-of-freedom robotic manipulator within the Webots simulation environment.
📊
Key Findings & Data:Two mutually orthogonal semicircular scanning trajectories are employed to increase surface coverage, while the camera optical axis is continuously directed toward the object center throughout the scan.
💡
Conclusion & Impact:The comparison showed that increasing the number of waypoints reduced the median point spacing and improved cross-trajectory consistency approximately in proportion to the square root of the point density, while leaving the pose errors and geometric accuracy essentially unchanged.
🔬Design:Modeling & Simulation
📍Scope:Comprehensive Sample
⚙️Variables:RGB-D camera · 3D point cloud · multi-view reconstruction

Abstract

In this study, a multi-view RGB-D-based three-dimensional point cloud acquisition system is developed using a six-degree-of-freedom robotic manipulator within the Webots simulation environment. The robotic manipulator is actuated along semicircular arcs to position the RGB-D camera at predefined viewing angles, enabling simultaneous acquisition of color and depth images at each waypoint. Two mutually orthogonal semicircular scanning trajectories are employed to increase surface coverage, while the camera optical axis is continuously directed toward the object center throughout the scan. At each stable waypoint, RGB-D data and camera pose information are synchronously recorded. The resulting point cloud provides a geometrically consistent representation of the scanned object without requiring an additional point-cloud registration procedure. The results were first obtained for n = 7 waypoints per semicircular trajectory, and the corresponding error statistics were analyzed in detail. This analysis showed that the median distance between the point cloud generated using the ground-truth poses and the reference surface remained on the order of micrometers, indicating that the fusion pipeline introduced no measurable geometric error beyond that contained in the provided pose information. When the same frames were fused using the commanded poses, the mean surface error increased to 0.45 mm, corresponding to approximately 0.62 times the mean pose position error. Therefore, the scanning accuracy is determined by the positioning performance of the manipulator rather than the sensor model. Subsequently, the same trajectory geometry was rescanned with n=37 waypoints to evaluate the effect of sampling density comparatively. The comparison showed that increasing the number of waypoints reduced the median point spacing and improved cross-trajectory consistency approximately in proportion to the square root of the point density, while leaving the pose errors and geometric accuracy essentially unchanged.

Özet

Bu çalışmada, Webots benzetim ortamında altı serbestlik dereceli bir robotik manipülatör kullanılarak çoklu-görüş RGB-D tabanlı üç boyutlu nokta bulutu elde etme sistemi geliştirilmiştir. Robotik manipülatör, RGB-D kamerayı önceden tanımlanan bakış açılarına konumlandırmak amacıyla yarım dairesel yaylar boyunca hareket ettirilmiş ve her bir durak noktasında renk ve derinlik görüntülerinin eşzamanlı olarak elde edilmesi sağlanmıştır. Yüzey kapsamasını artırmak amacıyla birbirine dik iki yarım dairesel tarama yörüngesi kullanılmış ve tarama süresince kameranın optik ekseni sürekli olarak nesne merkezine yönlendirilmiştir. Her bir kararlı durak noktasında RGB-D verileri ile kamera poz bilgileri eşzamanlı olarak kaydedilmiştir. Elde edilen nokta bulutu, ek bir nokta bulutu hizalama işlemine gereksinim duymadan taranan nesnenin geometrik olarak tutarlı bir temsilini sağlamıştır. Sonuçlar ilk olarak her bir yarım dairesel yörünge için n=7 durak noktası kullanılarak elde edilmiş ve ilgili hata istatistikleri ayrıntılı olarak analiz edilmiştir. Bu analiz, gerçek pozlar kullanılarak oluşturulan nokta bulutu ile referans yüzey arasındaki medyan uzaklığın mikrometre mertebesinde kaldığını ve füzyon işlem hattının sağlanan poz bilgisinin içerdiği hataya ek olarak ölçülebilir düzeyde herhangi bir geometrik hata oluşturmadığını göstermiştir. Aynı görüntülerin, komut edilen pozlar kullanılarak birleştirilmesi durumunda ise ortalama yüzey hatası yaklaşık 0. 45 mm seviyesine yükselmiş ve bu değer ortalama pozisyon hatasının yaklaşık 0. 62 katına karşılık gelmiştir. Dolayısıyla tarama doğruluğunun sensör modelinden ziyade manipülatörün konumlandırma başarımı tarafından belirlendiği görülmüştür. Daha sonra, örnekleme yoğunluğunun etkisini karşılaştırmalı olarak değerlendirmek amacıyla aynı yörünge geometrisi n = 37 durak noktası kullanılarak yeniden taranmıştır. Karşılaştırma sonuçları, durak noktası sayısının artırılmasının medyan nokta aralığını azalttığını ve çapraz yörünge tutarlılığını nokta yoğunluğunun karekökü ile yaklaşık orantılı olarak iyileştirdiğini; buna karşılık poz hataları ile geometrik doğruluğun değişmeden kaldığını göstermiştir.

🔖

Cite This Article

Boztaş, G., Çalik, M. F., Pirinççi, N. (2026). Multi-View RGB-D 3D Point Cloud Reconstruction Using a Robotic Manipulator. Balkan Journal of Electrical and Computer Engineering, Volume: 14. https://doi.org/10.17694/bajece.2047162

📄 Full Text PDF Viewer

📄

PDF görüntüleyici hazırlanıyor...

References (30)

  1. Fei, B., Xu, J., Li, Y., Yang, W., Zhou, Q., Liu, L., Luo, T., & He, Y. (2026). Self-Supervised Learning for Pre-Training 3D Point Clouds: A Survey. Computational Visual Media, 12(3),509-573. https://doi.org/10.26599/CVM.2025.9450514
  2. Imran, M. M., Kim, Y., Jung, G., De Silva, L. C., Suh, J.-H., Abas, P. E., & Kim, Y. B. (2023). In-Situ Process Monitoring and Defects Detection Based on Geometrical Topography With Streaming Point Cloud Processing in Directed Energy Deposition. IEEE Access, 11, 131319- 131337. https://doi.org/10.1109/ACCESS.2023.3335130
  3. Tychola, K.A., Vrochidou, E., & Papakostas, G.A. (2024). Deep learning based computer vision under the prism of 3D point clouds: a systematic review. The Visual Computer, 40, 8287–8329. https://doi.org/10.1007/s00371-023-03237-7
  4. Wang, C., Wang, C., Li, W., & Wang, H. (2021). A brief survey on RGB-D semantic segmentation using deep learning. Displays, 70, 102080. https://doi.org/10.1016/j.displa.2021.102080
  5. Xu, Y., Tong, X., & Stilla, U. (2021). Voxel-based representation of 3D point clouds: Methods, applications, and its potential use in the construction industry. Automation in Construction, 126, 103675. https://doi.org/10.1016/j.autcon.2021.103675
  6. Komori, J., & Hotta, K. (2019, December). AB-PointNet for 3D Point Cloud Recognition. 2019 Digital Image Computing: Techniques and Applications (DICTA) (pp. 1- 6). IEEE. https://doi.org/10.1109/DICTA47822.2019.8945926
  7. Luo, K., Yu, H., Chen, X., Yang, Z., Wang, J., Cheng, P., & Mian, A. (2024). 3D point cloud-based place recognition: a survey. Artificial Intelligence Review, 57. https://doi.org/10.1007/s10462-024-10713-6
  8. Liu, Y., Yuan, H., Zhao, X., Fan, C., & Cheng, M. (2023). Fast reconstruction method of three-dimension model based on dual RGB-D cameras for peanut plant. Plant Methods, 19. https://doi.org/10.1186/s13007-023-00998-z
  9. Ren, H., Yang, M., & Velipasalar, S. (2025). DG-MVP: 3-D Domain Generalization via Multiple Views of Point Clouds for Synthetic-to-RGBD Sensor Data Classification. IEEE Sensors Journal, 25(20), 38133-38144. https://doi.org/10.1109/JSEN.2025.3605134
  10. Chen, G., Cui, G., Jin, Z., Wu, F., & Chen, X. (2019). Accurate Intrinsic and Extrinsic Calibration of RGB-D Cameras With GP-Based Depth Correction. IEEE Sensors Journal, 19(7), 2685-2694. https://doi.org/10.1109/JSEN.2018.2889805
  11. Wang, Q., Li, S., Qin, H., & Hao, A. (2016). Super- Resolution of Multi-Observed RGB-D Images Based on Nonlocal Regression and Total Variation. IEEE Transactions on Image Processing, 25(3), 1425-1440. https://doi.org/10.1109/TIP.2016.2521180
  12. Chu, P. M., Sung, Y., & Cho, K. (2019). Generative Adversarial Network-Based Method for Transforming Single RGB Image Into 3D Point Cloud. IEEE Access, 7, 1021-1029. https://doi.org/10.1109/ACCESS.2018.2886213
  13. Yu, Q., Xiao, J., Lu, H., & Zheng, Z. (2018). Hybrid- Residual-Based RGBD Visual Odometry. IEEE Access, 6, 28540-28551. https://doi.org/10.1109/ACCESS.2018.2836928
  14. Liu, Y., Jia, T., Zhang, H., Yang, G., Wang, H., & Chen, D. (2025). RASP: Robot Active Scene Perception With Joint Viewpoint Planning and Depth Completion in Cluttered Environments. IEEE Transactions on Automation Science and Engineering, 22, 20611-20623. https://doi.org/10.1109/TASE.2025.3604455
  15. Fukai, H. & Xu, G. (2011, August). Fast and robust registration of multiple 3D point clouds. 2011 RO-MAN (pp. 331-336). IEEE. https://doi.org/10.1109/ROMAN.2011.6005295
  16. Lieberknecht, S., Huber, A., Ilic, S., & Benhimane, S. (2011, October). RGB-D camera-based parallel tracking and meshing. 2011 10th IEEE International Symposium on Mixed and Augmented Reality (pp. 147-155). IEEE. https://doi.org/10.1109/ISMAR.2011.6092380
  17. Pan, H., Guan, T., Luo, Y., Duan, L., Tian, Y., Yi, L., Zhao, Y., & Yu, J. (2016). Dense 3D reconstruction combining depth and RGB information. Neurocomputing, 175, 644-651. https://doi.org/10.1016/j.neucom.2015.10.104
  18. Ylimäki, M., Heikkilä, J., & Kannala, J. (2018, August). Accurate 3-D Reconstruction with RGB-D Cameras using Depth Map Fusion and Pose Refinement. 2018 24th International Conference on Pattern Recognition (ICPR) (pp. 1977-1982). IEEE. https://doi.org/10.1109/ICPR.2018.8545508
  19. Teng, X., Zhou, G., Wu, Y., Huang, C., Dong, W., & Xu, S. (2021). Three-Dimensional Reconstruction Method of Rapeseed Plants in the Whole Growth Period Using RGB-D Camera. Sensors, 21(14), 4628. https://doi.org/10.3390/s21144628
  20. Park, B.-S., Kim, W., Kim, J.-K., Kim, D.-W., & Seo, Y.-H. (2022). Iterative extrinsic calibration using virtual viewpoint for 3D reconstruction. Signal Processing, 197, 108535. https://doi.org/10.1016/j.sigpro.2022.108535
  21. Wang, C., & Guo, X. (2017). Feature-based RGB-D camera pose optimization for real-time 3D reconstruction. Computational Visual Media, 3(2), 95–106. https://doi.org/10.1007/s41095-016-0072-2
  22. Michel, O. (2004), Cyberbotics Ltd. WebotsTM: Professional Mobile Robot Simulation. International Journal of Advanced Robotic Systems. 1(1), 39-42. https://doi.org/10.5772/5618
  23. Yin, S., Ren, Y., Zhu, J., Yang, S., & Ye, S. (2013). A Vision-Based Self-Calibration Method for Robotic Visual Inspection Systems. Sensors, 13(12), 16565-16582. https://doi.org/10.3390/s131216565
  24. Abdel-Malek, K., & Othman, S. (1999). Multiple sweeping using the Denavit–Hartenberg representation method. Computer-Aided Design, 31(9), 567-583. https://doi.org/10.1016/S0010-4485(99)00053-6
  25. Tsai, R. (1987). A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV cameras and lenses. IEEE Journal on Robotics and Automation, 3(4), 323-344. https://doi.org/10.1109/JRA.1987.1087109
  26. Szeliski, R. (2010). Computer Vision: Algorithms and Applications. Springer.
  27. LaValle, S. M. (2020). Virtual Reality. Cambridge University Press.
  28. Sugihara, T. (2011). Solvability-Unconcerned Inverse Kinematics by the Levenberg–Marquardt Method. IEEE Transactions on Robotics, 27(5), 984-991. https://doi.org/10.1109/TRO.2011.2148230
  29. Proença, P. F., & Gao, Y. (2018). Probabilistic RGB-D odometry based on points, lines and planes under depth uncertainty. Robotics and Autonomous Systems, 104, 25-39. https://doi.org/10.1016/j.robot.2018.02.018
  30. Xu, H., Chen, G., Wang, Z., Sun, L., & Su, F. (2019). RGB-D-Based Pose Estimation of Workpieces with Semantic Segmentation and Point Cloud Registration. Sensors, 19(8), 1873. https://doi.org/10.3390/s19081873