Digital twin based photogrammetry field-of-view evaluation and 3D layout optimisation for reconfigurable manufacturing systems

Wang, Zi; Wang, Likun; Martínez-Arellano, Giovanna; Griffin, Joseph; Sanderson, David; Ratchev, Svetan

Abstract

Photogrammetry is extensively used in manufacturing processes due to its non-contact nature and rapid data acquisition. Positioning photogrammetry cameras requires knowledge of the manufacturing process and time in manual field-of-view (FoV) adjustment. Such a lengthy and labour-intensive process is not suitable for modern manufacturing systems, where automation, robotics and dynamic reconfigurable layout are used to shorten production time and adapt to demand changes. Hence, there exists the need for a fast layout planning approach ensuring manufacturing process feasibility and maximising camera FoV effectiveness. This paper introduces a digital twin based FoV evaluation method and a computationally efficient 3D layout optimisation framework for reconfigurable manufacturing systems. The framework computes optimal layout for photogrammetry cameras and the object of interest (OOI). The automated nature of the proposed framework can speed up planning processes and shorten dynamic system commissioning time. At a technical level, the framework takes advantage of a 3D digital twin, and uses point clouds to represent the camera FoV. Iterative Closest Point (ICP) registration and K-Dimensional Tree (KDTree) intersection techniques are applied to calculate OOI visibility and target coverage ratio. Experimental validation attested to a digital-physical similarity exceeding 93%, indicating a high level of fidelity and the feasibility of station-level 3D layout design in digital twin environments. Feeding into the 3D layout planning, the optimisation framework considers robot reachability, FoV effectiveness, and estimated uncertainty. Given characteristics of the objective function, genetic algorithm, simulated annealing, and Bayesian optimisation were evaluated within a computational budget (100 function calls). The optimised results are compared against a baseline best obtained through brute force grid search. All tested algorithms achieved results within 98% of the grid search’s best solution within 5 min. Genetic algorithm and simulated annealing outperformed the baseline best by 0.16% and 0.25% respectively for OOI visibility, and Bayesian optimisation exceeded the baseline best by 0.12% for target coverage. These findings emphasise the feasibility of the proposed approach and the efficiency of the overall framework, highlighting its applicability across various development stages from design to execution in a dynamic manufacturing environment.

Citation

Wang, Z., Wang, L., Martínez-Arellano, G., Griffin, J., Sanderson, D., & Ratchev, S. (2024). Digital twin based photogrammetry field-of-view evaluation and 3D layout optimisation for reconfigurable manufacturing systems. Journal of Manufacturing Systems, 77, 1045-1061. https://doi.org/10.1016/j.jmsy.2024.11.001

• Journal Article Type: Article
• Acceptance Date: Nov 4, 2024
• Online Publication Date: Nov 20, 2024
• Publication Date: 2024-12
• Deposit Date: Nov 28, 2024
• Publicly Available Date: Nov 12, 2024
• Journal: Journal of Manufacturing Systems
• Print ISSN: 0278-6125
• Electronic ISSN: 0278-6125
• Publisher: Elsevier
• Peer Reviewed: Peer Reviewed
• Volume: 77
• Pages: 1045-1061
• DOI: https://doi.org/10.1016/j.jmsy.2024.11.001
• Keywords: Digital twin, Photogrammetry, Field-of-view, Measurement-assisted manufacturing, Layout optimisation, Reconfigurable manufacturing systems, Heuristic methods
• Public URL: https://nottingham-repository.worktribe.com/output/42479660
• Publisher URL: https://www.sciencedirect.com/science/article/pii/S0278612524002565?via%3Dihub