Image processing algorithm to determine an optimised 2D laser cutting trajectory

Laser cutting processes offer high-quality and fast cutting capability across a wide variety of materials, including metals, plastics and organic tissues. To enable 2D laser cutting process, a set of (x, y) Cartesian coordinates that form a cutting trajectory have to be given to a machine controller to perform the cutting process. Automatically determining the cutting trajectory from an image of materials with inhomogeneous, crease and transparency characteristics, for example biomaterials, is difficult.In this paper, an image processing algorithm for determining and optimising the trajectory of a 2D laser cutting process is presented. Using this optimised 2D trajectory, uncut material wastes from the laser cutting process can be substantially reduced. The waste reductions are mainly obtained from optimised cutting area allocation and defective cut avoidance by manual cutting. In addition, the presented algorithm accommodates different cutting shapes, determined by a user, to maximise material cut from the laser cutting process.Case studies of thin and transparent amnion biomaterials cutting are presented to demonstrate the proposed algorithm to optimise the 2D laser cutting trajectory of the biomaterials. The algorithm has been tested to determine the optimised 2D cutting trajectory for a rectangle, circle and random shape amnion biomaterials. Results show that uncut materials can be minimised up to 2%, 3% and 5% of the total material of rectangle, circle and random shapes, respectively, by using this algorithm.


I. INTRODUCTION
Laser cutting nowadays is one of the most used industrial laser applications, for example, in Japan, laser cutting processes count up to 80% from the total use of industrial lasers [1], [2]. The main advantages of laser cutting process can be divided into two categories: cut quality and process characteristics.
The advantages related to cut quality [1] include relatively narrow kerf width compared to other conventional machining methods, clean cutting edge, blind cut capability, relatively small heat affected zone (HAZ) and no edge burr. Meanwhile, the advantages related to process characteristics [1] include relatively faster cutting speed compared to other cutting methods, minimal part fixturing, no tool wear, no tool changes, flexible cutting and ability to cut almost any materials, including organic tissues. Especially for the cutting speed, a 2D cutting trajectory has to be determined so that a minimum cutting path and a maximum material yield can be obtained.
This paper presents the optimisation of a 2D laser cutting trajectory for organic tissue cutting. The main goal of the trajectory optimisation is to significantly increase the production throughput of amnion-based material that is Omnigen ® , which is the trademark of NuVision ® Biotherapies Ltd (NuVision). This material is the first augmented dry-preserved amnion (amniotic membrane)-derived regenerative medicine in the UK and is developed as a directly accessible biological matrix that can be directly applied and reconstituted in vivo without any biological loss [3]. Omnigen is currently used as an effective regenerative medicine for the treatment of various ocular surface disease (OSD) conditions, including disease and trauma [4], [5]. An example of Omnigen product that is cut in a circular shape with diameter of 10 mm, for the use on dry eyes, is shown in Fig. 1.
There is a significance increase on the use of Omnigen in a clinical setting both in the UK and internationally, so that a substantial manufacturing scale up and production throughout are required. Currently, the production method of Omnigen is performed by manual cutting that is labour and resource intensive. Additionally, the manual cutting uses medical grade cookie cutters, increasing costs and yielding low production throughput with high waste and high rates of product quality rejection.
The challenge of this case study is to 2D laser cut the dry-preserved amnions into different predefined shapes: circle, square and rectangle from a large sheet of dried-preserved amnions. Prior to cutting, an optimised trajectory for the 2D laser cutting process needs to be determined.
Many works have been reported to solve problems related to trajectory optimisations. Aubry et al. proposed a trajectory correction method for robot welding process by mounting a laser scanning system on the robot's end effector and tracking the welding seam [6]. An automatic cutting trajectory algorithm, by cross-sectioning a 3D computer model, for laminated tooling has been proposed by Yoo and Walczyk [7]. Zaeh and Vogl proposed a method to manually determine robot trajectory by utilising augmented-reality for robot teaching [8]. A zig-zag trajectory algorithm for laser hardening process has been proposed by Martinez et al. [9].
More specifically, work related to trajectory planning for laser cutting have been proposed. Dolqui and Pashkevisch proposed an algorithm to optimise 6-axis robot trajectory so that the robot has a smooth continuous motion from a given path [10]. An optimisation of a laser cutting head (mounted on a robot) orientation based on minimising the consuption of laser energy has been proposed by Xu et al. [11]. Huang et al. proposed a simple 2D contouring accuracy compensation to improve the accuracy of a robot following a given trajectory [12]. Qi et al. proposed an algorithm to reduce a 3D shape distortion of a part cut by galvanometric-based laser system [13]. From all the mentioned works, the proposed solutions are not applicable for the problem presented in this paper. Because, these works focusing on optimising motion from a given trajectory, optimising cutting head orientation or requiring a 3D measuring instruments. In out case, an optimised 2D trajectory needs to be determine by using only a 2D imaging system. This paper presents an algorithm based on image processing for 2D laser cutting trajectory (in-plane) optimisation to cut amnions. The algorithm is presented as a multi-objective optimisation to generate an optimised 2D cutting trajectory that accommodates different cutting shapes. The trajectory optimisation is carried out by image processing method. This paper is structured as follows. Section 2 presents the model of the optimisation problem to maximise cutting yields and an algorithm to solve the cutting yield optimisation problem. The implementation of the algorithm to define and optimise cutting trajectories is presented in Section 3. Finally, Section 4 presents the conclusion and highlights the future work of the paper.

II. OPTIMISATION MODEL AND ALGORITHM FOR 2D
LASER CUTTING TRAJECTORY OPTIMISATION In this section, an optimisation model to determine and optimise the 2D laser cutting trajectory of a large sheet amnion into smaller sheets with three different shapes: circle, square and rectangle is presented based on image processing and multi-objective optimisation. After the model explanation, an algorithm to solve the optimisation problem to obtain an optimised trajectory, that maximises cutting yields, is proposed.

A. Optimisation model
The main objective of the optimisation problem is the maximisation of amnion cutting yields subject to three different cutting shapes and sizes. In our case study, cutting time is not considered in the model because the laser cutting process is fast and the size of the cutting area is of (250 × 250) mm so that different cutting trajectories do not significantly affect the total cutting time. The maximisation model of the 2D laser cutting trajectory is formulated as follows: The explanation of the model is as follows. p i;i={x,y} is the i−th pixel at location (x, y) on an amnion image that represents an area where an amnion material exists. p i;i={x,y} is a binary variable with value = 1 meaning the pixel has been allocated for cutting, and otherwise for value = 0.
Eq. (1) shows the model for the maximisation of cutting yields. In (1), the total area of cutting with different shape 1, shape 2, and shape 3, corresponding to circle, rectangle and square respectively, have to be maximised. Variable i explain pixel allocations for each shape 1, shape 2 and shape 3, respectively. C j is the proportion of cutting area for shape j − th. N j is maximum number of pixels for shape j − th and N p i;i={x,y} is the maximum number of pixels on an image. r is the radius for the circle cutting shape. ξ j is the maximum cutting area for shape j −th. l 1 and w are the length and width, respectively, for the rectangular cutting shape. l 2 is the length and width for the square cutting shape. δ is the kerf width of the laser cutting process.
The maximisation model is subject to constraints shown in (2) to (9). To limit cutting yields for all the three shapes circle, rectangle and square respectively and not exceed the available amnion material area on an image, (2) to (4) are deployed. Eq. (5) explains that all allocated pixels for the three cutting shapes should belong to the pixels on a given image. Eq. (6) explains that the total number of allocated pixels for cutting should not exceed the total number of pixels on a given image. The constraint to set the decision variables p i;i={x,y} as binary is shown in (7). Eq. (8) constraints if a pixel p i;i={x,y} has been allocated for a cutting shape, then this variable cannot be allocated for other cutting shapes. In another word, each pixel p i;i={x,y} that has value = 1 can only be allocated once. Finally, the constraint in (9) is to consider the kerf (cutting) width for adjacent pixels of different cutting trajectories with the same or different shapes.

B. Algorithm
To solve the maximisation problem, an algorithm is proposed and is presented in Algorithm 1. the explanation of the proposed algorithm is as follows.
The first step is image preprocessing and thresholding to obtained a binary image from a colour image. the thresholding process is based on Otsu's method [14]. From this binary image, amnion image segmentation can be applied and its contour can be found by implementing the border-following algorithm from [15]. This method to find the contour is selected, instead of a complex and sophisticated segmentation algorithm such as [16], due the fact that it is a straightforward and fast contour finding algorithm and it is effective for our case.
After the segmentation, the orientation of the segmented amnion is calculated with respect to the x-axis of the image. By knowing the orientation, the scan direction (on later steps) can be determined. After the amnion has been segmented and its orientation calculated, the total area A (in unit pixel) of the amnion can be calculated (by counting pixels inside the segmented region). This total area is equal to the uncut area Au due to the fact that there is no material has been allocated for cutting trajectories at this initial stage. The segmented Algorithm 1 Algorithm for the determination and optimisation of the 2D laser cutting trajectory for three different cutting shapes Input: Amnion image Output: 2D laser cutting trajectories for three different cutting shapes 1: Image preprocessing and thresholding 2: Amnion image segmentation 3: Find the amnion's contour/edge 4: Find the orientation of the segmented amnion with respect to the horisontal axis 5: Calculate A = the total amnion area 6: Uncut area Au = A 7: Duplicate the segmented amnion image and the contour/edge 8: for k = l to number of shapes= 3 do

9:
Scanning the image along its orientation  Reduce the cutting area to consider the laser kerf width 28: end for 29: end for 30: return T = set of cutting trajectories for all shapes amnion image is duplicated into another image that is used to track the pixel allocation during trajectory determination and optimisation processes (see Fig. 2a-right).
Then, for each cutting shape, a window scanning process through the entire pixel of the image is applied. The window is the shape of a cutting area (circle, square or rectangle) with a predetermined size (defined by a user). The illustration of the scanning processes is shown in Fig.2. For each window scanning step, if the area is inside the segmented amnion (see Fig. 2a) and the same area of the window in the duplicate image still not yet being allocated (see Fig. 2b) and the total proportion of the shape is still less than its maximum, then allocate the window of a shape as a laser cutting trajectory (see Fig. 2c-left) and also set the pixel on the duplicate image at the same location of the window scanning to one, meaning the pixel is already allocated (see Fig. 2c-rigth). otherwise, ignore the current scanned window. Each allocated cutting area is reduced to consider the laser kerf width. The window scanning process is repeated until all pixel of the image have been scanned by a window from all the three shapes or all the required trajectories have been determined. During the determination of cutting trajectories, a major defect, such as big hole or heavy crease, is identified and, if exists, avoided. is not yet allocated for trajectories (c) Set current scanning window into a trajectory set (left) and assign one, meaning the pixel has been allocated to a cut shape and other cutting cannot take the same area anymore, to pixel on the duplicate image at the same location of the current scanning window (right). The scanning process is carried out for all cutting shapes (circle, rectangular and square) until all material is cut or the cutting area is already achieved the number of acquired cutting areas set by a user.

III. EXPERIMENT AND EVALUATION
The proposed algorithm to solve the cutting yield maximisation problem is implemented in C/C++ programming language with a graphical user interface (GUI) using Qt library [18]. For basic image processing, such as accessing pixels, the OpenCV library is used [17]. Fig 3 shows the developed software with the GUI. The software can obtain an image loaded from a file or directly streamed from a complementary metal-oxide semiconductor (CMOS) camera's buffer memory (live image capturing). Main user inputs provided by the software are the input to define maximum material proportion for each cutting shape and the priority for cutting shapes, cut width separation for each adjacent shape to consider the laser's kerf width, camera control (to adjust exposure, contrast and pixel gain) and image thresholding parameters for segmentation.
This section is divided into two subsections: experiment from already available images and experiment from a live streamed image from a vision system. In these experiments, the size of the cutting shapes, determined from user inputs, is in unit pixel.

A. Experiment using available images
The first testing of the implemented algorithm is by using available amnion images. In this experiment, the optimisation of 2D laser cutting trajectories is tested on two amnion sheets with rectangular and circular shapes. The actual dimension for the large rectangular sheet is approximately (225 × 225) mm and for the circle sheet is approximately 50 mm diameter. For this experiment, the proportion of the cutting area for each shape is set for: 50% for circular cutting, 25% for rectangular cutting and 25% for square cutting of the total amnion sheet area. The diameter of the circle is set to be 80 unit pixel; The length of the square shape is set to be 20 unit pixel; The length and width of the rectangular shape are set to be 40 and 20 unit pixel, respectively. In addition the kerf width is set to zero assuming a hypothetical infinitely small laser spot diameter for cutting. The cutting priority for these experiments is ordered as follow circle, square and, then, rectangular shape.
Results of the experiments are shown in Fig.4 and Fig.5. In Fig.4, the optimised 2D laser cutting trajectories with three different cutitng shapes is shown for the rectangular amnion sheet. As can be observed in Fig.4, there are a few void areas on the rectangular sheet that are not allocated for cutting. This problem is due to the fact that the amnion material is transparent and wrinkled so that the algorithm detects that area as "non-material". In Fig.5, the optimised cutting trajectories is shown for the circular sheet. Cutting areas at the bottom of the circle area shown in Fig.5 touches the border of the circle causing and invalid cutting trajectories. This problem is caused by invalid edge definition at the bottom of the circle area during the segmentation process. All the calculation time for the trajectory determination and optimisation is < 6s

B. Experiment using a vision setup
In the second type of the experiment, a vision system is set up to capture amnion images in real time. The vision system consists of a 5 MP CMOS camera, spot lights and a base to place amnion samples. Fig. 6 shows the vision system constructed for this experiment.
In this experiment, an amnion image, live streamed from the CMOS camera on the vision system, was used for the optimisation of 2D laser cutting trajectories. In addition, this experiment used an amnion with random shape that imitates better the real situation the algorithm and software are to be implemented on.
In this experiment, the proportion of the cutting area for each shape and priority were set as for the previous tests. Here though, the diameter of the circle is set to be 85 unit pixel. The length of the square shape is set to be 20 unit pixel. The length and width of the rectangular shape are set to be 40 and 20 unit pixel, respectively. Fig.7 shows the random-shape amnion image streamed from the CMOS sensor as well as the resulted 2D laser cutting trajectories. As can be seen from Table I, approximately 50% of the cutting area are circular. The results from Fig. 7 suggest that the algorithm is robust to be implemented in the real vision system. The CMOS camera used on the vision system has low pixel noise so that the pixel variation will not significantly affect the algorithm [19].The calculation time for this 2D trajectory determination and optimisation is < 4s. A summary of all the results from the experiment using available images and the experiment using the vision system is presented in Table I. From Table I, with the optimised 2D cutting trajectory, material waste can be reduced up to 2%, 3% and 5% of the total material of rectangle, circle and random shapes, respectively.  Fig. 4), Circular (see Fig. 5) and Random (see Fig. 7).

IV. CONCLUSION AND FUTURE WORK
In this paper, an image processing algorithm to determine and optimise 2D laser cutting trajectories for cutting of amnion sheets has been presented. In addition, a maximisation model explaining the problem is also presented. The algorithm has been implemented in C/C++ programming language and has been tested with available amnion images and with an image live streamed from a CMOS camera from a vision system. Results from the experiments show that the algorithm is robust for different material shapes and only needs very fast calculation time for < 6s. Moreover, material wastes can be reduced up to 2% from the total cutting area.
Future work will include the improvement of the segmentation problem to identify amnion areas that are transparent and to robustly detect the amnion edge. A machine learning algorithm will also be studied to enhance the amnion segmentation problem.