Improved Efficiency of Bridge Inspection with Bridge Inspection Robots
- Use of 3D model -

Figure 1. Scene of capturing images with a bridge inspection robot

1. Introduction

  There are approximately 720,000 road bridges in Japan, half of which have been in service for more than 40 years. To ensure safe and secure use of these aging bridges, every five years inspection engineers perform visual inspection on every element of the bridges, assess the soundness of the bridges, and take appropriate measures. However, this is both time and cost consuming. To improve the efficiency of bridge inspections, the use of bridge inspection robots including UAVs (unmanned aerial vehicles commonly known as drones) is currently being promoted (see Figure 1). Meanwhile, Structure from Motion (SfM) software that generates 3D models (point cloud models) from inspection images, and 3D CAD software that can display these models are already being widely used.

  Regarding bridge inspections using robots, this report introduces two points involving the use of SfM software and other programs: 1) advantages in managing inspection results with 3D models, and 2) necessity of 3D Bridge Modeling documentations.

2. Advantages in Managing Inspection Results with 3D Models

  During a robot inspection, an inspection engineer looks at images to determine the level of damage. In doing so, the inspection engineer has an image of three-dimensional position relationships in his head such as the "area, direction, and range (size) of images". Thus, there are problems in sharing locations of damage and other information with others. The user can grasp the inspection results intuitively because SfM software and other programs allow the user to (1) create a 3D bridge model, (2) estimate the image shooting positions, (3) record damages, and (4) display the 3D bridge model, image, and damages in 3D space (see Figure 2). Also, such software offers advantages in that the user can "grasp inspection status (completeness) of a target bridge", "reproduce the damage size by geometric transformation", "easily utilize the data for VR and AR because it is managed in the form of three-dimensional models" and "offer the data to other bridge inspection robots because different types of data have 3D coordinates".

Figure 2. Example of utilizing the 3D model

3. Necessity of 3D Bridge Model Creation Manuals

  In order to create a 3D bridge model, know-how is necessary for capturing images and operating SfM software. For this reason, PWRI is preparing a manual for creating 3D models without the need for reworking. The manual consists of 3 parts: "Basics of 3D bridge models", "Method of capturing images of bridges" and "Creating 3D bridge models" (See Figure 3). This autumn, the manual will be released on PWRI's website for your use.

Figure 3. Manual for creating a 3D model

4. Closing Remark

  Depending on size and location, some bridges are suitable to utilize robots while some are more efficient in visual inspection by human. We are going to build effective mixed-initiatives human-robot interaction to achieve the efficiency of the bridge management.

(Contact: Advanced Technology Research Team)

Study on Maintenance Techniques for Rockfall Prevention Facilities Consisting of Steel Members

Photo 1 Rockfall prevention fences

  In Japan, there are various rockfall prevention facilities in place to ensure the safety of coastline and mountain roads.

  Rockfall prevention nets and fences (Photo 1) are made of steel members that are relatively easy to obtain, such as wire ropes, diamond-shaped wire net, and H-shaped steel columns. Such relatively small-scale rockfall prevention facilities have been installed for many roads across the country.

  However, they can deteriorate over time due to corrosion and other factors especially in corrosive environments like coastlines. In the near future, the maintenance of rockfall prevention facilities made of steel members will be an important issue.

Photo 2 Wire ropes collected from an abandoned road section and their cross-sections

  The Structures Research Team has been promoting joint research with private companies to develop technical documents on the maintenance of rockfall prevention facilities, such as the ones described above.

  In this project, we collected data on the deterioration of and damage to rockfall prevention facilities to determine which members were most degraded, and conducted various loading tests on deteriorated members collected from the site (Photos 2), including the wire ropes and wire nets (Photos 3 and 4), to confirm how much their load-bearing capacity had been reduced.

Photo 3 Tensile test of the above wire rope

Photo 4 Heavy-weight impact test on corroded wire net

  We plan to carry out loading tests on the corroded and deteriorated H-shaped steel columns as well as more detailed analysis using the collected data, so that we can develop an efficient on-site inspection method, repair methods for corroded or deteriorated components, component replacement procedures (including the timing of replacement), and safety evaluation methods for rockfall prevention facilities with deteriorated and corroded members.

(Contact: Structures Research Team, CERI)

Study on Pavement Damage Countermeasures for Snowmelt Season

Photo 1 Potholes in the snowmelt season

1. Introduction

  Various damages can occur to pavements in cold, snowy regions during the snowmelt season due to the effects of snowmelt water and freeze-thaw actions. With many existing roads having deteriorated over the years, potholes like those in Photo 1 have been spotted in many locations in recent years. Since potholes can greatly affect the usability of roads, there is an urgent need to develop effective countermeasures as well as techniques to fundamentally prevent the occurrence of potholes.

  The Road Maintenance Research Team has been promoting projects to develop a wide range of techniques to address this issue, including techniques for surveying the conditions of the occurrence of potholes, for preventing them from occurring, for inspection and diagnosis, and for improving pavement durability. Below, we will share the details of a few such projects.

Photo 2 An example of survey using thermal imaging

2. Development of Prior Pothole Detection Technique Using Thermal Imaging Cameras

  We are developing a non-destructive diagnostic technique that can estimate areas where potholes are likely to occur during the snowmelt season, using thermal imaging cameras. We investigated the relationship between thermal images captured in the fall and actual potholes during the subsequent snowmelt season. As a result, it was found that there were local abnormalities in the thermal images from the fall in the areas where potholes occurred during the snowmelt season (these areas tended to be cooler during the daytime and warmer at night).

  In addition, water impregnation and fragmentation of the asphalt mixture were also observed inside the pavements where the temperature abnormalities were observed in the thermal images (Photo 2).

  However, there were also many areas with temperature abnormalities in the fall where potholes did not occur later in the snowmelt season. We are now exploring measures to enhance the reliability of the prior detection system.

Figure 1 Relationship between the scores and potholes

3. Development of Pothole Detection Technique Using Deep Learning

  Deep learning has been in the spotlight as a powerful technology in the artificial intelligence (AI) field in recent years. We have also been promoting the development of automatic pothole detection technology using in-vehicle cameras and the deep learning technology.As part of this project, we created a processing model and fed pothole images collected by in-vehicle cameras as teacher data for automatic detection of potholes.

  Then, the processing model was applied to unknown validation data. As shown in Figure 1, the model tended to give higher scores for images with potholes, suggesting that it was capable of automatically detecting potholes to an extent.

  At the same time, however, there were also many instances of omission and misdetections. The model is still incapable of accurately detecting potholes from road surface images captured in different environments. As deep learning is a new technology, we need to gain more technological know-how in order to use it for the evaluation of paved surfaces and enhance the model's detection capability.

(Contact: Road Research Maintenance Team, CERI)

Study on Deterioration of Concrete Structures in Icy Waters and Repair and Reinforcement Techniques

Photo 1 A Concrete Structure in the Harsh Natural Environment in Icy Waters

1. Objectives

  Infrastructures play an important role in supporting our lives. However, many existing infrastructures have deteriorated over the years. Photo 1 shows a harbor in icy waters. Structures like this one may deteriorate quicker than expected due to exposure to the harsh natural environment. In addition to salt damage and sand abrasion, there are some unique factors that contribute to the deterioration of infrastructures in icy waters, such as frost damage and sea ice abrasion. Deterioration is caused by the combined effect of all of these factors. The Port and Coast Research Team is studying the complex deterioration mechanisms of concrete structures in icy waters as well as repair and reinforcement techniques.

Photo 2 Example of a Deteriorated Concrete Structure in Icy Waters

2. An Example of a Deteriorated Concrete Structure in Icy Waters

  Photo 2 shows an example of a deteriorated concrete structure. This is a pillar of an ice boom, a facility that inhibits the entry of sea ice into a coastal lagoon, which is located at the mouth of Lake Saroma(brackish/150km2). You can see that the reinforced concrete structure is significantly worn out around the sea level. The area near the sea surface is most exposed to water and salt supply from the sea. Expansion due to freezing of the water and corrosion of rebar have deteriorated the concrete (frost and salt damage). In addition, sea ice abrasion may have also contributed to the significant deterioration.

3. Details of the Study

  This study focuses on deterioration due to the combined effect of frost and sea ice abrasion. It is considered that this type of deterioration is caused by the cycle of Steps 1 and 2 shown in Fig. 1 (left).

  Step 1: The concrete structure repeatedly undergoes a cycle of freezing and thawing, which causes small cracks in the surface where water is most absorbed.

  Step 2: The frictional force of the sea ice moving on the weakened concrete surface causes the concrete material to flake off.

  In order to recreate the conditions of Step 2, an abrasion tester, shown in Fig. 1 (right), was developed. The tank was filled with an antifreeze solution (3M Fluorinert FC-770) and supplied with ice and concrete. We measured the amount of wear on the concrete specimens after wearing them down with a reciprocating motion. Because the ice does not lose volume in fluorinert due to factors other than wear and tear, it was possible to continue testing for a significantly longer period of time than when testing in atmosphere. We have developed new techniques such as this one as we promote research on repair and reinforcement techniques.

Fig. 1 Estimated Deterioration Mechanism (Left) and Ice Abrasion Test Using Antifreeze on Concrete Structures (Right)

(Contact: Port and Coast Research Team, CERI)

AI-based Measures against Road Bridge Concrete Slab Deterioration

  Horizontal cracks and deterioration in concrete, which have been observed recently in the slabs of road bridges, tend to damage the flatness of road surfaces at an early point. However, because the top of slabs is covered with asphalt pavement, it is difficult to detect a deteriorating concrete slab under the pavement at an early stage before damage becomes extensive (Figure 1).

Figure 1. Example of deterioration of road bridge concrete slab

Figure 2. AI-based analysis of electromagnetic radar measurement results(Top: analysis image, Bottom: an example of analysis results, indicating the range of water caused slab deterioration)

  Non-destructive examination technologies such as electromagnetic radar have been developed in order to detect deterioration on slab top surfaces without removing the pavement. High hopes are placed on these technologies as technologies for detecting concrete slab deterioration from the top of the pavement, but it is not easy to repair a slab after deterioration has occurred. For this reason, with the aim of performing preventive maintenance of slabs, we are focusing on non-destructive examination technology that uses electromagnetic wave radar to detect water under pavement, which is the cause of slab deterioration, at an early stage. However, the amount of data obtained is enormous, so involves complicated analysis. Therefore, we are examining AI-based methods for analyzing the data accurately and efficiently (Figure 2).

  In so doing, it is necessary to verify that the information obtained from non-destructive examination is correct, and to obtain direct data representing the actual phenomena for a sound state and a damaged state.

Photo 1. Non-destructive examination of road bridge slab(On-site exchange of opinions with young engineers)

  Against this background, PWRI has been working with MLIT and local governments to share the issues of slab deterioration. Through these collaborations, we are also smoothly conducting on-site surveys and surveys of removed members (Photo 1).

  Regarding concrete slab deterioration in road bridges, for which there are increasingly more reports, there are still many things to be clarified, including the cause of deterioration. In addition to on-site improvements, investigation and examination must be supported continually by many people. With the intention of expanding the scope of cooperation, we're working hard to popularize measures by compiling the current status of concrete slab deterioration in road bridges at construction sites into documents.

  Note) Materials are now available on our website.

  https://www.pwri.go.jp/caesar/profile/pdf/d4398.pdf

  (PWRI document, No.4398, 2020.3)

(Contact: Bridge and Structural Engineering Research Group)