|
Navigating Covid-19: An Image Analysis Robotics Expert
31 August 2020
With the Covid-19 wreaking havoc worldwide, every country faces the dilemma of having to choose between controlling the pandemic and reopening its economy before a vaccine becomes available. Businesses and public institutions are striving to minimise crowd gathering and unnecessary social contact by changing the way they operate. Devices like body temperature sensors have become essential devices for epidemic control in public places.
Based in the Hong Kong Science Park, Insight Robotics specialises in AI-assisted high-resolution image analysis technology and services. Since its inception, it has been using unmanned aerial vehicles (UAVs) and wildfire detection robots to guard some 3 million hectares of forests and land in Hong Kong, mainland China and other parts of the world from the threat of wildfire. During the Covid-19 outbreak, it has also assisted the Hong Kong Science Park and Hong Kong Red Cross Society in setting up body temperature screening systems at cost.
Capitalising on the strengths of the Guangdong-Hong Kong-Macao Greater Bay Area (GBA) in the mechatronic industry as well as Hong Kong’s close links with the global technology supply chain, Insight Robotics is able to offer one-stop hardware and software services to clients. Following the realignment of global supply chains induced by the pandemic, the company has had to adapt and explore different modes of operation. It now focuses on the research, development and promotion of AI software technology. Speaking recently to HKTDC Research, Rex Sham, Co-founder and Chief Science Officer of Insight Robotics, discussed the application and prospects of his company’s image analysis technology during the pandemic and beyond.
Group Temperature Screening Systems
Fever is a common symptom of Covid-19. Thus, operating temperature scanners at the entrances to public spaces has become a standard practice, in addition to wearing face masks and using alcohol-based hand wash. However, taking people’s temperatures one by one in high-footfall locations is impractical and exposes the frontline staff to health risks. As a result, some commercial buildings are using group temperature screening systems. Most of these systems need regular calibration and maintenance to prevent factors such as room temperature and intensity of sunlight affecting the screening accuracy.
Dual temperature black body calibrator.
In order to solve these problems, Insight Robotics has developed a group temperature screening system using its expertise in the fields of infrared image capture and AI analytics technology. “Most of the infrared thermal imagers in the market require daily calibration to accurately spot people with fever. Our dual temperature black body calibrator with absolute temperature reference allows the thermal imaging camera to automatically make adjustments according to environmental factors,” said Sham.
“Our system is not only capable of making automatic calibration adjustments, it has additional custom features. For example, the imager can pixelate the faces of the people being screened for security and privacy reasons. This is important for airports, to protect the identity of international travellers. For international brands, protecting customer privacy is also important for their reputation and to maintain customer loyalty. Our system will pixelate the faces of people without fever and delete the images after spotting people with fever,” he said.
Visible light imaging can pixelate the faces of people without fever.
Insight Robotics’ group temperature screening system is now installed at the Hong Kong Science Park and several branches of the Youth and Volunteer Department of the Hong Kong Red Cross. Sham said they are doing this mainly as a public service to fight the virus together. With this in mind, they have specially developed a temperature screening system with basic functions and a price tag of HK$20,000-30,000. This price is far below that of similar non-AI-assisted models currently available in the market, which are usually priced at HK$50,000-60,000.
Insight Robotics has also launched three regular models for commercial customers. With a price tag of over HK$80,000, these systems are designed for different screening environments (such as corridors and open spaces) and multiple detection points (maximum 20 persons or 40 persons). Their most recent clients include an airport in Indonesia and a casino in Portugal.
Group temperature screening systems of Insight Robotics.
Reasons for Engaging in Research on Image Analysis
The business of developing temperature screening systems aside, the “main business” of Insight Robotics is to use UAVs and AI-assisted wildfire detection robots to collect infrared thermal images of land surfaces from high altitude for large-scale ground image analysis. The company’s research and business goal since its inception in 2009 has been to protect forests and prevent wildfires.
The research path of Sham and his team has not been plain-sailing. “Back in 2010 when the internet-of-things was still not widely known, we planned to use wireless sensor networks to detect wildfires by placing a large number of sensors in forests. However, we discovered that sensors only have a detection radius of a few dozen metres, which means thousands of sensors are needed to cover even a small forest. We reckoned that several billion sensors would be needed to cover all forests worldwide. The difficulty and maintenance cost of such a project is comparable to the use of 5G technology. Large quantities of sensors would also trigger an electronic waste crisis and lead to an ecological disaster, which is contrary to our principles.
“We also considered using cameras on satellites to detect wildfires. But satellite images are more obscured by clouds than we thought, and so not very useful for tropical rain forests where there is a great deal of water vapour and the forests are frequently shrouded in rain clouds. The clear geographic pictures we see on online maps are prepared by service developers using synthetic satellite images taken over many years. In other words, ‘god’s perspective’ is actually the view of the world from the satellite. Most of the time there is not much to see.”
Sham and his team finally decided on the use of infrared thermal imaging technology after considering the feasibility from the technical and business points of view. He said: “Images can mainly be divided into infrared thermal images and visible light images. Visible light imaging analysis involves higher technical requirements. Morning mist is often mistaken for wildfire smoke and may trigger false alarms. It was only after accumulating years of research experience that we finally had the confidence to launch smoke detection and analysis products. We were lucky enough to work with relevant government departments in Hong Kong and Guangdong in infrared thermal imaging detection technology and we acquired a lot of valuable experimental data for wildfires over the years.”
“Thanks to the referral of the Agriculture, Fisheries and Conservation Department, we were able to collect data on local farm workers’ burning of garbage and wild grass on private plots. We also co-operated with the Guangdong Academy of Forestry in conducting field sampling of controlled forest fires. The Academy set aside for us a pest-infested forest about 1 sq km in size and set it on fire under the protection of good fire belts. Using detectors and thermal imaging sensors, we collected data on the source and spread of fires. This made us the world’s only research team for wildfire detection and prevention that has undertaken a live hands-on experiment on a forest fire,” he said.
The NASA Earth Observation Laboratory pointed out in its research that 92% of wildfires are caused by humans. Sham therefore believes that fire prevention by means of image analysis is a cost-effective method. “Wildfires mostly occur in the vicinity of human activities” he points out. “In other words, there are usually telecoms networks and power grids near the source of wildfires. By placing robots with thermal imaging sensors near these telecoms networks and power grids, we can use the power grids to power our equipment and transmit data through the telecoms networks. When wildfires are spotted, the robots can automatically transmit the fire alarm to nearby bases via 4G, 5G or Wi-fi networks and notify the fire departments via the internet. However, if wildfires break out in remote locations, fire alarms cannot be transmitted via ground stations and we have no alternative but to use expensive satellite networks.”
“Pictures taken from the air or on the ground are two-dimensional,” Sham explains. “Firefighters need to know the latitude, longitude and altitude to accurately locate wildfires. For this reason, we have built a three-dimensional electronic model that combines the angles, bearings and pixels observed from all sources to determine accurate fire co-ordinates. In other words, two-dimensional images can be converted into three-dimensional co-ordinates. We have obtained patents for this geographical information system (GIS), which we named Insight Globe, in Hong Kong, mainland China, Indonesia and the US.”
Insight Robotics’ wildfire detection system monitors round-the-clock ignition sources as small as 6 sq m in size within a radius of 8 km and transmits the fire co-ordinates to the disaster relief departments within 10 minutes.
Going Global
Sham said the experience they gained in Hong Kong and mainland China helps them promote their technology in overseas markets. He said: “Very few people realise that Hong Kong leads the world in technical applications and management technology in agriculture and fisheries. Hong Kong has a rich natural flora as well as farmed crops. Hong Kong is skilled in tree planting and has good planning for forest belts, firebreaks and fireproofing species, which explains why hill fires rarely occur in local forests. As in neighbouring regions, fires are divided into urban fires and rural fires. Urban fires are the responsibility of the Fire Services Department while rural fires are the responsibility of the Agriculture, Fisheries and Conservation Department. Since most wildfires would spread from the urban to the rural areas, these two departments have to work closely together to promptly discover and put out “cross-regional” wildfires. This long-established and efficient wildfire control system underscores Hong Kong’s important position in agricultural and fisheries circles worldwide.”
Mainland China, which boasts huge tracts of forests, is an important base for Insight Robotics. Here the company can perfect its systems and demonstrate their performance for overseas clients. Sham said: “We have visited places like Jinan and Guangzhou with our overseas clients to see how our AI-assisted system can help local fire control centres to prevent and control wildfires. Based on our capability and experience in handling forests as big as these on the mainland, we can convince them that we have no problem coping with their forests and plantations.”
Indonesia is a major producer and exporter of palm oil and timber. It is also one of Insight Robotics’ most important markets after China. “In Indonesia, there are occasional outbreaks of fire in the plantations of forestry companies that may spread to the primary forests nearby,” said Sham. “After the primary forests have been destroyed, forestry companies would try to buy up these lands at a low price while paying compensation. These companies have large plantations of a single species, which increases the risk of pests and diseases. The companies sometimes let wildfires spread unchecked in order to be able to lease more land for planting more trees at a low price.”
In order to crack down on illegal acts of arson and protect primitive forests, the Indonesian government launched the world’s first “Timber Legality Verification System” (SVLK) in 2016. The system regulates forestry and timber exports by strict licensing and requires licence holders to establish fire-fighting measures in their plantations. Companies caught for arson can have their licence revoked and lose their export permit. Insight Robotics has rapidly become an important partner of the Indonesian government’s wildfire prevention department as well as of private forestry companies.
Sham said: “The Ministry of Environment and Forestry of Indonesia bought our wildfire detection system in order to control arson. Some forestry companies have also installed our system to ensure that they have sufficient evidence to handle matters of liability when they are affected by fires in adjacent plantations.”
Insight Robotics was able to establish its leading position in the industry during a forest fire detection competition in South Africa. “The Union of Forestry Companies in Africa put its wildfire detection system out to tender,” explains Sam. “The bidding was in the form of a fortnight-long competition. The organiser repeatedly started fires on the competition grounds and required the contestants to detect the precise location of the fire sources within the shortest time. We were the only company that used robots. In this ‘battle between humans and robots’, it turned out that robots are far superior to humans in response and precision.”
Insight Robotics’ wildfire detection system tested in South Africa.
Apart from agricultural countries and regions, Insight Robotics is also favoured by insurance companies and financial institutions in many countries. Sham said: “Although forest fires in developed countries may not actually affect forestry production, they do great harm to property. A reinsurance company in Switzerland had to pay US$60 billion in insured losses for power grids, wireless networks and other infrastructure facilities destroyed by wildfires. Theoretically, insurance companies would only insure against damage or loss caused by natural disasters, but very often there is no evidence to determine whether the fire is caused by natural or human factors. Through co-operation with us, insurance companies are able to link our wildfire detection system with their insurance plans. This makes it much easier for them to effectively determine their statutory responsibilities and make reasonable compensation and manage risks.”
He added: “Some financial companies with business in agricultural and forestry production are also using real-time monitoring of the impact of pests and diseases, floods and fires on the supply of raw materials and therefore on futures prices. When their production business is affected by natural disasters, they can predict the impact on businesses in the same region and make timely futures trading to offset risks and even benefit from their investment.”
Sustainable AI Development
Pointing out that his company’s business model encourages sustainable AI development, Sham said: “AI is like a baby and needs learning materials to develop new skills. Insight Robotics constantly collects images from drones and wildfire detection robots on some 3 million hectares of land, and we are able to see not just fires but also images requiring further analysis. These are unique learning materials, they are our assets for future development, and they are crucial to the extension of AI capabilities to the analysis of different images.”
For example, Insight Robotics has developed a tree analysis tool by studying data collected in Indonesia over the years. From images taken by drones, AI can count the number of trees and detect the species, height, size, fallen leaves and health conditions of each tree. He pointed out: “Generally speaking, trees avoid touching one another. In plantations, workers would plant trees at intervals. There is the phenomenon of “crown shyness” in natural forests, meaning that some trees would keep a distance from one another. This allows drones to capture the images of individual trees like a scanner from above. We would instruct our staff to identify on site the image of each tree so AI can learn how to analyse the images of different trees.”
According to Sham, tree analysis is now well received by forestry companies and has become a major source of business for Insight Robotics. Explaining the need for this service, he said: “While a plantation company will plant and cut 2 million trees every day, the total number of trees planted has been falling in recent years, declining by about 10% of the total. The reason may be due to theft, insect pests or mistakes in manual counting. If they use radio frequency identification (RFID) or other types of active sensors to manage their plantations, they will have to make an astronomical investment. Our image analysis just needs a drone to scan a plantation from time to time, collecting information that is difficult to obtain manually, such as the height of trees. With this valuable data, the plantation company can cut trees according to height regulations.”
Insight Robotics improves the efficiency of managing plantation forests through terrain mapping and crop assessment by drones.
Crisis and Opportunity under Covid-19
Insight Robotics has been carrying out R&D on AI in the Hong Kong Science Park and relying on its production chain in the GBA for the production of drones, wildfire detection systems and components used in data collection. “We would ask our drone manufacturers in the GBA to dispatch orders overseas directly,” explains Sham. “We ask the manufacturers to provide us with information on raw materials suppliers before shipment and send our staff to inspect the quality of both the raw materials and the finished products. We must ensure the quality of the raw materials used, otherwise, the drones may disintegrate in mid-air and the lithium batteries may explode when they hit the ground and start a forest fire.”
After the outbreak of Covid-19, Insight Robotics was unable to dispatch staff to the GBA for product testing. To ensure product quality, they therefore decided to carry out the final assembly of the drones in Hong Kong. Since parts are also in short supply, they asked their clients to use small four-rotor aerial cameras instead. “The large fixed-wing drones we assemble have a cruising range of 2,000 hectares, whereas the small four-rotor aerial cameras can generally only cover 1,000 hectares,” says Sham.
Fixed-wing drone of Insight Robotics.
Insight Robotics sources the crucial infrared thermal imager from different suppliers depending on client requirements. The company typically utilises Dali Technology Co products on behalf of its clients in mainland China, which could detect wildfires within a radius of 5km. For other countries and regions, European and North American products are generally used, which are said to be able to detect any wildfires within a radius of 8km. But in practice, the company has successfully detected ignition sources some 20 km away for a client in Mexico.
Sham admits that the infrared thermal imager is a relatively sensitive technology, with both civil and military applications. Expanding upon the potential of the technology, he said: “The best infrared thermal imagers can collect 30 images per second and can be used to detect heat-seeking missiles and rockets. We only use infrared thermal imagers with a frame rate of nine images per second at best. Nevertheless, even civilian infrared thermal imagers are subject to complicated import and export inspections because of the sensitivity of the technology. A Singaporean company that recently settled in the Hong Kong Science Park is able to manufacture infrared thermal imagers with a lower rating. This is a good start although this type of imager is only suitable for temperature guns.”
Sham also pointed out that Covid-19 has turned cutting-edge technologies such as image analysis, cloud computing and AI from something irrelevant to people’s daily life into necessities. “In the past, forestry companies in Indonesia usually hired workers to inspect plantations to provide some local employment. Indonesia is extremely hot in summer, with temperatures at times exceeding 40°C. During the epidemic, it was impossible for workers to work in plantations growing on steep hillsides with their masks on. These companies have no alternative but turn to AI for assistance.”
The e-sports industry has been growing apace in recent years. As a result, a number of related technologies – notably graphic cards and graphic accelerators – have also developed rapidly. In the case of Image analysis and learning, this can now be completed in a matter of hours than in up to three days as used to be the case. Sham is convinced that imaging technology and applications associated with the e-sports industry are just beginning.
Sham believes that the epidemic has deepened people’s understanding of cloud computing, making them see that private clouds, hybrid clouds and shared clouds can satisfy the requirements of different businesses for data management and privacy protection. This could hasten the development of relevant technologies and applications by as much as four years. Insight Robotics has been able to significantly improve its handling capacity through cloud computing. For most companies, including forestry companies, production estimates are important business secrets. It is necessary to create a hybrid cloud setup to facilitate linkage with the shared cloud while storing the business secrets of clients in their private cloud servers.
Sham and his research team have been supplying AI analysis software to public and private institutions in Hong Kong in recent months. Their services include helping clients to find sections of road that need repair, analysing the flow of different types of vehicles on roads, and measuring the height of buildings. The Lands Department, which offers services such as mapping information and land surveying, has also become a client of Insight Robotics. “Every three months, the Lands Department uses the aerial images taken by the Government Flying Services to record topographical and land use changes across Hong Kong. It used to manually identify the land use changes using GIS software, which not only requires considerable time and human resources, but is also difficult to operate. Now, however, our AI-assist aerial survey graphic analysis system automatically labels such objects as skyscrapers, houses and cars, allowing the department to make better use of its manpower,” says Sham. “For our part, we are responsible for the maintenance of the AI-assisted computers used by the Lands Department. In line with its needs and the latest technological developments, we also provide the department with new software functions and value-added services, including such facilities as being able to distinguish taxis of different colours or even determining the location of parasols or solar panels.”
He also disclosed that thanks to better than expected AI software sales during the epidemic, Insight Robotics’ year-on-year business performance has risen rather than dropped in the first half of this year.
- Technology
- Technology
- Technology
- Technology
- Technology
