NEC Develops Vision-Language Models – NEC Corporation (NEC; TSE: 6701) has developed two new technologies for Vision-Language Models (VLMs), VLM-based Local Region Understanding (*1) and Continual Learning for VLMs (*2), which are expected to be utilized in rescue operations and recovery efforts by local governments and other organizations. VLM-based Local Region Understanding accurately identifies small-scale damage visible within images collected from disaster sites. Continual Learning for VLMs enables new types of damage information to be added as search criteria while maintaining the VLM’s original recognition capabilities. These technologies are the result of joint research with the Okatani Laboratory at Tohoku University in Japan.
VLMs are expected to support rescue and recovery efforts by enabling users to search for large volumes of images using natural language and efficiently narrow down the images that match specific situations. NEC’s technologies not only reduce the likelihood of overlooking small-scale damage but also allow search criteria to be added later. In NEC’s verification, they reduced the time required to identify specific disaster conditions to less than an hour, which previously took approximately 7 hours by visual inspection (*3). Through these technologies, NEC aims to contribute to speeding up disaster response by local governments and other organizations, with practical implementation targeted for fiscal year 2027.
Background
In recent years, as natural disasters have become more severe and frequent, the importance of minimizing damage and making rapid decisions has increased. There are often tens of thousands of on-site images immediately after a disaster occurs, but it takes an enormous amount of effort and time for staff to visually inspect them. Against this backdrop, there are high expectations for VLMs that can identify relevant images based on textual queries to enable even small teams to quickly assess disaster situations.
However, many types of damage critical to rescue and recovery decisions appear only as small details within images, occupying 10% or less of an image’s area, and conventional VLMs sometimes overlook them (*4). Furthermore, while VLMs excel at recognizing common objects and situations, they can struggle to identify previously unseen phenomena, such as region-specific events like landslides and flooding, or newly identified damage. Moreover, additional training can reduce the accuracy of previously recognized events (forgetting).
To address these challenges, NEC developed VLM-based Local Region Understanding, which comprehends even the finest details of images, and Continual Learning for VLMs, which adds new information while retaining existing knowledge. These research findings were accepted as papers and presented at the international computer vision conferences “ECCV 2024” and “ECCV 2026.”
Value Proposition
These technologies reduce the likelihood of overlooking small-scale damage visible in field images and cut the time required to assess specific damage conditions to less than an hour, which previously took personnel approximately 7 hours to complete visually. During the initial response phase immediately following a disaster, they accelerate assessment through text-based searches (e.g., “roads blocked by landslide debris”). When making recovery decisions, they help prioritize on-site inspections by sorting out images of damage to roads and buildings. Furthermore, search criteria can be continuously added to enable flexible responses to unforeseen disaster risks—such as flooding following an earthquake—and to local conditions.
Features of This Technology
1. High-precision detection of even the finest details in images, such as small areas and narrow regions
NEC’s proprietary VLM-based Local Region Understanding incorporates region information of objects within an image into the VLM’s feature extraction process, enabling it to extract information by focusing not only on the entire image but also on small areas and narrow regions. As a result, NEC has confirmed that even small areas accounting for 10% or less of an image can be identified with the same level of accuracy that conventional VLMs achieve for areas accounting for about 30–40% of an image. This enables the technology to accurately identify even small-scale damage, such as partial structural damage to buildings or cracks in roads, and to generate detailed descriptions.
2. New information can be added while maintaining the recognition capabilities of the existing VLM
NEC’s proprietary Continual Learning for VLMs uses not only additional training data, but also a diverse range of descriptions generated by the VLM itself before additional training. By learning from both sources together, the system can recognize new events while minimizing any decline in the accuracy for previously recognized events. Since there is no need to prepare a large volume of historical training data, search criteria can be added repeatedly at a later stage. This allows images to be narrowed down using both newly learned disaster-related events and existing features such as mountains and rivers as clues; in NEC’s verification experiments, this reduced the number of images requiring final human verification from 5,035 to an average of 544 (about one-ninth), cutting the review time to less than an hour (*3).
Through these technologies, NEC will help local governments and other organizations quickly assess disaster situations. Going forward, in addition to ground-level site images, NEC will combine these technologies with the analysis of wide-area observation data, such as optical and synthetic aperture radar (SAR) satellite imagery, to support a multifaceted understanding of disaster conditions, and will advance research and development aimed at preventing disasters before they occur.
For more information visit: https://www.nec.com/.
