The invasion of Ukraine in late-February 2022 has taught important lessons about things like maintaining Situational Awareness (SA) and the ability of Remotely Piloted Air Systems (RPAS) to make a huge difference in combat.

But these insights aren’t new to Melbourne-based Sentient Vision Systems which has been working since 2018 on a system which can conduct Wide Area Surveillance on land with an extremely high probability of detecting moving man-size targets. The company has done this development work under a AUD$5 million Defence Innovation Hub (DIH) contract awarded by the Australian Department of Defence.

Sentient Vision Systems has developed ViDAR Land – ViDAR stands for Visual Detection And Ranging and is said to be the world’s first true Optical Radar. This is a Wide Area Motion Imagery (WAMI) system whose technology has already been proven at sea in service with the U.S. Coast Guard, Royal Australian Navy (RAN), Australian Maritime Safety Authority (AMSA) and other agencies worldwide. ViDAR uses Sentient AI, a combination of Artificial Intelligence and Computer Vision, to examine every pixel in every frame in an optical sensor’s imagery feed – in real time – and detect targets that a human operator would miss, regardless of the resolution of the sensor.

Thanks to its Sentient AI, ViDAR Land can autonomously detect, track, classify and filter thousands of objects and help the operator focus solely on targets of interest, from columns of vehicles to individual humans trying to be inconspicuous. It’s designed to provide real-time SA in a range of applications including wide area monitoring, Intelligence Surveillance and Reconnaissance (ISR), force protection, border security and law enforcement.

ViDAR Land can be packaged in an external pod or integrated into a crewed or uncrewed fixed- or rotary-wing platform. And it’s scalable, so Size, Weight and Power (SWaP) aren’t issues.

The ViDAR processor scans the incoming imagery feed and when it detects an object it is disseminated to the control station or Mission Management System (MMS) in real-time. Detected objects are displayed to the operator as markers on the geographic map interface at their detected locations. Detailed information can be viewed for each detection, including classification, track, and video history. Multiple detections are grouped or represented as a heatmap for easy batch processing or review. Sentient have developed a user interface (UI) that is both easy to use and intuitive and critically, the operator is consistently supported by the AI and other interactive features presenting information in a way that suits the task at hand and reduces operator workload at every step.

ViDAR Land’s data output is STANAG 4609-compliant so can be shared with other advanced Processing, Exploitation and Dissemination (PED) systems, enhancing data fusion to help commanders’ decision-making.

One of the advantages of a ViDAR Land pod is that the onboard sensors and processors undertake all the detection, processing and classification of target data, leaving the platform’s primary EO/IR sensor free to conduct other tasks. That sensor is cross-cued to ViDAR Land, however, so that it will slew when commanded to inspect a target more closely. The vast number of sensor pixels analysed in realtime by ViDAR Land, coupled with a ground resolution measured in centimetres, enables ViDAR Land to provide multiple virtual turrets, augmenting the system’s ability to provide full video streams of several areas of interest concurrently from a single airborne sensor platform.

The Australian Department of Defence has invested in ViDAR Land with two particular tactical surveillance projects in mind: Project LAND 129 Ph 3, in which the Insitu Integrator has been named the preferred supplier, to replace the Army’s existing Textron Shadow 200 RPAS system with a new Tactical Uncrewed Aerial System (TUAS); and Project AIR 7003 which, until it was cancelled in March 2022, was intended to see the introduction of a fleet of MQ-9A Sky Guardians for the RAAF’s Medium Altitude Long Endurance (MALE) armed RPAS requirement. Although this project has been terminated, Sentient Vision Systems’ development work means that ViDAR Land could be integrated with MALE RPASs fielded by other operators. Adding ViDAR Land to each of these systems would enhance their ISR capabilities enormously. Maritime ViDAR already equips the Integrator’s stablemate, Insitu’s ScanEagle, in both USCG and RAN services.

In the U.S.A. and Europe, Sentient Visions Systems believes ViDAR Land can make a significant difference to the surveillance of things like borders as well on a dynamic, fast-moving battlefield where ongoing, real-time SA is vital. Its best sales pitch is the footage coming daily from Ukraine.

Adapting this technology for land use hasn’t been easy. There are significant differences between the land and maritime domains and the company has had to work carefully on its Sentient AI to deal with a very different operating environment.

For a start there may be thousands of potential targets in the area of interest – vehicles and individual humans; secondly, the land environment is ‘super-cluttered’– targets may be masked or partially concealed by static vegetation or buildings; thirdly, they are likely to leave tracks or traces of their presence even after they have moved on, which maritime targets don’t.

So the challenge has been to detect targets whilst simultaneously developing classification systems and filters to eliminate those that are of no interest as well as false alarms. This is no easy task, but Sentient Vision Systems has been developing this AI and Computer Vision technology since the mid-1990s.

Development and trials have shown that the combination of arrays of ultra-high resolution cameras, on-board processing and a powerful user interface provides an ISR system with capacity far beyond human capability. Advanced AI and filtering features including geo-fencing allow the operator to focus solely on objects of interest across large areas of terrain that are awash with clutter. The system provides direct access to object data such as video imagery, location, speed and bearing, tracks and classification, both in real time and forensically. Importantly, it also operates at medium altitude, making it an important enhancement for MALE RPAS platforms.

Sentient currently has a AUD$1.5M contract from the Defence Innovation Hub to increase ViDAR Land’s full capability so that it can detect targets from altitudes of 25,000 ft, so watch this space.

ViDAR Land is designed to perform three different missions using the same technology: Overwatch; Area Scan; and Patrol. In each case, it contributes to and strengthens a battlefield commander’s Common Operating Picture (COP).

In an Overwatch mission an area is held under persistent observation. This might be to provide ISR support for a battlefield or reconnaissance to build up a ‘pattern of life’ picture. Geofencing allows monitoring of Areas of Interest; tracking and classification providing real-time information on ground movements, both hostile and friendly forces, whilst filtering out neutrals, and relaying a tactical view of the area in real-time. Not only do operators get live streams, if they want, the system can also provide a historical view of personnel and vehicle movements.

Area Scan is like conducting a search: an operator or commander may have no idea what’s out there in a new area of operations, so an airborne platform flying a routine search pattern can build up an extremely detailed picture of activities and movements within a new area of interest. Importantly, everything the system sees, the operator can return to later in the MMS or in another live pass over the target. Once a section of terrain has been mapped properly, the mission might change to Overwatch and the search for patterns of life.

Patrol enables an operator or commander to maintain a permanent watch on things like roads, pipelines, and borders and to track targets – this could also be an extension of both the Overwatch and Area Scan missions. The search is more focused and the operator-selectable search parameters may be more specific.

Importantly, in all of these cases, imagery processing is done aboard the airborne platform. This reduces significantly the bandwidth and power demands of the Tactical Datalink, which is important for small RPASs and also for users on the ground using small hand-held devices. And it provides local intelligence for dispersed commanders, reducing their dependence on data from more centralised sources and enhancing their freedom of decision-making and action.

Even more importantly, an EO sensor is passive – it doesn’t alert the surveillance subject, unlike an active radar or laser. And a small RPAS at the right altitude and speed is virtually invisible and inaudible so doesn’t betray its location (and interest in the target) and invite a potentially lethal response. This is one of the lessons the ADF, via the Defence Innovation Hub, was keen to apply, so ViDAR Land is designed to operate outside the detectable range, which is why Sentient Vision Systems has developed its AI so that it can detect a human-size target from altitudes of greater than 12,000ft.

The podded ViDAR Land system can become an item of role equipment for existing aircraft and RPAS; but Sentient Vision can also conduct a more elaborate integration as it has done with several maritime ViDAR customers.

Whatever the case, ViDAR Land is now through its development and is ready for operational use.