The EU-funded CREXDATA project has announced a major technological milestone with the successful development and field testing of its new Augmented Reality (AR) ecosystem. Designed to bridge the gap between extreme-scale predictive analytics and on-site decision-making, the system provides first responders and maritime operators with real-time, immersive situational awareness.
CREXDATA is building a “Prediction-as-a-Service” (PaaS) platform to manage streaming data of immense scale and complexity.
The Architecture: Wearables Meets Extreme-Scale Data
The technical core of the system consists of a robust communication and data handling server that interfaces with wearable hardware.
- Primary Hardware: The system utilizes the Microsoft HoloLens 2 to deliver hands-free, gesture-controlled overlays.
- Data Streaming: High-velocity data feeds, including real-time sensor data and predictive models, are managed via a Kafka-based server architecture.
- Outdoor Localization: To solve the challenge of precise outdoor positioning for AR, the project developed a companion Android application. This app acts as a high-precision GPS provider, streaming location data to the headset via a local Wi-Fi or 5G hotspot.
Intelligent Visual Strategies
The AR overlays are designed to prioritize rapid comprehension in high-pressure environments. Key visual features include:
- Context-Aware POIs: Points of Interest (POIs) such as hazard zones or resource locations are color-coded by risk and include confidence indicators to reflect the uncertainty of predictive models.
- Geospatial Anchoring: Real-time drone feeds and hazard zones are anchored to the physical world, allowing operators to see “through” obstructions.
- Trajectory Corridors: For maritime navigation, the system renders expanding “uncertainty cones” that visually encode the potential variability in a vessel’s path over time.
Multi-Domain Application Overview
The CREXDATA AR system is currently being validated across three primary operational domains:
| Domain | Key AR Capabilities | Technical Highlight |
| Weather Emergencies | Flood level visualization, fire front tracking, and drone feed integration. | 3D terrain mapping with real-time hazard overlays. |
| Maritime Navigation | AIS data visualization and predictive trajectory corridors. | Hands-free bridge operation with situational “heads-up” data. |
| Health & Safety | Real-time mental fatigue prediction and team monitoring. | Integrated eye-tracking to assess cognitive load. |
Real-Time Fatigue Monitoring
A standout technical feature is the Mental Fatigue Prediction component. Utilizing HoloLens 2’s integrated eye-tracking sensors, the system monitors physiological markers to classify an operator’s fatigue into four distinct states. If a responder’s cognitive load reaches critical levels, the system can trigger alerts for team leaders, ensuring that task allocation remains aligned with the safety of the personnel.
Field Readiness and Future Outlook
The system has undergone two rigorous rounds of field trials in flood and wildfire contexts. Feedback from first responders led to technical refinements, including faster access to key data layers and improved legibility of information panels under harsh operational conditions.
Flood visualization examples with navigation arrows, POI panels, water level indicators, and drone feed overlay:
