Event Annotation Platform
Event Annotation Platform
Overview
Data Annotation Pipeline - Unified Event Annotation & Visual Intelligence
📅 Sep 2020 – Jan 2023
🏛️ University of Skövde, Sweden; Örebro University, Sweden; Constructor University, Germany
👥 Collaborators: Mehul Bhatt, Jakob Suchan
This establishes a universal framework for annotating and analyzing visual events across diverse domains using a unified ontological structure. The pipeline transforms raw video data from classrooms, public spaces, films, and driving scenarios into structured knowledge for advanced visual intelligence applications.
Pipeline Stages
flowchart TD
A[Input] --> B[Manual Annotation]
B --> C[Automated Processing]
C --> D1[Taxonomy Generation]
C --> D2[Bounding Box Detection]
C --> D3[Eye-tracking Integration]
D1 --> E[Knowledge Export]
D2 --> E
D3 --> E
E --> F[Query Processing]
F --> G[Visualization]
classDef stage fill:#f0f7ff,stroke:#4a90e2,stroke-width:2px;
classDef process fill:#e6f7ff,stroke:#7ec4cf,stroke-width:2px;
classDef output fill:#e6ffe6,stroke:#50c878,stroke-width:2px;
class A,B stage;
class C,F process;
class D1,D2,D3,G output;
subgraph Automated_Processing["Automated Processing"]
direction TB
D1["Taxonomy generation
via event patterns"] D2["Bounding box detection
for AOIs"] D3["Eye-tracking data
integration"] end
via event patterns"] D2["Bounding box detection
for AOIs"] D3["Eye-tracking data
integration"] end
Annotation Components
| Component | Type | Description |
|---|---|---|
| Controlled Vocabulary | Automated | From input .info |
| Annotation Templates | Automated | Taxonomy generation from input .info |
| Scene Elements | Manual | Labeling objects/actors via ELAN |
| Scene Activities | Manual | Labeling interactions/events via ELAN |
| AOIs | Manual | Drawing Bounding box |
| Low-level Attention | Automated | From eye-tracking data |
| High-level Attention | Automated | From eye-tracking data & AOIs |
Query Pattern Examples
| Query Type | Description | Output Format |
|---|---|---|
| Spatial Relations | Objects in specific proximity | Video segments + heatmaps |
| Temporal Sequences | Event A followed by Event B | Annotated timeline |
| Attention Focus | Primary AOI dwell time | Statistical report |
| Behavioral Anomalies | Expected vs actual attention | Highlight reels |
Dashboard Features
-
Visualization Tools:
- Dynamic attention heatmap overlays
- Scan path animations
- AOI comparison matrices
-
Segment Extraction:
- Query-based video clipping
- Frame export with attention metadata
- Query-based highlight generation
Key Innovations
- Unified Event Model: Consistent ontology across domains
- Hybrid Annotation: Combines manual precision with automated efficiency
- Attention-Aware Queries: Merge visual behavior with event logic
- Knowledge Export: Multiple formats for different analysis needs
Collaboration Opportunities
Open to collaboration or discussion on the technical specifications. Happy to exchange ideas and explore new perspectives.