Transforming Railway Scene Understanding with LiDAR & 2D Annotation
Discover how Shaip’s LiDAR, camera, and 2D data annotation services help AI models detect objects, understand railway environments, and improve autonomous mobility perception.
Project Overview
Modern railway perception systems demand more than generic object labeling. They require rail-specific context, accurate geometry, and reliable cross-sensor consistency across LiDAR point clouds and synchronized camera views.
To address this need, Shaip was engaged for a LiDAR rail data annotation project covering railway infrastructure scenes. The scope included annotation of LiDAR point clouds supported by five synchronized 2D reference images per scene, using a 29-class taxonomy with mandatory and optional
attributes for each class.
The goal was to create a structured, quality-controlled dataset capable of training and validating railway perception models for object detection, scene understanding, track awareness, and infrastructure interpretation.
Key Stats
Sensor Setup
LiDAR point cloud
Taxonomy
29 Rail-specific classes
Quality Targets
≥99% overall annotation accuracy
Cross-Sensor Consistency
≥99% LiDAR – 2D alignment
Challenges
- Annotating multimodal rail scenes consistently across LiDAR and 2D imagery
- Managing a rail-specific taxonomy of 29 classes with detailed attributes and object relationships
- Maintaining cross-sensor consistency between 3D and 2D annotations
- Preserving geometric precision for linear structures such as tracks and cables
- Meeting strict accuracy thresholds for class labels, geometry, and attributes in a pilot environment
Solution
Shaip designed a railway-focused annotation workflow tailored to the project’s technical requirements.
Data Strategy
Defined a multimodal annotation pipeline covering LiDAR point cloud data and five synchronized 2D reference images per LiDAR frame.
Annotation Framework
Applied multiple annotation methods based on object type and scene structure:
- 3D Cuboid Annotation for discrete volumetric objects
- 3D Polyline Annotation for linear rail-related structures
- 2D Bounding Box for object localization in images
- 2D Polygon for irregular object boundaries
- 2D Polyline for rails, cables, and markings
- 3D Cuboid in 2D views where orientation and depth context were required
Rail-Specific Taxonomy Design
Annotated 29 predefined classes spanning people, personal items, vehicles, trains, wagons, animals, tracks, switches, catenary poles, signal poles, signals, signal bridges, buffer stops, reflective test objects, and other rail-scene entities.
Attribute-Rich Annotation
Captured both mandatory and optional attributes including:
- Occlusion levels
- State and type
- Connectivity via connectedTo
- Rail-side and track identifiers
- Object order and orientation
- Signal face, aspect, and structural properties
ID & Relationship Integrity
Ensured every annotation had a unique annotation ID and each object had a unique object ID in UUID format, with bidirectional connectedTo relationships in a sensor-aware manner.
Precision Rules & QA
Implemented annotation rules including:
- Occlusion estimation across 2D and 3D
- 2D labeling threshold of objects ≥25 px in height or width
- 2D tolerance of ±3 px
- 3D tolerance of up to 10 cm
- Rotation alignment on the z-axis for object-facing consistency
- 3-layer QA with correction loops for sub-threshold outputs
Project Scope
| Dataset Element | Specification |
|---|---|
| Primary Modality | LiDAR point cloud data |
| Reference Modality | 5 synchronized 2D images per LiDAR frame: 3 IR images and 5 RGB images |
| Total Classes | 29 rail-specific classes |
| 3D Annotation Types | 3D Cuboid, 3D Polyline |
| 2D Annotation Types | 2D Bounding Box, 2D Polygon, 2D Polyline, 3D Cuboid in 2D views |
| Pilot Volume | 2 sequences |
| ID Standard | UUID-based annotation ID and object ID |
| Accuracy Targets | ≥99% overall, ≥99% attribute, ≥99% cross-sensor consistency |
Example Class Coverage
| Category | Sample Classes | Annotation Type |
|---|---|---|
| Humans & Mobility | person, wheelchair, scooter, bicycle, crowd | 2D BB / 4-point polygon / 3D BB |
| Rail Assets | train, train_front, wagons, track, switch, drag_shoe | outline polygon / polyline / 3D BB |
| Signaling & Overhead | signal, signal_pole, signal_bridge, catenary_pole | 2D BB / outline polygon / 3D BB |
| Environment & Misc. | road_vehicle, animal, group_of_animals, reflective_test_objects, smoke, flame | varied by class |
Outcomes
- Established a rail-specific multimodal annotation framework for LiDAR and image data
- Defined a 29-class railway taxonomy with detailed attribute handling
- Set quality benchmarks of ≥99% across annotation accuracy, attributes, & cross-sensor consistency
- Created a pilot-ready foundation for building high-reliability railway perception models
In strategic terms, this project positions the client train AI systems with stronger scene understanding across rail environments, where geometry, object continuity, and infrastructure context are critical.
Shaip’s multimodal annotation approach helped us structure a highly specialized railway dataset with the precision needed for perception model development. Their focus on taxonomy depth, geometric accuracy, & cross-sensor consistency created a strong foundation for our rail AI pipeline.
– Head of Computer Vision
