Master Seminar "Multimodal Temporal Data Processing in Autonomous Driving"

Organizer: Emec Ercelik, Burcu Karadeniz

Contact: emec.ercelik(at)

Modul: IN2107

Registration: Via Matching System

Type: MasterSeminar

Semester: Winter Semester 2019/2020

ECTS: 5.0/4.0

Time & Location: Friday, 10.00-12.00 @ 03.07.011


News (Registration)

  • For the slides of introductory session, please click here.
  • Please check the preliminary topics added below.
  • The first session will be on 18.10.2019. We will introduce and distribute the topics.
  • Please provide a CV and a motivation letter that states your achievements and aims related to this seminar until the end of 23.07.2019 ( Please send your documents to "emec.ercelik(at)" with the subject line "Seminar: Multimodal Temporal Data Processing in Autonomous Driving" ).
  • There is no scheduled preliminary session.


Control systems that steer cars autonomously on the road should have the capability of making sense of the environment. It is a remaining problem that sensors mounted on cars only function in certain conditions. Therefore, the control system should consider a variety of sensory information (images, depth, velocity, map, etc.) at the same time to give rise to a reasonable control output. In addition, human drivers constantly observe the environment while driving, taking into account the past observations with the future predictions. In this seminar, we will be looking into the methods to have an understanding of the environment with processing multivariate sensory information at the same time with temporal information.

In this seminar course, students gain knowledge in sensor fusion and temporal data processing methods, challenges in autonomous driving related tasks, and how learning is applied to problems in this domain.


  • Stereo vision
  • Sensor fusion
  • Recurrent Neural Networks
  • Temporal Data Processing
  • Object Detection

Examples of sensor types that will be considered in this seminar

  • Camera
  • Lidar
  • Radar
  • GPS

Datasets that contain multimodal temporal data

Preliminary topics

  • Sequential multimodal data processing for 3D object detection using RNNs
  • Sequential multimodal data processing for 3D object detection using convolutional networks
  • Object tracking networks for multimodal data to improve object detection results
  • Object matching algorithms and metrics in successively sampled autonomous driving data