Offline Reinforcement Learning in Autonomous Driving

Pascal Schindler

Abstract

Current online reinforcement algorithms struggle to utilize large and diverse datasets. In contrast, offline reinforcement learning algorithms offer an efficient solution for this problem. This paves the way for data-driven reinforcement learning. With the help of offline reinforcement learning algorithms, it is now possible to apply reinforcement learning in costly environments such as healthcare or autonomous driving. For this reason, we tested one of the latest offline reinforcement learning algorithm, CQL, in the autonomous driving environments CarRacing-v0 and Carla. We evaluated the CQL performance on different datasets with different α values. The α value controls the conservatism of the algorithm. Thereby, we tested the hypothesis that higher α values perform better the better the dataset and lower α values perform better the worse the dataset. To this end, we created expert datasets with excellent trajectories and imperfect datasets with noisy trajectories. Furthermore, we evaluated the CQL performance in contrast to behavior cloning and the state-of-the-art online reinforcement learning algorithm SAC.

Topics
Reinforcement Learning Offline Reinforcement Learning AI ML Autonomous Driving
Research Methods

Publication Data

Author: Pascal Schindler
Thesis Type: Bachelor's Thesis
Pages: 67
Language: English
DOI:
About the Author:
Major / Study Program: Industrial Engineering
Primary Field of Study:
Additional Study Interests:
License: CC BY 4.0
Date of Publication: 12/01/22
Status: Available
Date of Grading: 09/13/21
Institution: Institut für Angewandte Informatik und Formale Beschreibungsverfahren (AIFB) (Institut für Angewandte Informatik und Formale Beschreibungsverfahren (AIFB), Germany)

Endorsements

Thesis Documents and Supplemental Materials

01/28/23 10:05:39 AM
# Description Type Upload Date Location
1 Thesis Document PDF (29.74MB) 10/22/22 01:00:00 AMIPFS Download Raw