Smart transaction picking in tier-to-tier SBS/RS by deep Q-learning

Abstract

By the rapid growth of e-commerce, the intralogistics sector is facing new challenges. Intralogistics sector requires more flexible, scalable processes with maximum reliability and availability. They are complicated and interconnected systems, whose all components are required to be perfectly coordinated with each other for optimal functionality. In this work, we study an intralogistics technology, shuttle-based storage and retrieval system (SBS/RS), where shuttles are tier-to-tier. In this novel system design, in an effort to increase shuttle utilization as well as decrease initial investment cost, shuttles are designed in a more flexible travel manner so that they can change their tiers within an aisle by using a separate lifting mechanism. Due to the complexity of such system design as well as aiming to obtain fast transaction process time by the decreased number of shuttles in the system, we implement a Deep Q-Learning (DQL) approach to let shuttles select the best transaction to process based on its targets. We compare the performance of the DQL by the average cycle time per transaction performance metric with the other well-known selection rules, First-in-First-Out (FIFO) and Shortest Process Time (SPT). Results show that Deep Q-Learning approach produces better results than those FIFO and SPT.