Hostel Is All You Need