Yes. By tradition another member who did get elected will be asked to step aside for the party leader. Most will do so. However, in a past election, I believe Chretien tried that and the encumbent told him to get lost. Which did great things for the politician's career, and ability to promote his issues in caucus.
But that's what happens.