It is correct that the point of contact with the wall is moving down infinitely fast when the plank hits the floor. You can think of the base being fixed and the wall moving away from the base. The infinite speed comes because when the wall hits the floor, the plank is perpendicular to the wall, so the angle of the plank is not constrained.