Coastal surveillance video helps officials to obtain on-site visual information on maritime traffic situations, which benefits building up the maritime transportation detection infrastructure. The previous ship detection methods focused on detecting distant small ships in maritime videos, with less attention paid to the task of ship detection from coastal surveillance video. To address this challenge, a novel framework is proposed to detect ships from coastal maritime images in three typical traffic situations in three consecutive steps. First the Canny detector is introduced to determine the potential ship edges in each maritime frame. Then, the self-adaptive Gaussian descriptor is employed to accurately rule out noisy edges. Finally, the morphology operator is developed to link the detected separated edges to connected ship contours. The model's performance is tested under three typical maritime traffic situations. The experimental results show that the proposed ship detector achieved satisfactory performance (in terms of precision, accuracy and time cost) compared with other state-of-the-art algorithms. The findings of the study offer the potential of providing real-time visual traffic information to maritime regulators, which is crucial for the development of intelligent maritime transportation.