An easy solution for crawling sites and saving to CSV. Open Source.
What is ping-webcrawler?
ping-webcrawler is an open source software maintained by Tsubasa Kato at Inspire Search that allows you to crawl websites by pinging the site first (not ICMP).
It is made with Flask, so you can host it locally or have it hosted on the server or cloud.