Hey All,

Long time no blog. During a recent OSINT investigation, I found that I needed to pull all domains found from my query on crt.sh. The problem I had, however, was that the results weren’t all that usable without a lot of copying, pasting, and cleaning.

To address this problem, and to save time in the future, I created a Jupyter Notebook to programmatically query the crt.sh website, dump the results into a pandas data frame (thinking that I’ll want to further enrich the data at a later date), and then printing out the unique list of results to the screen.

The code is written in Python 3, and relies on BeautifulSoup4, Pandas, and NumPy.

I’m calling it CrtShcrape (pronounced cert-shcrape) and you can download it from my GitHub here: https://github.com/andrewsmhay/CrtShcrape.

Hopefully, you can get some use from it. Until next time!