Develop AirBnb Parser in Python

So I am starting a new scraping series, called, ScrapeTheFamous, in which I will be parsing some famous websites and will discuss my development process. The posts will be using Scraper API for parsing purposes which makes me free from all worries blocking and rendering dynamic sites since Scraper API takes care of everything.

Anyways, the first post is about Airbnb. We will be scraping some important data points from it. We will be scraping a list of rental URL and fetch and store data in JSON format. So let’s start!

The URL we will be using is here: https://www.airbnb.com/s/Karachi-Sindh-Pakistan/homes?query=Karachi%2C%20Sindh%2C%20Pakistan

Above is the screenshot of the listing for the city of Karachi. Though you can pick as much data as you want for example purpose, I am only picking a price, number of guests, listing URL and, number of bedrooms.

You are seeing a few methods here: get_price(), get_guests and get_bed. These functions will be returning our required data. Also, I am passing HTML data of each element which will then be used to extract the data.

I am not showing all methods here, you can download the code from Github, I am just giving code of get_bed here.

As you can see I am using RegEx here. Parsing is not all about using Beautifulsoup. You can any tool that helps to give you data. If you want you can use Beautifulsoup. All up to you. The data is then saved in records list which then converts to a JSON structure and saves in a .json file. If all goes well, it will generate a file like below:

Looks good, no?

Conclusion

In this post, you learned how you can create an Airbnb parser in Python using Scraper API. You do not have to worry about Proxy IPs either nor you have to pay hundreds of dollars, especially when you are an individual or working in a startup. The company I work with spend 100s of dollars on a monthly basis just for the proxy IPs.

Oh if you sign up here with my referral link or enter promo code adnan10, you will get a 10% discount on it. In case you do not get the discount then just let me know via email on my site and I’d sure help you out.

The code is available on Github.

Planning to write a book about Web Scraping in Python. Click here to give your feedback

Originally published at http://blog.adnansiddiqi.me on February 10, 2020.