This post going to be a bit longer as I am going to cover multiple concepts. I will be covering the following things:
According to Investopedia:
A smart contract is a self-executing contract with the terms of the agreement between buyer and seller being directly written into lines of code. The code and…
So I was exploring Telegram APIs for a project someone asked me to work on it. The script was actually a cron job that would be sending messages on daily basis. While working on it I found that you could come up with your own commands that can pull data from some remote API and display the results to Telegram users. I found this an opportunity for my next blog post which I am writing here 🙂
Telegram is very much similar to WhatsApp for communication and it is quite popular among Crypto lovers. …
In this post, I am going to talk about Proto Buffers and how you can use them in Python for passing messages across networks. Protocol Buffers or Porobuf in short, are used for data serialization and deserialization. Before I discuss Protobuf, I would like to discuss data serialization and serialization first.
According to Wikipedia
Serialization is the process of translating a data structure or object state into a format that can be stored (for example, in a file or memory data buffer) or transmitted (for example, over a computer network) and reconstructed later (possibly in a different computer environment)
I had written about Elasticsearch almost 3 years ago in June 2018. During this time a new Elasticsearch version launched which has some new features and changes. I’d be repeating some concepts again in this post so one does not have to go to the old post to learn about it. So, let’s begin!
ElasticSearch (ES) is a distributed and highly available open-source search engine that is built on top of Apache Lucene. It’s open-source which is built in Java thus available for many platforms. You store unstructured data in JSON format which also makes it a NoSQL database. …
Usually, when you start developing a scraper to scrape loads of records, your first step is usually to go to the page where all listings are available. You go to the page by page, fetch individual URLs, store in DB or in a file and then start parsing. Nothing wrong with it. The only issue is the wastage of resources. Say there are 100 records in a certain category. Each page has 10 records. Ideally, you will write a scraper that will go page by page and fetch all links. Then you will switch to the next category and repeat…
This post is part of the FastAPI series.
This is another post related to FastAPI(indirectly) in which I am going to discuss how to use GraphQL based APIs to access and manipulate data. I already have discussed how you can make Rest API in the FastAPI framework in the previous post. We will be learning some basics of GraphQL and how graphene helps us to use Python for writing a GraphQL based application. So, let’s get started!
From the official website:
GraphQL is a query language for APIs and a runtime for fulfilling those queries with your existing data. GraphQL…
This post is part of the FastAPI series.
In the first post, I introduced you to FastAPI and how you can create high-performance Python-based applications in it. In this post, we are going to work on Rest APIs that interact with a MySQL DB. We will also be looking at how we can organize routers and models in multiple files to make them maintainable and easier to read.
In this post, I am going to introduce FastAPI: A Python-based framework to create Rest APIs. I will briefly introduce you to some basic features of this framework and then we will create a simple set of APIs for a contact management system. Knowledge of Python is very necessary to use this framework.
Before we discuss the FastAPI framework, let’s talk a bit about REST itself.
Representational state transfer (REST) is a software architectural style that defines a set of constraints to be used for creating Web services. Web services that conform to the REST architectural style, called…
In this post of ScrapingTheFamous, I am going o write a scraper that will scrape data from eBay. eBay is an online auction site where people put their listing up for selling stuff based on an auction.
Like before, we will be writing the two scripts, one to fetch listing URs and store in a text file and the other to parse those links. The data will be stored in JSON format for further processing.
I will be using Scraper API service for parsing purposes which makes me free from all worries blocking and rendering dynamic sites since it takes…
In this post of ScrapingTheFamous, I am going o write a scraper that will scrape data from Amazon. I do not need to tell you what is Amazon. You are here because you already know about it 🙂
So, we are going to write two different scripts: one would be
fetch.py that would be fetching URLs of individual listings and save in a text file. Later another script,
parse.py that will have a function taking an individual listing URL, scrape data, and save in JSON format.
I will be using Scraper API service for parsing purposes which makes me free…