TIL: benchmarking a single GET request with 5 different Python HTTP libraries

Photo by Chris Liverani on Unsplash

Today I needed to figure out which Python HTTP library would give me the best performance for a plain and simple HTTP GET request, so I though I’d write up my approach and results for future reference.

Libraries

I tested five different libraries:

• urllib, which is in the standard library
• urllib3, which is presented asurllib on steroids
• requests or “HTTP for Humans™ ”
• aiohttp, an asynchronous HTTP Client/Server for asyncio and Python
• httpx, the only library to offer both HTTP/1.1 and HTTP/2

For the record, I am aware that aiohttp (and to some extent httpx too) is specifically geared towards async requests, which is in fact only a disadvantage when testing with a single request, but I wanted to get the numbers anyway…

System

These tests were performed on machine with the following specifications.

Operating system:       Linux Mint 21.2 Cinnamon
Linux Kernel:           5.15.0-78-generic
Processor:              Intel(R) Core(TM) i7-8550U CPU @ 1.80GHz x 4
RAM memory:             15.3 GiB
Hard Drives:            518.3 GB
System:                 Dell XPS 13…