[Python] requests module

requests is a simple HTTP library for Python.

Let's find out about it.

Installation

$ pip install requests

Test Server

I will use some test servers for example.

• httpbin.org
  • httpbin.org is a simple HTTP request and response service.
• jsonplaceholder.typicode.com
  • jsonplaceholder.typicode.comis a fake online REST API server.

Usage

You can simply use this.

import requests

rsp = requests.post("http://httpbin.org/post", data={"key":"value"})
rsp = requests.get("http://httpbin.org/get")
rsp = requests.put("http://httpbin.org/put", data={"key":"value"})
rsp = requests.patch("http://httpbin.org/patch", data={"key":"value"})
rsp = requests.delete("http://httpbin.org/delete")

Yes, it is done. You will get the response from the server!!

Attributes

- headers

Case insensitive dictionary of response headers.

- encoding

Encoding to decode with when accessing response.text.

- text

Content of the response, in Unicode.

- content

Content of the response, in bytes.

- status_code

Integer code of responded HTTP status.

- stream

Stream response content default.

- cookies

A CookieJar of cookies the server sent back.

Methods

- json()

Returns the json-encoded content of a response, if any.

- raise_for_status()

Raises HTTPError, if one occurred.

Example

GET Requests

import requests

url = "http://httpbin.org/get"
with requests.Session() as s:
    rsp = s.get(url)
    print(rsp.headers)
    print(rsp.text)
    print(rsp.status_code)

Open a session for HTTP communication, and send GET method to the URL.

You will get a response from the server.

Please run the code and watch the result.

 

You can also request it with headers.

import requests

url = "http://httpbin.org/get"
headers = {"user-agent": "Chrome"}
with requests.Session() as s:
    rsp = s.get(url, headers=headers)
    print(rsp.headers)
    print(rsp.text)
    print(rsp.status_code)

 

import requests

url = "https://jsonplaceholder.typicode.com/posts/1"
with requests.Session() as s:
    rsp = s.get(url)
    print(rsp.json())
    print(rsp.json().keys())
    print(rsp.json().values())
    print(rsp.encoding)
    print(rsp.content)

As you can see in the above code, you can also get the response with JSON type.

Streaming Requests

You can easily iterate over streaming APIs.

import json
import requests

url = "http://httpbin.org/stream/20"
rsp = requests.get(url, stream=True)
for line in rsp.iter_lines():
    if line:
        decoded_line = line.decode('utf-8')
        print(json.loads(decoded_line))

When using decode_unicode=True with Response.iter_lines() or Response.iter_content(), you'll want to provide a fallback encoding in the event the server doesn't provide one.

import json
import requests

url = "http://httpbin.org/stream/20"
rsp = requests.get(url, stream=True)

if rsp.encoding is None:
    rsp.encoding = 'utf-8'

for line in rsp.iter_lines(decode_unicode=True):
    if line:
        l = json.loads(line)
        for k in l.keys():
            print(f"key: {k}, values: {l[k]}")
        print()

Requests with Cookie

import requests

jar = requests.cookies.RequestsCookieJar()
jar.set("key", "value", domain="httpbin.org", path="/cookies")

rsp = requests.get("http://httpbin.org/cookies", cookies=jar)
rsp.raise_for_status()
print(rsp.text)

rsp = requests.post("http://httpbin.org/post", data={"name": "hoya"}, cookies=jar)
rsp.raise_for_status()
print(rsp.text)

Requests with Timeouts

You can tell requests to stop waiting for a response after a given number of seconds with the timeout parameter.

import requests

rsp = requests.get("https://github.com", timeout=0.05)
print(rsp.text)

POST, PUT, and PATCH with Data

import json
import requests

payload1 = {"key1": "value1", "key2": "value2"}
payload2 = (("key1", "value1"), ("key2", "value2"))

rsp = requests.post("http://httpbin.org/post", data=payload1)
print(rsp.text)

rsp = requests.put("http://httpbin.org/put", data=payload2)
print(rsp.text)

rsp = requests.patch("http://httpbin.org/patch", data=json.dumps(payload1))
print(rsp.text)

rsp = requests.patch("http://httpbin.org/patch", data=json.dumps(payload2))
print(rsp.text)

If you directly use the payloads, it returns in the form field.

If you use the payloads with json.dumps, it returns in the data field.