w3resource

Python: Send a request to a web page, and print the response text, content, raw data


3. Response Text, Content, and Raw Socket

Write a Python program to send a request to a web page, and print the response text and content. Also get the raw socket response from the server.

Sample Solution:

Python Code:

import requests
res = requests.get('https://www.google.com/')
print("Response text of https://google.com/:")
print(res.text)
print("\n==============================================================================")
print("\nContent of the said url:")
print(res.content)
print("\n==============================================================================")
print("\nRaw data of the said url:")
r = requests.get('https://api.github.com/events', stream = True)
print(r.raw)
print(r.raw.read(15))

Sample Output:

Response text of http://example.com/:
<!doctype html>
<html>
<head>
    <title>Example Domain</title>

    <meta charset="utf-8" />
    <meta http-equiv="Content-type" content="text/html; charset=utf-8" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
    <style type="text/css">
    body {
        background-color: #f0f0f2;
        margin: 0;
        padding: 0;
        font-family: -apple-system, system-ui, BlinkMacSystemFont, "Segoe UI", "Open Sans", "Helvetica Neue", Helvetica, Arial, sans-serif;
        
    }
    div {
        width: 600px;
        margin: 5em auto;
        padding: 2em;
        background-color: #fdfdff;
        border-radius: 0.5em;
        box-shadow: 2px 3px 7px 2px rgba(0,0,0,0.02);
    }
    a:link, a:visited {
        color: #38488f;
        text-decoration: none;
    }
    @media (max-width: 700px) {
        div {
            margin: 0 auto;
            width: auto;
        }
    }
    </style>    
</head>

<body>
<div>
    <h1>Example Domain</h1>
    <p>This domain is for use in illustrative examples in documents. You may use this
    domain in literature without prior coordination or asking for permission.</p>
    <p><a href="https://www.iana.org/domains/example">More information...</a></p>
</div>
</body>
</html>


==============================================================================

Content of the said url:
b'<!doctype html>\n<html>\n<head>\n    <title>Example Domain</title>\n\n    <meta charset="utf-8" />\n    <meta http-equiv="Content-type" content="text/html; charset=utf-8" />\n    <meta name="viewport" content="width=device-width, initial-scale=1.0" />\n    <style type="text/css">\n    body {\n        background-color: #f0f0f2;\n        margin: 0;\n        padding: 0;\n        font-family: -apple-system, system-ui, BlinkMacSystemFont, "Segoe UI", "Open Sans", "Helvetica Neue", Helvetica, Arial, sans-serif;\n        \n    }\n    div {\n        width: 600px;\n        margin: 5em auto;\n        padding: 2em;\n        background-color: #fdfdff;\n        border-radius: 0.5em;\n        box-shadow: 2px 3px 7px 2px rgba(0,0,0,0.02);\n    }\n    a:link, a:visited {\n        color: #38488f;\n        text-decoration: none;\n    }\n    @media (max-width: 700px) {\n        div {\n            margin: 0 auto;\n            width: auto;\n        }\n    }\n    </style>    \n</head>\n\n<body>\n<div>\n    <h1>Example Domain</h1>\n    <p>This domain is for use in illustrative examples in documents. You may use this\n    domain in literature without prior coordination or asking for permission.</p>\n    <p>More information...</p>\n</div>\n</body>\n</html>\n'

==============================================================================

Raw data of the said url:
<urllib3.response.HTTPResponse object at 0x0FE4ACF0>
b'\x1f\x8b\x08\x00\x00\x00\x00\x00\x00\xff}TMs\xdb'

For more Practice: Solve these Related Problems:

  • Write a Python program that sends a request to a given URL, prints both the response text and content, and then saves the raw socket response to a file.
  • Write a Python script to send a GET request, then compare and print the differences between the response text (decoded) and the raw byte content.
  • Write a Python function that sends a request and retrieves the raw socket response using the .raw attribute, then prints the first 100 bytes.
  • Write a Python program to fetch a web page, print the response text, and then analyze the raw binary content to identify the character encoding used by the server.

Go to:


Previous: Write a Python code to check the status code issued by a server in response to a client's request made to the server. Print all of the methods and attributes available to objects on successful request.
Next: Write a Python code to send a request to a web page, and print the information of headers. Also parse these values and print key-value pairs holding various information.

Python Code Editor:


Have another way to solve this solution? Contribute your code (and comments) through Disqus.

What is the difficulty level of this exercise?

Test your Programming skills with w3resource's quiz.



Follow us on Facebook and Twitter for latest update.