w3resource

Python BeautifulSoup: Extract all the text from a given web page

BeautifulSoup: Exercise-12 with Solution

Write a Python program to extract all the text from a given web page.

Sample Solution:

Python Code:

import requests
from bs4 import BeautifulSoup
url = 'https://www.python.org/'
reqs = requests.get(url)
soup = BeautifulSoup(reqs.text, 'lxml')
print("Text from the said page:")
print(soup.get_text())

Sample Output:

Text from the said page:
Welcome to Python.org
     {
       "@context": "https://schema.org",
       "@type": "WebSite",
       "url": "https://www.python.org/",
       "potentialAction": {
         "@type": "SearchAction",
         "target": "https://www.python.org/search/?q={search_term_string}",
         "query-input": "required name=search_term_string"
       }
     }
    

    var _gaq = _gaq || [];
    _gaq.push(['_setAccount', 'UA-39055973-1']);
    _gaq.push(['_trackPageview']);

    (function() {
        var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
        ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
        var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
    })();
    
Notice: While Javascript is not essential for this website, your interaction with the content will be limited. Please turn Javascript on for the full experience. 

...........

▲ Back to Top

Help & General Contact
Diversity Initiatives
Submit Website Bug

Status 

Copyright ©2001-2019.
                             Python Software Foundation
                            Legal Statements
                             Privacy Policy
                             Powered by Heroku

window.jQuery || document.write('<script src="/static/js/libs/jquery-1.8.2.min.js"><\/script>')

Python Code Editor:

Have another way to solve this solution? Contribute your code (and comments) through Disqus.

Previous: Write a Python program to a list of all the h1, h2, h3 tags from the webpage python.org.
Next: Write a Python program to print the names of all HTML tags of a given web page going through the document tree.

What is the difficulty level of this exercise?

Test your Python skills with w3resource's quiz



Python: Tips of the Day

Getting the last element of a list:

some_list[-1] is the shortest and most Pythonic.

In fact, you can do much more with this syntax. The some_list[-n] syntax gets the nth-to-last element. So some_list[-1] gets the last element, some_list[-2] gets the second to last, etc, all the way down to some_list[-len(some_list)], which gives you the first element.

You can also set list elements in this way. For instance:

>>> some_list = [1, 2, 3]
>>> some_list[-1] = 5 # Set the last element
>>> some_list[-2] = 3 # Set the second to last element
>>> some_list
[1, 3, 5]

Note that getting a list item by index will raise an IndexError if the expected item doesn't exist. This means that some_list[-1] will raise an exception if some_list is empty, because an empty list can't have a last element.

Ref: https://bit.ly/3d8TfFP