w3resource
Python Web Scraping Exercises

Python Web Scraping - Exercises, Practice, Solution

Web Scraping

Web scraping or web data extraction is data scraping used for extracting data from websites. Web scraping softwares are used to access the World Wide Web directly using the Hypertext Transfer Protocol, or through a web browser. While web scraping can be done manually by a software user, the term typically refers to automated processes implemented using a bot or web crawler. It is a form of copying, in which specific data is gathered and copied from the web, typically into a central local database or spreadsheet, for later retrieval or analysis.

Python request module :

Requests allows user to send organic, grass-fed HTTP/1.1 requests, without the need for manual labor. There’s no need to manually add query strings to your URLs, or to form-encode your POST data.

Python BeautifulSoup module :

Beautiful Soup is a Python library for pulling data out of HTML and XML files. It works with your favorite parser to provide idiomatic ways of navigating, searching, and modifying the parse tree. It commonly saves programmers hours or days of work.

Python Web Scraping [25 exercises with solution]

[An editor is available at the bottom of the page to write and execute the scripts.]

1. Write a Python program to test if a given page is found or not on the server. Go to the editor
Click me to see the sample solution

2. Write a Python program to download and display the content of robot.txt for en.wikipedia.org. Go to the editor
Click me to see the sample solution

3. Write a Python program to get the number of datasets currently listed on data.gov. Go to the editor
Click me to see the sample solution

4. Write a Python program to convert an address (like "1600 Amphitheatre Parkway, Mountain View, CA") into geographic coordinates (like latitude 37.423021 and longitude -122.083739). Go to the editor
Geocodin: Geocoding is the process of converting addresses (like "1600 Amphitheatre Parkway, Mountain View, CA") into geographic coordinates (like latitude 37.423021 and longitude -122.083739), which you can use to place markers on a map, or position the map.
Click me to see the sample solution

5. Write a Python program to display the name of the most recently added dataset on data.gov. Go to the editor
Click me to see the sample solution

6. Write a Python program to extract h1 tag from example.com. Go to the editor
Click me to see the sample solution

7. Write a Python program to extract and display all the header tags from en.wikipedia.org/wiki/Main_Page. Go to the editor
Click me to see the sample solution

8. Write a Python program to extract and display all the image links from en.wikipedia.org/wiki/Peter_Jeffrey_(RAAF_officer). Go to the editor
Click me to see the sample solution

9. Write a Python program to get 90 days of visits broken down by browser for all sites on data.gov. Go to the editor
Click me to see the sample solution

10. Write a Python program to that retrieves an arbitary Wikipedia page of "Python" and creates a list of links on that page. Go to the editor
Click me to see the sample solution

11. Write a Python program to check whether a page contains a title or not. Go to the editor
Click me to see the sample solution

12. Write a Python program to list all language names and number of related articles in the order they appear in wikipedia.org. Go to the editor
Click me to see the sample solution

13. Write a Python program to get the number of people visiting a U.S. government website right now. Go to the editor
Source: https://analytics.usa.gov/data/live/realtime.json
Click me to see the sample solution

14. Write a Python program get the number of security alerts issued by US-CERT in the current year. Go to the editor
Source: https://www.us-cert.gov/ncas/alerts
Click me to see the sample solution

15. Write a Python program to get the number of Pinterest accounts maintained by U.S. State Department embassies and missions. Go to the editor
Source: https://www.state.gov/r/pa/ode/socialmedia/
Click me to see the sample solution

16. Write a Python program to get the number of followers of a given twitter account. Go to the editor
Click me to see the sample solution

17. Write a Python program to get the number of following on Twitter. Go to the editor
Click me to see the sample solution

18. Write a Python program to get the number of post on Twitter liked by a given account. Go to the editor
Click me to see the sample solution

19. Write a Python program to count number of tweets by a given Twitter account. Go to the editor
Click me to see the sample solution

20. Write a Python program to scrap number of tweets of a given Twitter account. Go to the editor
Click me to see the sample solution

21. Write a Python program to find the live weather report (temperature, wind speed, description and weather) of a given city. Go to the editor
Click me to see the sample solution

22. Write a Python program to display the date, days, title, city, country of next 25 Hackevents. Go to the editor
Click me to see the sample solution

23. Write a Python program to download IMDB's Top 250 data (movie name, Initial release, director name and stars). Go to the editor
Click me to see the sample solution

24. Write a Python program to get movie name, year and a brief summary of the top 10 random movies. Go to the editor
Click me to see the sample solution

25. Write a Python program to get the number of magnitude 4.5+ earthquakes detected worldwide by the USGS. Go to the editor
Click me to see the sample solution

More to Come !

Python Code Editor:



ankara escort
kızılay escort