Fetch and Extract HTML Title from URL

  • Share this:

Code introduction


This function takes a URL as an argument, uses the requests library to retrieve the content of the URL, then checks if the content type is HTML. If it is HTML, it uses BeautifulSoup to parse the HTML document and get the title. If it is not HTML, it returns non-HTML content.


Technology Stack : requests, BeautifulSoup

Code Type : The type of code

Code Difficulty : Intermediate


                
                    
import requests
from random import choice

def fetch_random_url(url):
    response = requests.get(url)
    content_type = response.headers.get('Content-Type')
    if 'text/html' in content_type:
        soup = fromstring(response.content)
        title = soup.find('title').text
        return title
    else:
        return "Non-HTML content"