Не получается спарсить dotabuff

Question

Написал код, который должен парсить страницу игроков на dotabuff и затем возвращать всю нужную мне информацию, но вместо полной html страницы я получаю

<!DOCTYPE html>

<html>
<head>
<title>DOTABUFF - Too Many Requests</title>
<meta content="width=device-width, initial-scale=1" name="viewport"/>
<style>
    html {
      background: #1C242D;
      color: #C4C4C4;
      font-family: "Lucida Grande", Tahoma, Verdana, Arial, sans-serif;
      font-size: 14px;
      width: 100%;
      height: 100%;
    }
    body {
      margin: 21% auto;
      text-align: center;
    }
    p a {
      color: #A9CF54;
      text-decoration: none
    }
    #logo {
      background: #ED3B1C;
      color: #FFFFFF;
      display: inline-block;
      font-family: Arial;
      font-weight: 800;
      padding: 6px 12px;
    }
    #logo a {
      color: #FFFFFF;
      text-decoration: none;
    }
    #status {
      font-weight: 200;
      color: #FFFFFF;
    }
  </style>
</head>
<body>
<h1 id="logo"><a href="/">DOTABUFF</a></h1>
<h2 id="status">Too Many Requests</h2>
<p>You have been rate limited for making too many requests. Please try again later.</p>
<p><a href="/">Home</a> | <a href="/pages/about#support">Support</a></p>
</body>
</html>

вот пример кода для team = TeamSpirit, player = Yatoro

def get_winrate(team, player, hero):
    url = f'https://www.dotabuff.com/search?q={team.replace(" ", "+")}+{player.replace(" ", "+")}&commit=Search'
    page = requests.get(url)
    soup = BeautifulSoup(page.text, "html.parser")
    print(soup)

Answer 1

Можно использовать selenium

from selenium import webdriver
from bs4 import BeautifulSoup

driver = webdriver.Chrome()
driver.get('https://ru.dotabuff.com/players')

soup = BeautifulSoup(driver.page_source, 'lxml')

Answer 2

import requests
from bs4 import BeautifulSoup

headers = {'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:120.0) Gecko/20100101 Firefox/120.0'}
response = requests.get('https://ru.dotabuff.com/search?q=TSpirit.Yatoro&commit=Search', headers=headers)
soup = BeautifulSoup(response.text, 'lxml')
print(soup.text)

БЛОГ НА HUSL

Не получается спарсить dotabuff

Ответы (2 шт):