Файл записывается в сломанной кодировке
Я паршу Twitter через библиотеку requsets и мне надо посмотреть что мне возвращает запрос:
from bs4 import BeautifulSoup
import requests
import json
url = 'https://api.x.com/graphql/E3opETHurmVJflFsUBVuUQ/UserTweets?variables=%7B%22userId%22%3A%2244196397%22%2C%22count%22%3A20%2C%22includePromotedContent%22%3Atrue%2C%22withQuickPromoteEligibilityTweetFields%22%3Atrue%2C%22withVoice%22%3Atrue%2C%22withV2Timeline%22%3Atrue%7D&features=%7B%22rweb_tipjar_consumption_enabled%22%3Atrue%2C%22responsive_web_graphql_exclude_directive_enabled%22%3Atrue%2C%22verified_phone_label_enabled%22%3Afalse%2C%22creator_subscriptions_tweet_preview_api_enabled%22%3Atrue%2C%22responsive_web_graphql_timeline_navigation_enabled%22%3Atrue%2C%22responsive_web_graphql_skip_user_profile_image_extensions_enabled%22%3Afalse%2C%22communities_web_enable_tweet_community_results_fetch%22%3Atrue%2C%22c9s_tweet_anatomy_moderator_badge_enabled%22%3Atrue%2C%22articles_preview_enabled%22%3Atrue%2C%22responsive_web_edit_tweet_api_enabled%22%3Atrue%2C%22graphql_is_translatable_rweb_tweet_is_translatable_enabled%22%3Atrue%2C%22view_counts_everywhere_api_enabled%22%3Atrue%2C%22longform_notetweets_consumption_enabled%22%3Atrue%2C%22responsive_web_twitter_article_tweet_consumption_enabled%22%3Atrue%2C%22tweet_awards_web_tipping_enabled%22%3Afalse%2C%22creator_subscriptions_quote_tweet_preview_enabled%22%3Afalse%2C%22freedom_of_speech_not_reach_fetch_enabled%22%3Atrue%2C%22standardized_nudges_misinfo%22%3Atrue%2C%22tweet_with_visibility_results_prefer_gql_limited_actions_policy_enabled%22%3Atrue%2C%22rweb_video_timestamps_enabled%22%3Atrue%2C%22longform_notetweets_rich_text_read_enabled%22%3Atrue%2C%22longform_notetweets_inline_media_enabled%22%3Atrue%2C%22responsive_web_enhance_cards_enabled%22%3Afalse%7D&fieldToggles=%7B%22withArticlePlainText%22%3Afalse%7D'
bearer_token = x
headers = {
'Authorization': f'Bearer {bearer_token}',
'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 7_7_6) AppleWebKit/537.16 (KHTML, like Gecko) Chrome/48.0.2560.172 Safari/537',
'Accept-Language': 'en-US,en;q=0.ru-RU,ru;q=0.9,en-US;q=0.8,en;q=0.7',
'Accept-Encoding': 'gzip, deflate, br',
'X-Guest-Token': 'x'
}
cookies = {
'night_mode': '2',
'd_prefs': 'MToxLGNvbnNlbnRfdmVyc2lvbjoyLHRleHRfdmVyc2lvbjoxMDAw',
'dnt': '1',
'guest_id': 'v1%3A171938068735478688',
'guest_id_ads': 'v1%3A171938068735478688',
'guest_id_marketing': 'v1%3A171938068735478688',
'external_referer': 'padhuUp37zjgzgv1mFWxJ12Ozwit7owX|0|ziZgIoZIK4nlMKUVLq9KcnBFms0d9TqBqrE%2FyjvSFlFJR45yIlYF%2Bw%3D%3D',
'gt': '1834602927027028255',
'att': '1-LNmtOuvQMOtp2Yq9FnDKpSCQtofUjZnbSwSCA6V1',
'personalization_id': 'v1_d7rvvqhpguRq9Z/6SgEevg==',
}
s = requests.Session()
s.headers.update(headers)
s.cookies.update(cookies)
response = s.get(url)
if response.status_code == 200:
with open('page.html', 'w', encoding='utf-8') as f:
f.write(response.text)
else:
print(f"Ошибка при запросе страницы: {response.status_code}")
После записи текста ответа в файл, в фале появляется вот это:
[2�njd��@3p\�u�W�~�����nܻ��R(< yH �f�@@7���'bb�ՖT����Ἷ�M���$��D٬�U� qf7KI����]¯O�I�Z���'�ׯ!u7H�r HRN %�0���so��F � ʤE���@��jVKc�-��i7��{x�O�z�7���?���� ?����$��M�4m����.b�/cٗgu�v���J�B�Ǘ���C�G�w���2��X�2��ɮ������e@B��V+��:��4w��u6D L��S��v0k�ż�t;�"ɝ�B-gU]�?PLL")D+;����ŠN�s'��o�)��I��}G�@?���^-���N:W�?���a��Ym�r� ��PT��e����h������t;R?�g���y��m�p��;����p�*睝�c��T�����d���"�Q��S��6�2�^��� #$��q��O]�y�RG��O���[�/���B.7���&�A�a���V.k���-�}�$�.��ieg�Z-Зc-6��ΎO����\�����}��Zǥk/^��M5�� 3<�g��:y��@�2{f,O��k5��;-����A�̲Z(l�����-�?�dm�C�K2q�( k����Tm������cs9�t�\�ӹ8����ng�Zfeg��$��xf���Z�2���1a�v1�h'��IF'��٪���^�<�WB�;+p�ժUE��O ��ko���+��5�N7�\�Ѭ��>��5F��rb�W���;��,���NgR�}dK��%�&c�j����b�,㓙Um� /�]�v:��X)� �[���) �y8k�t�n���>��ꋪ��a����հ���e��R���Ng�e۾Q��.2��p��:�ė���� �C%5�(Y�z9�h�����8��#+e�}��S��?��CG�g+�U�lu��b��rƷ,˲��ie|~hjW�ʤ���/5��d�p�np�Ig�S�x��G^c���q�Z��/����ʶ������ɹj�&a64��2�o{��� и т.д.
По идее запрос должен возвращать json но с ним тоже возникают проблемы. Как мне исправить ситуацию?
P.S Когда прописываю строку:
print(response.encoding)
Возвращает utf-8