问答

python爬虫程序报错:ConnectionResetError

作者:admin 2021-04-21 我要评论

程序如下,爬取过程中会报错,报错时间随机,有时候爬几百条就报错,有时候爬几千条报错: ` ConnectionResetError: [Errno 54] Connection reset by peer def g...

在说正事之前,我要推荐一个福利:你还在原价购买阿里云、腾讯云、华为云服务器吗?那太亏啦!来这里,新购、升级、续费都打折,能够为您省60%的钱呢!2核4G企业级云服务器低至69元/年,点击进去看看吧>>>)

程序如下,爬取过程中会报错,报错时间随机,有时候爬几百条就报错,有时候爬几千条报错:`
ConnectionResetError: [Errno 54] Connection reset by peer


    def get_page(url):
        headers = {
            "User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.96 Safari/537.36"
     }
        reqeust = urllib.request.Request(url=url, headers=headers)
     try:
            response = urllib.request.urlopen(reqeust)
            page = response.read().decode('utf-8')
            # print(html)
     except urllib.error.URLError as e:
            if hasattr(e, "code"):
                print(e.code)
            if hasattr(e, "reason"):
                print(e.reason)
     return page
     
     
     File "/Users/chinalife/Desktop/source/flaskTiantian/views/getdata.py", line 33, in get_data
        page = get_page(url)
      File "/Users/chinalife/Desktop/source/flaskTiantian/views/getdata.py", line 21, in get_page
        page = response.read().decode('utf-8')
      File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 471, in read
        s = self._safe_read(self.length)
      File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 612, in _safe_read
        data = self.fp.read(amt)
      File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/socket.py", line 704, in readinto
        return self._sock.recv_into(b)

版权声明:本文转载自网络,遵循 CC 4.0 BY-SA 版权协议,转载请附上原文出处链接和本声明。本站转载出于传播更多优秀技术知识之目的,如有侵权请联系QQ/微信:153890879删除

相关文章
  • python爬虫程序报错:ConnectionResetEr

    python爬虫程序报错:ConnectionResetEr

  • vue原理:如果子组件发生变化重新渲染

    vue原理:如果子组件发生变化重新渲染

  • 关于c++智能指针源码实现的疑问

    关于c++智能指针源码实现的疑问

  • 谁到底是Ant Design

    谁到底是Ant Design

腾讯云代理商
海外云服务器