首页 > pyspider +mongodb 部署有问题

pyspider +mongodb 部署有问题

如题 pyspider -c config.json 后报错:

[W 151225 17:29:08 run:398] phantomjs not found, continue running without it.
[I 151225 17:29:10 result_worker:49] result_worker starting...
[I 151225 17:29:11 tornado_fetcher:429] fetcher starting...
[I 151225 17:29:11 scheduler:453] loading projects
[I 151225 17:29:11 processor:208] processor starting...
Exception AttributeError: "'Cursor' object has no attribute '_Cursor__id'" in <bound method Cursor.__del__ of <pymongo.cursor.Cursor object at 0x7fb94b5dadd0>> ignored
[E 151225 17:29:11 scheduler:463] __init__() got an unexpected keyword argument 'fields'
    Traceback (most recent call last):
      File "/root/miniconda2/lib/python2.7/site-packages/pyspider/scheduler/scheduler.py", line 458, in run
        self.run_once()
      File "/root/miniconda2/lib/python2.7/site-packages/pyspider/scheduler/scheduler.py", line 442, in run_once
        self._update_projects()
      File "/root/miniconda2/lib/python2.7/site-packages/pyspider/scheduler/scheduler.py", line 84, in _update_projects
        for project in self.projectdb.check_update(self._last_update_project):
      File "/root/miniconda2/lib/python2.7/site-packages/pyspider/database/mongodb/projectdb.py", line 61, in check_update
        for project in self.get_all(fields=('updatetime', 'name')):
      File "/root/miniconda2/lib/python2.7/site-packages/pyspider/database/mongodb/projectdb.py", line 49, in get_all
        for each in self.collection.find({}, fields=fields):
      File "/root/miniconda2/lib/python2.7/site-packages/pymongo/collection.py", line 1137, in find
        return Cursor(self, *args, **kwargs)
    TypeError: __init__() got an unexpected keyword argument 'fields'
Exception AttributeError: "'Cursor' object has no attribute '_Cursor__id'" in <bound method Cursor.__del__ of <pymongo.cursor.Cursor object at 0x7fb94b5dadd0>> ignored
[E 151225 17:29:11 scheduler:463] __init__() got an unexpected keyword argument 'fields'
    Traceback (most recent call last):
      File "/root/miniconda2/lib/python2.7/site-packages/pyspider/scheduler/scheduler.py", line 458, in run
        self.run_once()
      File "/root/miniconda2/lib/python2.7/site-packages/pyspider/scheduler/scheduler.py", line 442, in run_once
        self._update_projects()
      File "/root/miniconda2/lib/python2.7/site-packages/pyspider/scheduler/scheduler.py", line 84, in _update_projects
        for project in self.projectdb.check_update(self._last_update_project):
      File "/root/miniconda2/lib/python2.7/site-packages/pyspider/database/mongodb/projectdb.py", line 61, in check_update
        for project in self.get_all(fields=('updatetime', 'name')):
      File "/root/miniconda2/lib/python2.7/site-packages/pyspider/database/mongodb/projectdb.py", line 49, in get_all
        for each in self.collection.find({}, fields=fields):
      File "/root/miniconda2/lib/python2.7/site-packages/pymongo/collection.py", line 1137, in find
        return Cursor(self, *args, **kwargs)
    TypeError: __init__() got an unexpected keyword argument 'fields'

二选一

  1. 升级 git 版本pyspider

  2. 使用 pymongo<3.0


已解决,多谢!

【热门文章】
【热门文章】