Add better proxy support for the middleware.#33
Add better proxy support for the middleware.#33zhangtemplar wants to merge 4 commits intoclemfromspace:developfrom
Conversation
More specifically, user can configure the proxy for each of their request. If the proxy is not yet used, a new driver will be created.
amit-nath
left a comment
There was a problem hiding this comment.
Hi,
I tried installing your scrapy-selenium package and getting below error. Could you please help to fix it?
Traceback (most recent call last):
File "c:\users\amit\downloads\projects\mywork\lib\site-packages\twisted\internet\defer.py", line 1418, in _inlineCallbacks
result = g.send(result)
File "c:\users\amit\downloads\projects\mywork\lib\site-packages\scrapy\core\downloader\middleware.py", line 40, in process_request
% (method.self.class.name, response.class.name)
scrapy.exceptions._InvalidOutput: Middleware SeleniumMiddleware.process_request must return None, Response or Request, got WebDriver
Installed the package with -
pip install git+https://github.com/zhangtemplar/scrapy-selenium
|
any news on this PR? @zhangtemplar @amit-nath |
|
I no longer have time to work on this. |
More specifically, user can configure the proxy for each of their request. If the proxy is not yet used, a new driver will be created.