Add generic SOCKS proxy support, variable renaming
Now you can run your workers through a SOCKS proxy with the -S flag! Authentication at the command-line through fully-qualified URLs as well as interactive authentication are now supported. I may revert the try/except block in LinodeProxy's constructor should it be found to be non-advantageous in further testing. Performance in creating large quantities of Linode proxies is still dog slow, may use concurrent.futures to speed this up in the future (hah) -- this will require a major rework of the downloader main().
This commit is contained in:
parent
b98fba1508
commit
cf2a50ba93
5 changed files with 332 additions and 59 deletions
38
squid_dl/proxy.py
Normal file
38
squid_dl/proxy.py
Normal file
|
|
@ -0,0 +1,38 @@
|
|||
"""
|
||||
squid_dl Proxy base class (for typing)
|
||||
|
||||
⣀⣠⣴⣷⣾⣿⣿⣿⣶⣦⣠
|
||||
⣴⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⡖
|
||||
⣰⣿⣿⠟⠉⣀⠄⠻⣿⣿⣿⣿⣿⣿⡟
|
||||
⣠⣿⣿⡯ ⠜⠁ ⢰⣿⣿⣿⣿⣿⡿⠋
|
||||
⢀⣠⣼⣿⣿⣿⣿⣯⡄⡀⣠⣤⡿⡿⢿⣿⠿⠋
|
||||
⣀⣶⣿⣿⣿⡿⠿⠿⠿⢿⢿⣿⣿⣿⣿⣿⣿⣶
|
||||
⣠⣿⣿⣿⣿⡿⠗⠁ ⠙⣿⣿⣿⣿⣿⣿⣷⡆
|
||||
⢀⣴⣿⣿⣿⣿⣿ ⢻⣿⣿⣿⣿⣿⣿⣿⣥⠄
|
||||
⣴⣿⣿⣿⣿⣿⡃ ⢨⣿⣿⣿⣿⣿⣿⣿⣿⡅
|
||||
⠼⣿⣿⣿⣿⠋ ⠈⣿⣿⣿⣿⣿⣿⣿⣿⣗⠄
|
||||
⠹⠛⠏⠃ ⠸⣿⣿⣿⣿⣿⣿⣿⣿⣿⡁
|
||||
⣨⣟⣿⣿⣿⣿⣿⡏⣿⣿⣗
|
||||
⢠⣾⢷⣿⣿⣿⣿⣿⣯⢿⣿⣿⡂
|
||||
⣠⣾⣟⣾⣿⣿⣿⣿⣿⣿⣸⣿⣿⣧⠄
|
||||
⢀⣴⣿⣿⣽⣿⣿⣿⣿⣿⣿⣿⣧⢿⣿⣿⣷⣵⣀
|
||||
⢾⣿⣻⣾⣿⣿⣿⣿⣿⣿⣿⣿⡟⠘⢿⣿⣿⣿⣷⠄
|
||||
⢀⣤⣷⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⡗ ⠈⠘⠉⠋⠁
|
||||
⢀⣴⣿⣿⡟⠋⠉⠁⠁⠁⠉⠹⣻⣿⣿⣧⡀
|
||||
⣰⣿⣿⡟⠉ ⢻⣿⣿⡄
|
||||
⢠⣾⣿⣿⠯ ⣻⣿⣷⣆
|
||||
⢠⣿⣿⣿⠩ ⠙⣿⣿⣿⣦⡀
|
||||
⢠⣿⣿⣟⡷⡕ ⠈⢻⣿⣿⣿⣷⡀
|
||||
⢰⣿⣿⡳⡩⣇ ⢙⣿⣿⣿⣿⣞⡄
|
||||
⢸⣿⣿⢧⡲⣧ ⢺⣿⣿⣿⣿⡽⡆
|
||||
⠻⣿⣯⠛ ⠘⠟⠿⠿⡙
|
||||
|
||||
"""
|
||||
|
||||
|
||||
class Proxy:
|
||||
proxy_url = None
|
||||
exclusive = False
|
||||
|
||||
def cleanup(self) -> None:
|
||||
pass
|
||||
Loading…
Add table
Add a link
Reference in a new issue