(hI @ s d Z d d l Z d d l Z d d l Z d d l Z d d l Z d d l Z d d l Z d d l Z d d l Z d d l
m Z d d l m
Z
d d l m Z m Z m Z m Z d d l Z d d l m Z m Z m Z m Z m Z m Z m Z m Z m Z m Z m Z m Z m! Z! d d l m" Z" d d l# m$ Z$ d d l% m& Z& d d
l' m( Z( d d l) m* Z* d d l+ m, Z, d d
l- m. Z. e/ Z0 e j1 d Z2 e j1 d e j3 Z4 e j1 d Z5 e j1 d e j3 j6 Z7 d j8 Z9 d d d d g Z: d Z; d Z< e< j= d d j= e j> d e Z? d d Z@ d d ZA d d ZB d d! d ZC d d" d# ZD d d$ d% ZE d e d d& d ZF d d' d( ZG d) d* ZH e j1 d+ e j3 ZI eH d, d- ZJ Gd. d/ d/ ZK Gd0 d1 d1 eK ZL Gd2 d d e ZM e j1 d3 jN ZO d4 d5 ZP d6 d7 ZQ d d8 d9 ZR d: d; ZS Gd< d= d= ZT Gd> d? d? e jU ZV e jW jX d@ dA ZY dB dC ZZ eR e; eY ZY dD dE Z[ dF dG Z\ d S)Hz#PyPI and direct package downloading N)wraps)six)urllib http_client configparsermap)
CHECKOUT_DIST Distribution BINARY_DISTnormalize_path SOURCE_DIST Environmentfind_distributions safe_name safe_version to_filename Requirement DEVELOP_DISTEGG_DIST) ssl_support)log)DistutilsError) translate)get_all_headers)unescape)Wheelz^egg=([-A-Za-z0-9_.+!]+)$zhref\s*=\s*['"]?([^'"> ]+)z([^<]+)\n\s+\(md5\)z([-+.a-z0-9]{2,}):z .tar.gz .tar.bz2 .tar .zip .tgz PackageIndexdistros_for_urlparse_bdist_wininstinterpret_distro_name z/tmp/pip-build-kmztzv_d/setuptools/setuptools/package_index.pyparse_requirement_arg5 s
r( c C s | j } d \ } } } | j d r | j d rR | d d } d } n | j d d r | d d } | d d } d } n` | j d
r | d d } d } n8 | j d
d r | d d } | d d } d } | | | f S)z=Return (base,pyversion) or (None,None) for possible .exe nameNz.exez
.win32.exe
win32z .win32-py z.win-amd64.exe z win-amd64z
.win-amd64-py )NNNiiiiiiir0 i)lowerendswith
startswith)namer1 basepy_verplatr& r& r' r > s$ c C s t j j | } | \ } } } } } } t j j | j d d } | d k r | d k r t j j | j d d } d | k r | j d d \ } } | | f S) N/ zsourceforge.netdownload #)r r" urlparseunquotesplit) urlpartsschemeserverpathZ
parametersqueryfragmentr5 r& r& r' egg_info_for_urlV s rI c c s t | \ } } x t | | | D] } | Vq% W| r t j | } | r x. t | | j d | d t D] } | Vqq Wd S)zEYield egg or source distribution objects that might be found at a URLr9
precedenceN)rI distros_for_location EGG_FRAGMENTmatchr groupr )rB metadatar5 rH distrM r& r& r' r a s "c
C sG | j d r | d d
} | j d rP d | k rP t j | | | g S| j d r d | k r t | } | j s g St d | d | j d | j d
t d g S| j d r t | \ } } } | d k r t | | | | t
| SxD t D]< } | j | r| d t | } t | | | SqWg S)z:Yield egg or source distribution objects based on basenamez.egg.zipNr- z.egg-z.whllocation project_nameversionrJ r9 z.exer0 )
r2 r
from_locationr Z
is_compatiblerS rT r r r r
EXTENSIONSlen)rR basenamerO wheelZwin_baser6 platformextr& r& r' rK o s.
rK c C s" t t | t j j | | S)zEYield possible egg or source distribution objects based on a filename)rK r osrF rX )filenamerO r& r& r' distros_for_filename s r^ c c s | j d } | r= t d d | d d D r= d Sxm t d t | d D]R } t | | d j | d | d j | | d d | d | d | VqW Wd S)
zGenerate alternative interpretations of a source distro name
Note: if `location` is a filesystem filename, you should call
``pkg_resources.normalize_path()`` on it before passing it to this
routine!
rQ c s s! | ] } t j d | Vq d S)z py\d\.\d$N)rerM ).0pr& r& r' s z(interpret_distro_name..r; Nr9
py_versionrJ rZ )rA anyrangerW r join)rR rX rO rc rJ rZ rC ra r& r& r' r s * 5 c c s t } | j } | d k rS xj t j j | j | D] } | | | Vq7 Wn8 x5 | D]- } | | } | | k rZ | | | VqZ Wd S)zHList unique elements, preserving order. Remember all elements ever seen.N)setaddr moves filterfalse __contains__)iterablekeyseenZseen_addelementkr& r& r' unique_everseen s
rq c s" t f d d } | S)zs
Wrap a function returning an iterable such that the resulting iterable
only ever yields unique items.
c s t | | S)N)rq )argskwargs)funcr& r' wrapper s z unique_values..wrapper)r )rt ru r& )rt r'
unique_values s rv z(<([^>]*\srel\s*=\s*['"]?([^'">]+)[^>]*)>c c s x t j | D] } | j \ } } t t t j | j j d } d | k sg d | k r x: t j | D]) } t
j j | t
| j d Vqw Wq Wxd d D]\ } | j | } | d k r t j | | } | r t
j j | t
| j d Vq Wd S)
zEFind rel="homepage" and rel="download" links in `page`, yielding URLs,Zhomepager: r9
| Home Page | Download URLN)rx ry r= )RELfinditergroupsrg r strstripr1 rA HREFr r" urljoin
htmldecoderN findsearch)rB pagerM tagrelZrelsposr& r& r' find_external_links s '+
r c @ s: e Z d Z d Z d d Z d d Z d d Z d S) ContentCheckerzP
A null content checker that defines the interface for checking content
c C s d S)z3
Feed a block of data to the hash.
Nr& )selfblockr& r& r' feed s zContentChecker.feedc C s d S)zC
Check the hash. Return False if validation fails.
Tr& )r r& r& r' is_valid s zContentChecker.is_validc C s d S)zu
Call reporter with information about the checker (hash name)
substituted into the template.
Nr& )r reportertemplater& r& r' report s zContentChecker.reportN)__name__
__module__ __qualname____doc__r r r r& r& r& r' r s r c @ sa e Z d Z e j d Z d d Z e d d Z d d Z d d Z
d
d Z d S)
HashCheckerzK(?Psha1|sha224|sha384|sha256|sha512|md5)=(?P[a-f0-9]+)c C s( | | _ t j | | _ | | _ d S)N) hash_namehashlibnewhashexpected)r r r r& r& r' __init__ s zHashChecker.__init__c C sR t j j | d } | s# t S| j j | } | sB t S| | j S)z5Construct a (possibly null) ContentChecker from a URLr9 r= )r r" r? r patternr groupdict)clsrB rH rM r& r& r' from_url s zHashChecker.from_urlc C s | j j | d S)N)r update)r r r& r& r' r s zHashChecker.feedc C s | j j | j k S)N)r hexdigestr )r r& r& r' r # s zHashChecker.is_validc C s | | j } | | S)N)r )r r r msgr& r& r' r & s
zHashChecker.reportN) r r r r_ compiler r classmethodr r r r r& r& r& r' r
s r c s e Z d Z d Z d dJ d d d d Z d d d
Z d d d Z d d
d Z d d Z d d Z d d Z
d d Z d d d Z d d Z
d f d d Z d d Z d d Z d! d" Z d# d$ Z d% d& Z d d d d d' d( Z d d d) d* Z d+ d, Z d- Z d. d/ Z d0 d1 Z d d2 d3 Z d4 d5 Z d6 d7 Z d8 d9 Z d: d; Z d< d= Z e d d> d? Z! d@ dA Z" dB dC Z# dD dE Z$ dF dG Z% dH dI Z& S)Kr z;A distribution index that scans web pages for download URLszhttps://pypi.org/simple/*NTc O s t j | | | | d d | j d | _ i | _ i | _ i | _ t j d j t
t | j | _
g | _ | o t j o | p t j } | r t j | | _ n t j j | _ d S)Nr8 |)r
r r2 index_url scanned_urls fetched_urls
package_pagesr_ r rf r r rM allowsto_scanr Z is_availableZfind_ca_bundleZ
opener_foropenerr requesturlopen)r r hostsZ ca_bundleZ
verify_sslrr kwZuse_sslr& r& r' r . s ! ' zPackageIndex.__init__Fc C s | | j k r | r d Sd | j | .)filterr\ rF r r itertoolsstarmap
scan_egg_link)r search_pathdirsZ egg_linksr& r& r' scan_egg_links s
zPackageIndex.scan_egg_linksc
C s t t j j | | ( } t t d t t j | } Wd QRXt | d k r\ d S| \ } } xQ t
t j j | | D]4 } t j j | | | _ t | _
| j | q Wd S)Nr; )openr\ rF rf r r r r} r~ rW r rR r rJ rh )r rF r Z raw_lineslinesZegg_pathZ
setup_pathrP r& r& r' r s ( " zPackageIndex.scan_egg_linkc
s f d d } xX t j | D]G } y, | t j j | t | j d Wq" t k
rh Yq" Xq" W| | \ } } | rxv t | | D]e } t | \ } } | j
d r | r | r | d | | f 7} n
j | j | q Wt
j d d | Sd Sd S)
z#Process the contents of a PyPI pagec s | j j r t t t j j | t j d j d } t | d k r d | d k r t | d } t
| d } d j j | j
i | .scanr9 z.pyz
#egg=%s-%sc S s d | j d d d S)Nz%sr9 r; )rN )mr& r& r' s z,PackageIndex.process_index..r N)r r{ r r" r r rN r# r rI r2 need_version_infoscan_urlPYPI_MD5sub)
r rB r r rM r r new_urlr5 fragr& )r r' r s$ ,
zPackageIndex.process_indexc C s | j d | d S)NzPPage at %s links to .py file(s) without version info; an index scan is required.)scan_all)r rB r& r& r' r s z PackageIndex.need_version_infoc G sI | j | j k r5 | r( | j | | | j d | j | j d S)Nz6Scanning index of all packages (this may take a while))r r r r r )r r rr r& r& r' r s zPackageIndex.scan_allc C s | j | j | j d | j j | j sK | j | j | j d | j j | j sm | j | x3 t | j j | j f D] } | j | q Wd S)Nr8 ) r r unsafe_namer r rm rS not_found_in_indexr )r requirementrB r& r& r'
find_packages s
%zPackageIndex.find_packagesc sk | j | j | x8 | | j D]) } | | k r; | S| j d | | q% Wt t | j | | S)Nz%s does not match %s)prescanr rm r superr obtain)r r installerrP ) __class__r& r' r s
zPackageIndex.obtainc C sf | j | j d | | j sb | j t j | t d | j j t j j
| f d S)z-
checker is a ContentChecker
z Validating %%s checksum for %sz7%s validation failed for %s; possible download problem?N) r r r r r\ unlinkr r r4 rF rX )r checkerr] tfpr& r& r'
check_hash s
zPackageIndex.check_hashc C sr xk | D]c } | j d k sJ t | sJ | j d sJ t t | rZ | j | q | j j | q Wd S)z;Add `urls` to the list that will be prescanned for searchesNzfile:)r r r3 r r r append)r urlsrB r& r& r' add_find_links s
zPackageIndex.add_find_linksc C s/ | j r" t t | j | j d | _ d S)z7Scan urls scheduled for prescanning (e.g. --find-links)N)r r r r )r r& r& r' r s zPackageIndex.prescanc C sN | | j r | j d } } n | j d } } | | | j | j d S)Nz#Couldn't retrieve index page for %rz3Couldn't find index page for %r (maybe misspelled?))rm r r r r )r r methr r& r& r' r & s
z PackageIndex.not_found_in_indexc C s t | t s t | } | ry | j | j d | | } t | \ } } | j d ru | j | | | } | St j j
| r | St | } t | j
| | d d S)a Locate and/or download `spec` to `tmpdir`, returning a local path
`spec` may be a ``Requirement`` object, or a string containing a URL,
an existing local filename, or a project/version requirement spec
(i.e. the string form of a ``Requirement`` object). If it is the URL
of a .py file with an unambiguous ``#egg=name-version`` tag (i.e., one
that escapes ``-`` as ``_`` throughout), a trivial ``setup.py`` is
automatically created alongside the downloaded file.
If `spec` is a ``Requirement`` object or a string containing a
project/version requirement spec, this method returns the location of
a matching distribution (possibly after downloading it to `tmpdir`).
If `spec` is a locally existing file or directory name, it is simply
returned unchanged. If `spec` is a URL, it is downloaded to a subpath
of `tmpdir`, and the local filename is returned. Various errors may be
raised if a problem occurs during downloading.
r9 z.pyrR N)r r r
_download_urlrN rI r2 gen_setupr\ rF r r( r fetch_distribution)r r$ tmpdirrD foundr5 rH r& r& r' r: 0 s zPackageIndex.downloadc s: j d | i d } d f d d } | rf j j | | | } | r | d k r | | | } | d k r j d k r j | | } | d k r | r j | | | } | d k r j d rd p d | n# j d | | j d | j Sd S)
a| Obtain a distribution suitable for fulfilling `requirement`
`requirement` must be a ``pkg_resources.Requirement`` instance.
If necessary, or if the `force_scan` flag is set, the requirement is
searched for in the (online) package index as well as the locally
installed packages. If a distribution matching `requirement` is found,
the returned distribution's ``location`` is the value you would have
gotten from calling the ``download()`` method with the matching
distribution's URL or filename. If no matching distribution is found,
``None`` is returned.
If the `source` flag is set, only source distributions and source
checkout links will be considered. Unless the `develop_ok` flag is
set, development and system eggs (i.e., those using the ``.egg-info``
format) will be ignored.
zSearching for %sNc s | d k r } x | | j D] } | j t k re re | k r j d | d | .findz:No local packages or working download links found for %s%sza source distribution of r zBest match: %srR )r r r r r cloner ) r r r
force_scanr r Z local_indexrP r r& )r r r r r r' r R s0 !
z PackageIndex.fetch_distributionc C s/ | j | | | | } | d k r+ | j Sd S)a3 Obtain a file suitable for fulfilling `requirement`
DEPRECATED; use the ``fetch_distribution()`` method now instead. For
backward compatibility, this routine is identical but returns the
``location`` of the downloaded distribution instead of a distribution
object.
N)r rR )r r r r r rP r& r& r' fetch s zPackageIndex.fetchc
C s\ t j | } | r= d d t | | j d d D p@ g } t | d k r-t j j | } t j j | | k r t j j | | } d d l
m } | | | s t j
| | | } t t j j | d d ? } | j d | d j | d j t j j | d f Wd QRX| S| rLt d | | f n t d
d S) Nc S s g | ] } | j r | q Sr& )rT )r` dr& r& r'
s z*PackageIndex.gen_setup..r9 r )samefilezsetup.pywzIfrom setuptools import setup
setup(name=%r, version=%r, py_modules=[%r])
zCan't unambiguously interpret project/version identifier %r; any dashes in the name or version should be escaped using underscores. %rzpCan't process plain .py files without an '#egg=name-version' suffix to enable automatic setup script generation.)rL rM r rN rW r\ rF rX dirnamerf Z setuptools.command.easy_installr shutilcopy2r writerS rT splitextr )
r r] rH r rM r rX dstr r r& r& r' r s2 !"zPackageIndex.gen_setupi c C s{ | j d | d } zMt j | } | j | } t | t j j rk t d | | j | j
f | j } d } | j } d } d | k r t | d } t
t t | } | j | | | | | t | d w }
xZ | j | } | r?| j | |
j | | d 7} | j | | | | | q Pq W| j | | |
Wd QRX| SWd | rv| j Xd S) NzDownloading %szCan't download %s: %s %sr r9 zcontent-lengthzContent-Lengthwbr= )r r r r r r r r r r r dl_blocksizer maxr int
reporthookr r r r r r ) r rB r] fpr r blocknumbssizeZsizesr r r& r& r' _download_to s:
zPackageIndex._download_toc C s d S)Nr& )r rB r] r$ Zblksizer& r& r& r' r" s zPackageIndex.reporthookc C s | j d r t | Sy t | | j SWnt t j f k
r } zV d j d d | j D } | r | j | | n t
d | | f | WYd d } ~ XnRt j j
k
r } z | SWYd d } ~ Xn$t j j k
r>} z= | r| j | | j n t
d | | j f | WYd d } ~ Xn t j k
r} z= | ro| j | | j n t
d | | j f | WYd d } ~ Xnc t j t j f k
r} z7 | r| j | | n t
d | | f | WYd d } ~ Xn Xd S)Nzfile: c S s g | ] } t | q Sr& )r} )r` argr& r& r' r s z)PackageIndex.open_url..z%s %szDownload error for %s: %sz;%s returned a bad status line. The server might be down, %s)r3
local_openopen_with_authr r# r
InvalidURLrf rr r r r r r URLErrorreason
BadStatusLineline
HTTPExceptionsocket)r rB warningvr r& r& r' r s8
+( zPackageIndex.open_urlc C sK t | \ } } | rL x7 d | k rH | j d d j d d } q Wn d } | j d rq | d d } t j j | | } | d k s | j d r | j | | S| d
k s | j d r | j | | S| j d r | j | | S| d
k r't
j j t
j
j | d S| j | d | j | | Sd S)Nz...\_Z__downloaded__z.egg.zipr- svnzsvn+gitzgit+zhg+r r; Tr0 )rI replacer2 r\ rF rf r3
_download_svn
_download_git _download_hgr r url2pathnamer" r? r _attempt_download)r rD rB r r4 rH r] r& r& r' r s$ % zPackageIndex._download_urlc C s | j | d d S)NT)r )r rB r& r& r' r = s zPackageIndex.scan_urlc C sK | j | | } d | j d d j k rC | j | | | S| Sd S)Nr z content-typer )r' r r1 _download_html)r rB r] r r& r& r' r? @ s z PackageIndex._attempt_downloadc C s t | } xT | D]L } | j r t j d | r^ | j t j | | j | | SPq W| j t j | t d | d S)Nz ([^- ]+ - )?Revision \d+:z Unexpected HTML page found at ) r r~ r_ r r r\ r r; r )r rB r r] r r0 r& r& r' r@ G s
zPackageIndex._download_htmlc C sy t j d t | j d d d } d } | j j d rHd | k rHt j j | \ } } } } } } | rH| j d rHd | d
d k rH| d
d j d d \ } } t | \ }
} |
rHd |
k r|
j d d \ } }
d | |
f } n
d
|
} | } | | | | | | f } t j j
| } | j d | | t j
d | | | f | S)Nz"SVN download support is deprecatedr< r9 r r zsvn:@z//r8 r; :z --username=%s --password=%sz --username=z'Doing subversion checkout from %s to %szsvn checkout%s -q %s %s)warningsr UserWarningrA r1 r3 r r" r?
_splituser
urlunparser r\ system)r rB r] credsrD netlocrF ra qr authhostuserpwrC r& r& r' r; V s& !$,"
zPackageIndex._download_svnc C s t j j | \ } } } } } | j d d d } | j d d d } d } d | k rw | j d d \ } } t j j | | | | d f } | | f S)N+r9 r< r rA r r= )r r" urlsplitrA rsplit
urlunsplit)rB
pop_prefixrD rI rF rG r revr& r& r' _vcs_split_rev_from_urll s ! !z$PackageIndex._vcs_split_rev_from_urlc C s | j d d d } | j | d d \ } } | j d | | t j d | | f | d k r | j d | t j d | | f | S)
Nr< r9 r rS Tz Doing git clone from %s to %szgit clone --quiet %s %szChecking out %sz git -C %s checkout --quiet %s)rA rU r r\ rG )r rB r] rT r& r& r' r< ~ s zPackageIndex._download_gitc C s | j d d d } | j | d d \ } } | j d | | t j d | | f | d k r | j d | t j d | | f | S)
Nr< r9 r rS Tz Doing hg clone from %s to %szhg clone --quiet %s %szUpdating to %szhg --cwd %s up -C -r %s -q)rA rU r r\ rG )r rB r] rT r& r& r' r= s zPackageIndex._download_hgc G s t j | | d S)N)r r )r r rr r& r& r' r s zPackageIndex.debugc G s t j | | d S)N)r r )r r rr r& r& r' r s zPackageIndex.infoc G s t j | | d S)N)r r )r r rr r& r& r' r s zPackageIndex.warn)r )'r r r r r r r r r r r r r r r r r r r r: r r r r r' r" r r r r? r@ r; staticmethodrU r< r= r r r r& r& )r r' r + sL 5
+
#J
) $ # z!&(#(\d+|x[\da-fA-F]+)|[\w.:-]+);?c C s | j d } t | S)Nr )rN r )rM whatr& r& r'
decode_entity s rX c C s
t t | S)a
Decode HTML entities in the given text.
>>> htmldecode(
... 'https://../package_name-0.1.2.tar.gz'
... '?tokena=A&tokenb=B">package_name-0.1.2.tar.gz')
'https://../package_name-0.1.2.tar.gz?tokena=A&tokenb=B">package_name-0.1.2.tar.gz'
)
entity_subrX )textr& r& r' r s r c s f d d } | S)Nc s f d d } | S)Nc s? t j } t j z | | SWd t j | Xd S)N)r2 getdefaulttimeoutsetdefaulttimeout)rr rs Z old_timeout)rt timeoutr& r' _socket_timeout s
z@socket_timeout.<locals>._socket_timeout.<locals>._socket_timeoutr& )rt r^ )r] )rt r' r^ s z'socket_timeout.<locals>._socket_timeoutr& )r] r^ r& )r] r' socket_timeout s r_ c C sI t j j | } | j } t j | } | j } | j d d S)aq
A function compatible with Python 2.3-3.3 that will encode
auth from a URL suitable for an HTTP header.
>>> str(_encode_auth('username%3Apassword'))
'dXNlcm5hbWU6cGFzc3dvcmQ='
Long auth strings should not cause a newline to be inserted.
>>> long_auth = 'username:' + 'password'*10
>>> chr(10) in str(_encode_auth(long_auth))
False
r )r r" r@ encodebase64 b64encoder r: )rK Zauth_sZ
auth_bytesZ
encoded_bytesencodedr& r& r' _encode_auth s
re c @ s: e Z d Z d Z d d Z d d Z d d Z d S)
Credentialz:
A username/password pair. Use like a namedtuple.
c C s | | _ | | _ d S)N)usernamepassword)r rg rh r& r& r' r s zCredential.__init__c c s | j V| j Vd S)N)rg rh )r r& r& r' __iter__ s zCredential.__iter__c C s d t | S)Nz%(username)s:%(password)s)vars)r r& r& r' __str__ s zCredential.__str__N)r r r r r ri rk r& r& r& r' rf s rf c @ sF e Z d Z d d Z e d d Z d d Z d d Z d S)
PyPIConfigc C sr t j d d d g d } t j j | | t j j t j j d d } t j j | rn | j
| d S)z%
Load from ~/.pypirc
rg rh
repositoryr ~z.pypircN) dictfromkeysr RawConfigParserr r\ rF rf
expanduserr r )r defaultsrcr& r& r' r s
!zPyPIConfig.__init__c s5 f d d j D } t t j | S)Nc s. g | ]$ } j | d j r | q S)rm )r r~ )r` section)r r& r' r s z2PyPIConfig.creds_by_repository.<locals>.<listcomp>)sectionsro r _get_repo_cred)r Zsections_with_repositoriesr& )r r' creds_by_repository s z PyPIConfig.creds_by_repositoryc C sO | j | d j } | t | j | d j | j | d j f S)Nrm rg rh )r r~ rf )r ru repor& r& r' rw s zPyPIConfig._get_repo_credc C s7 x0 | j j D] \ } } | j | r | Sq Wd S)z
If the URL indicated appears to be a repository defined in this
config, return the credential for that repository.
N)rx itemsr3 )r rB rm credr& r& r' find_credential s zPyPIConfig.find_credentialN)r r r r propertyrx rw r| r& r& r& r' rl s rl c C s t j j | } | \ } } } } } } | j d rH t j d | d
k ri t | \ } }
n d } | s t j | } | r t | } | j
| f } t j d | | r d t
| } | |
| | | | f }
t j j |
} t j j | } | j d | n t j j | } | j d t | | } | rt j j | j \ } } } } } } | | k r| |
k r| | | | | | f }
t j j |
| _ | S) z4Open a urllib2 request, handling HTTP authenticationrB znonnumeric port: ''httphttpsNz*Authenticating as %s for %s (from .pypirc)zBasic
Authorizationz
User-Agent)zhttpzhttps)r r" r? r2 r r, rE rl r| r} rg r r re rF r Request
add_header
user_agentrB )rB r parsedrD rI rF paramsrG r rK addressr{ r rC r r r# s2h2path2Zparam2Zquery2Zfrag2r& r& r' r+ s8 'r+ c C s. | j d \ } } } | r$ | n d | f S)zNsplituser('user[:passwd]@host[:port]')
--> 'user[:passwd]', 'host[:port]'.rA N)
rpartition)rL rM delimr& r& r' rE I s rE c C s | S)Nr& )rB r& r& r'
fix_sf_urlT s r c C s t j j | \ } } } } } } t j j | } t j j | rX t j j | S| j d rNt j j
| rNg } x t j | D] } t j j | | }
| d k r t
|
d } | j } Wd QRXPn t j j
|
r | d 7} | j d j d | q Wd }
|
j d | d d
j | } d \ } } n d \ } } } d d i } t j | } t j j | | | | | S)z7Read a local path, with special support for directoriesr8 z
index.htmlrNz<a href="{name}">{name}</a>r4 zB<html><head><title>{url}{files} |