Python爬虫有哪些通用方法可以应用？

本文共计157个文字，预计阅读时间需要1分钟。

python获取本地域名LocalDomain=self.request.headers['Host']

从header中读取IPimport osurl='http://www.**.net/images/logo.gif'filename=os.path.basename(url)print(filename)

从URL中提取文件名path=str(i) + /count

LocalDomain = self.request.headers[‘Host’] 服务器能够从header中读取IP

import os

url = 'www.**.net/images/logo.gif'
filename = os.path.basename(url)
print(filename)

python 从url中提取文件名

path = str(i)+"/"
count = 1
for files in os.listdir(path):
print(files)
with open(path+files, 'rb') as f:
print(f)
sha1obj = hashlib.sha256()
sha1obj.update(f.read())
hash = sha1obj.hexdigest()
print(hash)
os.rename(os.path.join(path,files),os.path.join(path,str(hash)+'.jpg'))
count+=1

文件名批量修改

print(re.findall('<div id="content">(.+?)</div>',chapter_detail['soup'])[0])