requests支持以下新特性：
- 支持http2，默认以http2进行连接，连接失败后会进行退化而进行http1.1连接
- 支持JA3指纹修改
- 支持http2+JA3指纹
- 支持修改TLS指纹
- 支持修改http2指纹
此模块参考于Python的requests模块
如果你想跟更多志同道合的人一起学习，那么加入群聊来吧！
go get github.com/wangluozhe/requestsgo get github.com/wangluozhe/[email protected]迫不及待了吗？本页内容为如何入门 Requests 提供了很好的指引。其假设你已经安装了 Requests。如果还没有，去安装看看吧。
首先，确认一下：
- Requests已安装
- Requests是最新的
让我们从一些简单的示例开始吧。
使用 requests 发送网络请求非常简单。
一开始要导入requests 模块：
import (
"github.com/wangluozhe/requests"
"github.com/wangluozhe/requests/url"
)然后，尝试获取某个网页。本例子中，我们来获取 Github 的公共时间线：
r, err := requests.Get("https://api.github.com/events", nil)现在，我们有一个名为 r 的 Response 对象。我们可以从这个对象中获取所有我们想要的信息。
Requests 简便的 API 意味着所有 HTTP 请求类型都是显而易见的。例如，你可以这样发送一个 HTTP POST 请求：
data := url.NewData()
data.Set("key","value")
r, err := requests.Post("http://httpbin.org/post", &url.Request{Data: data})漂亮，对吧？那么其他 HTTP 请求类型：PUT，DELETE，HEAD 以及 OPTIONS 又是如何的呢？都是一样的简单：
data := url.NewData()
data.Set("key","value")
r, err := requests.Post("http://httpbin.org/post", &url.Request{Data: data})
r, err = requests.Delete("http://httpbin.org/delete")
r, err = requests.Head("http://httpbin.org/get")
r, err = requests.Options("http://httpbin.org/get")都很不错吧，但这也仅是 requests 的冰山一角呢。
requests设置代理也非常简单，只需给req.Proxies传入代理ip即可。
req := url.NewRequest()
req.Proxies = "http://127.0.0.1:32768"
// 设置带账号密码的代理：req.Proxies = "http://username:password@ip:port"
r, err := requests.Get("https://api.github.com/events", req)你也许经常想为 URL 的查询字符串(query string)传递某种数据。如果你是手工构建 URL，那么数据会以键/值对的形式置于 URL 中，跟在一个问号的后面。例如， httpbin.org/get?key=val。 Requests 允许你使用 params 关键字参数，以一个字符串字典来提供这些参数。举例来说，如果你想传递 key1=value1 和 key2=value2 到 httpbin.org/get ，那么你可以使用如下代码：
params := url.NewParams()
params.Set("key1","value1")
params.Set("key2","value2")
r, err := requests.Get("http://httpbin.org/get",&url.Request{Params: params})
if err != nil {
fmt.Println(err)
}或者：
// map[string]type，type = string|int|float64|interface{}|[]string|[]int|[]float64|[]interface{}
req := url.NewRequest()
//params := map[string]string{
// "page": "1",
// "limit": "20",
// "skip": "5",
//}
//params := map[string]int{
// "page": 1,
// "limit": 20,
// "skip": 5,
//}
//params := map[string][]string{
// "page": []string{"1", "2"},
// "limit": []string{"20"},
// "skip": []string{"5"},
//}
//params := map[string][]int{
// "page": []int{1, 2},
// "limit": []int{20},
// "skip": []int{5},
//}
params := map[string]interface{}{
"page": []interface{}{"1", 2},
"limit": []string{"20"},
"skip": []int{5},
}
req.Params = url.ParseParams(params)
r, err := requests.Get("https://httpbin.org/get", req)
if err != nil {
fmt.Println(err)
}
fmt.Println(r.Text)或者：
params := url.ParseParams("key1=value1&key2=value2")
r, err := requests.Get("http://httpbin.org/get",&url.Request{Params: params})
if err != nil {
fmt.Println(err)
}通过打印输出该 URL，你能看到 URL 已被正确编码：
fmt.Println(r.Url)
http://httpbin.org/get?key1=value1&key2=value2你还可以使Params有多个值传入：
params := url.NewParams()
params.Set("key1","value1")
params.Add("key1","value2")
params.Set("key2","value2")
r, err := requests.Get("http://httpbin.org/get",&url.Request{Params: params})
if err != nil {
fmt.Println(err)
}
fmt.Println(r.Url)
http://httpbin.org/post?key1=value1&key1=value2&key2=value2我们能读取服务器响应的内容。再次以 GitHub 时间线为例：
package main
import (
"fmt"
"github.com/wangluozhe/requests"
)
func main() {
r, err := requests.Get("https://api.github.com/events", nil)
if err != nil {
fmt.Println(err)
}
fmt.Println(r.Text)
}
[{"repository":{"open_issues":0, "url":"https://github.com/...requests 会自动解码来自服务器的内容。大多数 unicode 字符集都能被无缝地解码。
你也能以字节数组的方式访问请求响应体，对于非文本请求：
fmt.Println(r.Content)
[91 123 34 105 100 34 58 34 50 48 55 49 52 50 51 57 56 48 53 34 44 34 116 121 112 101 34 58 34...Requests 会自动为你解码 gzip 和 deflate 以及br传输编码的响应数据。
例如，以请求返回的二进制数据创建一张图片，你可以使用如下代码：
package main
import "github.com/wangluozhe/requests"
func main(){
r, err := requests.Get("图片URL", nil)
if err != nil {
fmt.Println(err)
}
jpg,_ := os.Create("1.jpg")
io.Copy(jpg,r.Body) // 第一种
//jpg.Write(resp.Content) // 第二种
}Requests 中也有一个内置的 JSON 解码器，助你处理 JSON 数据（或者使用第三方库go-simplejson）：
package main
import (
"fmt"
"github.com/wangluozhe/requests"
)
func main(){
r, err := requests.Get("https://api.github.com/events", nil)
if err != nil{
fmt.Println(err)
}
json, err := r.Json() // 推荐使用r.SimpleJson，自带的json解码使用太过于复杂
// json, err := r.SimpleJson()
fmt.Println(json, err)
}
{"Accept-Ranges":["bytes"],"Access-Control-Allow-Origin":["*"],"Access-Control...如果 JSON 解码失败， r.Json() 就会返回一个异常。例如，响应内容是 401 (Unauthorized)，尝试访问 r.Json() 将会抛出 map[] invalid character '(' after top-level value 异常。
需要注意的是，成功调用 r.Json() 并不意味着响应的成功。有的服务器会在失败的响应中包含一个 JSON 对象（比如 HTTP 500 的错误细节）。这种 JSON 会被解码返回。要检查请求是否成功，请检查 r.StatusCode 是否和你的期望相同。
在罕见的情况下，你可能想获取来自服务器的原始套接字响应，那么你可以访问 r.Body。 具体你可以这么做：
r, err := requests.Get("http://www.baidu.com", nil)
if err != nil {
fmt.Println(err)
}
fmt.Println(r.Body)
// 返回的是io.ReadCloser类型但一般情况下，你应该以下面的模式将文本流保存到文件：
f, _ := os.Create("baidu.txt")
io.Copy(f,r.Body)如果你想为请求添加 HTTP 头部，只要简单地url.NewHeaders() 给 Headers 参数就可以了。
例如，在前一个示例中我们没有指定 content-type:
rawurl := "https://api.github.com/some/endpoint"
headers := url.NewHeaders()
headers.Set("user-agent", "my-app/0.0.1")
req := url.NewRequest()
req.Headers = headers
r, err := requests.Get(rawurl, req)或者
rawurl := "https://api.github.com/some/endpoint"
req := url.NewRequest()
req.Headers = url.ParseHeaders(map[string]interface{}{
":authority": "spidertools.cn",
":method": "GET",
":path": "/",
":scheme": "https",
"accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9",
"accept-encoding": "gzip, deflate, br",
"accept-language": "zh-CN,zh;q=0.9",
"cache-control": "no-cache",
"cookie": "_ga=GA1.1.630251354.1645893020; Hm_lvt_def79de877408c7bd826e49b694147bc=1647245863,1647936048,1648296630; Hm_lpvt_def79de877408c7bd826e49b694147bc=1648296630",
"pragma": "no-cache",
"sec-ch-ua": "\" Not A;Brand\";v=\"99\", \"Chromium\";v=\"98\", \"Google Chrome\";v=\"98\"",
"sec-ch-ua-mobile": "?0",
"sec-ch-ua-platform": "\"Windows\"",
"sec-fetch-dest": "document",
"sec-fetch-mode": "navigate",
"sec-fetch-site": "same-origin",
"sec-fetch-user": "?1",
"upgrade-insecure-requests": "1",
"user-agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/98.0.4758.80 Safari/537.36",
}) // 注意：此方式生成的headers是无序的，因为map是无序的。
r, err := requests.Get(rawurl, req)注意: 定制 header 的优先级低于某些特定的信息源，例如：
Content-Length请求头不能随便设置，可能会有错误发生。- 如果被重定向到别的主机，授权 header 就会被删除。
- 代理授权 header 会被 URL 中提供的代理身份覆盖掉。
- 在我们能判断内容长度的情况下，header 的 Content-Length 会被改写。
更进一步讲，Requests 不会基于定制 header 的具体情况改变自己的行为。只不过在最后的请求中，所有的 header 信息都会被传递进去。
注意: 所有的 header 值必须是 string。
如果你想让你的请求头变为有序的话，请添加一个"Header-Order:"参数即可，值为有序请求头数组，注：值必须全部为小写。
例如：
package main
import (
"github.com/wangluozhe/requests/url"
)
func main(){
headers := url.NewHeaders()
headers.Set("Path", "/get")
headers.Set("User-Agent", "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.198 Safari/537.36")
headers.Set("accept-language", "zh-CN,zh;q=0.9")
headers.Set("Scheme", "https")
headers.Set("accept-encoding", "gzip, deflate, br")
//headers.Set("Content-Length", "100") // 注意，不能随便改变Content-Length大小
headers.Set("Host", "httpbin.org")
headers.Set("Accept", "application/json, text/javascript, */*; q=0.01")
(*headers)["Header-Order:"] = []string{ // 请求头排序，值必须为小写
"user-agent",
"path",
"accept-language",
"scheme",
"connection",
"accept-encoding",
"content-length",
"host",
"accept",
}
req.Headers = headers
r, err := requests.Get("https://httpbin.org/get", req)
if err != nil {
fmt.Println(err)
}
fmt.Println("text:", r.Text)
// 最好用fiddler抓包工具查看一下
}或者：
package main
import (
"github.com/wangluozhe/requests/url"
)
func main(){
req := url.NewRequest()
req.Headers = url.ParseHeaders(`
:authority: spidertools.cn
:method: GET
:path: /
:scheme: https // 不会被加入到头部中
// accept: text/html // 不会被加入到头部中，因为被注释了
accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9
accept-encoding: gzip, deflate, br
accept-language: zh-CN,zh;q=0.9
cache-control: no-cache
cookie: _ga=GA1.1.630251354.1645893020; Hm_lvt_def79de877408c7bd826e49b694147bc=1647245863,1647936048,1648296630; Hm_lpvt_def79de877408c7bd826e49b694147bc=1648296630
pragma: no-cache
sec-ch-ua: " Not A;Brand";v="99", "Chromium";v="98", "Google Chrome";v="98"
sec-ch-ua-mobile: ?0
sec-ch-ua-platform: "Windows"
sec-fetch-dest: document
sec-fetch-mode: navigate
sec-fetch-site: same-origin
sec-fetch-user: ?1
upgrade-insecure-requests: 1
user-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/98.0.4758.80 Safari/537.36
`) // 注意：这是反引号，不是单引号
r, err := requests.Get("https://httpbin.org/get", req)
if err != nil {
fmt.Println(err)
}
fmt.Println("text:", r.Text)
// 最好用fiddler抓包工具查看一下
}通常，你想要发送一些编码为表单形式的数据——非常像一个 HTML 表单。要实现这个，只需简单地url.NewData()给 Data 参数。你的数据字典在发出请求时会自动编码为表单形式：
data := url.NewData()
data.Set("key1","value1")
data.Set("key2","value2")
req := url.NewRequest()
req.Data = data
r, err := requests.Post("http://www.baidu.com",req)
if err != nil {
fmt.Println(err)
}
fmt.Println(r.Text)
...
"form": {
"key1": "value1",
"key2": "value2"
}
...或者：
req := url.NewRequest()
// 跟url.ParseParams同一用法，只是名称不同而已。
// data := map[string]interface{}{
// "page": []interface{}{"1", 2},
// "limit": []string{"20"},
// "skip": []int{5},
// }
// req.Data = url.ParseData(data)
req.Data = url.ParseData("key1=value1&key2=value2")
r, err := requests.Post("http://www.baidu.com",req)
if err != nil {
fmt.Println(err)
}
fmt.Println(r.Text)
...
"form": {
"key1": "value1",
"key2": "value2"
}
...你还可以为 data在表单中多个元素使用同一 key 的时候，这种方式尤其有效：
data := url.NewData()
data.Set("key1","value1")
data.Add("key1","value3")
data.Set("key2","value2")
data.Add("key2","value4")
req := url.NewRequest()
req.Data = data
r, err := requests.Post("http://httpbin.org/post",req)
if err != nil {
fmt.Println(err)
}
fmt.Println(r.Text)
...
"form": {
"key1": [
"value1",
"value3"
],
"key2": [
"value2",
"value4"
]
}
...很多时候你想要发送的数据并非编码为表单形式的。如果你想传递一个 json 数据那么用下面的方法。
你可以使用 Json 参数直接传递，然后它就会被自动编码。
package main
import (
"fmt"
"github.com/wangluozhe/requests"
"github.com/wangluozhe/requests/url"
)
func main() {
req := url.NewRequest()
req.Json = map[string]interface{}{ // 修正老版map[string]string
"some": "data",
"name": "测试",
"colors": []string{"蓝色", "绿色", "紫色"},
"data": map[string]interface{}{
"json": true,
"age": 15,
},
}
r, err := requests.Post("http://httpbin.org/post", req)
if err != nil {
fmt.Println(err)
}
fmt.Println(r.Text)
}
{
"args": {},
"data": "{\"colors\":[\"\u84dd\u8272\",\"\u7eff\u8272\",\"\u7d2b\u8272\"],\"data\":{\"age\":15,\"json\":true},\"name\":\"\u6d4b\u8bd5\",\"some\":\"data\"}",
"files": {},
"form": {},
"headers": {
"Accept": "*/*",
"Accept-Encoding": "gzip, deflate, br",
"Content-Length": "99",
"Content-Type": "application/json",
"Host": "httpbin.org",
"User-Agent": "golang-requests 1.0",
"X-Amzn-Trace-Id": "Root=1-6241ac54-4345fed071127ed54d2ae83b"
},
"json": {
"colors": [
"\u84dd\u8272",
"\u7eff\u8272",
"\u7d2b\u8272"
],
"data": {
"age": 15,
"json": true
},
"name": "\u6d4b\u8bd5",
"some": "data"
},
"origin": "220.249.16.210",
"url": "http://httpbin.org/post"
}requests 使得上传多部分编码文件变得很简单：
files := url.NewFiles()
// SetFile(name,fileName,filePath,contentType)
// name为字段名，fileName为上传的文件名，filePath为上传文件的绝对路径，contentType为上传的文件类型
// 如果contentType设置为""，则默认为"application/octet-stream"
files.SetFile("api","api","D:\\Go\\github.com\\wangluozhe\\requests\\api.go","")
req := url.NewRequest()
req.Files = files
r, err := requests.Post("http://httpbin.org/post",req)
if err != nil {
fmt.Println(err)
}
fmt.Println(r.Text)
...
"files": {
"api": "文件内容"
}
...requests使得FormData的使用也方便多了：
files := url.NewFiles()
// SetFile(name,value)
// name为字段名，value为值
files.SetField("name","value")
req := url.NewRequest()
req.Files = files
r, err := requests.Post("http://httpbin.org/post",req)
if err != nil {
fmt.Println(err)
}
fmt.Println(r.Text)
...
"form": {
"name": "value"
}
...requests支持post传输普通文本信息，默认content-type=text/plain，也可以自定义content-type。
package main
import (
"fmt"
"github.com/wangluozhe/requests"
"github.com/wangluozhe/requests/url"
)
func main() {
req := url.NewRequest()
headers := url.NewHeaders()
headers.Set("User-Agent", "123")
req.Headers = headers
req.Body = "testdata"
r, _ := requests.Post("https://httpbin.org/post", req)
fmt.Println(r.Text)
}我们可以检测响应状态码：
r, err := requests.Get("http://httpbin.org/get", nil)
fmt.Println(r.StatusCode)
200为方便引用，可以直接使用此方法：
fmt.Println(r.StatusCode == http.StatusOK)
True如果发送了一个错误请求(一个 4XX 客户端错误，或者 5XX 服务器错误响应)，我们可以通过 Response.RaiseForStatus() 来抛出异常：
r, err := requests.Get("http://httpbin.org/status/404", nil)
fmt.Println(r.StatusCode)
404
fmt.Println(r.RaiseForStatus())
404 Client Error但是，由于我们的例子中 r 的 StatusCode 是 200 ，当我们调用 RaiseForStatus() 时，得到的是：
fmt.Println(r.RaiseForStatus())
nil一切都挺和谐哈。
我们可以查看以一个 http.Header形式（实际是一个map[string][]string类型）展示的服务器响应头：
fmt.Println(r.Headers)
map[Access-Control-Allow-Credentials:[true] Access-Control-Allow-Origin:[*] Connection:[keep-alive] Content-Length:[1976] Content-Type:[application/json] Date:[Sat, 12 Mar 2022 15:59:05 GMT] Server:[gunicorn/19.9.0]]但是这个类型比较特殊：它是仅为 HTTP 头部而生的。根据 RFC 2616， HTTP 头部是大小写不敏感的。
因此，我们可以使用任意大写形式来访问这些响应头字段：
fmt.Println(r.Headers["Content-Type"][0])
application/json
fmt.Println(r.Headers.Get("content-type"))
application/json它还有一个特殊点，那就是服务器可以多次接受同一 header，每次都使用不同的值。但 Requests 会将它们合并，这样它们就可以用一个映射来表示出来，参见 RFC 7230:
A recipient MAY combine multiple header fields with the same field name into one "field-name: field-value" pair, without changing the semantics of the message, by appending each subsequent field value to the combined field value in order, separated by a comma.
接收者可以合并多个相同名称的 header 栏位，把它们合为一个 "field-name: field-value" 配对，将每个后续的栏位值依次追加到合并的栏位值中，用逗号隔开即可，这样做不会改变信息的语义。
如果某个响应中包含一些 cookie，你可以快速访问它们：
url := "https://www.baidu.com"
r, err := requests.Get(url, nil)
fmt.Println(r.Cookies)
[BD_NOT_HTTPS=1; Path=/; Max-Age=300 BIDUPSID=7DB59A8D47E763943295969C33979837; Path=/; Domain=baidu.com; Max-Age=2147483647 PSTM=1647233990; Path=/; Domain=baidu.com; Max-Age=2147483647 BAIDUID=7DB59A8D47E7639495833AF6370F9985:FG=1; Path=/; Domain=baidu.com; Max-Age=31536000]要想发送你的cookies到服务器，可以使用 cookies 参数：
req := url.NewRequest()
cookies,_ := cookiejar.New(nil)
// cookies := url.NewCookies() // 推荐使用这种
urls, _ := url.Parse("http://httpbin.org/cookies")
cookies.SetCookies(urls,[]*http.Cookie{&http.Cookie{
Name: "cookies_are",
Value: "working",
}})
req.Cookies = cookies
r, err := requests.Get("http://httpbin.org/cookies", req)
if err != nil {
fmt.Println(err)
}
fmt.Println(r.Text)
{"cookies": {"cookies_are": "working"}}或者：
package main
import (
"fmt"
"github.com/wangluozhe/requests"
"github.com/wangluozhe/requests/url"
)
func main() {
rawUrl := "http://httpbin.org/cookies"
req := url.NewRequest()
//cookies := map[string]string{
// "name": "zhangsan",
// "age": "10",
// "language": "en",
//}
//cookies := map[string]int{
// "age": 20,
// "page": 2,
// "limit": 10,
//}
cookies := map[string]interface{}{
"Hm_lpvt_def79de877408c7bd826e49b694147bc": 1648301329,
"Hm_lvt_def79de877408c7bd826e49b694147bc": "1647245863,1647936048,1648296630",
"_ga": "GA1.1.630251354.1645893020",
}
req.Cookies = url.ParseCookies(rawUrl, cookies)
//req.Cookies = url.ParseCookies(rawUrl,"_ga=GA1.1.630251354.1645893020; Hm_lvt_def79de877408c7bd826e49b694147bc=1647245863,1647936048,1648296630; Hm_lpvt_def79de877408c7bd826e49b694147bc=1648301329")
r, err := requests.Get(rawUrl, req)
if err != nil {
fmt.Println(err)
}
fmt.Println(r.Text)
}
{
"cookies": {
"Hm_lpvt_def79de877408c7bd826e49b694147bc": "1648301329",
"Hm_lvt_def79de877408c7bd826e49b694147bc": "1647245863,1647936048,1648296630",
"_ga": "GA1.1.630251354.1645893020"
}
}默认情况下，除了 HEAD, Requests 会自动处理所有重定向。
可以使用响应对象的 history 方法来追踪重定向。
Response.History 是一个 Response 对象的列表，为了完成请求而创建了这些对象。这个对象列表按照从最老到最近的请求进行排序。
例如，Github 将所有的 HTTP 请求重定向到 HTTPS：
r, err := requests.Get("http://github.com", nil)
fmt.Println(r.Url)
https://github.com/
fmt.Println(r.StatusCode)
200
fmt.Println(r.History)
[0xc0001803f0]如果你使用的是GET、OPTIONS、POST、PUT、PATCH 或者 DELETE，那么你可以通过 allow_redirects 参数禁用重定向处理：
注：跳转链接请到headers中自行获取
req := url.NewRequest()
req.AllowRedirects = false
r, err := requests.Get("http://github.com", req)
if err != nil {
fmt.Println(err)
}
fmt.Println(r.StatusCode)
301
fmt.Println(r.History)
[]如果你使用了 HEAD，你也可以启用重定向：
req := url.NewRequest()
req.AllowRedirects = true
r, err := requests.Get("http://github.com", req)
if err != nil {
fmt.Println(err)
}
fmt.Println(r.Url)
https://github.com/
fmt.Println(r.History)
[0xc0001803f0]你可以告诉 requests 在经过以 Timeout 参数设定的秒数时间之后停止等待响应。基本上所有的生产代码都应该使用这一参数。如果不使用，你的程序可能会永远失去响应：
req := url.NewRequest()
req.Timeout = 1 * time.Millisecond
r, err := requests.Get("http://github.com",req)
if err != nil {
fmt.Println(err)
}
panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xc0000005 code=0x0 addr=0x0 pc=0x253460]
goroutine 1 [running]:
main.main()
D:/Go/github.com/wangluozhe/requests/examples/test.go:27 +0xc0注意
Timeout 仅对连接过程有效，与响应体的下载无关。 Timeout 并不是整个下载响应的时间限制，而是如果服务器在 Timeout 秒内没有应答，将会引发一个异常（更精确地说，是在 Timeout 秒内没有从基础套接字上接收到任何字节的数据时）If no timeout is specified explicitly, requests do not time out.
许多要求身份认证的web服务都接受 HTTP Basic Auth。这是最简单的一种身份认证，并且 requests 对这种认证方式的支持是直接开箱即可用。
以 HTTP Basic Auth 发送请求非常简单：
package main
import (
"fmt"
"github.com/wangluozhe/requests"
"github.com/wangluozhe/requests/url"
)
func main() {
req := url.NewRequest()
req.Auth = []string{"user","password"}
r, err := requests.Get("http://httpbin.org/basic-auth/user/password",req)
if err != nil{
fmt.Println(err)
}
fmt.Println("text:",r.Text)
}
{
"authenticated": true,
"user": "user"
}你也可以指定一个本地证书用作客户端证书，可以是一个包含两个文件路径的数组（cert，key）或一个包含三个文件路径的数组（cert，key，根证书）：
req := url.NewRequest()
req.Cert = []string{"cert","key"}
// req.Cert = []string{"cert","key","rootca"}
r, err := requests.Get("xxx",req)
if err != nil{
fmt.Println(err)
}或者保持在会话中：
session := requests.NewSession()
session.Cert = []string{"cert","key"}requests也支持JA3指纹的修改，可以让你在访问的时候使用你自己定义的JA3指纹进行TLS握手访问，但是请注意，JA3指纹必须符合要求，不能随便更改，最好使用wireshark或者ja3er.com获取标准指纹，而不是随便输入一串数字。
req := url.NewRequest()
headers := url.NewHeaders()
headers.Set("User-Agent", "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.198 Safari/537.36")
req.Headers = headers
req.Ja3 = "771,4865-4866-4867-49195-49199-49196-49200-52393-52392-49171-49172-156-157-47-53,0-23-65281-10-11-35-16-5-13-18-51-45-43-27-21,29-23-24,0"
r, err := requests.Get("https://ja3er.com/json", req)
if err != nil {
fmt.Println(err)
}
fmt.Println(r.Text)
{"ja3_hash":"b32309a26951912be7dba376398abc3b", "transport": "771,4865-4866-4867-49195-49199-49196-49200-52393-52392-49171-49172-156-157-47-53,0-23-65281-10-11-35-16-5-13-18-51-45-43-27-21,29-23-24,0", "User-Agent": "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.198 Safari/537.36"}requests支持你随意修改TLS指纹信息
package main
import (
"fmt"
http "github.com/wangluozhe/chttp"
"github.com/wangluozhe/requests"
"github.com/wangluozhe/requests/transport"
"github.com/wangluozhe/requests/url"
)
func main() {
req := url.NewRequest()
headers := &http.Header{
"User-Agent": []string{"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:109.0) Gecko/20100101 Firefox/112.0"},
"accept": []string{"text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8"},
"accept-language": []string{"zh-CN,zh;q=0.8,zh-TW;q=0.7,zh-HK;q=0.5,en-US;q=0.3,en;q=0.2"},
"accept-encoding": []string{"gzip, deflate, br"},
"upgrade-insecure-requests": []string{"1"},
"sec-fetch-dest": []string{"document"},
"sec-fetch-mode": []string{"navigate"},
"sec-fetch-site": []string{"none"},
"sec-fetch-user": []string{"?1"},
"te": []string{"trailers"},
http.PHeaderOrderKey: []string{
":method",
":path",
":authority",
":scheme",
},
http.HeaderOrderKey: []string{
"user-agent",
"accept",
"accept-language",
"accept-encoding",
"upgrade-insecure-requests",
"sec-fetch-dest",
"sec-fetch-mode",
"sec-fetch-site",
"sec-fetch-user",
"te",
},
}
req.Headers = headers
req.Ja3 = "771,4865-4866-4867-49195-49199-49196-49200-52393-52392-49171-49172-156-157-47-53,0-23-65281-10-11-35-16-5-13-18-51-45-43-27-21,29-23-24,0"
es := &transport.Extensions{
SupportedSignatureAlgorithms: []string{
"ECDSAWithP256AndSHA256",
"ECDSAWithP384AndSHA384",
"ECDSAWithP521AndSHA512",
"PSSWithSHA256",
"PSSWithSHA384",
"PSSWithSHA512",
"PKCS1WithSHA256",
"PKCS1WithSHA384",
"PKCS1WithSHA512",
"ECDSAWithSHA1",
"PKCS1WithSHA1",
},
//CertCompressionAlgo: []string{
// "brotli",
//},
RecordSizeLimit: 4001,
DelegatedCredentials: []string{
"ECDSAWithP256AndSHA256",
"ECDSAWithP384AndSHA384",
"ECDSAWithP521AndSHA512",
"ECDSAWithSHA1",
},
SupportedVersions: []string{
"1.3",
"1.2",
},
PSKKeyExchangeModes: []string{
"PskModeDHE",
},
KeyShareCurves: []string{
"X25519",
"P256",
},
}
tes := transport.ToTLSExtensions(es)
req.TLSExtensions = tes
r, err := requests.Get("https://tls.peet.ws/api/all", req)
if err != nil {
fmt.Println(err)
}
fmt.Println(r.Request.Headers)
fmt.Println("url:", r.Url)
fmt.Println("headers:", r.Headers)
fmt.Println("text:", r.Text)
}requests支持HTTP2指纹信息的修改
package main
import (
"fmt"
http "github.com/wangluozhe/chttp"
"github.com/wangluozhe/requests"
"github.com/wangluozhe/requests/transport"
"github.com/wangluozhe/requests/url"
)
func main() {
req := url.NewRequest()
headers := &http.Header{
"User-Agent": []string{"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:109.0) Gecko/20100101 Firefox/112.0"},
"accept": []string{"text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8"},
"accept-language": []string{"zh-CN,zh;q=0.8,zh-TW;q=0.7,zh-HK;q=0.5,en-US;q=0.3,en;q=0.2"},
"accept-encoding": []string{"gzip, deflate, br"},
"upgrade-insecure-requests": []string{"1"},
"sec-fetch-dest": []string{"document"},
"sec-fetch-mode": []string{"navigate"},
"sec-fetch-site": []string{"none"},
"sec-fetch-user": []string{"?1"},
"te": []string{"trailers"},
http.PHeaderOrderKey: []string{
":method",
":path",
":authority",
":scheme",
},
http.HeaderOrderKey: []string{
"user-agent",
"accept",
"accept-language",
"accept-encoding",
"upgrade-insecure-requests",
"sec-fetch-dest",
"sec-fetch-mode",
"sec-fetch-site",
"sec-fetch-user",
"te",
},
}
req.Headers = headers
req.Ja3 = "771,4865-4866-4867-49195-49199-49196-49200-52393-52392-49171-49172-156-157-47-53,0-23-65281-10-11-35-16-5-13-18-51-45-43-27-21,29-23-24,0"
h2s := &transport.H2Settings{
Settings: map[string]int{
"HEADER_TABLE_SIZE": 65536,
//"ENABLE_PUSH": 0,
//"MAX_HEADER_LIST_SIZE": 262144,
//"MAX_CONCURRENT_STREAMS": 1000,
"INITIAL_WINDOW_SIZE": 131072,
"MAX_FRAME_SIZE": 16384,
},
SettingsOrder: []string{
"HEADER_TABLE_SIZE",
"INITIAL_WINDOW_SIZE",
"MAX_FRAME_SIZE",
},
ConnectionFlow: 12517377,
HeaderPriority: map[string]interface{}{
"weight": 42,
"streamDep": 13,
"exclusive": false,
},
PriorityFrames: []map[string]interface{}{
{
"streamID": 3,
"priorityParam": map[string]interface{}{
"weight": 201,
"streamDep": 0,
"exclusive": false,
},
},
{
"streamID": 5,
"priorityParam": map[string]interface{}{
"weight": 101,
"streamDep": 0,
"exclusive": false,
},
},
{
"streamID": 7,
"priorityParam": map[string]interface{}{
"weight": 1,
"streamDep": 0,
"exclusive": false,
},
},
{
"streamID": 9,
"priorityParam": map[string]interface{}{
"weight": 1,
"streamDep": 7,
"exclusive": false,
},
},
{
"streamID": 11,
"priorityParam": map[string]interface{}{
"weight": 1,
"streamDep": 3,
"exclusive": false,
},
},
{
"streamID": 13,
"priorityParam": map[string]interface{}{
"weight": 241,
"streamDep": 0,
"exclusive": false,
},
},
},
}
h2ss := transport.ToHTTP2Settings(h2s)
req.HTTP2Settings = h2ss
r, err := requests.Get("https://tls.peet.ws/api/all", req)
if err != nil {
fmt.Println(err)
}
fmt.Println(r.Request.Headers)
fmt.Println("url:", r.Url)
fmt.Println("headers:", r.Headers)
fmt.Println("text:", r.Text)
}JA4是什么，怎么组成的，请看华总的文章JA4概要
package main
import (
"fmt"
http "github.com/wangluozhe/chttp"
"github.com/wangluozhe/requests"
"github.com/wangluozhe/requests/transport"
"github.com/wangluozhe/requests/url"
)
func main() {
req := url.NewRequest()
headers := &http.Header{
"Accept": []string{"text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9"},
"Accept-Encoding": []string{"gzip, deflate, br"},
"Accept-Language": []string{"zh-CN,zh;q=0.9"},
"Cache-Control": []string{"no-cache"},
"Connection": []string{"keep-alive"},
"Host": []string{"gospider2.gospiderb.asia:8998"},
"Pragma": []string{"no-cache"},
"sec-ch-ua": []string{"\".Not/A)Brand\";v=\"99\", \"Google Chrome\";v=\"103\", \"Chromium\";v=\"103\""},
"sec-ch-ua-mobile": []string{"\"?0\""},
"sec-ch-ua-platform": []string{"\"Windows\""},
"Sec-Fetch-Dest": []string{"document"},
"Sec-Fetch-Mode": []string{"navigate"},
"Sec-Fetch-Site": []string{"none"},
"Sec-Fetch-User": []string{"\"?1\""},
"Upgrade-Insecure-Requests": []string{"1"},
"User-Agent": []string{"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.0.0 Safari/537.36"},
http.HeaderOrderKey: []string{
"HOST",
"connection",
"pragma",
"Cache-Control",
"sec-ch-ua",
"sec-ch-ua-mobile",
"sec-ch-ua-platform",
"upgrade-insecure-requests",
"user-agent",
"accept",
"sec-fetch-site",
"sec-fetch-mode",
"sec-fetch-user",
"sec-fetch-dest",
"accept-encoding",
"accept-language",
},
http.UnChangedHeaderKey: []string{
"sec-ch-ua",
"sec-ch-ua-mobile",
"sec-ch-ua-platform",
},
}
req.Headers = headers
req.Ja3 = "772,4865-4866-4867-49195-49199-49196-49200-52393-52392-49171-49172-156-157-47-53,0-23-65281-10-11-35-16-5-13-18-51-45-43-27-17513-21-41,29-23-24,0"
req.ForceHTTP1 = true
es := &transport.Extensions{
SupportedSignatureAlgorithms: []string{
"ecdsa_secp256r1_sha256",
"rsa_pss_rsae_sha256",
"rsa_pkcs1_sha256",
"ecdsa_secp384r1_sha384",
"rsa_pss_rsae_sha384",
"rsa_pkcs1_sha384",
"rsa_pss_rsae_sha512",
"rsa_pkcs1_sha512",
},
CertCompressionAlgo: []string{
"brotli",
},
RecordSizeLimit: 4001,
DelegatedCredentials: []string{
"ECDSAWithP256AndSHA256",
"ECDSAWithP384AndSHA384",
"ECDSAWithP521AndSHA512",
"ECDSAWithSHA1",
},
SupportedVersions: []string{
"GREASE",
"1.3",
"1.2",
},
PSKKeyExchangeModes: []string{
"PskModeDHE",
},
KeyShareCurves: []string{
"GREASE",
"X25519",
},
}
tes := transport.ToTLSExtensions(es)
req.TLSExtensions = tes
r, err := requests.Get("https://gospider2.gospiderb.asia:8998/", req)
if err != nil {
fmt.Println(err)
}
fmt.Println(r.Request.Headers)
fmt.Println("url:", r.Url)
fmt.Println("headers:", r.Headers)
fmt.Println("text:", r.Text)
}
// 模拟后的结果
// JA4: t13d1516h1_dea800f94266_27c5da80acb3
// JA4H: ge11nn13zhcn_d8f538a17def_e3b0c44298fc_e3b0c44298fcrequests支持一些常用的编码方式，并且命名更使人易懂。传递的参数可为字符串（string）或字符数组（[]byte）类型。
package main
import (
"fmt"
"github.com/wangluozhe/requests/utils"
)
func main() {
url := "https://www.baidu.com"
hexen := utils.HexEncode(url)
fmt.Println(hexen)
fmt.Println(string(hexen))
hexde := utils.HexDecode(hexen)
fmt.Println(hexde)
fmt.Println(string(hexde))
}
[54 56 55 52 55 52 55 48 55 51 51 97 50 102 50 102 55 55 55 55 55 55 50 101 54 50 54 49 54 57 54 52 55 53 50 101 54 51 54 102 54 100]
68747470733a2f2f7777772e62616964752e636f6d
[104 116 116 112 115 58 47 47 119 119 119 46 98 97 105 100 117 46 99 111 109]
https://www.baidu.compackage main
import (
"fmt"
"github.com/wangluozhe/requests/utils"
)
func main() {
url := "https://www.baidu.com?page=10&abc=123&name=你好啊"
encode := utils.EncodeURIComponent(url)
fmt.Println(encode)
decode := utils.DecodeURIComponent(encode)
fmt.Println(decode)
encode1 := utils.EncodeURI(url)
fmt.Println(encode1)
decode1 := utils.DecodeURI(encode1)
fmt.Println(decode1)
escape := utils.Escape(url)
fmt.Println(escape)
unescape := utils.UnEscape(escape)
fmt.Println(unescape)
}
https%3A%2F%2Fwww.baidu.com%3Fpage%3D10%26abc%3D123%26name%3D%E4%BD%A0%E5%A5%BD%E5%95%8A
https://www.baidu.com?page=10&abc=123&name=你好啊
https://www.baidu.com?page=10&abc=123&name=%E4%BD%A0%E5%A5%BD%E5%95%8A
https://www.baidu.com?page=10&abc=123&name=你好啊
https%3A//www.baidu.com%3Fpage%3D10%26abc%3D123%26name%3D%u4f60%u597d%u554a
https://www.baidu.com?page=10&abc=123&name=你好啊package main
import (
"fmt"
"github.com/wangluozhe/requests/utils"
)
func main() {
url := "https://www.baidu.com?page=10&abc=123&name=你好啊"
base32en := utils.Base32Encode(url)
fmt.Println(base32en)
base32de := utils.Base32Decode(base32en)
fmt.Println(base32de)
base64en := utils.Base64Encode(url)
fmt.Println(base64en)
base64de := utils.Base64Decode(base64en)
fmt.Println(base64de)
// 或者像JavaScript语言一样
btoa := utils.Btoa(url)
fmt.Println(btoa)
atob := utils.Atob(btoa)
fmt.Println(atob)
}
NB2HI4DTHIXS653XO4XGEYLJMR2S4Y3PNU7XAYLHMU6TCMBGMFRGGPJRGIZSM3TBNVST3ZF5UDS2LPPFSWFA====
https://www.baidu.com?page=10&abc=123&name=你好啊
aHR0cHM6Ly93d3cuYmFpZHUuY29tP3BhZ2U9MTAmYWJjPTEyMyZuYW1lPeS9oOWlveWVig==
https://www.baidu.com?page=10&abc=123&name=你好啊
aHR0cHM6Ly93d3cuYmFpZHUuY29tP3BhZ2U9MTAmYWJjPTEyMyZuYW1lPeS9oOWlveWVig==
https://www.baidu.com?page=10&abc=123&name=你好啊requests支持一些常用的加密算法，并且命名更使人易懂。传递的参数可为字符串（string）或字符数组（[]byte）类型。
MD系列：
package main
import (
"fmt"
"github.com/wangluozhe/requests/utils"
)
func main() {
m := utils.MD4("123")
fmt.Println("MD4:", m)
m = utils.RIPEMD160("123")
fmt.Println("RIPEMD160:", m)
m = utils.MD5("123")
fmt.Println("MD5:", m)
}
MD4: c58cda49f00748a3bc0fcfa511d516cb
RIPEMD160: e3431a8e0adbf96fd140103dc6f63a3f8fa343ab
MD5: 202cb962ac59075b964b07152d234b70SHA系列：
package main
import (
"fmt"
"github.com/wangluozhe/requests/utils"
)
func main() {
s1 := utils.SHA1("123")
b64 := utils.Btoa(s1)
h16 := utils.HexEncode(s1)
fmt.Println("SHA1-base64:", b64)
fmt.Println("SHA1-hex:", string(h16))
s2 := utils.SHA224("123")
b64 = utils.Btoa(s2)
h16 = utils.HexEncode(s2)
fmt.Println("SHA224-base64:", b64)
fmt.Println("SHA224-hex:", string(h16))
s2 = utils.SHA256("123")
b64 = utils.Btoa(s2)
h16 = utils.HexEncode(s2)
fmt.Println("SHA256-base64:", b64)
fmt.Println("SHA256-hex:", string(h16))
s5 := utils.SHA384("123")
b64 = utils.Btoa(s5)
h16 = utils.HexEncode(s5)
fmt.Println("SHA384-base64:", b64)
fmt.Println("SHA384-hex:", string(h16))
s5 = utils.SHA512("123")
b64 = utils.Btoa(s5)
h16 = utils.HexEncode(s5)
fmt.Println("SHA512-base64:", b64)
fmt.Println("SHA512-hex:", string(h16))
}
SHA1-base64: QL0AFWMIX8NRZTKeof9cXsvbvu8=
SHA1-hex: 40bd001563085fc35165329ea1ff5c5ecbdbbeef
SHA224-base64: eNgEXWhKvS7s6SN1jzzXgUid86SOEniYJGYBfw==
SHA224-hex: 78d8045d684abd2eece923758f3cd781489df3a48e1278982466017f
SHA256-base64: pmWkWSBCL51Bfkhn79xPuKBKHz//H6B+mY6G9/eieuM=
SHA256-hex: a665a45920422f9d417e4867efdc4fb8a04a1f3fff1fa07e998e86f7f7a27ae3
SHA384-base64: mgqC8MDPMUcNev/t40BsyaqEEGcVILcnBE7aFbTCVTKptc2Kr5zsSRnXYlW2v7AP
SHA384-hex: 9a0a82f0c0cf31470d7affede3406cc9aa8410671520b727044eda15b4c25532a9b5cd8aaf9cec4919d76255b6bfb00f
SHA512-base64: PJkJr+wlNU1VHa4hWQuybjjVPyFzuNPcPu5MBH56scHri4UQPjvnumE7MbtcnDYhTcnxSkL9ei/bhIVrylxEwg==
SHA512-hex: 3c9909afec25354d551dae21590bb26e38d53f2173b8d3dc3eee4c047e7ab1c1eb8b85103e3be7ba613b31bb5c9c36214dc9f14a42fd7a2fdb84856bca5c44c2Hmac系列：
package main
import (
"fmt"
"github.com/wangluozhe/requests/utils"
)
func main() {
md4 := utils.HmacMD4("123", "123")
bs64 := utils.Btoa(md4)
hex := utils.HexEncode(md4)
fmt.Println("HmacMD4-base64:", bs64)
fmt.Println("HmacMD4-hex:", string(hex))
r160 := utils.HmacRIPEMD160("123", "123")
bs64 = utils.Btoa(r160)
hex = utils.HexEncode(r160)
fmt.Println("HmacRIPEMD160-base64:", bs64)
fmt.Println("HmacRIPEMD160-hex:", string(hex))
md5 := utils.HmacMD5("123", "123")
bs64 = utils.Btoa(md5)
hex = utils.HexEncode(md5)
fmt.Println("HmacMD5-base64:", bs64)
fmt.Println("HmacMD5-hex:", string(hex))
sha1 := utils.HmacSHA1("123", "123")
bs64 = utils.Btoa(sha1)
hex1 := utils.HexEncode(sha1)
fmt.Println("HmacSHA1-base64:", bs64)
fmt.Println("HmacSHA1-hex:", string(hex1))
sha224 := utils.HmacSHA224("123", "123")
bs64 = utils.Btoa(sha224)
hex2 := utils.HexEncode(sha224)
fmt.Println("HmacSHA224-base64:", bs64)
fmt.Println("HmacSHA224-hex:", string(hex2))
sha256 := utils.HmacSHA256("123", "123")
bs64 = utils.Btoa(sha256)
hex3 := utils.HexEncode(sha256)
fmt.Println("HmacSHA256-base64:", bs64)
fmt.Println("HmacSHA256-hex:", string(hex3))
sha384 := utils.HmacSHA384("123", "123")
bs64 = utils.Btoa(sha384)
hex4 := utils.HexEncode(sha384)
fmt.Println("HmacSHA384-base64:", bs64)
fmt.Println("HmacSHA384-hex:", string(hex4))
sha512 := utils.HmacSHA512("123", "123")
bs64 = utils.Btoa(sha512)
hex5 := utils.HexEncode(sha512)
fmt.Println("HmacSHA512-base64:", bs64)
fmt.Println("HmacSHA512-hex:", string(hex5))
}
HmacMD4-base64: u3owGP+65If/IdHlG8klJg==
HmacMD4-hex: bb7a3018ffbae487ff21d1e51bc92526
HmacRIPEMD160-base64: Wt1vTR04ulq64/WbCZsoGMY1wM8=
HmacRIPEMD160-hex: 5add6f4d1d38ba5abae3f59b099b2818c635c0cf
HmacMD5-base64: sqHsDz4GBwmdfzl5HATppA==
HmacMD5-hex: b2a1ec0f3e0607099d7f39791c04e9a4
HmacSHA1-base64: o8Ak8BzMs7Y0V9hIsNL4nB90Sj0=
HmacSHA1-hex: a3c024f01cccb3b63457d848b0d2f89c1f744a3d
HmacSHA224-base64: +S2OBxrerlFMGsD5RQLnW7XCi4F+DgoB52I0jw==
HmacSHA224-hex: f92d8e071adeae514c1ac0f94502e75bb5c28b817e0e0a01e762348f
HmacSHA256-base64: PK/kD5K+asd9J5K0smfC2hHj8wh7k7sZxsUTN4aYS0Q=
HmacSHA256-hex: 3cafe40f92be6ac77d2792b4b267c2da11e3f3087b93bb19c6c5133786984b44
HmacSHA384-base64: b2i1J57kVppw+dC071Gk25fXfLkBmcILY38gkOOXgPjuyE8wK+LuP+w2pzTZ5tpb
HmacSHA384-hex: 6f68b5279ee4569a70f9d0b4ef51a4db97d77cb90199c20b637f2090e39780f8eec84f302be2ee3fec36a734d9e6da5b
HmacSHA512-base64: BjT9BDgLuvUGnIxGp0x9Id90FIiNmAwnoW1eJiy4yQWROcIS0JJgAPrwJuSDkEzvri9enZvV9R+8KsTE3lGBFQ==
HmacSHA512-hex: 0634fd04380bbaf5069c8c46a74c7d21df7414888d980c27a16d5e262cb8c9059139c212d0926000faf026e483904cefae2f5e9d9bd5f51fbc2ac4c4de518115