简洁大气企业网站欣赏,成都企业注册信息查询,什么是网络营销?网络营销的常用方法有哪些?,网站关键词词库怎么做注意#xff1a; 本文内容于 2024-11-09 19:20:07 创建#xff0c;可能不会在此平台上进行更新。如果您希望查看最新版本或更多相关内容#xff0c;请访问原文地址#xff1a;SSL证书以及实现HTTP反向代理。感谢您的关注与支持#xff01;
之前写的HTTP反向代理工具…注意 本文内容于 2024-11-09 19:20:07 创建可能不会在此平台上进行更新。如果您希望查看最新版本或更多相关内容请访问原文地址SSL证书以及实现HTTP反向代理。感谢您的关注与支持
之前写的HTTP反向代理工具在实际使用时碰到反代失败的问题。跟踪了一下才发现是由于对方使用了自签名SSL证书导致发起HTTP请求时验证失败。因此简单记录一下。
针对该问题的复现从两个方面来展开
理解SSL忽略SSL
一、理解SSL
1.1 HTTPS与SSL
SSL是用于加密传输的协议也是最初的加密标准目前已被TLS取代但由于历史原因大家还是会称为SSL。
HTTPS是HTTP上实现加密传输的协议依赖SSL/TLS来确保安全性。
从不求甚解的角度来理解HTTPSHTTPSSL/TLS
1.2 证书分类
SSL常见的证书分类有两种
CA证书自签名证书
这两者的区别如下
特性自签名证书CA证书签发机构由证书持有者自己签发由受信任的证书颁发机构CA签发信任级别默认不被浏览器或操作系统信任需手动安装信任被大多数浏览器和操作系统默认信任身份验证无身份验证持有者自行生成证书CA会对证书持有者进行身份验证安全性安全性较低可能被伪造或滥用高安全性通过身份验证保障证书真实性应用场景适用于开发、测试和内部网络适用于生产环境和面向互联网的服务成本免费需要付费费用根据证书类型和CA机构不同而异浏览器警告会弹出“不安全连接”警告不会弹出警告用户信任度高管理复杂度管理简单但不适合公开环境管理较复杂需要向CA申请和续期
互联网服务使用的一般都是CA证书由于操作系统已经内置了一系列根证书当访问一个使用CA签发证书的HTTPS网站时就不会出现“不安全连接”的警告。
而自签名证书由于操作系统缺少对其的信任访问就会被拦截了。此时服务提供方需要给调用方提供自签名证书以便调用方可以信任该连接。
1.3 OpenSSL生成自签名证书
1.3.1 扩展名说明
像我购买的CA证书部署到Nginx时一般都是.pem和.key文件。但在自己生成证书的过程中发现还有.crt文件。直观的感受是这些扩展名特别的混乱。经过查阅资料下面简单记录这些扩展名的区别。
crt: 存储证书(公钥)。该证书可提供给第三方使用比如HTTPS客户端key: 私钥。该私钥文件只应给服务提供者使用。csr: 向证书颁发机构申请签署密钥的请求不包含密钥本身。pem: 基于Base64编码的文本格式。它可以是上述任何文件。der: 基于二进制编码的文本格式。它可以是上述任何文件。
参考
ssl - Difference between pem, crt, key files - Stack Overflow
Difference between .pem and .crt and how to use them - Help - Let’s Encrypt Community Support
1.3.2 自签名证书
下面使用OpenSSL生成自签名的公钥和私钥证书。
# 生成一个2048位的RSA私钥并保存到private.key文件中
openssl genrsa -out private.key 2048# 根据私钥生成一个证书签名请求
openssl req -new -key private.key -out request.csr# X.509是SSL/TLS中最常用的公钥证书标准
# 通过私钥和证书签名请求生成一个时效为365天的证书
openssl x509 -req -days 365 -in request.csr -signkey private.key -out public.pem# 验证证书内容
openssl x509 -in public.pem -noout -text二、忽略SSL
2.1 服务端部署证书
2.1.1 Nginx
以Nginx为例部署证书
worker_processes 1;events {worker_connections 1024;
}http {include mime.types;default_type application/octet-stream;server {listen 443 ssl;server_name 10.0.0.1;ssl_certificate /usr/local/nginx/conf/cert/public.pem;ssl_certificate_key /usr/local/nginx/conf/cert/private.key;location / {root /usr/local/nginx/html;index index.html;}}
}2.1.2 Vertx
使用Java中的Vertx 4.5.10版本开启HTTPServer
PemKeyCertOptions pemKeyCertOptions new PemKeyCertOptions()//使用自签名证书开启ssl.addCertPath(/usr/local/nginx/conf/cert/public.pem).addKeyPath(/usr/local/nginx/conf/cert/private.key);
FutureHttpServer serverFuture vertx.createHttpServer(new HttpServerOptions().setSsl(true).setKeyCertOptions(pemKeyCertOptions))//注册路由.requestHandler(router).listen(port);
serverFuture.onComplete(re - {if (re.succeeded()) {log.info(http server started on port {}, port);} else {log.error(http server failed to start, re.cause());}
});2.2 客户端忽略校验
2.2.1 CURL
curl忽略ssl校验比较简单添加-k参数即可。
curl -k https://10.0.0.10:4432.2.2 Apache HttpPClient and OkHttpClient
设置忽略SSL的核心逻辑如下具体的写法还需根据框架而定。
/*** 信任所有SSL证书包括CA证书和自签名证书。实现效果类似于curl -k* see a hrefhttps://blog.csdn.net/qq_20683411/article/details/142996223Apache HttpClient 4.3.2 和 4.5.13 - 忽略证书问题_apache 4.3.5 忽略ssl-CSDN博客/a*/
public SSLContext trustAllCerts() {try {TrustManager[] trustManagers {new X509TrustManager() {Overridepublic void checkClientTrusted(X509Certificate[] x509Certificates, String s) throws CertificateException {}Overridepublic void checkServerTrusted(X509Certificate[] x509Certificates, String s) throws CertificateException {}Overridepublic X509Certificate[] getAcceptedIssuers() {return new X509Certificate[0];}}};SSLContext sslContext SSLContext.getInstance(SSL);sslContext.init(null, trustManagers, new SecureRandom());return sslContext;} catch (Exception ignore) {}return null;
}public HostnameVerifier getNoopHostnameVerifier() {return new HostnameVerifier() {Overridepublic boolean verify(final String s, final SSLSession sslSession) {return true;}Overridepublic final String toString() {return NO_OP;}};
}三、HTTP反向代理
这个主要是带着学习的目的实现的。
java8springboot2.5.14okhttp3
直接上源码meethigher/http-proxy-boot: 使用SpringBoot实现的开箱即用的HTTP反向代理工具
import okhttp3.*;
import okio.BufferedSink;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;import javax.net.ssl.*;
import javax.servlet.ServletException;
import javax.servlet.ServletInputStream;
import javax.servlet.ServletOutputStream;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import java.io.IOException;
import java.io.InputStream;
import java.security.SecureRandom;
import java.security.cert.CertificateException;
import java.security.cert.X509Certificate;
import java.util.*;
import java.util.concurrent.TimeUnit;/*** 该内容主要是想着学习一下底层HTTP反向代理的实现** author a hrefhttps://meethigher.topchenchuancheng/a* see a hrefhttps://github.com/mitre/HTTP-Proxy-Servletmitre/HTTP-Proxy-Servlet: Smiley#39;s HTTP Proxy implemented as a Java servlet/a* since 2024/11/09 22:43*/
public class ProxyServlet extends HttpServlet {protected static final Logger log LoggerFactory.getLogger(ProxyServlet.class);protected final OkHttpClient client;protected final String targetUrl; // 目标服务器信息protected final boolean corsControl; // 跨域控制。当为true时跨域信息都由自身服务管理protected final boolean allowCORS; // 是否允许跨域。当corsControl为true时该参数方可生效。protected final boolean logEnable; // 启用日志protected final boolean forwardIp; // 遵循代理规范将实际调用方的ip和protocol传给目标服务器protected final boolean preserveHost; // 保留原host这个仅对请求头有效。protected final boolean preserveCookie; // 保留原cookie。这个对请求头和响应头均有效。protected final String logFormat; // 日志格式/*** 跨域相关的响应头*/protected final ListString allowCORSHeaders Arrays.asList(access-control-allow-origin,//指定哪些域可以访问资源。可以是特定域名也可以是通配符 *表示允许所有域访问。access-control-allow-methods,//指定允许的HTTP方法如 GET、POST、PUT、DELETE 等。access-control-allow-headers,//指定允许的请求头。access-control-allow-credentials,//指定是否允许发送凭据如Cookies。值为 true 表示允许且不能使用通配符 *。access-control-expose-headers,//指定哪些响应头可以被浏览器访问。access-control-max-age,//指定预检请求的结果可以被缓存的时间以秒为单位。access-control-request-method,//在预检请求中使用指示实际请求将使用的方法。access-control-request-headers//在预检请求中使用指示实际请求将使用的自定义头。);/*** 不应该被复制的逐跳标头*/protected final String[] hopByHopHeaders new String[]{Connection, Keep-Alive, Proxy-Authenticate, Proxy-Authorization,TE, Trailers, Transfer-Encoding, Upgrade};/*** 默认的日志格式*/public static final String LOG_FORMAT_DEFAULT {method} -- {userAgent} -- {remoteAddr}:{remotePort} -- {source} -- {target} -- {statusCode} consumed {consumedMills} ms;protected void doLog(HttpServletRequest httpServletRequest, HttpServletResponse httpServletResponse, long startMills) {if (logEnable) {String queryString httpServletRequest.getQueryString();String logInfo logFormat.replace({method}, httpServletRequest.getMethod()).replace({userAgent}, httpServletRequest.getHeader(User-Agent)).replace({remoteAddr}, httpServletRequest.getRemoteAddr()).replace({remotePort}, String.valueOf(httpServletRequest.getRemotePort())).replace({source}, queryString null ? httpServletRequest.getRequestURL() : httpServletRequest.getRequestURL() ? queryString).replace({target}, rewriteUrlFromRequest(httpServletRequest)).replace({statusCode}, String.valueOf(httpServletResponse.getStatus())).replace({consumedMills}, String.valueOf(System.currentTimeMillis() - startMills));log.info({}: {}, getServletName(), logInfo);}}/*** 是否包含逐跳标头*/protected boolean containsHopByHopHeader(String name) {for (String header : hopByHopHeaders) {if (header.equalsIgnoreCase(name)) {return true;}}return false;}/*** 不考虑contextPath* 获取代理请求url不包含queryparams*/protected String getTargetUrl(HttpServletRequest request) {//request.getRequestURI();//包含contextPath的uriString uri request.getPathInfo();//不包含contextPath的uriif (uri null || uri.isEmpty()) {return targetUrl;} else {return targetUrl uri;}}/*** 获取代理请求完整url包含queryparams*/protected String rewriteUrlFromRequest(HttpServletRequest request) {String targetUrl getTargetUrl(request);String queryString request.getQueryString();return queryString null ? targetUrl : targetUrl ? queryString;}/*** 将重定向的url重写为代理服务器的地址*/protected String rewriteUrlFromResponse(HttpServletRequest request, String locationUrl) {String targetUrl getTargetUrl(request);if (locationUrl ! null locationUrl.startsWith(targetUrl)) {StringBuffer curUrl request.getRequestURL();int pos;if ((pos curUrl.indexOf(://)) 0) {if ((pos curUrl.indexOf(/, pos 3)) 0) {curUrl.setLength(pos);}}curUrl.append(request.getContextPath());curUrl.append(request.getServletPath());curUrl.append(locationUrl, targetUrl.length(), locationUrl.length());return curUrl.toString();}return locationUrl;}public ProxyServlet(OkHttpClient client, String targetUrl, boolean corsControl, boolean allowCORS, boolean logEnable, String logFormat, boolean forwardIp, boolean preserveHost, boolean preserveCookie) {this.client client;this.targetUrl targetUrl;this.corsControl corsControl;this.allowCORS allowCORS;this.logEnable logEnable;this.forwardIp forwardIp;this.preserveHost preserveHost;this.preserveCookie preserveCookie;this.logFormat logFormat;}public ProxyServlet(OkHttpClient client, String targetUrl) {this(client, targetUrl, false, false, true, LOG_FORMAT_DEFAULT, false, false, false);}/*** 根据代理的规定通过请求头进行真实信息的传递* X-Forwarded-For: 传输实际调用者ip* X-Forwarded-Proto: 传输实际调用者请求协议*/public void setXForwardedForHeader(HttpServletRequest request, Request.Builder requestBuilder) {if (forwardIp) {String forHeaderName X-Forwarded-For;String forHeader request.getRemoteAddr();String existingForHeader request.getHeader(forHeader);if (existingForHeader ! null) {forHeader existingForHeader , forHeader;}requestBuilder.header(forHeaderName, forHeader);String protoHeaderName X-Forwarded-Proto;String protoHeader request.getScheme();requestBuilder.header(protoHeaderName, protoHeader);}}Overrideprotected void service(HttpServletRequest httpServletRequest, HttpServletResponse httpServletResponse) throws ServletException, IOException {long start System.currentTimeMillis();try {if (httpServletRequest.getMethod().equalsIgnoreCase(options) corsControl allowCORS) {httpServletResponse.setHeader(Access-Control-Allow-Origin, httpServletRequest.getHeader(origin));httpServletResponse.setHeader(Access-Control-Allow-Methods, *);httpServletResponse.setHeader(Access-Control-Allow-Headers, *);httpServletResponse.setHeader(Access-Control-Allow-Credentials, true);httpServletResponse.setHeader(Access-Control-Expose-Headers, *);httpServletResponse.setStatus(HttpServletResponse.SC_OK);return;}String method httpServletRequest.getMethod();String proxyRequestUrl rewriteUrlFromRequest(httpServletRequest);Request.Builder requestBuilder getInitRequestBuilder(httpServletRequest, httpServletResponse);requestBuilder.url(proxyRequestUrl);if (get.equalsIgnoreCase(method)) {requestBuilder.method(method, null);} else {// 根据HTTP规定复制请求体RequestBody requestBody;if (httpServletRequest.getHeader(Content-Length) ! null || httpServletRequest.getHeader(Transfer-Encoding) ! null) {try {ServletInputStream inputStream httpServletRequest.getInputStream();requestBody new StreamingRequestBody(MediaType.parse(httpServletRequest.getContentType()), inputStream);} catch (Exception e) {writeGatewayError(httpServletResponse, e.getMessage());return;}} else {requestBody RequestBody.create(null, new byte[0]);}requestBuilder.method(method, requestBody);}copyRequestHeaders(httpServletRequest, requestBuilder);setXForwardedForHeader(httpServletRequest, requestBuilder);try (Response response client.newCall(requestBuilder.build()).execute()) {httpServletResponse.setStatus(response.code());copyResponseHeaders(httpServletRequest, httpServletResponse, response);if (response.code() 304) {// http状态码为304时表示当客户端发起请求时如果服务器发现请求的资源并没有自上次请求后发生任何更改就会返回 304 状态码同时不包含请求资源的实体内容。这意味着客户端可以继续使用缓存中的资源从而避免不必要的数据传输减少服务器负载和带宽消耗。httpServletResponse.setIntHeader(Content-Length, 0);} else {// 复制响应体ResponseBody responseBody response.body();if (responseBody ! null) {ServletOutputStream os httpServletResponse.getOutputStream();InputStream is responseBody.byteStream();int len;byte[] buffer new byte[8192];while ((len is.read(buffer)) ! -1) {os.write(buffer, 0, len);}os.flush();}}} catch (Exception e) {writeGatewayError(httpServletResponse, e.getMessage());return;}} finally {doLog(httpServletRequest, httpServletResponse, start);}}protected Request.Builder getInitRequestBuilder(HttpServletRequest request, HttpServletResponse response) {return new Request.Builder();}/*** 复制响应头*/protected void copyResponseHeaders(HttpServletRequest httpServletRequest, HttpServletResponse httpServletResponse, Response response) {SetString names response.headers().names();// httpServletResponse不支持移除请求头因此按需添加头MapString, String needSetHeaderMap new LinkedHashMap();for (String name : names) {if (containsHopByHopHeader(name)) {continue;}if (Location.equalsIgnoreCase(name)) {// 重写重定向头needSetHeaderMap.put(name, rewriteUrlFromResponse(httpServletRequest, response.header(name)));} else if (Set-Cookie.equalsIgnoreCase(name) || Set-Cookie2.equalsIgnoreCase(name)) {// 保存Cookie信息if (preserveCookie) {needSetHeaderMap.put(name, response.header(name));}} else {needSetHeaderMap.put(name, response.header(name));}}/*** 跨域控制*/if (corsControl) {/*** 1. 清空所有与跨域相关的响应头* 2. 如果允许跨域则添加跨域允许响应头*/IteratorMap.EntryString, String iterator needSetHeaderMap.entrySet().iterator();while (iterator.hasNext()) {Map.EntryString, String entry iterator.next();String keyIgnoreCase entry.getKey().toLowerCase(Locale.ROOT);if (allowCORSHeaders.contains(keyIgnoreCase)) {iterator.remove();}}if (allowCORS) {needSetHeaderMap.put(Access-Control-Allow-Origin, httpServletRequest.getHeader(origin));needSetHeaderMap.put(Access-Control-Allow-Methods, *);needSetHeaderMap.put(Access-Control-Allow-Headers, *);needSetHeaderMap.put(Access-Control-Allow-Credentials, true);needSetHeaderMap.put(Access-Control-Expose-Headers, *);}}for (String header : needSetHeaderMap.keySet()) {httpServletResponse.setHeader(header, needSetHeaderMap.get(header));}}/*** 复制请求头*/protected void copyRequestHeaders(HttpServletRequest httpServletRequest, Request.Builder requestBuilder) {EnumerationString headerNames httpServletRequest.getHeaderNames();while (headerNames.hasMoreElements()) {String key headerNames.nextElement();if (host.equalsIgnoreCase(key)) {if (preserveHost) {requestBuilder.header(key, httpServletRequest.getHeader(key));}} else if (cookie.equalsIgnoreCase(key)) {if (preserveCookie) {requestBuilder.header(key, httpServletRequest.getHeader(key));}} else {requestBuilder.header(key, httpServletRequest.getHeader(key));}}}protected void writeGatewayError(HttpServletResponse httpServletResponse, String msg) {httpServletResponse.setStatus(502);httpServletResponse.setContentType(text/html;charsetutf-8);try {httpServletResponse.getWriter().write(msg);} catch (Exception ignore) {}}/*** okhttpclient的流式请求体* 节省应用内存通过流式传输数据只会在网络传输过程中按需读取数据而不会将整个请求体加载到内存。*/class StreamingRequestBody extends RequestBody {private final MediaType contentType;private final InputStream inputStream;public StreamingRequestBody(MediaType contentType, InputStream inputStream) {this.contentType contentType;this.inputStream inputStream;}Overridepublic MediaType contentType() {return contentType;}Overridepublic void writeTo(BufferedSink sink) throws IOException {// 每次读8KB的数据byte[] buffer new byte[8192];int bytesRead;while ((bytesRead inputStream.read(buffer)) ! -1) {sink.write(buffer, 0, bytesRead);}}}private static OkHttpClient okHttpClient;public synchronized static OkHttpClient okHttpClient() {if (okHttpClient null) {OkHttpClient.Builder builder new OkHttpClient().newBuilder().connectionPool(new ConnectionPool(2, 60, TimeUnit.SECONDS)).connectTimeout(10, TimeUnit.SECONDS).readTimeout(30, TimeUnit.SECONDS).writeTimeout(30, TimeUnit.SECONDS);builder.followRedirects(false);builder.followSslRedirects(true);builder.hostnameVerifier(new HostnameVerifier() {Overridepublic boolean verify(final String s, final SSLSession sslSession) {return true;}Overridepublic final String toString() {return NO_OP;}});try {X509TrustManager x509TrustManager new X509TrustManager() {Overridepublic void checkClientTrusted(X509Certificate[] x509Certificates, String s) throws CertificateException {}Overridepublic void checkServerTrusted(X509Certificate[] x509Certificates, String s) throws CertificateException {}Overridepublic X509Certificate[] getAcceptedIssuers() {return new X509Certificate[0];}};TrustManager[] trustManagers {x509TrustManager};SSLContext sslContext SSLContext.getInstance(SSL);sslContext.init(null, trustManagers, new SecureRandom());builder.sslSocketFactory(sslContext.getSocketFactory(), x509TrustManager);} catch (Exception ignore) {}okHttpClient builder.build();}return okHttpClient;}
}
