Skip to content

[ISSUE #16053] Preserve UTF-8 HTTP responses that omit a charset - #16055

Closed
iyuan7 wants to merge 1 commit into
alibaba:developfrom
iyuan7:fix/issue-16053-http-response-bytes
Closed

iyuan7 wants to merge 1 commit into
alibaba:developfrom
iyuan7:fix/issue-16053-http-response-bytes

Conversation

@iyuan7

@iyuan7 iyuan7 commented Nov 15, 2026

Copy link
Copy Markdown
Contributor

中文

修改目的

修复 #16053。

同步 Apache HTTP 客户端读取未声明 charset 的 application/json UTF-8 响应时,会先把实体读成字符串,再按缺失 charset 时的 US-ASCII 编码回字节。中文和 emoji 会被替换为问号,二进制响应也可能损坏。显式声明 charset=UTF-8 时,相同内容可以完整读取。

字节保留方案和原始复现基于 asaid364 在 #16053 中的调查。

主要改动

  • 直接复制响应实体字节,避免先转字符串再编码回字节;保留既有状态码和响应头处理。
  • 增加真实本地 HTTP 服务回归测试,覆盖未声明和显式声明 UTF-8、二进制、显式非 UTF-8 字节、错误响应及连接复用。
  • 在中英文客户端规范中写明:同步 Apache HTTP adapter 必须保留响应字节。

验证情况

  • 新增测试在原实现上复现 UTF-8 JSON、二进制和错误响应体损坏;修复后通过。
  • mvn -B -pl common -am test:API 模块 1466 项、Common 模块 1057 项测试通过。
  • 全反应器的 Checkstyle、SpotBugs、RAT、Spotless 检查通过。
  • 测试在 Windows / JDK 17 上执行。本次只覆盖同步 Apache 适配器,不改变无响应实体的既有行为,也不声称修复上层文本解码或其他 HTTP 实现的字符集问题。

English

What is the purpose of the change

Fixes #16053.

The synchronous Apache HTTP adapter reads a response entity into a string and then encodes it back to bytes. When Content-Type has no charset, that encoding uses US-ASCII, so Chinese text and emoji in UTF-8 JSON become question marks and binary bodies can change. The same content is preserved when the response explicitly declares charset=UTF-8.

The byte-preservation approach and original reproduction build on the investigation in #16053 by asaid364.

Brief changelog

  • Copy response entity bytes directly instead of decoding them to a string and encoding them again; preserve existing status and header handling.
  • Add real local HTTP-server regression tests for UTF-8 with and without a declared charset, binary bytes, an explicitly declared non-UTF-8 charset, error responses, and connection reuse.
  • State in the English and Chinese client specs that the synchronous Apache HTTP adapter must preserve response bytes.

Verifying this change

  • The new tests reproduced corruption of UTF-8 JSON, binary, and error response bodies on the original implementation and passed with this fix.
  • mvn -B -pl common -am test: 1466 API tests and 1057 Common tests passed.
  • Reactor Checkstyle, SpotBugs, RAT, and Spotless checks passed.
  • Tests ran on Windows / JDK 17. This change covers the synchronous Apache adapter only. Existing no-entity behavior and higher-level text decoding are outside its scope.

Copy the Apache response entity as raw bytes so a missing charset cannot re-encode UTF-8 JSON or binary data as US-ASCII.
@github-actions

Copy link
Copy Markdown

Thanks for your this PR. 🙏
Please check again for your PR changes whether contains any usage/api/configuration change such as Add new API , Add new configuration, Change default value of configuration.
If so, please add or update documents(markdown type) in docs/next/ for repository nacos-group/nacos-group.github.io


感谢您提交的PR。 🙏
请再次查看您的PR内容,确认是否包含任何使用方式/API/配置参数的变更,如:新增API、新增配置参数、修改默认配置等操作。
如果是,请确保在提交之前,在仓库nacos-group/nacos-group.github.io中的docs/next/目录下添加或更新文档(markdown格式)。

@iyuan7 iyuan7 closed this Nov 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] Apache 同步 HTTP 客户端损坏未声明 charset 的 UTF-8 JSON 响应

1 participant