Skip to content

[Repo Assist] Change HTTP response default encoding from ISO-8859-1 to UTF-8#1691

Draft
github-actions[bot] wants to merge 2 commits intomainfrom
repo-assist/fix-issue-1251-http-utf8-v2-8d88c56436838833
Draft

[Repo Assist] Change HTTP response default encoding from ISO-8859-1 to UTF-8#1691
github-actions[bot] wants to merge 2 commits intomainfrom
repo-assist/fix-issue-1251-http-utf8-v2-8d88c56436838833

Conversation

@github-actions
Copy link
Contributor

@github-actions github-actions bot commented Mar 8, 2026

🤖 This is an automated pull request from Repo Assist, an AI assistant for this repository.

Implements the fix requested in #1251 — changes the HTTP response default encoding from ISO-8859-1 to UTF-8 to match System.Net.Http.HttpClient behaviour.

This is a clean rebase of #1665 on the current main branch (post-8.1.0 release). Please close #1665.

Closes #1251

Root Cause

HttpEncodings.ResponseDefaultEncoding was set to ISO-8859-1, following RFC 2616 §3.7.1. However, modern HTTP servers, browsers, and System.Net.Http.HttpClient all default to UTF-8 in practice when no charset is specified in the Content-Type response header. This caused garbled characters when fetching UTF-8 content from servers that don't include an explicit charset.

@dsyme endorsed this fix: "Agreed we should do the same as HttpClient and do this."

Change

  • HttpEncodings.ResponseDefaultEncoding changed from Encoding.GetEncoding("ISO-8859-1") to Encoding.UTF8
  • Updated XML doc comments across the type providers (XmlProvider, JsonProvider, CsvProvider, HtmlProvider) to reflect the new default
  • Updated the test HttpEncodings.ResponseDefaultEncoding returns ISO-8859-1 to reflect the new value
  • Added ## 9.0.0 section in RELEASE_NOTES.md (this is a breaking change: existing code relying on ISO-8859-1 default may need to set responseEncodingOverride)

The existing responseEncodingOverride parameter remains available for callers who need to force a specific encoding, including ISO-8859-1.

Test Status

  • ✅ Build: succeeded
  • ✅ All tests passed (FSharp.Data.Core.Tests, FSharp.Data.Tests, FSharp.Data.DesignTime.Tests, FSharp.Data.Reference.Tests)
  • ✅ Fantomas format check: passed
  • 1 test skipped (network test blocked by sandbox firewall — pre-existing infrastructure issue)

Generated by Repo Assist ·

To install this agentic workflow, run

gh aw add githubnext/agentics/workflows/repo-assist.md@8e6d7c86bba37371d2d0eee1a23563db3e561eb5

…#1251)

- ResponseDefaultEncoding changed from ISO-8859-1 to UTF-8 to match System.Net.Http.HttpClient behaviour
- Update test to expect UTF-8
- Update Encoding parameter doc comments in type providers to remove the ISO-8859-1 reference
- Add 9.0.0 release notes entry (breaking change: default encoding changes)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Problem with Http.RequestString detection of response content encoding

0 participants