Skip to content

Fix std.base64 for Unicode strings - #899

Open
sb123sb123 wants to merge 1 commit into
google:masterfrom
sb123sb123:fix/base64-utf8-887
Open

sb123sb123 wants to merge 1 commit into
google:masterfrom
sb123sb123:fix/base64-utf8-887

Conversation

@sb123sb123

Copy link
Copy Markdown

Problem

std.base64 rejected string inputs containing Unicode code points above 255 before encoding them. Emoji, CJK, and other non-Latin strings therefore failed instead of being encoded as UTF-8 bytes.

Fixes #887

Fix

String inputs are now passed directly through their UTF-8 byte representation before Base64 encoding. Array inputs retain their existing integer and byte-range validation.

Tests

  • Focused regression: go test . -run '^TestEval/builtinBase64_string_high_codepoint$' -count=1
  • Direct emoji and CJK checks, plus matching byte-array output.
  • Invalid byte-array controls still report the existing errors.
  • Full package compilation: go test ./... -run '^$' -count=1
  • go vet ./...
  • gofmt and git diff --check

Limitations

The complete golden test suite was also attempted on the required remote Windows Git Bash environment. Its pre-existing platform-sensitive golden expectations differ in Windows path separators and checkout line endings, so that suite reports unrelated failures; the focused regression and all-package compilation pass.

OpenAI Codex AI assistance disclosure

This pull request was prepared with OpenAI Codex AI assistance. Codex assisted with issue triage, reproduction, implementation, testing, and PR preparation; the submitting user reviewed the resulting changes.

Encode string inputs as UTF-8 bytes before Base64 encoding.

Fixes google#887

Signed-off-by: sb123sb123 <152394158+sb123sb123@users.noreply.github.com>
@google-cla

google-cla Bot commented Sep 22, 2026

Copy link
Copy Markdown

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

std.base64 fails on unicode strings with codepoints > 255 (emoji, CJK, etc.)

1 participant