Skip to content
GitLab
Projects Groups Snippets
  • /
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in / Register
  • pypy pypy
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
  • Issues 674
    • Issues 674
    • List
    • Boards
    • Service Desk
    • Milestones
  • Merge requests 13
    • Merge requests 13
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Schedules
  • Deployments
    • Deployments
    • Environments
    • Releases
  • Packages and registries
    • Packages and registries
    • Container Registry
  • Monitor
    • Monitor
    • Incidents
  • Analytics
    • Analytics
    • Value stream
    • CI/CD
    • Repository
  • Wiki
    • Wiki
  • Snippets
    • Snippets
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
Collapse sidebar
  • PyPyPyPy
  • pypypypy
  • Issues
  • #3291
Closed
Open
Issue created Sep 01, 2020 by Jean-Paul Calderone@exarkun

Add an interface to change the filesystem encoding

PyPy inspects its runtime environment and picks a "filesystem encoding" to use for converting between unicode and bytes when dealing with path names that have to be passed to or read from the underlying system.

The encoding is exposed via sys.getfilesystemencoding which returns a cached value from the object space (https://foss.heptapod.net/pypy/pypy/-/blob/893217a999926e7c33ca85f2b1e770d8607b8335/pypy/module/sys/interp_encoding.py#L43-69).

However, there is no way to change the filesystem encoding in use by the system. Also, a lot of interp-level code uses the value directly from the object space and doesn't go up to the Python sys.getfilesystemencoding to get the value (so monkey-patching isn't a viable way to change it either).

CPython had an API to change this value for a while but ultimately removed it in favor of strongly preferring UTF-8 everywhere. For older versions of Python where UTF-8 is not preferred everywhere, it's a real drag to have something like "ASCII" detected and used and have no way to override this.

It is possible to control this value on just about all versions of Python by setting LANG or other LC_... environment variables properly before the Python process is started. Changing these environment variables inside the process after startup (ie, using Python application code) makes no difference because the choice has already been made.

It would be great if PyPy offered an API that allowed this encoding to be changed to an application-supplied value (or just "UTF-8" which is so far the only value I ever want to change it to).

To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information
Assignee
Assign to
Time tracking