Skip to content

GitLab

  • Projects
  • Groups
  • Snippets
  • Help
    • Loading...
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in / Register
pypy
pypy
  • Project overview
    • Project overview
    • Details
    • Activity
    • Releases
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
  • Issues 582
    • Issues 582
    • List
    • Boards
    • Labels
    • Service Desk
    • Milestones
  • Merge Requests 12
    • Merge Requests 12
  • CI / CD
    • CI / CD
    • Pipelines
    • Jobs
    • Schedules
  • Operations
    • Operations
    • Incidents
    • Environments
  • Packages & Registries
    • Packages & Registries
    • Container Registry
  • Analytics
    • Analytics
    • CI / CD
    • Repository
    • Value Stream
  • Wiki
    • Wiki
  • Snippets
    • Snippets
  • Members
    • Members
  • Collapse sidebar
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards

This instance will be upgraded to Heptapod 0.19.0rc2 on 2021-01-18 at 11:00 UTC+1 (a few minutes of downtime)

  • PyPy
  • pypypypy
  • Issues
  • #3291

Closed
Open
Opened Sep 01, 2020 by Jean-Paul Calderone@exarkun

Add an interface to change the filesystem encoding

PyPy inspects its runtime environment and picks a "filesystem encoding" to use for converting between unicode and bytes when dealing with path names that have to be passed to or read from the underlying system.

The encoding is exposed via sys.getfilesystemencoding which returns a cached value from the object space (https://foss.heptapod.net/pypy/pypy/-/blob/893217a999926e7c33ca85f2b1e770d8607b8335/pypy/module/sys/interp_encoding.py#L43-69).

However, there is no way to change the filesystem encoding in use by the system. Also, a lot of interp-level code uses the value directly from the object space and doesn't go up to the Python sys.getfilesystemencoding to get the value (so monkey-patching isn't a viable way to change it either).

CPython had an API to change this value for a while but ultimately removed it in favor of strongly preferring UTF-8 everywhere. For older versions of Python where UTF-8 is not preferred everywhere, it's a real drag to have something like "ASCII" detected and used and have no way to override this.

It is possible to control this value on just about all versions of Python by setting LANG or other LC_... environment variables properly before the Python process is started. Changing these environment variables inside the process after startup (ie, using Python application code) makes no difference because the choice has already been made.

It would be great if PyPy offered an API that allowed this encoding to be changed to an application-supplied value (or just "UTF-8" which is so far the only value I ever want to change it to).

To upload designs, you'll need to enable LFS and have admin enable hashed storage. More information
Assignee
Assign to
None
Milestone
None
Assign milestone
Time tracking
None
Due date
None
Reference: pypy/pypy#3291