Skip to content

GitLab

  • Menu
Projects Groups Snippets
    • Loading...
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in / Register
  • pypy pypy
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
  • Issues 652
    • Issues 652
    • List
    • Boards
    • Service Desk
    • Milestones
  • Merge requests 13
    • Merge requests 13
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Schedules
  • Deployments
    • Deployments
    • Environments
    • Releases
  • Monitor
    • Monitor
    • Incidents
  • Packages & Registries
    • Packages & Registries
    • Container Registry
  • Analytics
    • Analytics
    • Value stream
    • CI/CD
    • Repository
  • Wiki
    • Wiki
  • Snippets
    • Snippets
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
Collapse sidebar
  • PyPy
  • pypypypy
  • Issues
  • #2052

Closed
Open
Created May 26, 2015 by Bitbucket Importer@bitbucket_importerMaintainer

Regex module memory leak/crash

Created originally on Bitbucket by magixx (Alex)

While using the regex module when parsing a lot of data (I have ~1GB of lines) I've notice that at a certain point the code balloons in memory usage to about 200MB/s. I highly suspect it has something to do with regex.finditer() as if I have that line of code the memory leak will occur. When parsing a small 30MB file this issue did not get a chance to arise.

The code looks like:

#!python
int_group_test = special_end_regex.search(string)
if int_group_test:
    new_string = string[:int_group_test.start()]
last_splitter = [m.end() for m in dash_regex.finditer(new_string)]

I have tried both 32 and 64 bit versions of PyPy with the same result. If the re module is used instead then this problem does not occur. I also used CPython with regex and re and both succeed without any such memory leaks.

To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information
Assignee
Assign to
Time tracking