Skip to content
Snippets Groups Projects
  • Paul Morelle's avatar
    44f5acfb9ad2
    aggressivemergedeltas: enabled the option by default · 44f5acfb9ad2
    Paul Morelle authored
    The option has been around for a while (August 2015) but was never turned on by
    default. In-depth testing shows large wins for having that on with no
    significant drawbacks.
    
    When enabled, revlog consider delta against both p1 and p2 at the same time when
    storing a revision. Selecting a delta against "p2" can produce better deltas and
    chain. This raise large benefit for all repositories, especially if they have a
    lot of merges.
    
    Comparison of `.hg/store/` size:
    
        mercurial (6.74% merges):
            before:    54,225,348 bytes
            after:     47,279,959 bytes -13%
        pypy (8.30% merges):
            before:   459,041,759 bytes
            after:    346,090,067 bytes -25%
        netbeans (34.21% merges):
            before: 2,468,041,333 bytes
            after:  1,364,077,645 bytes -45%
        mozilla-central (4.84% merges):
            before: 2,731,799,546 bytes
            after:  2,157,718,019 bytes -21%
    
    Comparison of `00manifest.d` size:
    
        mercurial (6.74% merges):
            before:    11,682,516 bytes
            after:      6,143,044 bytes -47%
        pypy (8.30% merges):
            before:   156,447,163 bytes
            after:     52,941,780 bytes -66%
        netbeans (34.21% merges):
            before: 1,250,363,851 bytes
            after:    130,088,982 bytes -90%
        mozilla-central (4.84% merges):
            before:   468,202,733 bytes
            after:    215,096,339 bytes -54%
    
    In addition, the better deltas help with the performance of multiple core
    operations. However, better chains mean longer chains, which can affect
    performance negatively (mostly manifest revision retrieval time). Chains length
    is a deeper problem that also affects linear repository too. Overall we think
    the benefits of using p2 as a diff target are bigger than the downsizes. In
    addition, we are also working on ways to improve the performance impact of chain
    length, so theses downsizes get fixed in the future.
    
    Below are interesting items from the full benchmark run:
    
        bundling 100 revisions from pypy:
            before: 670ms
            after:  480ms -28%
        bundle 10000 revisions from pypy:
            before: 1.38s
            after:  1.10s -54%
        bundle 10000 revisions from pypy:
            before: 16.1s
            after:  7.81s -52%
        bundle 10000 revisions from netbeans:
            before: 19.3s
            after:  15.5s -19%
    
        unbundle 1000 revisions to pypy:
            before: 641ms
            after:  315ms - 51%
    
        clone mercurial (http):
            before: 26.0s
            after:  22.6s -23%
    
        pulling 1000 revisions from pypy (shh):
            before: 2.07s
            after:  1.36s -44%
    
        pushing 1000 revision through http (pypy repository)
            before: 2.18s
            after:  1.35s -48%
    
        diff time in mozilla-central:
            before: 1.420s
            after:  0.983s -31%
    
        status time in mozilla-central:
            before: 1.260s
            after:  0.828s -34%
    
    Impact in other cases seems minimal (within a couple of percent in worse cases)
    and can be seen in both direction:
    
    Timing for a simple `hg commit`:
    
        mozilla-central:
            before: 3.37s
            after:  3.22s -4%
        pypy:
            before:  194ms
            after:   197ms +2%
    
    Timing for status (from tip to parent of tip):
        mercurial:
            before: 52.4ms
            after:  52.4ms (same)
        pypy:
            before: 55.2
            after:  56.9 +3%
    
    Timing for `hg update`
    
        mozilla-central, across 10 revisions:
            before: 4.82s
            after:  4.59s -5%
        mozilla-central, across 10000 revisions:
            before: 49.1s
            after:  49.9s +2%
        pypy, across 10 revisions:
            before: 213ms
            after:  216ms +1%
        pypy, across 10000 revisions:
            before: 5.31ms
            after:  5.24ms -1%
    
    The negative consequences are related to manifest fetch time:
    (timing for the tip revision tested by the benchmark)
    
        pypy-2018:
            before: 2.60ms
            after:  3.88ms +50%
        mozilla-central-2018:
            before: 565ms
            after:  652ms +15% (~+100ms)
        netbeans-2018:
            before: 101ms
            after:  250ms +48% (~+150ms)
    
    This shows up as a fixed overhead on some command we benchmarked:
    
        no-op push of mozilla-central:
            before:  945ms
            after:  1040ms +10% (~+100ms)
        pushing 10 changeset in netbeabs over ssh:
            before:  557ms
            after:   712ms +28% (+155ms)
        pushing 100 changeset in netbeabs over ssh:
            before: 592ms
            after:  771ms +30% (+179ms)
    44f5acfb9ad2
    History
    aggressivemergedeltas: enabled the option by default
    Paul Morelle authored
    The option has been around for a while (August 2015) but was never turned on by
    default. In-depth testing shows large wins for having that on with no
    significant drawbacks.
    
    When enabled, revlog consider delta against both p1 and p2 at the same time when
    storing a revision. Selecting a delta against "p2" can produce better deltas and
    chain. This raise large benefit for all repositories, especially if they have a
    lot of merges.
    
    Comparison of `.hg/store/` size:
    
        mercurial (6.74% merges):
            before:    54,225,348 bytes
            after:     47,279,959 bytes -13%
        pypy (8.30% merges):
            before:   459,041,759 bytes
            after:    346,090,067 bytes -25%
        netbeans (34.21% merges):
            before: 2,468,041,333 bytes
            after:  1,364,077,645 bytes -45%
        mozilla-central (4.84% merges):
            before: 2,731,799,546 bytes
            after:  2,157,718,019 bytes -21%
    
    Comparison of `00manifest.d` size:
    
        mercurial (6.74% merges):
            before:    11,682,516 bytes
            after:      6,143,044 bytes -47%
        pypy (8.30% merges):
            before:   156,447,163 bytes
            after:     52,941,780 bytes -66%
        netbeans (34.21% merges):
            before: 1,250,363,851 bytes
            after:    130,088,982 bytes -90%
        mozilla-central (4.84% merges):
            before:   468,202,733 bytes
            after:    215,096,339 bytes -54%
    
    In addition, the better deltas help with the performance of multiple core
    operations. However, better chains mean longer chains, which can affect
    performance negatively (mostly manifest revision retrieval time). Chains length
    is a deeper problem that also affects linear repository too. Overall we think
    the benefits of using p2 as a diff target are bigger than the downsizes. In
    addition, we are also working on ways to improve the performance impact of chain
    length, so theses downsizes get fixed in the future.
    
    Below are interesting items from the full benchmark run:
    
        bundling 100 revisions from pypy:
            before: 670ms
            after:  480ms -28%
        bundle 10000 revisions from pypy:
            before: 1.38s
            after:  1.10s -54%
        bundle 10000 revisions from pypy:
            before: 16.1s
            after:  7.81s -52%
        bundle 10000 revisions from netbeans:
            before: 19.3s
            after:  15.5s -19%
    
        unbundle 1000 revisions to pypy:
            before: 641ms
            after:  315ms - 51%
    
        clone mercurial (http):
            before: 26.0s
            after:  22.6s -23%
    
        pulling 1000 revisions from pypy (shh):
            before: 2.07s
            after:  1.36s -44%
    
        pushing 1000 revision through http (pypy repository)
            before: 2.18s
            after:  1.35s -48%
    
        diff time in mozilla-central:
            before: 1.420s
            after:  0.983s -31%
    
        status time in mozilla-central:
            before: 1.260s
            after:  0.828s -34%
    
    Impact in other cases seems minimal (within a couple of percent in worse cases)
    and can be seen in both direction:
    
    Timing for a simple `hg commit`:
    
        mozilla-central:
            before: 3.37s
            after:  3.22s -4%
        pypy:
            before:  194ms
            after:   197ms +2%
    
    Timing for status (from tip to parent of tip):
        mercurial:
            before: 52.4ms
            after:  52.4ms (same)
        pypy:
            before: 55.2
            after:  56.9 +3%
    
    Timing for `hg update`
    
        mozilla-central, across 10 revisions:
            before: 4.82s
            after:  4.59s -5%
        mozilla-central, across 10000 revisions:
            before: 49.1s
            after:  49.9s +2%
        pypy, across 10 revisions:
            before: 213ms
            after:  216ms +1%
        pypy, across 10000 revisions:
            before: 5.31ms
            after:  5.24ms -1%
    
    The negative consequences are related to manifest fetch time:
    (timing for the tip revision tested by the benchmark)
    
        pypy-2018:
            before: 2.60ms
            after:  3.88ms +50%
        mozilla-central-2018:
            before: 565ms
            after:  652ms +15% (~+100ms)
        netbeans-2018:
            before: 101ms
            after:  250ms +48% (~+150ms)
    
    This shows up as a fixed overhead on some command we benchmarked:
    
        no-op push of mozilla-central:
            before:  945ms
            after:  1040ms +10% (~+100ms)
        pushing 10 changeset in netbeabs over ssh:
            before:  557ms
            after:   712ms +28% (+155ms)
        pushing 100 changeset in netbeabs over ssh:
            before: 592ms
            after:  771ms +30% (+179ms)