Skip to content
Snippets Groups Projects
  1. Apr 21, 2020
    • Pierre-Yves David's avatar
      copies-rust: use immutable "OrdMap" to store copies information · 46bb4177
      Pierre-Yves David authored
      The large majority of time is currently spent coping and merging directories.
      the `IM` crate offer "immutable" Map, that use "copy on write" internally.  The
      new object use the same API as the standard HashMap.  So switching to it is
      trivial and it reduce copying cost significantly. More importantly, using
      immutable structure will unlock new possibility for a massive speed up of the
      "merging" part. This will came in a later changesets.
      
      Performance wise, we get very significant boost in the worst case. Below is some
      highlight of how we fare compared to the previous changeset.
      
      Repo            Cases                                 Source-Rev   Dest-Rev         Old-Time      New-Time     Difference   Factor
      ------------------------------------------------------------------------------------------------------------------------------------
      pypy            x0000_revs_x_added_0_copies           d1defd0dc478 c9cb1334cc78 :  62.468362 s,  33.527067 s, -28.941295 s, × 0.5367
      mozilla-central x000_revs_x000_added_x000_copies      7c97034feb78 4407bd0c6330 :   3.619850 s,   0.963905 s,  -2.655945 s, × 0.2663
      mozilla-central x0000_revs_xx000_added_x000_copies    f78c615a656c 96a38b690156 :  11.926587 s,   4.217003 s,  -7.709584 s, × 0.3536
      mozilla-try     x0000_revs_x_added_0_copies           63519bfd42ee a36a2a865d92 :  10.674920 s,   1.114864 s,  -9.560056 s, × 0.1044
      mozilla-try     x00000_revs_x00000_added_0_copies     dc8a3ca7010e d16fde900c9c :  19.647038 s,   1.442793 s, -18.204245 s, × 0.0734
      
      
      And we sometimes catch up with the performance of the python code as highlighted
      below:
      
      Repo            Cases                                 Source-Rev   Dest-Rev         Py-time       Rust-time    Difference   Factor
      ------------------------------------------------------------------------------------------------------------------------------------
      mozilla-try     x00000_revs_x00000_added_0_copies     dc8a3ca7010e d16fde900c9c :   1.074593 s,   1.442793 s,  +0.368200 s, × 1.3426
      
      However, multiple case remains significantly slower, as highlighted below
      
      Repo            Cases                                 Source-Rev   Dest-Rev         Py-time       Rust-time    Difference   Factor
      ------------------------------------------------------------------------------------------------------------------------------------
      mozilla-central x000_revs_x000_added_x000_copies      7c97034feb78 4407bd0c6330 :   0.190133 s,   0.963905 s,  +0.773772 s, × 5.0696
      mozilla-central x0000_revs_xx000_added_x000_copies    f78c615a656c 96a38b690156 :   0.440694 s,   4.217003 s,  +3.776309 s, × 9.5690
      mozilla-try     x0000_revs_x_added_0_copies           63519bfd42ee a36a2a865d92 :   0.370675 s,   1.114864 s,  +0.744189 s, × 3.0077
      pypy            x0000_revs_x_added_0_copies           d1defd0dc478 c9cb1334cc78 :   3.581556 s,  33.527067 s, +29.945511 s, × 9.3610
      
      Below are two different tables for full performance comparison
      - this changeset against the previous one (spoiler: it is much better)
      - this changeset against the python code  (spoiler: still slower, but it gets more comparable)
      
      This changeset compared to the previous one
      ===========================================
      
      Repo            Cases                                 Source-Rev   Dest-Rev         Old-Time      New-Time     Difference   Factor
      ------------------------------------------------------------------------------------------------------------------------------------
      mercurial       x_revs_x_added_0_copies               ad6b123de1c7 39cfcef4f463 :   0.000046 s,   0.000049 s,  +0.000003 s, × 1.0652
      mercurial       x_revs_x_added_x_copies               2b1c78674230 0c1d10351869 :   0.000173 s,   0.000179 s,  +0.000006 s, × 1.0347
      mercurial       x000_revs_x000_added_x_copies         81f8ff2a9bf2 dd3267698d84 :   0.006303 s,   0.006494 s,  +0.000191 s, × 1.0303
      pypy            x_revs_x_added_0_copies               aed021ee8ae8 099ed31b181b :   0.000229 s,   0.000339 s,  +0.000110 s, × 1.4803
      pypy            x_revs_x000_added_0_copies            4aa4e1f8e19a 359343b9ac0e :   0.000056 s,   0.000057 s,  +0.000001 s, × 1.0179
      pypy            x_revs_x_added_x_copies               ac52eb7bbbb0 72e022663155 :   0.000143 s,   0.000299 s,  +0.000156 s, × 2.0909
      pypy            x_revs_x00_added_x_copies             c3b14617fbd7 ace7255d9a26 :   0.001166 s,   0.001200 s,  +0.000034 s, × 1.0292
      pypy            x_revs_x000_added_x000_copies         df6f7a526b60 a83dc6a2d56f :   0.022931 s,   0.025120 s,  +0.002189 s, × 1.0955
      pypy            x000_revs_xx00_added_0_copies         89a76aede314 2f22446ff07e :   0.852446 s,   0.506921 s,  -0.345525 s, × 0.5947
      pypy            x000_revs_x000_added_x_copies         8a3b5bfd266e 2c68e87c3efe :   2.221824 s,   1.272060 s,  -0.949764 s, × 0.5725
      pypy            x000_revs_x000_added_x000_copies      89a76aede314 7b3dda341c84 :   1.194162 s,   0.690941 s,  -0.503221 s, × 0.5786
      pypy            x0000_revs_x_added_0_copies           d1defd0dc478 c9cb1334cc78 :  62.468362 s,  33.527067 s, -28.941295 s, × 0.5367
      pypy            x0000_revs_xx000_added_0_copies       bf2c629d0071 4ffed77c095c :   0.022116 s,   0.021970 s,  -0.000146 s, × 0.9934
      pypy            x0000_revs_xx000_added_x000_copies    08ea3258278e d9fa043f30c0 :   2.972788 s,   1.772094 s,  -1.200694 s, × 0.5961
      netbeans        x_revs_x_added_0_copies               fb0955ffcbcd a01e9239f9e7 :   0.000180 s,   0.000185 s,  +0.000005 s, × 1.0278
      netbeans        x_revs_x000_added_0_copies            6f360122949f 20eb231cc7d0 :   0.000123 s,   0.000135 s,  +0.000012 s, × 1.0976
      netbeans        x_revs_x_added_x_copies               1ada3faf6fb6 5a39d12eecf4 :   0.000315 s,   0.000329 s,  +0.000014 s, × 1.0444
      netbeans        x_revs_x00_added_x_copies             35be93ba1e2c 9eec5e90c05f :   0.001297 s,   0.001343 s,  +0.000046 s, × 1.0355
      netbeans        x000_revs_xx00_added_0_copies         eac3045b4fdd 51d4ae7f1290 :   0.024884 s,   0.029396 s,  +0.004512 s, × 1.1813
      netbeans        x000_revs_x000_added_x_copies         e2063d266acd 6081d72689dc :   0.032653 s,   0.040210 s,  +0.007557 s, × 1.2314
      netbeans        x000_revs_x000_added_x000_copies      ff453e9fee32 411350406ec2 :   4.230118 s,   4.556794 s,  +0.326676 s, × 1.0772
      mozilla-central x_revs_x_added_0_copies               3697f962bb7b 7015fcdd43a2 :   0.000197 s,   0.000199 s,  +0.000002 s, × 1.0102
      mozilla-central x_revs_x000_added_0_copies            dd390860c6c9 40d0c5bed75d :   0.000622 s,   0.000639 s,  +0.000017 s, × 1.0273
      mozilla-central x_revs_x_added_x_copies               8d198483ae3b 14207ffc2b2f :   0.000296 s,   0.000542 s,  +0.000246 s, × 1.8311
      mozilla-central x_revs_x00_added_x_copies             98cbc58cc6bc 446a150332c3 :   0.001626 s,   0.001685 s,  +0.000059 s, × 1.0363
      mozilla-central x_revs_x000_added_x000_copies         3c684b4b8f68 0a5e72d1b479 :   0.006218 s,   0.006954 s,  +0.000736 s, × 1.1184
      mozilla-central x_revs_x0000_added_x0000_copies       effb563bb7e5 c07a39dc4e80 :   0.132760 s,   0.132938 s,  +0.000178 s, × 1.0013
      mozilla-central x000_revs_xx00_added_0_copies         6100d773079a 04a55431795e :   0.029001 s,   0.008683 s,  -0.020318 s, × 0.2994
      mozilla-central x000_revs_x000_added_x_copies         9f17a6fc04f9 2d37b966abed :   0.005886 s,   0.005956 s,  +0.000070 s, × 1.0119
      mozilla-central x000_revs_x000_added_x000_copies      7c97034feb78 4407bd0c6330 :   3.619850 s,   0.963905 s,  -2.655945 s, × 0.2663
      mozilla-central x0000_revs_xx000_added_0_copies       9eec5917337d 67118cc6dcad :   0.058678 s,   0.049239 s,  -0.009439 s, × 0.8391
      mozilla-central x0000_revs_xx000_added_x000_copies    f78c615a656c 96a38b690156 :  11.926587 s,   4.217003 s,  -7.709584 s, × 0.3536
      mozilla-try     x_revs_x_added_0_copies               aaf6dde0deb8 9790f499805a :   0.001204 s,   0.001197 s,  -0.000007 s, × 0.9942
      mozilla-try     x_revs_x000_added_0_copies            d8d0222927b4 5bb8ce8c7450 :   0.001217 s,   0.001213 s,  -0.000004 s, × 0.9967
      mozilla-try     x_revs_x_added_x_copies               092fcca11bdb 936255a0384a :   0.000605 s,   0.000762 s,  +0.000157 s, × 1.2595
      mozilla-try     x_revs_x00_added_x_copies             b53d2fadbdb5 017afae788ec :   0.001876 s,   0.001909 s,  +0.000033 s, × 1.0176
      mozilla-try     x_revs_x000_added_x000_copies         20408ad61ce5 6f0ee96e21ad :   0.078190 s,   0.093021 s,  +0.014831 s, × 1.1897
      mozilla-try     x_revs_x0000_added_x0000_copies       effb563bb7e5 c07a39dc4e80 :   0.135428 s,   0.134536 s,  -0.000892 s, × 0.9934
      mozilla-try     x000_revs_xx00_added_0_copies         6100d773079a 04a55431795e :   0.029123 s,   0.009071 s,  -0.020052 s, × 0.3115
      mozilla-try     x000_revs_x000_added_x_copies         9f17a6fc04f9 2d37b966abed :   0.006141 s,   0.006206 s,  +0.000065 s, × 1.0106
      mozilla-try     x000_revs_x000_added_x000_copies      1346fd0130e4 4c65cbdabc1f :   4.857827 s,   1.150502 s,  -3.707325 s, × 0.2368
      mozilla-try     x0000_revs_x_added_0_copies           63519bfd42ee a36a2a865d92 :  10.674920 s,   1.114864 s,  -9.560056 s, × 0.1044
      mozilla-try     x0000_revs_x_added_x_copies           9fe69ff0762d bcabf2a78927 :   9.789462 s,   1.042658 s,  -8.746804 s, × 0.1065
      mozilla-try     x0000_revs_xx000_added_x_copies       156f6e2674f2 4d0f2c178e66 :   1.087890 s,   0.447402 s,  -0.640488 s, × 0.4113
      mozilla-try     x0000_revs_xx000_added_0_copies       9eec5917337d 67118cc6dcad :   0.060556 s,   0.051132 s,  -0.009424 s, × 0.8444
      mozilla-try     x0000_revs_xx000_added_x000_copies    89294cd501d9 7ccb2fc7ccb5 : killed         ,  83.508590 s
      mozilla-try     x0000_revs_x0000_added_x0000_copies   e928c65095ed e951f4ad123a : killed         ,  55.079813 s
      mozilla-try     x00000_revs_x00000_added_0_copies     dc8a3ca7010e d16fde900c9c :  19.647038 s,   1.442793 s, -18.204245 s, × 0.0734
      
      
      This changeset compared to the Python Code
      ==========================================
      
      
      Repo            Cases                                 Source-Rev   Dest-Rev         Py-time       Rust-time    Difference   Factor
      ------------------------------------------------------------------------------------------------------------------------------------
      mercurial       x_revs_x_added_0_copies               ad6b123de1c7 39cfcef4f463 :   0.000044 s,   0.000049 s,  +0.000005 s, × 1.1136
      mercurial       x_revs_x_added_x_copies               2b1c78674230 0c1d10351869 :   0.000138 s,   0.000179 s,  +0.000041 s, × 1.2971
      mercurial       x000_revs_x000_added_x_copies         81f8ff2a9bf2 dd3267698d84 :   0.005052 s,   0.006494 s,  +0.001442 s, × 1.2854
      pypy            x_revs_x_added_0_copies               aed021ee8ae8 099ed31b181b :   0.000219 s,   0.000339 s,  +0.000120 s, × 1.5479
      pypy            x_revs_x000_added_0_copies            4aa4e1f8e19a 359343b9ac0e :   0.000055 s,   0.000057 s,  +0.000002 s, × 1.0364
      pypy            x_revs_x_added_x_copies               ac52eb7bbbb0 72e022663155 :   0.000128 s,   0.000299 s,  +0.000171 s, × 2.3359
      pypy            x_revs_x00_added_x_copies             c3b14617fbd7 ace7255d9a26 :   0.001089 s,   0.001200 s,  +0.000111 s, × 1.1019
      pypy            x_revs_x000_added_x000_copies         df6f7a526b60 a83dc6a2d56f :   0.017407 s,   0.025120 s,  +0.007713 s, × 1.4431
      pypy            x000_revs_xx00_added_0_copies         89a76aede314 2f22446ff07e :   0.094175 s,   0.506921 s,  +0.412746 s, × 5.3828
      pypy            x000_revs_x000_added_x_copies         8a3b5bfd266e 2c68e87c3efe :   0.238009 s,   1.272060 s,  +1.034051 s, × 5.3446
      pypy            x000_revs_x000_added_x000_copies      89a76aede314 7b3dda341c84 :   0.125876 s,   0.690941 s,  +0.565065 s, × 5.4891
      pypy            x0000_revs_x_added_0_copies           d1defd0dc478 c9cb1334cc78 :   3.581556 s,  33.527067 s, +29.945511 s, × 9.3610
      pypy            x0000_revs_xx000_added_0_copies       bf2c629d0071 4ffed77c095c :   0.016721 s,   0.021970 s,  +0.005249 s, × 1.3139
      pypy            x0000_revs_xx000_added_x000_copies    08ea3258278e d9fa043f30c0 :   0.242367 s,   1.772094 s,  +1.529727 s, × 7.3116
      netbeans        x_revs_x_added_0_copies               fb0955ffcbcd a01e9239f9e7 :   0.000165 s,   0.000185 s,  +0.000020 s, × 1.1212
      netbeans        x_revs_x000_added_0_copies            6f360122949f 20eb231cc7d0 :   0.000114 s,   0.000135 s,  +0.000021 s, × 1.1842
      netbeans        x_revs_x_added_x_copies               1ada3faf6fb6 5a39d12eecf4 :   0.000296 s,   0.000329 s,  +0.000033 s, × 1.1115
      netbeans        x_revs_x00_added_x_copies             35be93ba1e2c 9eec5e90c05f :   0.001124 s,   0.001343 s,  +0.000219 s, × 1.1948
      netbeans        x000_revs_xx00_added_0_copies         eac3045b4fdd 51d4ae7f1290 :   0.013060 s,   0.029396 s,  +0.016336 s, × 2.2508
      netbeans        x000_revs_x000_added_x_copies         e2063d266acd 6081d72689dc :   0.017112 s,   0.040210 s,  +0.023098 s, × 2.3498
      netbeans        x000_revs_x000_added_x000_copies      ff453e9fee32 411350406ec2 :   0.660350 s,   4.556794 s,  +3.896444 s, × 6.9006
      netbeans        x0000_revs_xx000_added_x000_copies    588c2d1ced70 1aad62e59ddd :  10.032499 s, killed
      mozilla-central x_revs_x_added_0_copies               3697f962bb7b 7015fcdd43a2 :   0.000189 s,   0.000199 s,  +0.000010 s, × 1.0529
      mozilla-central x_revs_x000_added_0_copies            dd390860c6c9 40d0c5bed75d :   0.000462 s,   0.000639 s,  +0.000177 s, × 1.3831
      mozilla-central x_revs_x_added_x_copies               8d198483ae3b 14207ffc2b2f :   0.000270 s,   0.000542 s,  +0.000272 s, × 2.0074
      mozilla-central x_revs_x00_added_x_copies             98cbc58cc6bc 446a150332c3 :   0.001474 s,   0.001685 s,  +0.000211 s, × 1.1431
      mozilla-central x_revs_x000_added_x000_copies         3c684b4b8f68 0a5e72d1b479 :   0.004806 s,   0.006954 s,  +0.002148 s, × 1.4469
      mozilla-central x_revs_x0000_added_x0000_copies       effb563bb7e5 c07a39dc4e80 :   0.085150 s,   0.132938 s,  +0.047788 s, × 1.5612
      mozilla-central x000_revs_xx00_added_0_copies         6100d773079a 04a55431795e :   0.007064 s,   0.008683 s,  +0.001619 s, × 1.2292
      mozilla-central x000_revs_x000_added_x_copies         9f17a6fc04f9 2d37b966abed :   0.004741 s,   0.005956 s,  +0.001215 s, × 1.2563
      mozilla-central x000_revs_x000_added_x000_copies      7c97034feb78 4407bd0c6330 :   0.190133 s,   0.963905 s,  +0.773772 s, × 5.0696
      mozilla-central x0000_revs_xx000_added_0_copies       9eec5917337d 67118cc6dcad :   0.035651 s,   0.049239 s,  +0.013588 s, × 1.3811
      mozilla-central x0000_revs_xx000_added_x000_copies    f78c615a656c 96a38b690156 :   0.440694 s,   4.217003 s,  +3.776309 s, × 9.5690
      mozilla-central x00000_revs_x0000_added_x0000_copies  6832ae71433c 4c222a1d9a00 :  18.454163 s, killed
      mozilla-central x00000_revs_x00000_added_x000_copies  76caed42cf7c 1daa622bbe42 :  31.562719 s, killed
      mozilla-try     x_revs_x_added_0_copies               aaf6dde0deb8 9790f499805a :   0.001189 s,   0.001197 s,  +0.000008 s, × 1.0067
      mozilla-try     x_revs_x000_added_0_copies            d8d0222927b4 5bb8ce8c7450 :   0.001204 s,   0.001213 s,  +0.000009 s, × 1.0075
      mozilla-try     x_revs_x_added_x_copies               092fcca11bdb 936255a0384a :   0.000586 s,   0.000762 s,  +0.000176 s, × 1.3003
      mozilla-try     x_revs_x00_added_x_copies             b53d2fadbdb5 017afae788ec :   0.001845 s,   0.001909 s,  +0.000064 s, × 1.0347
      mozilla-try     x_revs_x000_added_x000_copies         20408ad61ce5 6f0ee96e21ad :   0.063822 s,   0.093021 s,  +0.029199 s, × 1.4575
      mozilla-try     x_revs_x0000_added_x0000_copies       effb563bb7e5 c07a39dc4e80 :   0.088038 s,   0.134536 s,  +0.046498 s, × 1.5282
      mozilla-try     x000_revs_xx00_added_0_copies         6100d773079a 04a55431795e :   0.007389 s,   0.009071 s,  +0.001682 s, × 1.2276
      mozilla-try     x000_revs_x000_added_x_copies         9f17a6fc04f9 2d37b966abed :   0.004868 s,   0.006206 s,  +0.001338 s, × 1.2749
      mozilla-try     x000_revs_x000_added_x000_copies      1346fd0130e4 4c65cbdabc1f :   0.222450 s,   1.150502 s,  +0.928052 s, × 5.1720
      mozilla-try     x0000_revs_x_added_0_copies           63519bfd42ee a36a2a865d92 :   0.370675 s,   1.114864 s,  +0.744189 s, × 3.0077
      mozilla-try     x0000_revs_x_added_x_copies           9fe69ff0762d bcabf2a78927 :   0.358020 s,   1.042658 s,  +0.684638 s, × 2.9123
      mozilla-try     x0000_revs_xx000_added_x_copies       156f6e2674f2 4d0f2c178e66 :   0.145235 s,   0.447402 s,  +0.302167 s, × 3.0805
      mozilla-try     x0000_revs_xx000_added_0_copies       9eec5917337d 67118cc6dcad :   0.037606 s,   0.051132 s,  +0.013526 s, × 1.3597
      mozilla-try     x0000_revs_xx000_added_x000_copies    89294cd501d9 7ccb2fc7ccb5 :   7.382439 s,  83.508590 s, +76.126151 s, × 11.3118
      mozilla-try     x0000_revs_x0000_added_x0000_copies   e928c65095ed e951f4ad123a :   7.273506 s,  55.079813 s, +47.806307 s, × 7.5727
      mozilla-try     x00000_revs_x00000_added_0_copies     dc8a3ca7010e d16fde900c9c :   1.074593 s,   1.442793 s,  +0.368200 s, × 1.3426
      mozilla-try     x00000_revs_x0000_added_x0000_copies  8d3fafa80d4b eb884023b810 :  27.746195 s, killed
      
      Differential Revision: https://phab.mercurial-scm.org/D9300
      46bb4177
  2. Sep 24, 2020
  3. Sep 02, 2020
  4. Sep 23, 2020
  5. Sep 04, 2020
  6. Sep 06, 2020
  7. Jul 07, 2020
    • Antoine Cezar's avatar
      rhg: add a limited `rhg root` subcommand · 18f8d3b3
      Antoine Cezar authored
      Clap has been choosen for argument parsing for the following reasons:
      
      - it's a wildly used and maintained crate
      - it can deal with OS encoding making it suitable for any encoding
      - it supports nonambiguous prefix matching as already available in hg
      - it will soon allow for structopts-style declarative pattern instead of the
        currently used builder pattern
      
      Differential Revision: https://phab.mercurial-scm.org/D8613
      18f8d3b3
  8. Jun 05, 2020
  9. Jun 15, 2020
  10. May 29, 2020
    • Raphaël Gomès's avatar
      rust: remove support for `re2` · 9f96beb9
      Raphaël Gomès authored
      With the performance issues with `regex` figured out and fixed in previous
      patches and `regex` newly gaining support for empty alternations, there is no
      reason to keep `re2` around anymore. It's only *marginally* faster at creating
      the regex which saves at most a couple of ms, but gets beaten by `regex` in
      every other aspect.
      
      This removes the Rust/C/C++ bridge (hooray!), the `with-re2` feature, the
      conditional code that goes with it, the documentation and relevant part of the
      debug/module output.
      
      Differential Revision: https://phab.mercurial-scm.org/D8594
      9f96beb9
    • Raphaël Gomès's avatar
      rust-dependencies: update `regex` to 1.3.9 · 4313a0d7
      Raphaël Gomès authored
      Version `1.3.8` introduces support for empty alternations, which makes
      previously disallowed patterns usable in `regex`.
      
      From a user's perspective, this means that glob patterns like `*.py{,c}` will
      no longer generate an "invalid" regex and will use the Rust path.
      
      `1.3.9` is a bugfix release, might as well update to the latest one.
      
      Differential Revision: https://phab.mercurial-scm.org/D8593
      4313a0d7
  11. Mar 23, 2020
    • Raphaël Gomès's avatar
      rust: update all dependencies · d31d1c06
      Raphaël Gomès authored
      We do this periodically to say up to date.
      
      No major versions were crossed this time per se, but the `rand` is still in v0,
      and their 0.7x series broke three things:
          - Some distribution-related elements were moved to a separate crate,
            flashing a deprecation warning
          - The `LogNormal::new` associated function now returns a `Result`
          - Certain RNGs were updated to sample a `u32` instead of `usize` when their
            upper-bound is less than `u32::MAX` for better portability, which changed
            the output for 2 tests.
      
      Moreover, the recent use of the `regex` crate for ignore mechanisms prompted
      some benchmarking that revealed that `regex` was slower at compiling big regex
      than `Re2`. The author of `regex` was very quick to discover an optimization
      that yielded a 30% improvement. It's still slower than `Re2` in that regard, but
      less so in the 1.3.6 release.
      
      Differential Revision: https://phab.mercurial-scm.org/D8320
      d31d1c06
    • Raphaël Gomès's avatar
      rust: update micro-timer dependency · f451a347
      Raphaël Gomès authored
      The new version uses a much more robust technique and should remove any existing
      risk of bad compiler error or performance hit.
      
      Differential Revision: https://phab.mercurial-scm.org/D8319
      f451a347
  12. Mar 18, 2020
  13. Mar 06, 2020
    • Raphaël Gomès's avatar
      hg-core: add function timing information · d880805d
      Raphaël Gomès authored
      This change makes use of the newly added logging infrastructure to trace the
      execution time of some important calls.
      
      This approach is very much complementary to using a profiler and will not
      guard against out-of-order execution or other kinds of compiler optimizations.
      That said, it is useful to get a rough high-level idea of where time is spent.
      
      Differential Revision: https://phab.mercurial-scm.org/D8253
      d880805d
    • Raphaël Gomès's avatar
      rust: add logging utils · 82f51ab7
      Raphaël Gomès authored
      This change adds the `log` crate, the community-approved logging facade backed
      by Rust core developers as well as the logging-consumer crate `simple_logger`
      to build a foundation for logging from Rust.
      
      Using this setup allows us to choose how to log depending on the way `hg-core`
      is used: if it's within the context of `hg-cpython`, we might not want to use
      it the same way as with a direct cli for example.
      
      Differential Revision: https://phab.mercurial-scm.org/D8252
      82f51ab7
    • Raphaël Gomès's avatar
      rust-status: traverse working directory in parallel · fe7d2cf0
      Raphaël Gomès authored
      Using `rayon` for this task ensures that we are using the same work-stealing
      threadpool for everything.
      
      This change introduces `crossbeam` as an explicit dependency, although it is
      already a dependency of `rayon`. It provides better structures for
      multi-threaded tasks than the stdlib.
      
      Differential Revision: https://phab.mercurial-scm.org/D8251
      fe7d2cf0
    • Pierre-Yves David's avatar
      setup-rust: remove the legacy 'direct-ffi' variant · 79ac59d3
      Pierre-Yves David authored
      This variant have been abandoned for a while. Keeping it around just get people
      confused.
      79ac59d3
  14. Feb 18, 2020
    • Georges Racinet's avatar
      rust-nodemap: pure Rust example · 8f7c6656
      Georges Racinet authored
      To run, use `cargo run --release --example nodemap`
      
      This demonstrates that simple scenarios entirely written
      in Rust can content themselves with `NodeTree<T>`.
      
      The example mmaps both the nodemap file and the changelog index.
      We had of course to include an implementation of `RevlogIndex`
      directly, which isn't much at this stage. It felt a bit
      prematurate to include it in the lib.
      
      Here are some first performance measurements, obtained with
      this example, on a clone of mozilla-central with 440000
      changesets:
      
        (create) Nodemap constructed in RAM in 153.638305ms
        (query CAE63161B68962) found in 22.362us: Ok(Some(269489))
        (bench) Did 3 queries in 36.418µs (mean 12.139µs)
        (bench) Did 50 queries in 184.318µs (mean 3.686µs)
        (bench) Did 100000 queries in 31.053461ms (mean 310ns)
      
      To be fair, even between bench runs, results tend to depend whether
      the file is still in kernel caches, and it's not so easy to
      get back to a real cold start. The worst we've seen was in the
      50us ballpark.
      
      In any busy server setting, the pages would always be in RAM.
      
      We hope it's good enough not to be significantly slower on any
      concrete Mercurial operation than the C nodetree when fully in RAM,
      and of course this implementation has the serious headstart advantage
      of persistence.
      
      Differential Revision: https://phab.mercurial-scm.org/D7797
      8f7c6656
  15. Jan 16, 2020
    • Raphaël Gomès's avatar
      rust-re2: add wrapper for calling Re2 from Rust · d8d4fa9a
      Raphaël Gomès authored
      This assumes that Re2 is installed following Google's guide. I am not sure
      how we want to integrate it in the project, but I think a follow-up patch would
      be more appropriate for such work.
      As it stands, *not* having Re2 installed results in a compilation error, which
      is a problem as it breaks install compatibility. Hence, this is gated behind
      a non-default `with-re2` compilation feature.
      
      Differential Revision: https://phab.mercurial-scm.org/D7910
      d8d4fa9a
  16. Jan 14, 2020
  17. Feb 05, 2020
  18. Jan 30, 2020
  19. Jan 17, 2020
  20. Jan 22, 2020
    • Georges Racinet's avatar
      rust-node: binary Node ID and conversion utilities · 7f86426f
      Georges Racinet authored
      The choice of type makes sure that a `Node` has the exact
      wanted size. We'll use a different type for prefixes.
      
      Added dependency: hexadecimal conversion relies on the
      `hex` crate.
      
      The fact that sooner or later Mercurial is going to need
      to change its hash sizes has been taken strongly in
      consideration:
      
      - the hash length is a constant, but that is not directly
        exposed to callers. Changing the value of that constant
        is the only thing to do to change the hash length (even
        in unit tests)
      - the code could be adapted to support several sizes of hashes,
        if that turned out to be useful. To that effect, only the
        size of a given `Node` is exposed in the public API.
      - callers not involved in initial computation, I/O and FFI
        are able to operate without a priori assumptions on the hash
        size. The traits `FromHex` and `ToHex` have not been directly
        implemented, so that the doc-comments explaining these
        restrictions would stay really visible in `cargo doc`
      
      Differential Revision: https://phab.mercurial-scm.org/D7788
      7f86426f
  21. Oct 14, 2019
    • Raphaël Gomès's avatar
      rust-performance: introduce FastHashMap type alias for HashMap · 5ac243a9
      Raphaël Gomès authored
      Rust's default hashing is slow, because it is meant for preventing collision
      attacks.
      For all of the current Rust code, we don't care about those attacks, because
      if an person with bad intentions has write access to your repo, you have other
      issues.
      
      I've chosen to use the TwoXHash crate because it was made by a reputable member
      of the Rust community and has very good benchmarks.
      
      For now it does not seem to improve performance by much for the current code,
      but it's something else to not worry about when benchmarking code: in a
      previous experiment with copytracing in Rust, it accounted for more than 10%
      of the time of the entire script.
      
      Differential Revision: https://phab.mercurial-scm.org/D7116
      5ac243a9
  22. Oct 11, 2019
    • Raphaël Gomès's avatar
      rust-dirstate-status: add first Rust implementation of `dirstate.status` · 99394e6c
      Raphaël Gomès authored
      Note: This patch also added the rayon crate as a Cargo dependency. It will
      help us immensely in making Rust code parallel and easy to maintain. It is
      a stable, well-known, and supported crate maintained by people on the Rust
      team.
      
      The current `dirstate.status` method has grown over the years through bug
      reports and new features to the point where it got too big and too complex.
      
      This series does not yet improve the logic, but adds a Rust fast-path to speed
      up certain cases.
      
      Tested on mozilla-try-2019-02-18 with zstd compression:
          - `hg diff` on an empty working copy:
              - c: 1.64(+-)0.04s
              - rust+c before this change: 2.84(+-)0.1s
              - rust+c: 849(+-)40ms
          - `hg commit` when creating a file:
              - c: 5.960s
              - rust+c before this change: 5.828s
              - rust+c: 4.668s
          - `hg commit` when updating a file:
              - c: 4.866s
              - rust+c before this change: 4.371s
              - rust+c: 3.855s
          - `hg status -mard`
              - c: 1.82(+-)0.04s
              - rust+c before this change: 2.64(+-)0.1s
              - rust+c: 896(+-)30ms
      
      The numbers are clear: the current Rust `dirstatemap` implementation is super
      slow, its performance needs to be addressed.
      This will be done in a future series, immediately after this one, with the goal
      of getting Rust to be at least to the speed of the Python + C implementation
      in all cases before the 5.2 freeze. At worse, we gate dirstatemap to only be used
      in those cases.
      
      Cases where the fast-path is not executed:
          - for commands that need ignore support (`status`, for example)
          - if subrepos are found (should not be hard to add, but winter is coming)
          - any other matcher than an `alwaysmatcher`, like patterns, etc.
          - with extensions like `sparse` and `fsmonitor`
      
      The next step after this is to rethink the logic to be closer to
      Jane Street's Valentin Gatien-Baron's Rust fast-path which does a lot less
      work when possible.
      
      Differential Revision: https://phab.mercurial-scm.org/D7058
      99394e6c
  23. Oct 13, 2019
  24. Jun 15, 2019
  25. Apr 24, 2019
    • Raphaël Gomès's avatar
      rust-filepatterns: add a Rust implementation of pattern-related utils · e8f3740c
      Raphaël Gomès authored
      This change introduces Rust implementations of two functions related to
      pattern handling, all located in `match.py`:
      
          - `_regex`
          - `readpatternfile`
      
      These utils are useful in the long-term effort to improve `hg status`'s
      performance using Rust. Experimental work done by Valentin Gatien-Baron
      shows very promising improvements, but is too different from the current
      Mercurial core code structure to be used "as-is".
      This is the first - albeit very small - step towards the code revamp
      needed down the line.
      
      Two dependencies were added: `regex` and `lazy_static`. Both of them
      will be useful for a majority of the Rust code that will be written,
      are well known and maintained either by the Rust core team, or by
      very frequent contributors.
      
      Differential Revision: https://phab.mercurial-scm.org/D6271
      e8f3740c
  26. May 06, 2019
    • Raphaël Gomès's avatar
      rust-dirstate: add rust-cpython bindings to the new parse/pack functions · e240bec2
      Raphaël Gomès authored
      This allows for Python code to call `parse/pack_dirstate` transparently.
      
      These bindings are heavy given the relatively simple task, as they are bound
      to implementation details of both the C and Python code. They will be slimmed
      down in future patches and eventually completely removed once more of the
      dirstate code has been refactored/rewritten in Rust.
      
      Both functions emulate the mutate-on-loop style of the Python and C
      implementations by looping over changed items in the compatibility layer,
      instead of at the core functions.
      
      Differential Revision: https://phab.mercurial-scm.org/D6349
      e240bec2
  27. Dec 02, 2018
    • Georges Racinet's avatar
      rust: translated random test of missingancestors · ee7b7bd4
      Georges Racinet authored
      This is a Rust implementation of the random
      DAG generator and related incrementalmissingancestors
      tests against a naive brute force implementation.
      
      It is provided as an integration test, so that it
      won't run by default if any unit test fails.
      
      In case of a failed example, all needed information
      for reproduction is included in the panic message,
      (this is how
      `test_remove_ancestors_from_case1()` has been generated),
      as well as the random seed.
      
      The whole test is rerunnable by passing the random seed
      in the TEST_RANDOM_SEED environment variable.
      The other parameters (numbers of iterations) can be passed
      in the TEST_MISSING_ANCESTORS environment variable.
      
      An alternative would have been to expose to Python
      MissingAncestors<VecGraphs> but that would have meant
      pollution of the release build used from Python,
      whereas we do it in this changeset within the tests submodule
      
      Differential Revision: https://phab.mercurial-scm.org/D5417
      ee7b7bd4
  28. Dec 17, 2018
  29. Dec 03, 2018
    • Georges Racinet's avatar
      rust-cpython: start cpython crate bindings · 5532823e
      Georges Racinet authored
      This changeset introduces the hg-cpython crate,
      that compiles as a shared library holding a whole
      Python package (mercurial.rustext), with only the empty
      'ancestor' submodule for now.
      
      Such bindings will be easier and safer to develop and maintain
      that those of `hg-direct-ffi`.
      They don't involve C code, only unsafe Rust that's mostly isolated
      within the cpython crate.
      
      The long-term goal would be to import the provided modules, such
      as rustext.ancestor with mercurial.policy.importmod, same as
      we already do with cext modules.
      
      Differential Revision: https://phab.mercurial-scm.org/D5434
      5532823e
    • Georges Racinet's avatar
      rust-cpython: exclude hgcli from workspace · 98a0fbda
      Georges Racinet authored
      hgcli uses a specific rust-cpython commit by indygreg, of which
      a PR has been derived which is not merged nor released yet.
      
      But we can't use several versions of the sys-python2.7 crate in
      a single workspace: it makes for a build error. Since hgcli does not at
      the time being need anything from hg-core, whereas the upcoming hg-cpython
      will. So for now we're moving hgcli aside, hoping we could base all of them
      on the same version of rust-cpython again in the future.
      
      Differential Revision: https://phab.mercurial-scm.org/D5433
      98a0fbda
  30. Sep 27, 2018
    • Georges Racinet's avatar
      rust: iterator bindings to C code · a36c5e23
      Georges Racinet authored
      In this changeset, still made of Rust code only,
      we expose the Rust iterator for instantiation and
      consumption from C code.
      
      The idea is that both the index and index_get_parents()
      will be passed from the C extension, hence avoiding a hard
      link dependency to parsers.so, so that the crate can
      still be built and tested independently.
      
      On the other hand, parsers.so will use the symbols
      defined in this changeset.
      a36c5e23
Loading