Skip to content
Snippets Groups Projects
  • Georges Racinet's avatar
    77e37a03
    CommitService.CommitLanguages implementation with tokei · 77e37a03
    Georges Racinet authored
    This introduces a new binary dependency on the
    [Tokei](https://crates.io/crates/tokei) code statistics tool.
    
    The path to the Tokei executable is configurable
    as `hgitaly.tokei-executable`, defaulting to `tokei`.
    
    Strict comparison with Gitaly is not fully possible, as Tokei
    does not produce the very same results as GitHub Linguist (or
    `go-enry` which is the library Gitaly is currently moving to).
    For instance `robots.txt` is detected as the `robots.txt` language
    by Linguist and as `Plain Text` by tokei. That being said, we
    can compare more frequent results, namely Python and Ruby.
    
    We vendor the `languages.json` file from GitHub Linguist to add the
    color information (not part of Tokei results), with the same defaulting
    logic as Gitaly (see the detailed docstrings), based on a hash of the
    language name. We were lucky to find a programming language (COBOL) to
    base our tests on.
    
    It is important to sort the results in decreasing order, because
    the Rails app considers only the 5 most frequent languages in the
    repository, and does so by truncating at the 5 first elements.
    77e37a03
    History
    CommitService.CommitLanguages implementation with tokei
    Georges Racinet authored
    This introduces a new binary dependency on the
    [Tokei](https://crates.io/crates/tokei) code statistics tool.
    
    The path to the Tokei executable is configurable
    as `hgitaly.tokei-executable`, defaulting to `tokei`.
    
    Strict comparison with Gitaly is not fully possible, as Tokei
    does not produce the very same results as GitHub Linguist (or
    `go-enry` which is the library Gitaly is currently moving to).
    For instance `robots.txt` is detected as the `robots.txt` language
    by Linguist and as `Plain Text` by tokei. That being said, we
    can compare more frequent results, namely Python and Ruby.
    
    We vendor the `languages.json` file from GitHub Linguist to add the
    color information (not part of Tokei results), with the same defaulting
    logic as Gitaly (see the detailed docstrings), based on a hash of the
    language name. We were lucky to find a programming language (COBOL) to
    base our tests on.
    
    It is important to sort the results in decreasing order, because
    the Rails app considers only the 5 most frequent languages in the
    repository, and does so by truncating at the 5 first elements.