[Nix-dev] Some bug tracker experience and RFC on improvements

[Luca Bruno]

On 16/03/2015 00:43, Austin Seipp wrote:
> Hi *,
>
> Over the past few days, while I've been waiting for things to compile,
> I've been casually triaging the NixOS pull request list and the actual
> bug list, adding labels to everything to categorize issues. I think
> all of the PRs open as of today are labeled, which is great. The first
> few pages of the bug list are also almost labeled, but I haven't gone
> much farther.
>
> I want to take a moment emphasize the importance of doing this, and
> some thoughts on improving it, based on my experience doing similar
> things for other projects. Please note, I'm talking almost exclusively
> about the 'nixpkgs' repository.
>
> As many of you may be aware of by now, GitHub has an issue tracker.
> It's integrated with the Pull Request capability. It's relatively
> simple by design. But this has some knock-on consequences - it has a
> relatively anemic ability to extend metadata (e.g. custom issue
> fields), and personally I've found the full-text search capabilities
> to be relatively poor. Their issue tracker is quite simple like I
> said, but what this means is that metadata is vitally important to
> triaging bugs and having maintainers get to them easily and in a
> timely manner.
>
> For example, I'm generally interested in all kernel bugs, and things
> like security bugs and issues so I can get CVEs fixed (I know, I
> haven't had much time for this recently.) But I only added the
> 'kernel' label just on Friday, I think!
I do something similar, e.g. finding update-package or new-package that
are not wip, not darwin and not haskell :P .
>
> Also, I want to know about all the bugs for the 15.05 release that
> need to be fixed, which are kernel/systemd bugs. And the ones that are
> critical priority, that haven't been fixed. And the ones that have
> been fixed, so we can add to the release notes. That way I know what
> to do about them.
>
> And, as I'm sure most of you are aware, GitHub's notification system
> is not very lean. If you 'watch' a repository, you get spammed for
> every update to it! This works OK for small projects where I do
> everything - it does not scale very nicely to more people, meaning in
> practice I have a GitHub filter in my email with hundreds of unread
> things for NixOS subsystems I don't maintain.
I like receiving notifications for every issue and PR currently.

>       There is a very clear warning at the top to read about how to
> contribute. This can have a quick note that says things like A)
> "Please label any bugs you file as you see fit" B) "Format your
> messages properly" C) "Brush your teeth before going to bed".
>
>       This doesn't guarantee people follow protocol, but it will
> hopefully remind people to file bugs following specific guidelines.
People can't label their own bugs, only collaborators can assign labels.

>
>      2) The current issue list is really, really confusing and
> haphazard. For example, there is a 'python' label but no 'perl' label,
> despite the fact I just labeled 5 perl PRs! And there's no 'systemd'
> label, or any label about, say, Xorg, the NixOS installer, etc etc. As
> well as no uniformity to color, either :) Also, labels for things like
> 'high' vs 'low' priority don't exist either (really, these should be
> first class concepts)
>
>       So, I'd recommend we take a play from the Rust people and handle
> GitHub's simpllistic tracker, and categorize all labels by a
> 'callsign' - a distinctive single letter - followed by the label name.
> For example, a reinvisioning of the current label list with some
> additions might look like:
>
>   "Blocker" issues/bugs:
>        B-low               -- Low priority
>        B-normal          -- (Default) normal priority
>        B-high              -- High priority
>        B-blocker         -- Blocking bug
>
>   "Infrastructure" issues/bugs:
>        I-security               -- Security updates
>        I-documentation     -- Documentation
>        I-mass-rebuild        -- Massive rebuild
>
>   "Language" issues/bugs:
>        L-python                -- Python packages
>        L-perl                     -- Perl packages
>        L-php                     -- PHP packages
>        L-haskell                -- Haskell packages
>        L-ruby                    -- Ruby packages
>
>   "Miscellaneous" issues/bugs:
>        M-rfc                     -- RFC (AKA 'feedback-requested')
>        M-wip                    -- Work-in-progress
>        M-question             -- Questions
>        M-meta                  -- Meta bugs for other issues.
>
>   "NixOS" issues/bugs:
>        N-installer              -- NixOS installer
>        N-systemd             -- systemd (NixOS only)
>        N-xorg                   -- Xorg/MESA/etc
>        N-gnome               -- GNOME packaging
>        N-KDE                  -- KDE packaging
>        N-kernel                -- Kernel packages
>
>   "Package" issues/bugs:
>        P-enhancement     -- Package enhancement, amendment, etc
>        P-update                -- Update version package
>        P-new                    -- New package
>
>  "System" issues/bugs:
>        S-linux                 -- (Default) Linux bugs
>        S-darwin              -- Darwin bugs
>        S-other                -- Other platforms
>
>   "Invalid/duplicate/wontfix" bugs
>        Z-einvalid               -- Invalid issue/bug; -EINVAL
>        Z-enotabug             -- Not a bug; -ENOTABUG
>        Z-ewontfix              -- Nope; -EWONTFIX
>        Z-duplicate             -- Duplicate issue
>
> Note that any issue may combine any of these together. It's also a
> much richer set of labels that express more constraints on a bug. A
> few things:
>
>   A) There is a specific 'M-meta' to describe metabugs. This should
> have no other labels associated, and should simply list other bugs in
> a checkbox-list specifying what's blocking them.
>
>   B) Labels are color-coated by *category*, not individually. This
> means that all 'P' labels are blue, all Z labels are grey, etc. This
> makes things visually clearer as to what they represent, and saves us
> recycling colors all the time.
In general I don't like too much granularity or too much tagging etc.,
but maybe we can try this for NixOS. I think I may like this.
> -----------------------------------
>
> Finally, there are some other associated things I was thinking of:
>
>   1) It would also be nice to adopt an official policy of maintainers
> for the source tree; e.g. who is responsible for certain NixOS or
> Nixpkgs subsystems, or Darwin/FreeBSD support, etc. This makes it
> easier to track down the right person, IMO.
IIRC there was a discussion already about maintainers of nixos modules.
I usually git blame the file, but probably adding a maintainers meta to
nixos modules is a good idea.
>
>   2) It would be nice if we could do a round-robin scheduling or
> something of maintainers who have to triage issues, even very very
> briefly. Every day one person is elected to just add labels to things,
> and set the ticket to a milestone. Some projects have done this in the
> past; in practice, I think if we had a random selection of people who
> could once a day add labels and set them, and we were diligent, most
> days would only see 5-10 issues needing labels. Which is not too hard;
> appropriate metadata is pretty easy to infer from just reading an
> issue.
I do it when I like to do it, already. So big "NO" to this kind job.