PyLint Utilities - A Long Time Coming

Summary¶

In my last article, I talked about taking my plugin work on the Project Summarizer project to its logical conclusion by adding proper plugin support to the project. In this article, I talk about the first external plugin: PyLint Utilities.

Introduction¶

I believe that most software development professionals have little packages of utilities that they use personally. Be it something simple to set up their environment consistently or a more complex utility to handle something that is either bothersome or error prone. We have all got fed up with having to do repeat something repeatedly. And once it hits a certain threshold in our brain, we decide to write something to deal with it.

Most of the time, those utilities never seen public consumption. I mean, they are OUR utilities. But sometimes, as with my PyLint Utilities project, they get to a point where we want to clean them up and share them. This is one of the utilities that I want to share.

Why A Long Time Coming?¶

Well, the easy answer to that question is that I have been working on the PyLint Utilities project for at least the last four months. According to the commit logs for the project, it was 2021 Nov 26 when I added the first commit to the project. Following that, I did work over the Christmas holidays to start padding it out. But I have been using it in various forms during that time to help guide me on how things are going with my projects, especially on the PyMarkdown project.

Between the start of the year and April, I made small modifications to the various utilities, tuning them and making them work better. Nothing significant, just minor changes to make them work better under various conditions. It was when I sat back and looked at the collection of utilities as a whole did I realize that I had an application which had utilities that might be useful to others. It was only then that added the necessary project files and scenario tests to bring the project up to a releasable level.

Up to that point, it essentially was just a place for me to “doodle with code” to try and address some issues that I was trying to solve. It meant a bit of a mindset change on how I looked at the project, but after a couple of weeks, I was okay with it. It just felt weird taking a proof-of-concept (what I would refer to as a “code doodle”) and releasing it as a project. I am getting used to it though.

So What Are Those Utilities?¶

To be honest, this project started out as a bit of a toy application that helped me with my PyLint suppressions. Each one of the concerns that I had was addressed with one of the utilities that I added. I did not consider any of the utilities noteworthy on their own, but they were still useful to me.

Balanced Suppressions¶

I measure quality in a project on a sliding scale. I have no problem with warnings of any quality analysis being suppressed if there is some accounting for those suppressions within a project. I am not talking about the rare 2-4 suppressions that I typically do in a project where the code coverage mechanism does not cover something that I know is covered. Those are what I would consider to be a “cost of doing business”. These are unique to the project warnings that are actual warning signals, not warning noise.

To that extent, I believe that each suppression should be added to the code with the smallest scope possible. If I am not excluding that suppression throughout the entire project in configuration files, that I feel that I should be expected to present a focused suppression around where that warning occurs. Otherwise, I feel that I am doing myself and the project a disservice.

And I am particularly good with following that policy around 95% of the time. However, I acknowledge that I am not perfect. There are times that I have disabled a warning on line 50 of a 900+ line Python file, only to forget to enable it again on line 90 when the function was finished. There are times where I have copied and pasted an enable suppression line to the start of a function, only to wonder why it is still showing up when I run PyLint, because I am 100% positive that when I read enabled, it said disabled. Yup… I hope I am not alone in making those mistakes. As an old college professor once reminded us “sometimes you cannot see the forest because you are too busy looking at the trees.”

To account for myself making those mistakes, the first utility that I added was for balanced suppressions. The rules are simple. Except for too-many-lines, any disable must be followed by an enable. Disabling something that is already disabled is not allowed, as well as enabling something that is already enabled. And yes, I have done each of those before, at least five times each… this month. The utility simple loads up each Python file, parses for suppression lines, and looks for regions that do not follow those rules. If any mismatch is found, an error is reported. Simple.

Now, I do realize that this kind of utility might seem silly to other developers, but for me it is about my confidence in my work. After I run that utility, I am confident that my suppressions are properly balanced. No guesswork involved. To me, that confidence is worth a bit of code and investment.

Generating Reports¶

With that utility completed, the next utility that I added was to report on the suppressions that exist in the code. As I measure quality on a sliding scale, it was important for me to know where I stood with PyLint suppressions in my various projects. To be clear, I do not believe there is any concrete “thou must have” or “thou must not” have rules in a code base. My thoughts are that “should have” things increase the quality and “should not have” things decrease the quality. As such, keeping track of the suppressions in the project helps me track where any suppression is on that scale.

This utility was simple to write, especially after my work in the previous utility to make verify that each set of suppressions are balanced. It ended up being a few lines of code added to the tracking of each disabled suppression in the code, keeping a running total of how many times the utility encountered each suppression and where that suppression was encountered. After that, it was just throwing the data into a JSON object and writing the file somewhere.

An example of that report file for the PyLint_Utils project itself is currently as follows:

{
    "disables-by-file": {
        "pylint_utils/__init__.py": {},
        "pylint_utils/__main__.py": {},
        "pylint_utils/file_scanner.py": {},
        "pylint_utils/main.py": {
            "broad-except": 1,
            "too-many-arguments": 1,
            "too-many-locals": 1
        },
        "pylint_utils/pylint_comment_scanner.py": {},
        "pylint_utils/simple_logging.py": {},
        "pylint_utils/version.py": {}
    },
    "disables-by-name": {
        "broad-except": 1,
        "too-many-arguments": 1,
        "too-many-locals": 1
    }
}

I use this report as more of a sanity check than a guide to refactoring. The JSON object that I write includes a disables-by-file section as well as a disables-by-name section. If something looks weird in the disables-by-name section, I can quickly look to see where the modules that use that section are and look at each one individually. From there, it is a judgement call on my part as to whether I should refactor.

The benefit of this utility to me is in having that information presented to me in a clear format that informs on my judgement calls. And removing some of the guesswork from that judgement call is a win in my books.

Finding Unused Suppressions¶

This is the interesting utility for me. When I do decide to take those steps to refactor a function, I want to have something simple to tell me if I have mitigated the need for a suppression. From experience, even if there is another reason for the refactoring, it typically will include a desire to reduce some manner of suppressions. And while I can run PyLint again and not see the warning, PyLint does not tell me if a suppression already in the code base is no longer needed.

Logically following that, I ended up writing something simple to rescan files with individual suppressions disabled. It is a bit terse, but it works. The utility simply takes that file, rewrites it without a given pair of suppression statements, and verifies that the warning is still emitted. If it is not emitted, then the suppression is no longer needed.

And for any readers of this blog, the benefit of that to me should be obvious. I love keeping my code clean!

Why Talk About It Now?¶

Given that information about the utilities I created, I noticed one thing about how I was using those utilities: I was using them to track PyLint suppressions and keep my usage of those suppression in check. Specifically, it occurred to me that since I am tracking the number of tests and code coverage for those tests, it just made sense to me to track the PyLint suppressions in an equivalent manner.

To roll things back a bit, let me start at the beginning. Collecting information about the tests that were run and their execution states was an easy decision for me. By using the Project Summarizer project, I can make sure to keep track of how many tests are in place and if any of them are failing. While I have always made sure all new tests and old tests were passing before committing code, test-driven development allows for committing failing tests that get enabled when they pass. I just work in small increments and use skip statements to achieve the same effect without failing tests. Just a small personal change to test-driven development that gets the same results.

Then there is the matter of collecting information about the code coverage metrics for those tests. Once again, the Project Summarizer project allows me to track the coverage percentage on two distinct levels (lines and branches) to ensure that the percentage is a healthy one for the project. While my absolute baseline percentage for a released project is 75%, I can drive that number up into the high 90s without that much effort. And as most of the projects that I work on are lower-level projects, it makes sense to me to ensure that code coverage percentages are more than 95%. For me, if I will not accept a package into any of my projects without decent testing or decent code coverage, I need to be showing that same respect to users of my projects.

And that brings me to the PyLint Utilities project. Both the first and third purposes of these utilities, ensuring balanced suppressions and finding unused suppressions, are actions that are taken to improve the projects. But generating reports, the second purpose, is one that fits in very nicely like the two existing plugins for the Project Summarizer project.

Tell Me More¶

Why is that? I believe I have said before that it is more important to track quality in a project than to get it to a “golden” state. Noting that a project only has a handful of tests is more important than immediately adding new tests. The most obvious reason for that is that it is best to understand what needs to be tested before adding those new tests.

That is where noting the code coverage for a project comes into play. I have joined teams with no quality measurement in place. The first thing I say to that team is not “shame, no code coverage”, but “we need to know what we are dealing with in order to move forward”. Besides coming across as a jerk with the first suggestions, it will [reduce their desire to] improve the code coverage. In addition, there is a chance that their code coverage is already decent for their project and the stage it is at. Only after doing a good analysis of the project can things move forward.

Which brings me to PyLint suppressions. As I have mentioned before, I use these suppressions to effectively confirm that I have seen the warning and know about it. To me, that warning itself is not a big thing, but just a thing. What concerns me is a pattern of suppressing the warnings without trying to improve on the code to make those warnings not needed. This philosophy merges in very well with my philosophy on code coverage. It is my belief that my comments in the previous paragraph about code coverage, knowing what we are dealing with, and moving forward are as equally applicable to suppressions as to code coverage.

From those similarities, it should be easy to see why I want to add a plugin for the Project Summarizer project to report this. I mean, if it is good enough for code coverage, it should be good enough for suppressions.

What Is Next?¶

Now that I have gone through the new utility project that I have just made public, next week I am going to start providing plugin integration to the Project Summarizer project. Stay tuned!

Like this post? Share on: Twitter ❄ Facebook ❄ Email