Six QA Tools that will make your Python code better

Six QA Tools that will make your Python code better…

Put simply, they are: Black, iSort, PyLint, MyPy, PyTest and Coverage.

When used regularly as part of your code – test – commit – push cycle, these tools combine to improve the quality of your Python code. No more will your team argue about trite matters of coding style freeing up their creativity for issues of design that truly matter.

We shall start by looking at some code fragments that contain common bugs that might have been trapped by the linting process.

Each of the linters brings an implementation overhead. Black and iSort are easy to deploy with their functionality often included in a modern IDE. These tools will change the structure of your code completely removing any white-space noise from your source repo.

PyLint is a static analysis tool scans for ‘code smells’. PyLint will detect issues in your code like “variable not used”, “variable name does not conform to snake_case” and “too many local variables”

MyPy checks your code for compatible types and will report issues in code like my_var = 1 + “2” before the code is executed.

PyTest is a framework that supports writing unit tests for your code and brings many productivity features like test fixtures, and auto created and cleaned up temp folders. To use PyTest well, you will have to structure your code for testability which brings a learning curve. A side benefit is that when your code can be thoroughly exercised by PyTest, it will be easier for a human to maintain.

Coverage is a tool that watches while your PyTest suite executes, and reports lines of code not exercised by your unit tests.

Each tool brings a higher cost of use than the previous one. Some can be executed in the IDE. Others on the desktop via a script, and on the build server.

In our team, we have a strict rule that all code going to prod must have been through all six tools, with 100% of code covered by unit tests.

This rule will not be appropriate in all circumstances, but if your code is going to last a long time, be worked on by a large team, and carries a high cost in the event of errors, they are worth considering.

Peter Aych

Peter has been with AEMO, the Australia Energy Market Operator since soon after market start. The Australian electricity grid has undergone two major transformations: First in the late 1990s when the national market was created, and the states were interconnected. Now the biggest transformation is now underway as we move towards a 100% renewable grid.

There are many software systems that support the electricity grid and market: Pascal, Fortran, C, Oracle and Java in the early days. Python, JavaScript & Vue, C# & C++ today.

Peter’s team supports a suite of tools called AMP – The AEMO Modelling Platform – a ‘digital twins’ of the grid used to predict its behavior, design it into the future, and analyze operation faults.

AMP Features a SQL Server & Python backend running on Windows, with Vue & JavaScript user interface. At the time of writing, we have 25,000 lines of Python code, all linted and unit tested with 100% coverage.

Some of the Pascal code we wrote in the late 90s is still in production delivering value to the industry. We are planning a 20 year or longer life for our Python code so have a somewhat manic focus on quality.