[Python-projects] pyLint: Speeding up pyLint

[Julian Scheid]

Hello,

first of all, many thanks for making pyLint available.

I'm working test-driven and I have most of my code covered by tests.
Still, pyLint has proved invaluable to me by catching the odd error in
a corner case that I forgot to add a test for.

The code for my current project is mission critical and therefore I am
running pyLint --errors-only before every commit. This works fine and,
as I said, has already saved my b*tt a couple of times.

The only problem with this is that pyLint is comparatively slow. In my
project, the unit tests take around 5s to run, pyLint takes 20s.

I understand that pyLint has to do extensive analysis of the source
code, type inference and checking hundreds of rules so this is fair
enough. But I am wondering if there is anything I could do - both as a
pyLint user and as a Python developer - to make pyLint run faster.

I tried playing with the --cache-size parameter but that doesn't seem
to help. So I wonder if pyLint code could be improved to speed things
up. For instance, I suspect that pyLint is mostly CPU-bound but runs
single-threaded, i.e. doesn't make use of multiple CPU cores - is this
correct?

Also, I assume that it will parse all third-party libraries used in my
project (there are quite a few) every time I run it. These libraries
are only updated infrequently, would it be possible to cache any
parsing/type inference/etc. pyLint is doing for these libraries?

My understanding of the pyLint codebase is currently zero, but I'd
like to offer my help if you don't have the time to look into
improving pyLint performance.

Thanks in advance,

Julian