[Python-projects] pyLint: Speeding up pyLint

[Nicolas Chauvat]

Julian,

On Fri, May 29, 2009 at 11:11:07PM +1200, Julian Scheid wrote:
> I've also patched pylint today to use the multiprocessing module in
> order to parallelize the checker runs. That turned out to be trivial
> and while it doesn't provide a linear speed-up (as is to be expected),
> it does improve performance significantly on multicore machines.

Nice!

> I'll tidy up the code and send over a patch soon.

Please use `hg bundle` or publish you hg repository somewhere so that
we can pull from it.

> Here's another idea that occurred to me. If checkers could be
> categorized into those that only work locally on a single file (e.g.
> check number of statements per function) and those that need
> information from other files (e.g. method not overridden) then for
> files that haven't changed between pylint runs (usually the majority
> of files) only the latter would have to be run. Unfortunately, from a
> quick look at the checkers it looks like local/global checks are often
> intermingled in the same source file. Still, this could be one
> approach to pursue. Thoughts?

If I recall correctly, the motivation for mixing several checks was to
speed things up...

> Thanks again for these pointers. I am acquainted with CI but I wasn't
> aware of Hudson and the pylint integration via the Violations plug-in
> yet. I'm playing with it at the moment and it works very well. I'll
> check out apycot next week.

You look like you will be doing some work on pylint, etc. Please mail
me in private if you want an account on logilab.org so that you can
add tickets, etc.

-- 
Nicolas Chauvat

logilab.fr - services en informatique scientifique et gestion de connaissances