[Python-projects] A first impression of pylint

[Marius Gedminas]

Hello!

Nicolas Chauvat invited me to join this list and share my (not very
good) first impressions about pylint.

A year or two ago I was looking for a Python code checker for a
medium-sized project (~140,000 lines of code) because we've recently been
bitten by the same kind of mistake (copying a unit test, changing the
code, forgetting to change the name and thus inadvertently disabling th
old test).

I looked at pychecker, pylint and pyflakes.

Pychecker was importing Python modules and introspecting objects in
memory, which would not have caught the mistake I wanted to catch, among
other reservations (I'd like a static code checker tool to have no
surprising side effects without trusting the code I'm checking).  There
was a pychecker2 branch what did source code analysis, but it wasn't
readyi then.  (I haven't been keeping track.  Is it ready now?)

Pylint scared me by producing a very verbose report full of ASCII-art
tables and fascist-looking warnings ("your naming conventions don't
match this arbitrary looking regexp!").

Pyflakes was probably the roughest and weakest of the alternatives, but
it was fast (20 seconds to check 140,000 lines of code), it reported
the errors I wanted to see, and the superfluous errors were not too
numerous that I could think about filtering them away in some way (grep
-v, perhaps).

IIRC none of the tools actually could handle our source code correctly
at the time, so I was facing the need to do some modification.  Zope 3
was caught by the doctest fashion by then, and almost all unit tests in
our project were written like this:

    import doctest

    from some.place import FooClass

    def doctest_this_or_that():
        """Test FooClass.barMethod

        We start with a fresh foo

            >>> foo = FooClass()

        and we do unspeakable things to it

            ...

        """

I believe all of the tools mistakenly thought the FooClass import was
unused.  My coworker Ignas Mikalajūnas wrote a patch for pyflakes to
compile and recursively descend into doctest examples while walking the
AST of the module (I'd previously written a tool to more or less
accurately track unused imports in the presence of doctests using this
technique -- http://pypi.python.org/pypi/findimports; that tool was
running on our buildbot machine and emailing reports every night, if it
had something to report, that is).  I also added previously-nonexistent
command line options to suppress some of the warnings by type/filename,
changed the output format to a more editor-friendly

  filename:lineno:message

and hooked it up to a nightly buildbot task that sent email when the
output was not empty.  (The code lives in a public branch on Launchpad,
and I plan to extract the various features as patches and send them to
pyflakes upstream Any Day Now.)

Back to pylint.  I'm pretty sure that it would be possible to achieve
the same with pylint, perhaps by crafting a config file and maybe
writing a plugin.  Tough maybe not the 20-second processing time---I ran
pylint on this codebase right now and it's still not done 14 minutes
later.

Why didn't I even try?  Pyflakes seemed to be simpler.  It's always
easier to add than it is to subtract.  Also, I seem to remember some
scary warnings.  I haven't kept notes at the time, but if I try to run
pyflakes on this codebase today, I see things like

  Exception RuntimeError: 'maximum recursion depth exceeded in __subclasscheck__' in <type 'exceptions.AttributeError'> ignored

and

  .../zope/interface/interface.py:206: RuntimeWarning: Python C API version mismatch for module _zope_interface_coptimizations: This Python has API version 1013, module _zope_interface_coptimizations has version 1012.

the latter being an indication that pylint *does* import user code, which
is something I don't like in my source code analyzers.  Also, I see a
lot of false positives (such as E: No name 'adapts' in module
'zope.component', which is completely false) that probably result from
the Python version mismatch.  Although, when I ran it on a different
(and much smaller project, namely http://mg.pov.lt/restview/), it also
complained about things that do exist:

  $ PYTHONPATH=src/ pylint -e src/restview/restviewhttp.py
  E:246:SyntaxHighlightingHTMLTranslator.visit_Text: Module 'pygments.lexers' has no 'PythonConsoleLexer' member
  E:250:SyntaxHighlightingHTMLTranslator.visit_Text: Module 'pygments.formatters' has no 'HtmlFormatter' member

  $ python
  >>> from pygments.lexers import PythonConsoleLexer
  >>> from pygments.formatters import HtmlFormatter

So there you have it: my first impression and why I did not choose
pylint as my favourite Python source code checker.

Actually, as I keep checking whether that pylint run is finally done, I
realize that speed may have been the decisive factor for my choice.  Our
full suite of unit and functional tests takes 20 minutes, which is an
eternity to me; pylint (yay it's finished at last!) took 17 minutes.

Marius Gedminas
-- 
Remember when the net was smart people sitting in front of dumb terminals?
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
URL: <http://lists.logilab.org/pipermail/python-projects/attachments/20090716/cc3ed61e/attachment.pgp>