[Python-projects] Check imported modules attributes

[Sylvain Thénault]

On Monday 20 June à 13:40, amaury.forgeotdarc at ubitrade.com wrote:
> Hello,

Hi,
 
> I just downloaded PyLint this morning, and tried it on my code.
> This tool is really impressive... Nice job!
> My first impression is that I have a lot of work to get a note that is not
> negative...

Well, pylint is a bit maniac ;)
 
> Then, I tried to make it detect some errors in my code,
> and I was surprised that PyLint doesn't detect simple (IMO)
> errors like
> 
>     import sys
>     sys.this_is_an_error
> 
> 
> So I hacked a bit, and came with an additional test, that checks
> attribute access: for each GetAttr node, if the left-hand side
> is a module, the right-hand name must belong to the module.
> 
> Here are my mods to the checkers package:
 
[snip patch]

That's really a valuable patch, many thanks ! I've checked in a modified
version so that imports themselves are checked too (ie handle From and
Import nodes).
 
> OK, this is straightforward and almost works.
> The difficult part is that the tests relies on the imported modules
> to be parsed  correctly. When using my new test on PyLint's code itself,
> I saw it report errors on missing "sys.stdout" and "re.finditer"...
> 
> The first one was easy to find: when building the astng from a living
> module,
> functions, classes, and descriptors are taken, but not "regular" variables.
> Here is a quick hack:
> 
> [in logilab/common/astng/builder.py, at the end of object_build:]
>             else:
>                 # All other objects: just give a name
>                 from compiler.ast import EmptyNode
>                 data = EmptyNode() # XXX is it the right node?
>                 data.name = name
>                 # Keeping a reference to any object may have unexpected
> results...
>                 # data.value = member
>                 node.add_local_node(data)

checked in too :)

> For "re.finditer", I don't have any solution yet. The function is defined
> like this:
> [in sre.py:]
>     if sys.hexversion >= 0x02020000:
>         def finditer(pattern, string):
>             ...
> An idea would be to import the module instead of trying to parse it.
> I know this was removed some releases ago, but I suggest that at least
> standard library modules could be "really" imported.

Hum... Actually pylint should deal with this. The problem is actually more 
hairy: re.py is filled by doing "from sre import *", so pylint parse
this module (sre) to get imported names, considering the __all__ variable. 
The problem is that the "finditer" name is dynamically appended to this
variable, so the astng builder missed it. I still would like to avoid
module imports (today, a module is imported only if there is no source
code available, ie usually built-in / compiled libraries). I'll try to
make a fix so that if the module is already imported, such as the re 
module, we take advantage of it.

> Well, sorry for this long post.
> But I think that this kind of checks would avoid a lot of errors, specially
> when imported modules are to change...

Yep. I would like more checks like this (type checking related), but I'm
missing time to work on this today...

> Thanks again for this tool,

Thanks again for your patch :)

regards
-- 
Sylvain Thénault                               LOGILAB, Paris (France).

http://www.logilab.com   http://www.logilab.fr  http://www.logilab.org