Comment

Comment on Python Performance: Why 'if not list' is 2x Faster Than Using len()

thebestaquaman@lemmy.world ⁨6⁩ ⁨days⁩ ago

I write a lot of Python. I hate it when people use “X is more pythonic” as some kind of argument for what is a better solution to a problem. I also have a hang up with people acting like python has any form of type safety, instead of just embracing duck typing.This lands us at the following:

The article states that “you can check a list for emptiness in two ways: if not mylist or if len(mylist) == 0”. Already here, a fundamental mistake has been made: You don’t know (and shouldn’t care) whether mylist is a list. These two checks are not different ways of doing the same thing, but two different checks altogether. The first checks whether the object is “falsey” and the second checks whether the object has a well defined length that is zero. These are two completely different checks, which often (but far from always) overlap. Embrace the duck type- type safe python is a myth.

source

Sort:hotnew top

iAvicenna@lemmy.world ⁨6⁩ ⁨days⁩ ago
isn’t the expected behaviour exactly identical on any object that has len defined:

“By default, an object is considered true unless its class defines either a bool() method that returns False or a len() method that returns zero, when called with the object.”

source
- CompassRed@discuss.tchncs.de ⁨6⁩ ⁨days⁩ ago
  It’s not the same, and you kinda answered your own question with that quote. Consider what happens when an object defines both dunder bool and dunder len. It’s possible for dunder len to return 0 while dunder bool returns True, in which case the falsy-ness of the instance would not depend at all on the value of len
  
  source
- thebestaquaman@lemmy.world ⁨6⁩ ⁨days⁩ ago
  Exactly as you said yourself: Checking falsieness does not guarantee that the object has a length. There is considerable overlap between the two, and if it turns out that this check is a performance bottleneck (which I have a hard time imagining) it can be appropriate to check for falsieness instead of zero length. But in that case, don’t be surprised if you suddenly get an obscure bug because of some custom object not behaving the way you assumed it would.
  
  I guess my primary point is that we should be checking for what we actually care about, because that makes intent clear and reduces the chance for obscure bugs.
  
  source
sugar_in_your_tea@sh.itjust.works ⁨5⁩ ⁨days⁩ ago

type safe python is a myth

Sure, but type hints provide a ton of value in documenting for your users what the code expects. I use type hints everywhere, and it’s fantastic! Yes, there’s no guarantee that the types are correct, but with static analysis and the assumption that your users want their code to work correctly, there’s a very high chance that the types are correct.

That said, I lie about types all the time. For example, if my function accepts a class instance as an argument, the intention is that the code accept any class that implements the same methods as the one I’ve defined in the parameter list, and you don’t necessarily have to pass an instance of that class in (or one of its sub-classes). But I feel like putting something reasonable in there makes a lot more sense than nothing, and I can clarify in the docstring that I really just need something that looks like that object. One of these days I’ll get around to switching that to Protocol classes to reduce type errors.

That said, I don’t type hint everything. A lot of private methods and private functions don’t have types, because they’re usually short and aren’t used outside the class/file anyway, so what’s the point?

source
- thebestaquaman@lemmy.world ⁨5⁩ ⁨days⁩ ago
  Type hints are usually great, as long as they’re kept up to date and the IDE interprets them correctly. Recently I’ve had some problems with PyCharm acting up and insisting that matplotlib doesn’t accept numpy arrays, leading me to just disable the type checker altogether.
  
  All in all, I’m a bit divided on type hints, because I’m unsure whether I think the (huge) value added from correct type hints outweighs the frustration I’ve experienced from incorrect type hints. Per now I’m leaning towards “type hints are good, as long as you never blindly trust them and only treat them as a coarse indicator of what some dev thought at some point.”
  
  source
  - sugar_in_your_tea@sh.itjust.works ⁨5⁩ ⁨days⁩ ago
    
    leading me to just disable the type checker altogether.
    
    The better option is to just put # type: ignore on the statements where it gets confused, and add hints for your code. I’ve done that for SQLAlchemy before they got proper type hinting, and it worked pretty well.
    
    That said, a type hint is just that, a hint. It shouldn’t be relied on to be 100% accurate (i.e. lots of foo: list should actually be foo: list | None), but if you use a decent static analysis tool, you should catch the worst of it. We use pyright, which is built in to the VSCode extension pylance. It works incredibly well, though it’s a bit too strict in many cases (e.g. when things can be None but generally aren’t).
    
    So yeah, never blindly trust type hints, but do use them everywhere. The more hints you have, the more the static analysis can help, and disabling them on a case-by-case basis is incredibly easy. You’ll probably still get some runtime exceptions that correct type checking could have caught, but it’s a lot better than having a bunch of verbose checks everywhere that make no sense. A good companion to type checks is robust unit test cases with reasonable data (i.e. try to exercise the boundaries of what users can input).
    
    As it stands, we very rarely get runtime exceptions due to poor typing because our type hints are generally pretty good and our unit test cases back that up. Don’t blindly trust it, and absolutely read the docs for anything you plan to use, but as long as you are pretty consistent, you can start making some assumptions about what your data looks like.
    
    source
    thebestaquaman@lemmy.world ⁨5⁩ ⁨days⁩ ago
    I really do agree on all your points, so at the end of the day I think a lot comes down to use-case and personal preference.
    
    My primary use cases for Python are prototyping and as a frontend/scripting tool for software written in C/C++/Fortran. I’ve written/worked on only one larger code base in pure Python, and my personal opinion became that I heavily prefer strictly typed languages once the code base exceeds a certain size. It just feels so much smoother to work with when I have actual guarantees that are enforced by the language.
    
    With that said, we were a bunch of people that are used to using Python for prototyping that developed this larger library, and it would probably have gone a lot better if we actually enforced use of proper type hinting from the start (which we were not used to).
    
    source
    -> View More Comments