Python Tuple and List Comprehension

As far as I know python does not support tuple comprehension.

For any readers not familiar with list comprehension, run this:

c = [i for i in range(10)]

It's a short hand way to create lists, if you currently would have done this:

c = []
for i in range(10):
  c.append(i)
#end for

STOP!

You are wasting valuable time and brain power, learn list comprehensions, they will be foreign at first (when you're doing complicated ones), but after struggling through it a few times they will become second nature.

Now as far as tuple comprehension, here is my gripe: Python can't do it (edit: read the end of this post to see otherwise)!

You have to do this:

c = tuple([i for i in range(10)])


But here is my real gripe... people that think they know all the ways people are using a given toolset, read this posting from a mailing list about tuple comprehensions:

From http://mail.python.org/pipermail/python-list/2002-April/139269.html

[quote]
.... Even if we *could* make the parser smart
enough to recognize a tuple-comp, it would be confusing to read,
at least IMO.

I also agree with Dave Brueck, that the cases in which tuples
hold uniform values (so that tuple comprehensions might make
sense) are relatively rare, compared to cases in which tuples
hold heterogeneous values. (This is exactly the opposite case of
lists, which may be heterogeneous but are usually homogenous.)
So ISTM that tuple comprehensions would be of limited utility.
[/quote]

The difference between tuples and lists is that you can't change tuples, once they're made, thats it, can't add, can't remove, can't even set an existing element equal to something else.

So that's the real reason why tuple comprehension doesn't make sense, when you expand it (in your head) it looks like this:

c = ()
for i in range(i):
  c.append(i)
#end for

And you can't append to a tuple, but anyway, the parser COULD be smart enough to expand it like this:

c = (0,1,2,3,4,5,6,7,8,9)

However this could only be done by the parser on very simple tuple comprehensions.

I'm not saying if it SHOULD or SHOULDN'T, just that it COULD.

Here is a slightly more complicated list comprehension, that I wish I could have done tuple comprehension on:

Background: on a mysqldb cursor object...
cursor.execute('select id from table')
all = cursor.fetchall()
print all
((1L,), (3L,), (4L,))

Ok, so we have a tuple of one length tuples contain the ids... I want to change this to just a tuple of ids, ie: (1,3,4)

so I do list comprehension and then create a tuple from the list:
c = tuple([id[0] for id in all])

There you go, a very real world problem where tuple comprehension would have been very nice.

Personally, I think if tuple comprehension was ever introduced to the language that it should actually use this syntax: tuple(i for i in range(10)), or tuple([i for i in range(10)]) (although this might be slightly misleading, it shouldn't break existing code and offer a performance boost). You couldn't use (i for i in range(10)) because its already used to create a generator.

-----

This is hilarious, after I wrote that, I thought, maybe the people who add features to python already have done this? Since that mail list post I referenced is from 2002 or something... guess what, looks like tuple(i for i in range(10)) works, so there you have it, tuple comprehension IS supported!

I'm going to leave this post as is, as a warning to all you know-it-alls who think, retarded things tuple generation would have "limited utility"

Python Tabs

I use tabs, real tabs, and only tabs for indenting.

If I download code I convert it, I hate spaces.

I write my code in windows, I use PSPad, I always have View | Special Chars enabled, so my code looks like this:

def filetolist(filename, clean = True):
->result = []
->f = open(filename, 'r')
->for line in f.xreadlines():
->->removedn = False
->->if clean:
->->->stripped = line.strip()
->->->if len(stripped) > 0:
->->->->result.append(stripped)
->->->#end if
->->else:
->->->if len(line) > 0 and line.endswith('\n'):
->->->->line = line[:-1] # remove 1 char
->->->->removedn = True
->->->#end if
->->->result.append(line)
->->#end if
->#end for
->if not clean and removedn:
->->result.append('')
->#end if
->return result
#end def

The only reason I use tabs, and hate spaces with a passion because every now and then I find myself editing a .py file in a text editor that does not support "smart" tabs, and when I used to spaces, it was the most tedious thing in the world to work on that .py file. Now when I find myself using a "dumb" editor, like notepad, or nano, the tabs are HUGE, and thats fine because I'll take the clarity of seeing my code blocks AND being able to manipulate code much quicker by tabbing quickly to my indentation points as opposed to holding down the space bar, and then deleting the few extra spaces I had put in any day.

And guess what, every "smart" text editor out there can support and display tabs any way you like, so you don't even have to know you're using them.

I know I should get off PSPad and start using a real Python editor, but I haven't had the time to focus on it, and I don't like doing anything half assed, so when I research my IDEs I want the best one for me.

I use PyLint, its brilliant. Its a pain to get going on windows but worth it.

Pylint scores my code very low, a negative number out of 10! But I'm not a fan of its default style, so I'm happy with that.

I run pylint -e filename.py it can't find all errors that are normally found at compile time but it does a very impressive job.

Python: Reversing a string, Replacing text in reverse

This blog is going to be filled with a lot of quick little hints, mostly for myself but others might find them useful too.

Strings in python don't have a built in (simple) way to reverse them, this is how its done:

'1234'[::-1]


And here are 2 functions, one so you don't have to remember how to reverse a string, and another to help with replacing in reverse.

def rev(s):
 return str(s)[::-1]
#end def

def replacerev(s,old,new,count=-1):
 "use count=1 to replace the last match of old with new in s, count = 2 for 2 replaces, and so on"
 return rev( rev(str(s)).replace(rev(old),rev(new),int(count)) )
#end def

rev('1234')
replacerev('FOO BAR FOO','FOO','QUUX',1)
replacerev('FOO BAR FOO','FOO','QUUX',2)

Python: How to paste all your code into the python console??

A beautiful thing about python is that it eliminates the need for the language to need characters to specify blocks of code, it does this with white space.

An incredibly annoying thing about python (when you're starting out) is that it uses whitespace to denote blocks of code.

Have you ever copied and pasted python code from a web page and had to reformat it properly? And was it annoying and a bitch, and even IMPOSSIBLE to do, without knowing what the logic was in the first place?

If you're starting out at python you have probably played around quite a bit in the interrupter to learn it, but when you started writing python code in files you wrote little programs to test the functions you're writing, which is great, but more on unit testing in another post.

Alot of python scripts I write are quick, dirty and I don't care about writing unit tests, I just want it working ASAP, so I save the code to a file, and then what? What is the quickest way to test it, PASTE THE ENTIRE FILE CONTENTS INTO THE PYTHON INTERPRETER, then just run the function you want to test to your heart's content.


Here is some sample code of mine, notice the comments denoting an ending of a block, I also use tabs, which will not be preserved when you copy and paste this out of your browser, go ahead, copy and paste it into your python interpreter, I even included 2 lines of code runing the function for you at the end.

def filetolist(filename, clean = True):
 result = []
 f = open(filename, 'r')
 for line in f.xreadlines():
  removedn = False
  if clean:
   stripped = line.strip()
   if len(stripped) > 0:
    result.append(stripped)
   #end if
  else:
   if len(line) > 0 and line.endswith('\n'):
    line = line[:-1] # remove 1 char
    removedn = True
   #end if
   result.append(line)
  #end if
 #end for
 if not clean and removedn:
  result.append('')
 #end if
 return result
#end def



a = filetolist('C:\\WINDOWS\\win.ini')
print a

The beautiful thing about this is you can paste an entire file into the python console, not to mention I can paste my code anywhere and not worry about html removing the visible spacing, for example I didn't include it in a pre tag here and the logic of the code is not lost due to the #end comments.


def filetolist(filename, clean = True):
result = []
f = open(filename, 'r')
for line in f.xreadlines():
removedn = False
if clean:
stripped = line.strip()
if len(stripped) > 0:
result.append(stripped)
#end if
else:
if len(line) > 0 and line.endswith('\n'):
line = line[:-1] # remove 1 char
removedn = True
#end if
result.append(line)
#end if
#end for
if not clean and removedn:
result.append('')
#end if
return result
#end def


It's still somewhat readable. As opposed to this, without the comments denoting the end of the blocks, just try to figure out what the heck is going on:



def filetolist(filename, clean = True):
result = []
f = open(filename, 'r')
for line in f.xreadlines():
removedn = False
if clean:
stripped = line.strip()
if len(stripped) > 0:
result.append(stripped)
else:
if len(line) > 0 and line.endswith('\n'):
line = line[:-1] # remove 1 char
removedn = True
result.append(line)
if not clean and removedn:
result.append('')
return result



One catch when writing code you want to paste into the python console, beware of blank lines, I purposefully left blank lines in here so you can paste it into your python interpreter and see the errors.

def filetolist(filename, clean = True):
 result = []
 f = open(filename, 'r')
 for line in f.xreadlines():
  removedn = False

  if clean:
   stripped = line.strip()
   if len(stripped) > 0:
    result.append(stripped)
   #end if

  else:

   if len(line) > 0 and line.endswith('\n'):
    line = line[:-1] # remove 1 char
    removedn = True
   #end if
   result.append(line)
  #end if
 #end for
 
 if not clean and removedn:
  result.append('')
 #end if
 
 return result
#end def

You can have blank lines, but the spacing has to be correct even for the blank lines, just remember the spaces denote the blocks of code, as soon as the spaces "de-dent" (opposite of indent) the block goes with it. Of course the actual python interpretter is much better at reading files, I've seen some really poorly formatted code be read properly.

I mentioned earlier I use tabs for spacing, more on this in another post.

Here I did the blank lines properly, preserving the block.

def filetolist(filename, clean = True):
 result = []
 f = open(filename, 'r')
 for line in f.xreadlines():
  removedn = False
  
  if clean:
   stripped = line.strip()
   if len(stripped) > 0:
    result.append(stripped)
   #end if
   
  else:
   
   if len(line) > 0 and line.endswith('\n'):
    line = line[:-1] # remove 1 char
    removedn = True
   #end if
   result.append(line)
  #end if
 #end for
 
 if not clean and removedn:
  result.append('')
 #end if
 
 return result
#end def