Caseless (case insensitive) Dictionaries

I had a need for a case insensitive dictionary, started out subclassing the dict class, then I discovered DictMixin. DictMixin is awesome! It lets you define the bare minimum to implement a dictionary, however if you just do that some of your operations might be slower than necessary, to speed your dict up you should write these functions too: __contains__, __iter__, iteritems.

import UserDict
class bareminimumoverloads(UserDict.DictMixin):
def __getitem__(self, key):
pass
#end def

def __setitem__(self, key, value):
pass
#end def

def __delitem__(self, key):
pass
#end def

def keys(self):
pass
#end def
#end class


I wrote two types of case insensitive dictionaries, they behave a little differently, you can modify them to suit your needs, here they are:


#locals
import UserDict

class caseless(UserDict.DictMixin):
def __init__(self): # could probably allow a normal dict to be passed in, and call self.update(adict)
self.caseless = True
self.m_dict = {}
#end def

def __getitem__(self, key):
"""Retrieve the value associated with 'key' (in any case)."""
return self.m_dict[key.lower()][1]
#end def

def __setitem__(self, key, value):
"""Associate 'value' with 'key'. If 'key' already exists, but
in different case, it will be replaced."""
self.m_dict[key.lower()] = (key, value)
#end def

def __delitem__(self, key):
del self.m_dict[key.lower()]
#end def

def keys(self):
return [v[0] for v in self.m_dict.values()]
#end def

def __contains__(self, key):
return self.m_dict.has_key(key.lower)
#end def

def __iter__(self):
for k,v in self.m_dict.iteritems():
yield v[0]
#end for
#end def

def iteritems(self):
for k,v in self.m_dict.iteritems():
yield v
#end for
#end def

def keys(self):
return [v[0] for v in self.m_dict.values()]
#end def
#end class

def iscaseless(adict):
return isinstance(adict, caselessdict)
#end def

class semi(UserDict.DictMixin):
def __init__(self): # could probably allow a normal dict to be passed in, and call self.update(adict)
self.m_lower = {} # each value is a dictionary of real keys in the real dict
self.m_real = {}
#end def

# case sensitive lookup
def __getitem__(self, key):
"""Retrieve the value associated with 'key' (in any case)."""
return self.m_real[key]
#end def

def __setitem__(self, key, value):
"""Associate 'value' with 'key'. If 'key' already exists, but
in different case, it will be replaced."""
keylower = key.lower()

# update the lower dictionary
if keylower in self.m_lower:
self.m_lower[keylower][key] = value
else:
self.m_lower[keylower] = {}
self.m_lower[keylower][key] = value
#end if

# update the real dictionary
self.m_real[key] = value
#end def

def __delitem__(self, key):
keylower = key.lower()
realkeys = self.m_lower[keylower]
for k in realkeys:
del self.m_real[k]
#end if
del self.m_lower[keylower]
#end def

def keys(self):
return self.m_real.keys()
#end def

def __contains__(self, key):
return key.lower() in self.m_lower
#end def

def __iter__(self):
return self.m_real.__iter__()
#end def

def iteritems(self):
return self.m_real.iteritems()
#end def
#end class

class semi2(semi):

# This version does a case insensitive lookup
def __getitem__(self, key):
"""Retrieve the value associated with 'key' (in any case)."""
# first try to look up the real key
if key in self.m_real:
return self.m_real[key]
else:
keylower = key.lower()
realkeys = self.m_lower[keylower]
for k in realkeys:
result = self.m_real[k]
break
#end for
#end if
return result
#end def
#end def

python -i

I used to write modules and copy and paste them into the python interactive shell to play around with them, see how they work see how I can use them, etc.

I recently discovered this option to python:

-i : inspect interactively after running script, (also PYTHONINSPECT=x)
and force prompts, even if stdin does not appear to be a terminal


this loads a module and then gives you the shell to play around.

I still do paste into the python shell to check for whitepsace, it is VERY non forgiving and I have caught bugs because of this pickyess.

New Laptops SUCK!

This is going to be a rant about all the retarded things about new laptops.

High gloss screens - sure they look fancy, but don't try doing any real work with a window behind you.

"Fancy" volume adjustments - My fancy shmancy HP dv7-1260us has "buttons" (in quotes because they aren't buttons, they are flush with the case, likely using the same technology as a touchpad to know when they are touched). Anyway these buttons obviously do not physically affect hardware, they require software to do their job. So when I'm remote desktop'd to another machine and press the mute "button" it doesn't actually mute my laptop, it doesn't do anything. Same with the volume control... I long for the days of a volume dial that directly affects the hardware of the laptop you're sitting at.

Keyboard layouts - They're all different, some have enter keys that are taller than they are wide, some are wider than they are tall, some swap the function key with the control key. Before you buy a laptop make sure you're going to enjoy typing on it, and that its the same or very similar to the keyboard you normally use.

Touchpads - The old school Dell's had the best touchpad. It was indented so your fingers could feel the edge of the touchpad, some touchpads are flush with the case so your finger will drift off of it and the only way for you to know that you're not on the touchpad anymore is the fact that the mouse cursor is no longer moving, this is just plain retarded.

Mouse Buttons - Most laptops only have 2 mouse buttons, how hard would it be to put in a middle one? I'd pay $100 more for a middle mouse button, and $100 more on top of that for 2 more mouse buttons to go forward and back. And $100 more for a mouse wheel. My $5 mouse has all of these things, why not a laptop? I'm paying good money for a laptop I'm going to have for a while, I want it to be usable as possible, I write code all day, I'll gladly get a lower performance laptop and pay more if the human-computer interface speeds up my work. Hell, I'd pay 2x the price of what it SHOULD cost if the physical interface was flawless.

Mouse Buttons - Mine are really hard to click, I have to use a lot of force to click them, this is going to cause unnecessary stress on my forearms, don't laugh, repetitive stress injury is serious and can seriously mess up your life if you are coding for 12 hours a day, its happened to me, I have to be careful with this stuff.

Touchpad / Buttons Closeness - I'm referring to how close the mouse buttons are to the touchpad. Some laptops are totally flush, this is RETARDED because you will be moving your finger down on the touchpad and since there is no distinct separation between the touchpad and the buttons your finger will glide onto the buttons and you will inadvertently click. Others, like mine there is too much separation, my mouse pad is indented, which I love, but the ridge should only be a millimeter thick, and then the buttons should be right there. My buttons are about a centimeter from the touchpad, this is too far.

All in all, I am more satisfied with my HP dv7-1260us than I was with the previous 3 laptops I tried and returned before this due to one or more of the above reasons.

If anyone from Lenovo reads this, if your function key wasn't swapped with your control key you would have had a sale and a very happy customer I think. The control key should be on the far bottom left as it is used more frequently and therefore should be easier to find by feel.

Python Tuple and List Comprehension

As far as I know python does not support tuple comprehension.

For any readers not familiar with list comprehension, run this:

c = [i for i in range(10)]

It's a short hand way to create lists, if you currently would have done this:

c = []
for i in range(10):
  c.append(i)
#end for

STOP!

You are wasting valuable time and brain power, learn list comprehensions, they will be foreign at first (when you're doing complicated ones), but after struggling through it a few times they will become second nature.

Now as far as tuple comprehension, here is my gripe: Python can't do it (edit: read the end of this post to see otherwise)!

You have to do this:

c = tuple([i for i in range(10)])


But here is my real gripe... people that think they know all the ways people are using a given toolset, read this posting from a mailing list about tuple comprehensions:

From http://mail.python.org/pipermail/python-list/2002-April/139269.html

[quote]
.... Even if we *could* make the parser smart
enough to recognize a tuple-comp, it would be confusing to read,
at least IMO.

I also agree with Dave Brueck, that the cases in which tuples
hold uniform values (so that tuple comprehensions might make
sense) are relatively rare, compared to cases in which tuples
hold heterogeneous values. (This is exactly the opposite case of
lists, which may be heterogeneous but are usually homogenous.)
So ISTM that tuple comprehensions would be of limited utility.
[/quote]

The difference between tuples and lists is that you can't change tuples, once they're made, thats it, can't add, can't remove, can't even set an existing element equal to something else.

So that's the real reason why tuple comprehension doesn't make sense, when you expand it (in your head) it looks like this:

c = ()
for i in range(i):
  c.append(i)
#end for

And you can't append to a tuple, but anyway, the parser COULD be smart enough to expand it like this:

c = (0,1,2,3,4,5,6,7,8,9)

However this could only be done by the parser on very simple tuple comprehensions.

I'm not saying if it SHOULD or SHOULDN'T, just that it COULD.

Here is a slightly more complicated list comprehension, that I wish I could have done tuple comprehension on:

Background: on a mysqldb cursor object...
cursor.execute('select id from table')
all = cursor.fetchall()
print all
((1L,), (3L,), (4L,))

Ok, so we have a tuple of one length tuples contain the ids... I want to change this to just a tuple of ids, ie: (1,3,4)

so I do list comprehension and then create a tuple from the list:
c = tuple([id[0] for id in all])

There you go, a very real world problem where tuple comprehension would have been very nice.

Personally, I think if tuple comprehension was ever introduced to the language that it should actually use this syntax: tuple(i for i in range(10)), or tuple([i for i in range(10)]) (although this might be slightly misleading, it shouldn't break existing code and offer a performance boost). You couldn't use (i for i in range(10)) because its already used to create a generator.

-----

This is hilarious, after I wrote that, I thought, maybe the people who add features to python already have done this? Since that mail list post I referenced is from 2002 or something... guess what, looks like tuple(i for i in range(10)) works, so there you have it, tuple comprehension IS supported!

I'm going to leave this post as is, as a warning to all you know-it-alls who think, retarded things tuple generation would have "limited utility"

Python Tabs

I use tabs, real tabs, and only tabs for indenting.

If I download code I convert it, I hate spaces.

I write my code in windows, I use PSPad, I always have View | Special Chars enabled, so my code looks like this:

def filetolist(filename, clean = True):
->result = []
->f = open(filename, 'r')
->for line in f.xreadlines():
->->removedn = False
->->if clean:
->->->stripped = line.strip()
->->->if len(stripped) > 0:
->->->->result.append(stripped)
->->->#end if
->->else:
->->->if len(line) > 0 and line.endswith('\n'):
->->->->line = line[:-1] # remove 1 char
->->->->removedn = True
->->->#end if
->->->result.append(line)
->->#end if
->#end for
->if not clean and removedn:
->->result.append('')
->#end if
->return result
#end def

The only reason I use tabs, and hate spaces with a passion because every now and then I find myself editing a .py file in a text editor that does not support "smart" tabs, and when I used to spaces, it was the most tedious thing in the world to work on that .py file. Now when I find myself using a "dumb" editor, like notepad, or nano, the tabs are HUGE, and thats fine because I'll take the clarity of seeing my code blocks AND being able to manipulate code much quicker by tabbing quickly to my indentation points as opposed to holding down the space bar, and then deleting the few extra spaces I had put in any day.

And guess what, every "smart" text editor out there can support and display tabs any way you like, so you don't even have to know you're using them.

I know I should get off PSPad and start using a real Python editor, but I haven't had the time to focus on it, and I don't like doing anything half assed, so when I research my IDEs I want the best one for me.

I use PyLint, its brilliant. Its a pain to get going on windows but worth it.

Pylint scores my code very low, a negative number out of 10! But I'm not a fan of its default style, so I'm happy with that.

I run pylint -e filename.py it can't find all errors that are normally found at compile time but it does a very impressive job.

Python: Reversing a string, Replacing text in reverse

This blog is going to be filled with a lot of quick little hints, mostly for myself but others might find them useful too.

Strings in python don't have a built in (simple) way to reverse them, this is how its done:

'1234'[::-1]


And here are 2 functions, one so you don't have to remember how to reverse a string, and another to help with replacing in reverse.

def rev(s):
 return str(s)[::-1]
#end def

def replacerev(s,old,new,count=-1):
 "use count=1 to replace the last match of old with new in s, count = 2 for 2 replaces, and so on"
 return rev( rev(str(s)).replace(rev(old),rev(new),int(count)) )
#end def

rev('1234')
replacerev('FOO BAR FOO','FOO','QUUX',1)
replacerev('FOO BAR FOO','FOO','QUUX',2)

Python: How to paste all your code into the python console??

A beautiful thing about python is that it eliminates the need for the language to need characters to specify blocks of code, it does this with white space.

An incredibly annoying thing about python (when you're starting out) is that it uses whitespace to denote blocks of code.

Have you ever copied and pasted python code from a web page and had to reformat it properly? And was it annoying and a bitch, and even IMPOSSIBLE to do, without knowing what the logic was in the first place?

If you're starting out at python you have probably played around quite a bit in the interrupter to learn it, but when you started writing python code in files you wrote little programs to test the functions you're writing, which is great, but more on unit testing in another post.

Alot of python scripts I write are quick, dirty and I don't care about writing unit tests, I just want it working ASAP, so I save the code to a file, and then what? What is the quickest way to test it, PASTE THE ENTIRE FILE CONTENTS INTO THE PYTHON INTERPRETER, then just run the function you want to test to your heart's content.


Here is some sample code of mine, notice the comments denoting an ending of a block, I also use tabs, which will not be preserved when you copy and paste this out of your browser, go ahead, copy and paste it into your python interpreter, I even included 2 lines of code runing the function for you at the end.

def filetolist(filename, clean = True):
 result = []
 f = open(filename, 'r')
 for line in f.xreadlines():
  removedn = False
  if clean:
   stripped = line.strip()
   if len(stripped) > 0:
    result.append(stripped)
   #end if
  else:
   if len(line) > 0 and line.endswith('\n'):
    line = line[:-1] # remove 1 char
    removedn = True
   #end if
   result.append(line)
  #end if
 #end for
 if not clean and removedn:
  result.append('')
 #end if
 return result
#end def



a = filetolist('C:\\WINDOWS\\win.ini')
print a

The beautiful thing about this is you can paste an entire file into the python console, not to mention I can paste my code anywhere and not worry about html removing the visible spacing, for example I didn't include it in a pre tag here and the logic of the code is not lost due to the #end comments.


def filetolist(filename, clean = True):
result = []
f = open(filename, 'r')
for line in f.xreadlines():
removedn = False
if clean:
stripped = line.strip()
if len(stripped) > 0:
result.append(stripped)
#end if
else:
if len(line) > 0 and line.endswith('\n'):
line = line[:-1] # remove 1 char
removedn = True
#end if
result.append(line)
#end if
#end for
if not clean and removedn:
result.append('')
#end if
return result
#end def


It's still somewhat readable. As opposed to this, without the comments denoting the end of the blocks, just try to figure out what the heck is going on:



def filetolist(filename, clean = True):
result = []
f = open(filename, 'r')
for line in f.xreadlines():
removedn = False
if clean:
stripped = line.strip()
if len(stripped) > 0:
result.append(stripped)
else:
if len(line) > 0 and line.endswith('\n'):
line = line[:-1] # remove 1 char
removedn = True
result.append(line)
if not clean and removedn:
result.append('')
return result



One catch when writing code you want to paste into the python console, beware of blank lines, I purposefully left blank lines in here so you can paste it into your python interpreter and see the errors.

def filetolist(filename, clean = True):
 result = []
 f = open(filename, 'r')
 for line in f.xreadlines():
  removedn = False

  if clean:
   stripped = line.strip()
   if len(stripped) > 0:
    result.append(stripped)
   #end if

  else:

   if len(line) > 0 and line.endswith('\n'):
    line = line[:-1] # remove 1 char
    removedn = True
   #end if
   result.append(line)
  #end if
 #end for
 
 if not clean and removedn:
  result.append('')
 #end if
 
 return result
#end def

You can have blank lines, but the spacing has to be correct even for the blank lines, just remember the spaces denote the blocks of code, as soon as the spaces "de-dent" (opposite of indent) the block goes with it. Of course the actual python interpretter is much better at reading files, I've seen some really poorly formatted code be read properly.

I mentioned earlier I use tabs for spacing, more on this in another post.

Here I did the blank lines properly, preserving the block.

def filetolist(filename, clean = True):
 result = []
 f = open(filename, 'r')
 for line in f.xreadlines():
  removedn = False
  
  if clean:
   stripped = line.strip()
   if len(stripped) > 0:
    result.append(stripped)
   #end if
   
  else:
   
   if len(line) > 0 and line.endswith('\n'):
    line = line[:-1] # remove 1 char
    removedn = True
   #end if
   result.append(line)
  #end if
 #end for
 
 if not clean and removedn:
  result.append('')
 #end if
 
 return result
#end def