merits of Lisp vs Python

Excerpts from my posts to the comp.lang.python thread about Lisp vs Python:



Date: Fri, Dec 8 2006 10:05 pm

Mark Tarver wrote:

> How do you compare Python to Lisp? What specific advantages do you

> think that one has over the other?



> Note I'm not a Python person and I have no axes to grind here. This is

> just a question for my general education.



> Mark



I've never programmed in Lisp but I have programmed in Cadence Skill a

Lisp inspired language with infix notation as an option. I found Skill

to be a very powerful language. At the time I new only AWK, C, Pascal,

Forth, Postcript, Assembler and Basic. Skill was superior and I came

to love it.

But that was a decade ago. Now, I'd rather a company integrate Python

into their product as I find Python to be less 'arcane' than Skill;

with more accessible power, and a great community.

.

Analogy time!

You need Pure Maths, but more mathematicians will be working applying

maths to real-world problems. You need research physicists, but more

physicists will be applying physics in the real world. It seems to me

that Lisp and its basis in maths makes people research and develop a

lot of new techniques in Lisp, but when it comes to applying those

techniques in the real world - switch to Python!



Lisp has a role to play, but maybe a language tuned to research and

with its user base would naturally find it hard to compete in the roles

in which dynamic languages such as Python are strongest.



- Paddy.



Date: Fri, Dec 8 2006 10:22 pm



What is it about Lisp that despite doing everything first, way before

any other language, people don't stop using anything else and

automatically turn to Lisp? Maybe there is more to this everything than

the Lisp community comprehends.

Maybe Lisp is to science, as Python is to engineering - with a slight

blurring round the edges?



- Paddy.



Date: Sun, Dec 10 2006 1:53 am



NOBODY expects the Lispers Inquisition!

Our chief weapon is age... age and macros...

...macros and age.

Our two weapons are age and macros....

And mathematical rigour...

Our THREE weapons are age, macros, and mathematical rigour...

...And an almost fanatical belief in Lisps superiority.

Our *four* ...no.

AMONGST our weapons...

Amongst our weaponry...

...Are such elements as fear, surprise.... I'll come in again.



Python is fun to use.

Easy to read.

Simple and powerful.

Easy to test.

Easy to maintain.

Fast. Very fast!



- Paddy.



Date: Sun, Dec 10 2006 1:20 am



Paul Rubin wrote:

> "mystilleef" writes:

> > Slow for users who aren't familiar with Psyco, Pyrex and C extensions,

> > sure.



> Anyway it's pretty lousy advocacy for a language to say "well if the

> language is too slow, don't use it, use another langauge like C instead".



Python can be used as a glue language. It is not solely a glue

language.

A lot of people find using Python to script libraries written in other

languages

a way to get things done. Ask the scipy guys or the biopython guys.

The Python community actively encourages groups writing useful

libraries

to maintain a Python port, or Python users might wrap libraries

themselves.



You don't always wrap a module in Python for reasons of speed of

execution.

Software testing may well be easier to do in Python than in the

native language of the wrapped library. The library itself may be

better

used in the dynamic environment of Pythons command line; or used

together

with other libraries already wrapped for/accessible from Python.



- Paddy.



Date: Mon, Dec 11 2006 7:22 am



Unlike Lisp, Python does not have a ubiquitous compiler. It is therefore made to interface nicely with compiled languages. Other compiled language users see the need for dynamic interpreted languages like Python and maintain links Python such as the Boost Python C++ wrapper. IronPython for .NET, Jython for Java.

Lisp is its own interpreter and compiler, which should be a great advantage, but only if you don't make the mistake of ignoring the wealth of code out there that is written in other languages.



> > Talk to these guys:

> > http://en.wikipedia.org/wiki/PyPy they have an interesting take on



> No, actually maybe you should talk to them since you seem to think that

> making Python run fast is dangerous, or at least unnecessary.



> > Python has this unsung module called doctest that neatly shows some of

> > the strengths of python: http://en.wikipedia.org/wiki/Doctest



> Now I'm *certain* that you're just pulling my leg: You guys document

> all your random ten-line hacks in Wikipedia?!?! What a brilliant idea!



Python is newbie-friendly. Part of that is being accessible.

Doctest is about a novel way of using a feature shared by Lisp, that is docstrings. Testing is important, usually not done enough, and doctests are a way to get people to write more tests by making it easier. Does Lisp have similar?

> Hey, you even have dead vaporware projects like uuu documented in

> Wikipedia! Cool! (Actually, I don't know that doctest is ten lines in

> Python, but it'd be about ten lines of Lisp, if that, so I'm just

> guessing here.)



Does Lisp have a doctest-like module as part of its standard

distribution?

Or are you saying that If you ever needed it, then it would be trivial to implement in Lisp, and you would 'roll your own'? There are advantages

to doctest being one of Pythons standard modules.



- Paddy.



Date: Wed, Dec 13 2006 2:03 am



Oh, you don't like Wikipedia.

There are a lot of people that use Wikipedia. I think some of them

might want to learn to program. I make it easier for them to find

Python by helping to maintain Python within Wikipedia.

If I am researching anything then I like to cross check with

information from multiple sites. that's just good practice.

Some people dislike Wikipedia which is fine. Some people dislike

Wikipedia and deliberately sabotage it, which is vandalism.



-Paddy.



Date: Wed, Dec 13 2006 1:15 am



Jesús Carrete Montaña wrote:

> > Fast. Very fast!



> > - Paddy.



> Well, Python certainly is faster than most people doing floating-point

> arithmetic by hand, but I don't think this is the correct argument to use

> against Lisp :-P.



Why not!

Lispers can indeed roll-their-own anything, many do it seems do just

that. But others look at the *time saving* libraries available to users

of Python and think hmm...

-Paddy.

Python Functions: Assignments And Scope

Explaining why this works:

n = [0]
def list_access():
   n[0] = n[0] + 1
   return n

try:
   print "\nlist_access:", list_access()
except UnboundLocalError, inst:
   print " ERROR:\n", inst

And this throws the exception:

m = 0
def int_access():
   m = m + 1
   return m

try:
   print "\nint_access:", int_access()
except UnboundLocalError, inst:
   print " ERROR:\n", inst

To execute a source program, the Python compiler compiles your
original source into 'byte codes' – a form of your program that
is easier for the Python interpreter to later run. In generating this
byte code, the byte code compiler will determine which variable names
in a function are local to that function, (so alowing it to optimise
accesses to such local names).

The rule for determining if a variable is local to a function is:

• If there is a global statement for the name in the function
  then the name is accessed from the global scope.
  
• If there is no global statement for the name, and if there
  are assignments to the 'bare' name within the function then the name
  is of local scope.
  ( A bare name assignment means assignment to a
  name, where the name occurs without attribute references,
  subscripts, or slicing s, just the bare name).
  
• Otherwise the name will be looked up in reverse order of all
  enclosing scopes, until it is found.

In the second example, function int_access; name m is flagged as
local by the byte code compiler as the bare name is being assigned
to. The interpreter therefore looks for a value of m to increment
only in the local scope, cannot find a value, then raises the
UnboundLocalError exception.
In function list_access, the bare
name n is not assigned to, so n is found when looking back
through enclosing scopes.

References

1. http://groups.google.com/group/comp.lang.python/browse_frm/thread/db9955da70c4e0ca

2. http://pyref.infogami.com/assignments

3. http://pyref.infogami.com/naming-and-binding

4. http://www.python.org/doc/2.4/ref/global.html

   
   

END.

Psycho re-run

I don't normally use Psycho, and did not have it installed with my python2.4 installation. Whilst searching for something else however, I came across this simple language speed comparison: Ruby, Io, PHP, Python, Lua, Java, Haskell, and Plain C Fractal Benchmark by Erik Wrenholt and wondered...

So, twenty minutes later I had installed Psycho and with the addition of an inserted line three to Eriks Mandlebrot.py example of:

import psyco; psyco.full()

The run time went down from 4.6 to 1 second.

The exercise was really to remind me that Psycho is available.

Function Attributes assigned by decorator

It appears to be straight forward to initialize function attributes by a decorator (the setup function).
The new function factory looks a lot like the class based version of my previous post, and performs similarly.

def make_standard_deviator2():
    '''Generate functions that return running standard deviations
    Uses function attributes applied by decorator
    '''

    def setup(func):
      func.N, func.X, func.X2 = [0.0]*3
      return func

    @setup
    def sd(x):
        '''returns standard deviation.
        Uses function attributes holding running sums
        '''
        from math import sqrt

        sd.N += 1       # Num values
        sd.X += x       # sum values
        sd.X2 += x**2   # sum squares

        if sd.N<2: return 0.0
        return sqrt((sd.N*sd.X2 - sd.X**2)
                    /(sd.N*(sd.N-1)))

    return sd

Python function attributes

Prompted by a presentation were someone said Python doesn't have full closures (noodle.odp), I looked at their example given and thought first that I could do that with funtion attributes, and then it hit me: I know of Pythons function attributes but had never used them.

I wrote a function generator that when called, returns a function that when it is called with successive numbers calculates and returns their standard deviation so far. The returned function stores accumulated data between calls as attributes.

The more usual Python way of doing this is to use a Clss instance. I created a Class based version too for comparison.

In speed terms, the Class based solution is only, (but consistently), just less than two percent faster than the function generator based solution. In terms of maintainability though both are readable, I expect most people to be trained in the Class based solution and so be more comfortable with it.
The function generator solution has the initializer section after the inner function definition which might be a minus, I wonder if a decorator could put the initializer 'up-front'.

#=== file: fn_attributes.py ===

def make_standard_deviator():
   '''Generate functions that return running standard deviations
    Uses function attributes
    '''
   def sd(x):
       '''returns standard deviation.
        Uses function attributes holding running sums
        '''
       from math import sqrt

       sd.N += 1       # Num values
       sd.X += x       # sum values
       sd.X2 += x**2   # sum squares

       if sd.N<2: return 0.0
       return sqrt((sd.N*sd.X2 - sd.X**2)
                   /(sd.N*(sd.N-1)))

   # Initialize attributes
   sd.N, sd.X, sd.X2 = [0.0]*3

   return sd

class Make_standard_deviator(object):
   '''Return running standard deviations
    when instance called as a function
    '''
   def __init__(self):
     self.N, self.X, self.X2 = [0.0]*3

   def __call__(self, x):
       '''Returns standard deviation.
        Uses instance attributes holding running sums
        '''
       from math import sqrt

       self.N += 1       # Num values
       self.X += x       # sum values
       self.X2 += x**2   # sum squares

       if self.N < color="#804040">return 0.0
       return sqrt((self.N*self.X2 - self.X**2)
                   /(self.N*(self.N-1)))

if __name__ == '__main__':
   import timeit

   print "function:",(
     timeit.Timer('[sd(x) for x in xrange(100000)][-1]',
     "from fn_attributes import *; sd = make_standard_deviator()").timeit(number=5)
     )

   print "   Class:",(
     timeit.Timer('[sd(x) for x in xrange(100000)][-1]',
     "from fn_attributes import *; sd = Make_standard_deviator()").timeit(number=5)
     )

What's wrong with Perl

It can be a refreshing tonic of a language when you move to perl
from something like basic, C or Java.
Perl can do what tens of Unix tools such as sed and awk can do from
the shell, but with more flexibility and higher order data structures
such as the perl 'hash' or 'associative array'.
You can sit and evolve a perl one-liner of X hundred characters
that will find and process millions of lines from thousands of files
from within hundreds of directories.
When you want to know how to do something else there is The Camel
, either the book or that human store of Perl Knowledge on the next
floor, a few cubicles over.
Before long, you have amassed a large personal perl knowledge
base, and can tackle most of your tasks, without having to consult
The Camel. You become more confident in perl and start to code for
others in your team, and write scripts in files rather than one
liners.

WAKE UP!

Languages have moved on.

Do you use perl references? Happy with them? Do you know that you
can get all that power without all the reference-gymnastics. Indeed,
you can get more power as you are freed from the mundane “how
do I create and use a hash of hashes”, to “O.K. The data
naturally starts as a hash of hashes so...”

Perl subroutines. Even AWK, one of the Unix stalwart languages
that perl was created to supplant has named arguments to its
functions. The better scripting languages can do a lot better,
allowing you to pass two lists into a function as easily as passing
in two integers. Or to return three hashes just as simply.

The perl slogan “There is more than one way to do it”
is often cited as one of perls strengths, and used by many
perl-mongers as their reason to stick with it. But how many of you
actually mean the extended form: “There is more than one way to
do it, and mine is the best”, or, “There is more than one
way to do it, but you will use mine!”, or more likely: “There
is more than one way to do it, and I don't know yours”, or
“There is more than one way to do it, and you will learn
mine!”, or “There is more than one way to do it, but I
don't know how to find any of them in Camels index”.

Perl still has goto! (and in its most evil of forms too, the
computed goto: where you have to interpret the program in your head
to work out where the goto might jump to from the listing because the
goto target is the result of an expression).

So much of perl documentation reads like 1001 recipes for using
1001 features, without any central theme to shape them.

Being an engineer, I am trained in finding patterns and
extrapolation. It is much easier for me to learn a small set of
powerful rules with a straightforward way of combining them. Perl,
unfortunately has a large number of 'gotchas' that bite when you try
to venture out from the recipes and examples given.

Conclusion

If you know perl, and have a need for programs greater than a few
tens of lines of code, then you should invest a week or two of your
time learning a different dynamic language. In that time, put aside
your perl knowledge and try to not just use the other languages
syntax, but learn how things are naturally done in that language.
Nothing may beat the 'perl -p -i -e' one-liner, but proper function
parameters and strict typing may be better for your longer scripts.

If you are thinking of learning Perl then think twice. Perls
'sweat-spot' is much diminished as new dynamic languages have emerged
with a less tacky support for new methodologies and standards such as
object oriented programming, program maintenance, functional
programming, XML, multi-language programming, and programming on
multiple frameworks such as dot-net and the Java Virtual Machine.

END.

Generate your way through the Verification quagmire

I was thinking some more about an idea I had when commenting on someones verification problem here:
http://verificationguild.com/modules.php?name=Forums&file=viewtopic&p=3312&highlight=#3312
It is very hard to verify todays chip designs, but they are already quite regular.
My idea is to design IP that has its size more fully parameterized and generatable, so you would not design a 16 bit UART, you would design an n-bit UART with n able to take a range of values.
If the design of the UART has no 'corner cases' for increasing values of n, i.e. the internal architectural algorithm doesn't change for increasing n, then new ways to test the IP become available:

• Formally test the IP generator performs its function over all n.
  It may be easier to formally prove the IP functionality for the IP generator over all n than to formally prove, for example, the functionality of a 16 bit version of the IP.
  Assertions would have to be written specifically for the genearator so that they too are parameterised by n.
  Questions: Are Assertion languages up to it? Are Formal checkers able to handle the complexity in the assertions?
• More thoroughly verify the IP for a sequence of much smaller values of n and infer the quality of the IP generator for large values of n for which verification, especially formal and simulation based verification, is much harder, if not impossible.

Another example could be in designing and verifying some next-generation 32-by-32 channel switch that will take hundreds of millions of gates to implement. Design an m-by-m switch instead. Maybe you will get the 2,3,4,6,and 8 by 8 versions onto an FPGA and are able to do throughput tests, that then allow you to extrapolate the throughput of the 32-by-32 bit target version.

In time, Systems design and verification could be transformed as the requisite IP arrives fully parameterised, and with information on how performance scales with parameter sizes. System simulations for, say, a 32 bit CPU connected to a 64 bit bus and with a 16 and 32 bit peripheral could be done by the system simulation engine working out and configuring a simulation of a much smaller system maybe a four bit CPU with an 8 bit bus and 2 and 4 bit peripherals. The Sytem simulator could insert monitors, do the reduced simulation, then extrapolate the performance of the larger system. The smaller simulation should still be able to show faults that would be in the larger system as the system as a whole is just scaled.
One performance result that might yield to such extrapolation could be system power calculations based on toggle counts of individual IP blocks.

Is anyone doing this at the moment? I know of configurable CPU generators but they seem to be tailoring the instruction set to the software that is to be run.

- Paddy