DVClub yah!

I attended the second (as well as the first), meeting of the Bristol chapter of DVClub yesterday.



This is a meeting, organised by and for Verification Engineers.
Held at a local company (my workplace this time around - Infineon
Technologies), it is a lunchtime meeting of approximately 55 engineers
this time and the subject for the day was Formal Verification (of
Hardware).







Once we all were seated, I couldn't help getting a feeling of
'belonging'. It isn't often you get to meet local engineers working in
similar fields to yourself. Mike Bartley, the organiser kept the event
on schedule, and the speakers knew their stuff. A question of
mine, lead to an interesting conversation with Douglas Fisher who is a
verification specialist at Synopsys - it's a rare opportunity when you
get to swap ideas with a developer from an EDA company so I must
thank DVClub and its Bristol members for a well organised gig.

Links to presented papers

Mike Bartley has all the presented papers available on his company's site for the Bristol Branch of the BCS








- Paddy.

word differ

My brother was over from France this new year and he asked me to write him a small script for the first time.



He had a problem in that he had some prose in a foreign language, and a list of words that the students should learn. He wanted stats on what words from the list were actually used in his prose.



I wrote the following small script. Interestingly enough, the bit that needed a bit of research was doing the graphical file opening for Windows - I really am that used to a Unix environment :-)



It is a very simple program but, on reflection, I can remember doing similar work twenty years ago in AWK: finding out what is different between two 'things' and printing the differences in a sorted order. Back then I would have used Suns original awk, keeping the two sets of data in associative arrays, doing the set arithmatic in explicit for loops, and calling out to the Unix sort utility to print sorted output. Such specialized diff'ing tasks have crept up regularly for me, and they are now very easy to do in Python (or Perl); and I have never had to resort to C for even the largest datasets I've encountered.

word_differ.pyw

Here is the program word_differ.pyw:

# -*- coding: cp1252 -*-

'''
  Find the difference in words used in two documents

  Asks for two files which then have their words extracted and compared.

  Most punctuation, and all words of just numbers are dropped.

  (C) 2009 Donald 'Paddy' McCarthy. paddy3118@gmail.com
'''

import Tkinter, tkFileDialog, re, datetime, sys
root = Tkinter.Tk()
root.withdraw()


def wordsinfile(f):
    words = set()
    txt = f.read()
    words = set( w.lower() for w in re.split(r"""[ \t\n\-,;:\.\?@#~=+_!"£$%^&\*\(\)<>|\\\[\]{}`]""", txt)
                 if w and not all( c in '0123456789' for c in w) )
    return words


f1 = tkFileDialog.askopenfile(parent=root,initialdir="/",title='First text file for word diffing')
if not f1: sys.exit()
f2 = tkFileDialog.askopenfile(parent=root,initialdir="/",title='Second text file for word diffing')
if not f2: sys.exit()
f3 = tkFileDialog.asksaveasfile(parent=root,initialdir="/",title='Save output as file (end with .txt to create a windows text file)')
if not f3: sys.exit()

set1 = wordsinfile(f1)
set2 = wordsinfile(f2)

print >>f3, "Output from program word_differ.py (Author Donald McCarthy: (C) 2009)."
print >>f3, "  Generated on: " + datetime.datetime.now().isoformat()

print >>f3, "  First File:  " + f1.name
print >>f3, "  Second File: " + f2.name
print >>f3, "  Output to:   " + f3.name

print >>f3, "\nWords in the first file that are not in the second:"
for w in sorted(set1 - set2):
    print >>f3, " ", w
print >>f3, "\nWords in the second file that are not in the first:"
for w in sorted(set2 - set1):
    print >>f3, " ", w
print >>f3, "\nWords common to both files:"
for w in sorted(set1 & set2):
    print >>f3, " ", w

f1.close(); f2.close(); f3.close()

Example word list: word list.txt

  back
  black
jump  
  bored
  hair
  missile
  lamb
  little
  played
  sent

Example prose: rhyme.txt

Mary had a little lamb
Who's hair was long and black.
She played with it,
Got bored with it
And then she sent it back!

By Paddy McCarthy - 2009-01-04.

Example output: diff.txt

Output from program word_differ.py (Author Donald McCarthy: (C) 2009).
  Generated on: 2009-01-04T07:20:18.890000
  First File:  C:/Documents and Settings/All Users/Documents/Paddys/word_differ/word list.txt
  Second File: C:/Documents and Settings/All Users/Documents/Paddys/word_differ/rhyme.txt
  Output to:   C:/Documents and Settings/All Users/Documents/Paddys/word_differ/diff.txt

Words in the first file that are not in the second:
  jump
  missile

Words in the second file that are not in the first:
  a
  and
  by
  got
  had
  it
  long
  mary
  mccarthy
  paddy
  she
  then
  was
  who's
  with

Words common to both files:
  back
  black
  bored
  hair
  lamb
  little
  played
  sent

Experiences converting a C lex/yacc parser for Verilog to pure Python

href="http://en.wiktionary.org/wiki/back_in_the_day">Back in
the day, simulators only had point delays, so all gate
libraries had buffers on their I/O. Now, we tend to use path, or
pin-to-pin delays in gate level models things have been made more
difficult for me.

My task was to randomly inject
some stuck-at faults in a large gate level Verilog design for
simulation. Individual gates of the design are connected by nets and
although I can force a net  I cannot force a node as all nodes
on a net are affected by a force on the net, and you cannot get at
individual nodes by looking to force a net  inside  a
particular gate model as their are no input buffers.  The
other problem with forcing nets/nodes using interactive commands of the
simulator is that you have to compile the design sub-optimally as all
the nets have to be accessible which stops the compiler/elaborator from
performing certain speed optimizations on what will be an execution
speed critical set of simulations.

Brainstorming
with a colleague we came up with the concept of modifying the netlist
to insert logic to either pass-through or force its out put to either 1
or 0 from an external control.

So off I went to href="http://embedded.eecs.berkeley.edu/Alumni/pinhong/scriptEDA/">ScriptEDA,
(great page - awful backdrop), and installed his href="http://www-cad.eecs.berkeley.edu/%7Epinhong/scriptEDA/pythongnd-0.35.tar.gz">Python
Interface for Gate-level Netlist Engine., which I then tried
on my gate level netlist.. Unfortunately our gate-level netlist failed
to parse because it used concatenations of nets which the parser
couldn't handle. I would need to extend the parser to support
 net concatenations.

Looking at the
source of the parser it was a C program using href="http://en.wikipedia.org/wiki/Flex_lexical_analyser">lex
and yacc
creating C data structures which were then interfaced to Python via a href="http://www.swig.org/">SWIG generated
wrapper. I don't like  to maintain C programs , and I needed
to flex my parsing abilities so decided to spend a short time
researching a pure Python solution.

PLY
The
obvious choice of  Python parsing tool seemed to be PLY as it
stands for Python Lex Yacc, so a download later and poking around in
the install, I came across href="http://code.google.com/p/ply/source/browse/trunk/example/yply/yply.py?r=43">yply.py:
style="margin-left: 40px;">

yply.py

This example implements a program yply.py that converts a UNIX-yacc
specification file into a PLY-compatible program.  To use, simply
run it like this:

   % python yply.py [-nocode] inputfile.y >myparser.py

The output of this program is Python code. In the output,
any C code in the original file is included, but is commented out.
If you use the -nocode option, then all of the C code in the
original file is just discarded.

To use the resulting grammer with PLY, you'll need to edit the
myparser.py file. Within this file, some stub code is included that
can be used to test the construction of the parsing tables. However,
you'll need to do more editing to make a workable parser.

Disclaimer:  This just an example I threw together in an afternoon.
It might have some bugs.  However, it worked when I tried it on
a yacc-specified C++ parser containing 442 rules and 855 parsing
states.

Debugging vs inserting print statements

After reading this blog entry from Corey Goldberg, you got some people saying use debuggers instead of print statements.



So why can't you put that same print statement in the debugger and have it show up as an inserted print statement in the source? Why can't you do that for any code to insert? Rather than a breakpoint appearing as a blob to the side, you should have the option of it appearing as inserted code, maybe made visually distinct via shading and containing the logic inherent in the breakpoint such as coded indication that it counts down and only fires on the 20'th pass and then executes this code without pausing the run (or not).





A debugger breakpoint that doesn't stop the simulation but prints some text and the value of a few variables could then look like the insertion of a print statement in your source.





If you could make the interface work the other-way round too, i.e. stick the IDE into Add Debug Text mode, then code added would automagically be turned into equivalent statements executed by the debugger, then you could really fight those pesky inserters of print statements for debugging purposes by giving them a debugger whose interface works that way anyway ;-)




</only_slightly_tongue_in_cheek>

Yet Another Blog Entry On Doctests.

Doctests are a current hot Python blog topic. I thought I would point out that many of the bloggers, especially those pointing out its problems, seem to be seasoned Pythoneers, aware of alternatives and so able to compare doctesting with other forms of testing..



I've found that there is a significant number of programmers out there who just don't write reusable and maintainable tests at all. They may test to ensure their program 'works' when they wrote it, but that is all - job done, finito!



Doctests perform the very important task of lowering the entry point to the subject of testing. Novice pythoneers ready for exposure to testing are likely to have used the python shell to "try things out"; it is a small cognitive step to point out that by cutting and pasting those explorations from the shell into a docstring and adding some "magic sauce" to invoke it, you get a test. A trainer can then move on to the "why" of tests with the student thinking "I do most of it anyway", rather than thinking that it is yet more to learn



I am not saying that doctest is without flaws. I am saying that it is still a good tool, without being perfection.

Knapsack Problem

The Rosetta Code Blog this week mentioned that entries starting with a K were missing so I reviewed my sources and added a knapsack type problem to RC together with the python solution.



It's not the first time that I've remembered that itertools.product can do arbitrary nested for loops. Before, I would have had to employ recursive function calls.



- Paddy.

New year. New PC

I'm thinking of retiring my trusty 17" laptop as my main PC and going quad core.I want to do it on the cheap, so was thinking of building from parts for hopefully arounf £400 and adding this monitor, or this TV/monitor. The screens are made by the same company and look very much the same except the TV version has extra inputs and a TV tuner. I wonder if anyone has this TV connected to a PC and can comment?

I don't play graphic-intensive games so can get away with a basic graphics card.

I want the quad cores to try out more parallel programming. I have a VCD to toggle count program that reads a large VCD file . I would like to work out how to accurately do this on the four cores and see how close to linear speed-up I can get. Our regression simulations at work leave behind approximately 20K simulation logs and coverage databases - I'm starting to think of ways of data-mining them which would be aided by parallel processing (most of works compute farm PC's have between 8 and 16 cores).

Priority would go to getting a little X-at-a-time script moved from bash to Python. Given a list of jobs to do - one per line in a file; the original bash script will execute X of them in the background, starting more, when others finish.

So, if you have any ideas of cheap bundles for creating quad-core PC's (In the UK), fire away :-)

- Paddy,

Smearing the Static vs Dynamic Typing Dichotomy

CPython
needs to ship with a compiler!
There, I've said it. But what
are my reasons?

I truly believe that your
programming language influences the way you go about solving a problem,
and their is a mindset to programming in a dynamic language like
Python/Perl/TCL/AWK/... that can lead to different types solutions (and
even perceived problems), than  programmers in say
Java/C/C++/C#. (OK some of the differences are also down to other
factors such as IDE's and OS which also affect how you solve an issue).

Lately
I have seen href="http://www.25hoursaday.com/weblog/2008/01/02/DoesC30BeatDynamicLanguagesAtTheirOwnGame.aspx">blog
posts  commenting on the 4.0 release of C# will be
adding dynamic type resolution capabilities via a keyword that will
allow C# to finally do Duck Typing.  The href="http://boo.codehaus.org/">Boo language has
done this trick from its outset. Thinking about it though, I don't
think it will make C# programmers style change to mimic that of
dynamic language programmers as they would have to stick their keyword
on every definition and leave behind their perceived safety of run-time
type checking. They will, I think,rather use it as another resource
that they can dip into, forming rules/design patterns on where it fits
best and use it from overwhelmingly statically typed source code.

When
I look critically at Python, (yep - I do do that from time to time),
 Python 3.0 has optional type decorations - but no compiler,
and Python distributions such as Enthoughts, include external modules
to help in creating compiled code to link to Python,, but we need to
have a complete solution out of the box i.e. CPython should be shipped
with a defined method of compiling 'Python-like' source code into
machine code as part of its standard library and enshrined in the
reference guide. That would allow Python programmers to do the opposite
of what the C# programmers can do: write a mainly dynamic program -
with dynamic language idioms, but be able to seamlessly have part of
their recognisable Python code compiled . Maybe all it would take is
the addition of a new static
keyword to apply to classes/functions/modules.

You
would need to think of static/dynamic as a continuum where instead of
having languages bunched at either end, you would have a smearing out
towards the middle.

Existing languages have too much
static vs dynamic baggage, but maybe a new language with type
inferencing and where you were forced to first define the default type
checking regime as either dynamic or static, then be given the option
of declaring particular 'things' as being at either run-time or
compile-time would lead to a  language that might breed new
programmers with more interesting ways to solve problems.

-
Paddy.

Comments on a "Python Tutorial"

Hi,
I thought I'd go through your tutorial on Python and make some notes if thats OK.



"No ++ / --":

I think this was the same kind of decision as Python not allowing assignments in if conditionals. There is a a high instance of problems caused by confusion over the misuse of increment/decrement operators in languages like C. They can make it harder to read a program.



"Slow in comparison to compiled languages" - (Here comes the dynamic language supporters usual reply): Yes, but often getting something right is much more important and often never requires the speep that Python gives you. People have prototyped in Python and shipped the prototype as it fits all quality metrics in a far shorter development time. I recently had a case of that myself here.



"Whitespace for indentation": Arguably a positive feature. It should read "whitespace for block delineation" or something. Run a pretty printer on say a C source file and it will try and indent it so that the visible level of indentation mirrors the logical structure of the code. Code is hard to read if the indentation does not follow the logic. Python recognises this and goes further in saying that the common use of block start and end delimeters such as {/} or begin/end are superfluous and only help in writing misleading code.



You seem to have missed the importance of docstrings in Python.



range "(good for indexing)": True, but in idiomatic Python you are much more likely to iterate over the members of a collection object rather than create an index using range:

for x in my_list:

rather than:

for i in range(len(my_list)):

....x = my_list[i]



"Functions work in python much like they do any other language": those with only Java and C experience might miss out on the extended argument passing of Python functions if not made aware.



"Classes are more basic than what you will be use to from java": In what way? There maybe less to learn to become equally proficient but I see this as a good thing. Note that Python is dynamic, meaning that, if you wanted to, you can modify classes and instances at run-time. Its not necessarily a X language does classes better than Y. Python does things differently and if you try and constrain Python to how you use OO in Java, you won't get the best from it.



"from os import *": It's a bad Python habit. You want to warn students off this, as maintenance can suffer.



"We can raise our own exceptions": It could be but shouldn't be a string. best to use an exception type.



"Numbers": There are decimals too, which are good for financial calculations.



"Tuples": If each position in a sequence has explicit meaning then you should use a tuple, e.g. a point on a surface might be the tuple (x,y). coordinates of a car would more likely be a list of points.



"Strings": Misses triple quoted strings.



Filter and map, although present, have declined in use due to the later addition of the powerful list comprehension and generator comprehension features.



One-liners in Python: Difficult to do. Usually people write multi-line scripts in an editor. In a shell such as bash which has multi-line string delimeters you might be able to use a multi-line single quote for bash and embed a multi-line python program using Pythons -c argument and restricting your Python to double quotes if needed.



Have fun with Python.



- Paddy.

November 2, 2008 5:26 AM

Paddy3118 said...

Ouch!

My notes are longer than your blog entry :-)

Dear Americans

Please vote for Obama.



Whoever you vote for - our British Prime Minister has to suck-up to support, and it has been painful, as a Brit, watching what our elected government has had to do these past few years. Unlike the French or the Germans, we have had to go along with so many of your current administrations schemes, and so have been seen as no more than a lickspittle.



It is time to make America great again! (We Brits can't stand the reflected ridicule).



P.S. The obligatory Python angle: If you are going to vote electronically, then use Open Source tools such as pvote.