Author Archives: admin

The Little Schemer in Clojure – Chapter 4 – Numbers Games

This is the fourth Chapter of a series of posts about porting The Little Schemer to Clojure. You may wish to read the intro.

You’ll notice that in the first three chapters, we introduced lists [Ch1], then we pulled things out of lists [Ch2], and then we built lists [Ch3]. The bigger principle of the book is to be able to operate on data structures with a minimal set of primitives using the power of recursion and lambdas.

Now we’re driving at the meta-circular interpreter with these principles. We also want to be able to do mathematical operations, and yet still keep a minimal number of primitives. Interestingly enough, we’ll just use equals, add1 and sub1 and build up to multiplication, division and exponents. So let’s get started!

Our first function is +_ (plus) – we’re going to define it terms of add1 and sub1. (We’ll rename the functions with an _ suffix so they don’t clash with the existing clojure definitions of + and zero?

(def zero_?
  (fn [n]
    (= 0 n)))

(def add1
  (fn [n]
    (+ 1 n)))

(def sub1
  (fn [n]
    (- n 1)))

(def +_
  (fn [n m]
    (cond
      (zero_? m) n
      true (add1 (+ n (sub1 m))))))

(println (+_ 1 2))

Now this is kind of frustrating. Why bother doing all this recursion for an operation that is already built into the CPU? Especially when you limit yourself to integers anyway?

This is about making you think about numeric implementations. Part of it is that LISPs generally implement their own ‘numeric tower’ – you can see what fresh thinking has done for Clojure’s ability to handle numbers – particularly irrational fractions.

The other reason is the purity of the metacircular interpreter that you’ll build – you’ll literally be able to count the number of primitives required to build it and have no holes. (There is an assumption that recursion is cheap.)

Our next function is – (minus) which is quite similar to the above:

(def -_
  (fn [n m]
    (cond
      (zero_? m) n
      true (sub1 (- n (sub1 m))))))

(println (-_ 2 1))

From here we extend our concept of the lat (a List-ATom) – being a non-nested list of atoms. We extend it to what the book calls a vec, which is a non-nested list of integers.

From here we arrive at [The Fourth Commandment]. This states When recurring on a list of atoms, lat, or a vec, ask two questions about them, and use (cdr lat) or (cdr vec) for the natural recursion.

It then goes on to apply this to lists of numbers as well. It goes on: When recurring on a number, n, ask two questions, and use (sub1 n) for the natural recursion.

Now we get to the addvec function. This is really like a sum function from sql land:

(def addvec
  (fn [vec]
    (cond
      (empty? vec) 0
      true (+_ (first vec) (addvec (rest vec))))))

(println (addvec '(1 2 3)))

Next we look at the multiply function *_.

(def *_
  (fn [n m]
    (cond
      (zero_? m) 0
      true (+_ n (*_ n (sub1 m))))))

(println (*_ 4 5))

Again – we’ve built this in terms of our existing primitives.

Now we get onto The Fifth Commandment. I know these could come across as boring and self-righteous. On the other hand, if you write recursive code from scratch (and don’t rely on a library to do it for you) you want to have absolute confidence that it will run and terminate as you expect. These Commandments give you that confidence.

The Fifth Commandment states: When building a value with +_ (plus), always use 0 for the value of the terminating line, for adding 0 does not change the value of an addition.

It goes on to say: when building a value with *_(multiply), always use 1 for the value of the terminating line, for multiplying by 1 does not change the value of a multiplication.

It then finishes with: when building a value with cons, always consider () for the value of the terminating line.

Now we’ll look at vec+. The big idea is to be able to add two lists of numbers matching each corresponding element in the sum operation. This has uses in matrix operations:

(def vec+
  (fn [vec1 vec2]
    (cond
      (empty? vec1) vec2
      (empty? vec2) vec1
      true (cons (+_ (first vec1) (first vec2))
             (vec+
               (rest vec1) (rest vec2))))))

(println (vec+ '(1 2 3) '(4 5 6)))

Now we’ll look at >_(greater than). We’ll define it in terms of our primitives.

(def >_
  (fn [n m]
    (cond
      (zero_? n) false
      (zero_? m) true
      true (>_ (sub1 n) (sub1 m)))))

(println (>_ 10 1))
(println (>_ 1 10))

We’ll reuse this for less-than. It actually turns out remarkably similar:

(def <_
  (fn [n m]
    (cond
      (zero_? m) false
      (zero_? n) true
      true (<_ (sub1 n) (sub1 m)))))

(println (>_ 10 1))
(println (>_ 1 10))

Next we’ll define equals:

 
(def =_   
  (fn [n m]     
    (cond       
      (>_ n m) false
      (<_ n m) false      
      true true)))

(println "=_")
(println (=_ 1 10))
(println (=_ 10 1))
(println (=_ 10 10))

Next we’ll look at exponents:

(def exp
  (fn [n m]
    (cond
      (zero_? m) 1
      true (*_ n (exp n (sub1 m))))))

(println (exp 2 3))

Now we’ll start using our numbers functions on atoms and lats – this gets used in our metacircular evaluator later. First we’ll start with length:

(def length
  (fn [lat]
    (cond
      (empty? lat) 0
      true (add1 (length (rest lat))))))

(println (length '(pasta with pesto and mushrooms)))

Now we’re going to write a function pick to get an item out of a list using an index.

(def pick
  (fn [n lat]
    (cond
      (empty? lat) '()
      (zero_? (sub1 n)) (first lat)
      true (pick (sub1 n) (rest lat)))))

(println (pick  3 '(pasta with pesto and mushrooms)))

Now we’re going to extend this to remove an indexed item from a list using rempick:

(def rempick
  (fn [n lat]
    (cond
      (empty? lat) '()
      (zero_? (sub1 n)) (rest lat)
      true (cons (first lat)
             (rempick
               (sub1 n) (rest lat))))))

(println (rempick  2 '(pasta with pesto and mushrooms)))

Now we’re going to write a function to strip all the numbers out of a list. We’ll cheat a little bit, as we’ll introduce a new primitive function number?. We’ll call our strip function no-nums.

(def no-nums
  (fn [lat]
    (cond
      (empty? lat) '()
      true (cond
             (number? (first lat)) (no-nums (rest lat))
             true (cons (first lat)
                    (no-nums (rest lat)))))))

(println (no-nums  '(1 potato 2 potato 3 potato 4)))

This time we’ll flip it over – and write a function to strip out all the non-numbers from the list:

(def all-nums
  (fn [lat]
    (cond
      (empty? lat) '()
      true (cond
             (number? (first lat)) (cons (first lat) (all-nums (rest lat)))
             true (all-nums (rest lat))))))

(println (all-nums  '(1 potato 2 potato 3 potato 4)))

Our final function eqan? is our first building block towards rewriting the equals function in terms of our own primitives. This function will get us about half way there:

(def eqan?
  (fn [a1 a2]
    (cond
      (number? a1)
        (cond
          (number? a2) (=_ a1 a2)
          true false)
      (number? a2) false
      true (= a1 a2))))

(println (eqan?  1 10))
(println (eqan?  10 1))
(println (eqan?  10 10))
(println (eqan?  'a 'b))
(println (eqan?  'a 'a))

Conclusion
This chapter we’ve introduced two new primtives add1 and sub1 – and used them along with all our existing primitives to build all the functions above.

In total our primitives so far are: atom?, null?, first, rest, cond, fn, def, empty?, =, cons, add1 and sub1. These are all the functions (and those in the chapters to come) that we’ll need to implement to get our metacircular interpreter working.

The Little Schemer in Clojure – Chapter 3 – Cons the Magnificent

This is the third Chapter of a series of posts about porting The Little Schemer to Clojure. You may wish to read the intro.

The first function we look at is rember.

(def rember
  (fn [ a lat]
    (cond
      (null? lat) '()
      true (cond
        (= (first lat) a) (rest lat)
        true (cons (first lat)
          (rember
            a (rest lat)))))))

(println (rember 'banana '(apple banana orange)))

The big idea here is list construction. We’re applying the second commandment of the Little Lisper – use cons to build lists [sequences].

The next function we look at is firsts.

(def firsts
  (fn [l]
    (cond
      (null? l) '()
      true (cons (first (first l))
        (firsts (rest l))))))

(println (firsts '((large burger)(fries coke)(chocolate sundae))))

Here we also meet the Third Commandment of TLS. This states when building a list, describe the typical element, and then cons it onto the natural recursion. This goes back to the discussion on lat? we were having in the previous chapter.  You have to identify the base case, and then build on it.

You can see it working here

The Little Schemer in Clojure – Chapter 2 – Do It, Do It Again

This is the second Chapter of a series of posts about porting The Little Schemer to Clojure. You may wish to read the intro.

This chapter introduces functions – and the idea of the list of atoms and the member function.

When looking at list atoms, known as a 'lat' we are introduced to writing functions in Scheme. We write the lat? function in Clojure to test for a non-nested list of atoms.

(def lat?
  (fn [l]
    (cond
      (null? l) true
      (and (seq? l)
           (atom? (first l)))
             (lat? (rest l))
      true false)))

There are a couple of things we’ve transposed here. The key one to raise is the usage of def... fn[] rather than defn. This is a convention in the book, and glues in the readers minds that functions are by default anonymous, and we give them labels after the fact. This enables the reader to get into the feel of passing anonymous functions around later.

The other thing that is introduced here is the application of the Ten Commandments (of The Little Schemer). The big ideas of the 10 Commandments is to help you solve problems recursively. The first one states that you should always ask null?  as the first question in expressing any function. In idiomatic Clojure we would use the empty? function for this – but for the sake of being closer to the book, we will not apply it in this function, but use null?.

The question behind this is what is a non-nested list of atoms and why do we care?  One of the big themes in this book is solving problems using recursion rather than iteration. The point is that before you can solve a problem using recursion (where you’ve got a nested list) you need to learn to solve it for the base case where there is no nested list. To validate the base case – we use the lat? function.

The next function we look at is the member? function.

(def member?
  (fn [a lat]
    (cond
      (null? lat) false
      true (or
        (= (first lat) a)
        (member? a (rest lat)))) ))

Why do we have this function? It’s the first useful function in the book. We also keep building our recursion skills, and it is a key building block for later functions.

You can see it working here

The Little Schemer in Clojure – Chapter 1 – Toys

This is the first Chapter of a series of posts about porting The Little Schemer to Clojure. You may wish to read the intro.

The big theme in Chapter 1 is Simplicity of Data. In The Little Schemer world there are two levels in the syntax that describes data – atoms and lists. Note that an atom in TLS refers to a a string of characters (perhaps you could consider this a list element – ie a non-list), wheras in Clojure an atom refers to a unit of a transaction.

To test for atoms – we’ll use the atom? function:

(def atom?
  (fn [a]
    (not (seq? a))))

Note that we’re not using the defn syntax in Clojure but def ... fn. This is intentional because the book uses this standard, and then makes use of it later in the book when it passes anonymous functions around.

Speaking of null – the original Scheme has a different concept of nil than what we see in Clojure today. To make Clojure act like the null tests in the book – we use the null? function:

(def null?
  (fn [a]
    (or
      (nil? a)
      (= () a))))

In TLS the thematic emphasis is also on the Simplicity of Syntax. All data structures are represented by lists, and lists of lists (ie nested listed). In Clojure we have several types of data structures – but the closest to what we’re dealing with here is the Sequence.  (Now it’s more complex than that – because in Clojure a sequence is more like an iterator interface – but we’ll start there.)

The other concept to get in Chapter 1 is that lists are treated as stacks rather than arrays. We push elements on the end and pop them off. Our primitives to do this in TLS are cons to push, car to pop and cdr to get the list remainder. In Clojure this corresponds to cons to push, first to pop and rest to get the sequence remainder. (Note that Clojure also has conj, but we’ll stick to sequences and using cons for now)

The last trick that Chapter 1 introduces are nested lists. The short summary of this is that if you push a list onto a list you get a nested list (using cons). To reverse this then you pop the nested list off the list (using car/first).

The Little Schemer in Clojure

The Little Schemer is a delightful book that has been the road to LISP for many people. Highlights include the YCombinator and the metacircular evaluator.

Positive reviews abound, with this book being the key to several ‘aha’ moments:

  • This is a book that literally blew my mind.” Anthony Lewis
  • I recently read The Little Schemer and had my mind blownJamie Strachan
  • I found Lisp to be a load of brackets till I read The Little Schemer. It blew my mind, was the best programming book I have ever read.a1212
  • “The Little LISPer blew my mind.” Sal

There is much buzz in the programmer zeitgeist about this book, and whilst the software development world has changed much since it was written – if anything it is coming closer to the ideals of this book.  In many ways, it is a timeless classic, and well worth giving a new lease of life in Clojure.

The book deliberately steers away from using library functions, and instead builds up on a minimal set of primitives, in order to be able to build a lisp interpreter for that minimal set of primitives. (In addition to teaching recursion.)

This series of posts is will communicate the big ideas of each Chapter of this book in Clojure as I work through it. I’m not aiming to cover every function or every question – just those that seem the most pertinent or the most useful to the big ideas in the chapter.

Questions

Is this really a good way to learn Clojure?

Fascinating question. I don’t believe the original book was ever intended to teach ‘The Scheme Language’ – but to teach just enough for the reader to get some big ideas in Computer science.

Is this a good way to learn Clojure? No. The book is about deliberately steering away from using library functions, and instead builds up on a minimal set of primitives, in order to be able to build a lisp interpreter for that minimal set of primitives. (In addition to teaching recursion.) You can read more about this idea here.

Where does this fit in the bigger picture?

The theme is Amazing Lisp Books living again in Clojure.

Chapters

Image

Amazing LISP Books living again in Clojure

The LISP family of languages has a rich heritage, with some epic tomes gracing Computer Science history (and our shelves) over the years.  Clojure is the latest entrant in the LISP family, bringing concurrency advances and the richness of the JVM ecosystem to the table. For Clojure this means there is much legacy code to plunder reuse.

The Little Schemer

An old favourite for many people who studied this in College or at home – The Little Schemer is the way many people have started the road to LISP.  Described as ‘mind blowing’ by some – particular highlights include the ycombinator and the metacircular interpreter.  In Clojure you can find the following online:

Practical Common LISP

Practical Common Lisp is the modern Common LISP Pickaxe – the guys at ITA software hand this to grads to get them up to speed.   The guys at Relevance have spent some time bringing this across:


Paradigms of Artificial Intelligence Programming

PAIP is a well known book that has aged well – and still remains a delight to flip through. The author Peter Norvig has served as the head of Computational Science at NASA and is now Director of Research at Google.  The opportunities for the Clojure community here are wide open because this book has some amazing stuff. In Clojure it looks like this:


On Lisp

Paul Graham has been an outstanding advocate and essayist for LISP, and many have explored LISP because of his writings. He’s also famous for his LISP startup Viaweb and it’s subsequent sale to Yahoo, plus his distinguished work in the YCombinator VC Fund.  One of his primary arguments in the essay ‘Beating the Averages’ is that LISP helps programmers by being more expressive (more powerful) and the tool that LISP has that other languages don’t is macros. How do we learn about macros? Paul Graham wrote The Book on LISP macros. Here is what it looks like in Clojure:

Lisp In Small Pieces

Lisp In Small Pieces

Lisp In Small Pieces is a wonderful book by Christian Queinnec. Lots of people have written a LISP interpreter. The author writes eleven interpreters and two compilers.

The opportunities for this to be ported to Clojure are wide open – and work has only just begun:

 

 

Structure and Interpretation of Computer Programs – SICP

 

Updated 25 May 2013

Peter Siebel Reaction

Running Checkstyle at Full Throttle on a Legacy Codebase

 TLDR;

So in summary:

You can turn on all checks for new checkins on a horrible old legacy code base in checkstyle by using checkstyle to generate a supressions file to ignore all the old mistakes.

Get new version of Checkstyle

Download the patched version here.

See the patch in Sourceforge here.

To run simply add the the following new task in your build file to generate the suppressions file from the build.xml here:

<target name="generate.suppressions"
 ...

Then you’ll need to add the following to your existing checkstyle task.

<property key="checkstyle.suppressions.file" 
file="target/suppressions.xml"/>

The Story

I think checkstyle is fantastic, because I work in a large team of developers on a large codebase, and I want to know we’re all on the same page with our coding styles.

“But it shouldn’t matter,” you might argue. “All it does is slow you down. That’s not providing benefits or reducing risks.” Fair enough. Suppose you find the following in your codebase.

...
if (inputString == "Expected String") {
...

Should the developer responsible have known? Should the code reviewer have picked it up? Is everybody on your team on the same page that This Is Not A Good Idea?

The team discussion

“But”, they argue [and here comes the broken window argument.] “This pattern is already all over the code base. The developer was just copying and pasting.”

[At this point you might groan internally and consider changing jobs. Alternatively you can acknowledge that there is real money in maintaining legacy codebases for profitable businesses, and try and find the opportunity in the murkiness.]

“That’s not a good enough reason.” You will respond. “There has to be a way for our tools to support this scenario.”

Enter Checkstyle

So checkstyle can run some checks over your code base. It comes with lots of fancy checks and it feels like it would address this situation.

So you run it with the default set of checks. You get back several thousand errors. So you alter the checks file to just the checks you’re interested in: StringLiteralEqualityCheck. Still your codebase comes back with 50 errors.

The Legacy Push-back

“We can do this,” you hope internally. We only have to touch 50 classes. Now imagine that this system has 20 000 users a day, which collectively transact about $1B a week through your system. The tolerance the organisation has for risk and defects is pretty close to zero, and changes requiring a regression test to system components not on the project will be looked on unfavourably by your project manager and user acceptance test manager. This Is Not The Way Forward.

“Well we can just use the supressions file!” you exuberantly exclaim. For these 50 errors, it should only take An Hour Of My Time to Manually Type In This XML. Because manually typing 150 lines of XML is the reason you took that Computer Science degree – right?

A New Way Forward

“Hang on a minute,” you think. “This doesn’t feel right. What I wanted was a way of working with my team where we can agree on a way of not writing crappy code. Here am I using an hour of my time to make one measly check work. What about the other checks?”

“Surely someone has done this before?” A quick search of Google and StackOverflow says no.

“What I want is for checkstyle to generate the supressions file for me, for all checks if I want. Surely generating a supressions file is about the same complexity as generating an errors file – and it already does that.”

Cracking open the checkstyle codebase one evening, it was surprisingly easy to do just that.

The Benefits

What you want is instant feedback for your developers. As soon as they save a file and run a checkstyle task in eclipse, or check it in and get an email from the Build Server – they should get some indication of whether their new code matches up with the existing codebase.

What you want is checkstyle running at full-throttle. You want all the checks possible. You don’t want to be held back by having to change legacy code to get there. You want the new code to be in line. This helps you get there.

Clojure GUI Demo of Ikeda Map (Sinkhole)

Clojure GUI Demo of Ikeda Map (Sinkhole)

The Beauty of the mathematics of a sinkhole – captured in a Clojure GUI

Lau B. Jensen wrote an amazing blog post called Theory vs Clojure. Here he described how to develop in Clojure a discrete-time dynamic system called an Ikeda map.

Rich Hickey then submitted his improvements to the code. This demo is Rich Hickey’s version.

You can launch it via web start here. You can read the code here.

Clojure GUI Demo of Ikeda Map (Sinkhole)

Clojure GUI demo of Asteroids Game

Clojure GUI Demo of Asteroids

Zach Tellman has done a masterful job on the Penumbra library – another example of this is the Asteroids game.

You can see the code on git hub here. You can click here to launch via web start.

Clojure GUI Demo of Asteroids game

Clojure GUI Demo of LED Timer

Webstart Launch of Clojure GUI demo of LED Timer

 

U2 sang a song called ‘Stuck in a Moment’. You can ponder this as you watch this LCD timer demo by icyrock.

You can launch it via web start here. You can see the code here.

Clojure GUI Demo of LED Timer