Hey look, this page is cited in Scientific American: 
The image below shows one in action:
The next image shows the principle of how the machine works:
But self-replication is not actually the main goal of von Neumann's work! There are far simpler cellular automata that support self-replicating structures - Langton's Loops, for example. Von Neumann was thinking about the process of life itself, how self-reproducing structures could ever grow in complexity. He saw that a good way to guarantee evolvability was to include a universal constructor in the body of the self-replicator - allowing it to not only make copies of itself (as Langton's Loops can do) but also to construct anything that the tape instructs it to, including self-replicators that are more complex than itself.
As a taste of what this might mean, the image below shows a replicator that has had a mutation added to it:
In this example the mutation has no effect on the operation of the machine but in theory it could have made an improvement, perhaps by making the machine more compact or faster or better at destroying other replicators. Other mutations would interfere with the working of the machine and would be fatal.
Actually of course talking of 'improvement' doesn't really make sense here, since the replicators are not competing. Von Neumann's design was never intended to demonstrate practical evolution - there is no room for each replicator to make more than one copy, so no competition; and no consideration for how replicators might interact. Instead von Neumann's sketched design is a solution to the problem of how complexity growth could occur even in theory. McMullin calls it a 'schematic' solution. (von Neumann mentions conflict and making room, and natural selection: 22.214.171.124 and 1.7.3 and 1.8 in  but never finishes this work.)
The challenge now is to find ways to embody some of the unlimited potential evolvability of von Neumann's designs in replicators that can compete for resources in a shared environment. These replicators might then exhibit the evolutionary growth of complexity through competition and natural selection. This is one of the main aims of the field of Artificial Life.
The ideas presented here are from this paper:
The Pesavento implementation of von Neumann's machine is presented here:
The design that Pesavento presented was created in collaboration with Renato Nobili, whose papers and software and machines you can download here: 
The [Golly] software supports JVN rules in versions 1.5 and later. If it isn't released yet, the CVS source code is available , or the following Windows binary might work:  - but these are alpha versions so beware. To import Nobili's machines into Golly, use the scripts posted here: . To run Pesavento's 1995 design in Golly, use the RLE file linked above, or you can make your own tape using the script posted here: .
To run the replicators in Golly, you need ideally 2GB of RAM or more, although it will work with less, just more slowly. Hit the "+" key a few times to increase the step to 8^5 or higher - until Golly updates only every second or so. This means that overall it runs fastest. Replication of the Pesavento design takes about 63 billion iterations (6.34e+10) but by the wonders of the hashlife algorithm this may only takes a few minutes, depending on your machine. Nobili's newer SR_CCN_AP.EVN machine is more compact, and is currently the fastest known von Neumann replicator. You can download it in his WJVN package.
The replicator with the flower mutation is available in the same zip as above: [zip (31k)].
(Some of the images above are now used on the wikipedia page:  )
(Everything on this site is public domain unless stated otherwise (some GPL software).)
Hey look, this page is cited in Scientific American: