Thursday, November 4, 2010

LED Display Board using the Propeller 8-core processor driven by the javax.comm and java.net Java APIs

Q: Do you find yourself after a long weekend asking - "What bugs, defects or enhancements was I working on?"  Then building this display and not having to run a query on the web or keeping a sticky note around should help you out.


    Project: Display periodic web-based data on an LED display using Java
I needed a relatively quick end-to-end solution where I could display web based data on an external circuit. 
The circuit consists of 4 lines of 8 (7-segment LED displays).  The display as configured is multiplexed by standard 74HC595 SIPO shift registers - 4 sets of 2 in total for independent column and data line drivers.  The microprocessor does not drive the displays directly - it only drives the shift|load|data lines of the 4 sets of shift registers.

The modules of the project are as follows...
  1. LED display multiplexer (4 lines)
  2. Microcontroller COMM port tranceiver
  3. Microcontroller serial to display logic
  4. PC COMM port tranceiver
  5. PC HTTP receiver
  6. PC HTML parser
The first 3 run on the microcontroller that drives the display, we therefore need 4-7 process threads running on the device which include 1-4 display drivers depending on how fast our chip is and if we want independent display lines.
The last 3 modules (hardware comms, http comms and html parser) can all be run from a single SE Java application on the host PC where the device is connected to a USB serial port.

Third Party Software Plugins:
  1. eclipselink.jar - EclipseLink 2.1.2 ORM implementation of JPA 2.0 API (part of JEE6)
  2. javax.persistence_2.0.3.v*.jar - The JPA 2.0 API specification interfaces jar
  3. derbyclient.jar - Derby Java Database 10.5.3.0 client jar
  4. comm.jar - The SUN Java Communications API jar (for serial/USB port comms)

BOM (Bill of Materials):
  • 1 - 40 pin DIP Parallax Propeller PAX32 8-core 32-bit Microcontroller running ASM or SPIN bytecode (with integrated 5MHz crystal, 32Kb EEPROM and FTDI USB controller/jack)
  • 8 - 16 pin 74HC595N DIP SIPO shift registers with separate shift/load registers from Parallax, Digikey.ca or futureelectronics.ca
  • 16 - blue 7-segment SURE electronics LED digit display
  • 16 - red 7-segment SURE electronics LED digit display
  • 5 - 63 row breadboards
  • 1 - 300-1000mF electrolytic power capacitor
  • 1 - 7803 3.3v power transistor with head sink (we get up to 45(c))
  • 1 - 7.5 - 9.0v 1A power supply (USB cannot power the board)
  • n - 0.1 inch (breadboard compatible) discrete LED from evilmadscience.com - an excellent electronics components distributor
persistence.xml

 
    org.eclipse.persistence.jpa.PersistenceProvider
       org.dataparallel.device.model.BugNumber
       
            
                        
            
            
            
                        
            
            
       
 

Java code:
Javaagent resource
Place the following file in your /src/META-INF/services/javax.persistence.spi.PersistenceProvider file.
org.eclipse.persistence.jpa.PersistenceProvider

SerialDriver.java
package org.dataparallel.device.driver;

import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.util.GregorianCalendar;
import java.util.TooManyListenersException;

// Sun's serial port driver
import javax.comm.CommPortIdentifier;
import javax.comm.PortInUseException;
import javax.comm.SerialPort;
import javax.comm.SerialPortEvent;
import javax.comm.SerialPortEventListener;
import javax.comm.UnsupportedCommOperationException;

/**
 * This code will writes to a full duplex serial port (a USB FTDI controller) that
 * is connected to a Parallax Propeller 8-core microcontroller that runs a 4 line 32 digit display.
 * See 
 *     http://www.dataparallel.com
 *     
 * References:
 *     Sun Java Communication API
 *         
 *     See the following tutorial by Rick Proctor for the Lego RCX Brick at 
 *         http://dn.codegear.com/article/31915
 *  Prerequisites:
*      Java Serial Support on Windows
*      See "PC Serial Solution" for the Lego(TM) RCX Brick at the following URL at MIT 
*      - it explains how to get the SUN javax.comm API working on Windows (it is officially supported only on Solaris and Linux).
*            http://llk.media.mit.edu/projects/cricket/doc/serial.shtml           
*
*      SUN/Oracle Java Communications API 3.0 (1998-2004)
*            http://www.oracle.com/technetwork/java/index-jsp-141752.html
*      - copy comm.jar and javax.comm.properties to both yourJDK and JRE lib directories
*      - copy win32com.dll to both your JDK and JRE bin directories - no need for a registration via regsvr32
*      - in your IDE (IE: eclipse.org) project add a library reference to comm.jar to get your javax.comm java code to compile
*         
* 20101025 : bi-directional command format 32 : 96 + address=0..31 : 48 + value=0..11 --> IE: put(0,9) --> 32:48:105
*                  Note: comm start will reset the Propeller chip - therefore make sure SPIN or ASM
*                  code is written to EEPROM and not just RAM. 
*
*/
public class SerialDriver implements SerialPortEventListener, Runnable {
    // http://download.oracle.com/docs/cd/E17802_01/products/products/javacomm/reference/api/javax/comm/CommPortIdentifier.html
    private static CommPortIdentifier portId;
    private InputStream inputStream;
    private OutputStream outputStream;
    private SerialPort serialPort;
    private Thread readThread;
    private int baud = 0;
    private String commPortName;
    private int[] propellerLoadMessage = {10,10,2,6,6,9,1,2, 10,3,1,6,5,1,3,10, 10,2,2,4,1,9,2,10, 3,1,4,5,1,9,10,10};
    
    // Display Device specific details
    public static final String DEFAULT_COMM_PORT_NAME = "COM15";
    public static final int DEFAULT_COMM_PORT_BAUD = 38400;
    public static final int NUMBER_DISPLAY_DIGITS = 32;
    public static final int NUMBER_DISPLAY_LINES = 4;
    public static final int NUMBER_DISPLAY_DIGITS_PER_LINE = 8;
    public static final int CHAR_CODE_BLANKING = 10;
    public static final int CHAR_CODE_DEC_POINT_ONLY = 11;
    private static final int PROTOCOL_COMMAND_CODE = 32;
    private static final int PROTOCOL_COMM_DELAY_MS = 80;
    private static final int PROTOCOL_INDEX_CHAR_OFFSET = 96;    
    
    public SerialDriver() {
        this(DEFAULT_COMM_PORT_NAME, DEFAULT_COMM_PORT_BAUD);
    }
    
    public SerialDriver(String portOverride) {
        this(portOverride, DEFAULT_COMM_PORT_BAUD);
    }
    
    public SerialDriver(String portOverride, int baudOverride) {
        if(baudOverride > 0 && baud == 0) {
            baud = baudOverride;
        }
        if(null == portOverride) {
            portOverride = DEFAULT_COMM_PORT_NAME;
        }
        boolean validPort = initialize(portOverride);
        if(validPort) {
            try {
                // Open appName=SerialDriver with timeout=2000 ms
                serialPort = (SerialPort) portId.open("SerialDriver", 2000);
                System.out.println(getTimeStamp() + ": " + portId.getName()
                    + " opened for Propeller chip communications");
            } catch (PortInUseException e) {
                e.printStackTrace();
            }
            if(null != serialPort) {
                // Get an input stream on the loader propeller
                try {
                    inputStream = serialPort.getInputStream();
                } catch (IOException e) {
                    e.printStackTrace();
                }
        
                // Add this class as the listener on the loader port
                try {
                    serialPort.addEventListener(this);
                } catch (TooManyListenersException e) {
                    e.printStackTrace();
                }

                // notify the loader port that we wish to capture events using this class
                serialPort.notifyOnDataAvailable(true);
                try {
                    // setup the loader port
                    serialPort.setSerialPortParams(baud, SerialPort.DATABITS_8,
                        SerialPort.STOPBITS_1, SerialPort.PARITY_NONE);
                    serialPort.setDTR(false);
                    serialPort.setRTS(false);
                } catch (UnsupportedCommOperationException e) {
                    e.printStackTrace();
                }

                readThread = new Thread(this);
                readThread.start();
            }
        } else {
            System.out.println("Unable to open port: " + portOverride);
        }
    }

    private boolean initialize(String portName) {
        commPortName = portName;
        try {
            portId = CommPortIdentifier.getPortIdentifier(commPortName);
        } catch (Exception e) {
            System.out.println(getTimeStamp() + ": " + commPortName + " " + portId);
            System.out.println(getTimeStamp() + ": " + e);
            e.printStackTrace();
        }
        if(null == portId) {
            return false;
        } else {
            return true;
        }
    }
    
    public void run() {
        try {
            Thread.sleep(1000);
        } catch (InterruptedException e) {
            e.printStackTrace();
        }
    }

    private static String getTimeStamp() {
        GregorianCalendar aDate = new GregorianCalendar();
        StringBuffer time = new StringBuffer();
        time.append(aDate.get(GregorianCalendar.YEAR));
        time.append(".");        
        time.append(aDate.get(GregorianCalendar.DAY_OF_MONTH));
        time.append(".");        
        time.append(aDate.get(GregorianCalendar.MONTH) + 1);
        time.append("_");
        time.append(aDate.get(GregorianCalendar.HOUR_OF_DAY));
        time.append(":");
        time.append(aDate.get(GregorianCalendar.MINUTE));
        time.append(":");
        time.append(aDate.get(GregorianCalendar.SECOND));
        time.append(".");
        time.append(aDate.get(GregorianCalendar.MILLISECOND));
        return time.toString();
    }
    
    public void setIndex(int index, int value) {
        propellerLoadMessage[index] = value;
    }
    
    public int getBaud() {
        return baud;
    }
    
    public String getCommPortName() {
        return commPortName;
    }
    
    private String getIndexValuePair() {
        StringBuffer buffer = new StringBuffer();
        int value, index;
        try {
        // get index
        value = inputStream.read();
        if(value == PROTOCOL_COMMAND_CODE) {
            value = inputStream.read();
        }
        buffer.append(Integer.toString(value));
        buffer.append("@");
        // get value
        index = inputStream.read();
        buffer.append(Integer.toString(index - PROTOCOL_INDEX_CHAR_OFFSET));
        } catch (IOException e) {
            e.printStackTrace();
        } finally {
            try {
                outputStream.flush();
                outputStream.close();
            } catch (IOException e2) {}// ignore exception on close
        }
        return buffer.toString();
    }
    
    public void serialEvent(SerialPortEvent event) {
        switch (event.getEventType()) {
        case SerialPortEvent.BI:
        case SerialPortEvent.OE:
        case SerialPortEvent.FE:
        case SerialPortEvent.PE:
        case SerialPortEvent.CD:
        case SerialPortEvent.CTS:
        case SerialPortEvent.DSR:
        case SerialPortEvent.RI:
        case SerialPortEvent.OUTPUT_BUFFER_EMPTY:
            break;
        case SerialPortEvent.DATA_AVAILABLE:
            StringBuffer readBuffer = new StringBuffer();
            int c;
            try {
                while ((c = inputStream.read()) != 13) {
                    // look for command
                    if (c == PROTOCOL_COMMAND_CODE) {
                        readBuffer.append(getIndexValuePair());
                        System.out.println("IN:  " + readBuffer.toString());
                        readBuffer = new StringBuffer();
                    }
                }
                inputStream.close();
            } catch (IOException e) {
                e.printStackTrace();
            } finally {
                try {
                    outputStream.flush();
                    outputStream.close();
                } catch (IOException e2) {}// ignore exception on close
            }
            break;
        }
    }

    public void push(int index, int value) {
        push(index, value, DEFAULT_COMM_PORT_NAME);
    }
    
    public void push(int index, int value, String portOverride) {
        if(null != serialPort) {
            try {
                outputStream = serialPort.getOutputStream();
                // protocol dictates that we write each byte twice
                // prefix command
                outputStream.write(PROTOCOL_COMMAND_CODE);            
                outputStream.write(PROTOCOL_COMMAND_CODE);
                // index 0-23 = 96-119
                outputStream.write(index + PROTOCOL_INDEX_CHAR_OFFSET);
                // value 48-57
                outputStream.write(value);
                // terminate command
                outputStream.write(13);            
                System.out.println("OUT: " + value + "@" + (index));            
                Thread.sleep(PROTOCOL_COMM_DELAY_MS);
                outputStream.flush();
                outputStream.close();
            } catch (IOException ioe) {
                ioe.printStackTrace();
            } catch (InterruptedException ie) {
                ie.printStackTrace();
            }
        }
    }    
    
    public void drive() {
        for(;;) {
            // push out 3 to get the stream started (first 2 are skipped)
            push(0, propellerLoadMessage[0]);
            push(0, propellerLoadMessage[0]);
            push(0, propellerLoadMessage[0]);
            for (int i=0;i<NUMBER_DISPLAY_DIGITS;i++) {
                push(i, propellerLoadMessage[i]);
            }
        }
    }

BugNumber.java
package org.dataparallel.device.model;

import java.io.Serializable;
import java.util.ArrayList;
import java.util.List;

import javax.persistence.Basic;
import javax.persistence.Column;
import javax.persistence.Entity;
import javax.persistence.GeneratedValue;
import javax.persistence.Id;
import javax.persistence.SequenceGenerator;
import javax.persistence.Table;
import javax.persistence.Version;

@Entity
@Table(name="DEVICE_BUGNUMBER")
public class BugNumber implements Serializable {
    private static final long serialVersionUID = 3132063814489287035L;

    @Id
    // keep the sequence column name under 30 chars to avoid an ORA-00972   
    @SequenceGenerator(name="DEV_SEQUENCE_BUGN", sequenceName="DEV_BUGN_SEQ", allocationSize=15)
    @GeneratedValue(generator="DEV_SEQUENCE_BUGN")
    @Column(name="BUG_ID")    
    private Long id;

    @Version
    @Column(name="BUG_VERSION")
    private int version;
    
    @Basic
    private long number;
    private int priority;
    private boolean assignedStatus;
    private String assignedTo;
    
    public BugNumber() {        
    }
    
    public BugNumber(String aNumber) {
        number = Integer.parseInt(aNumber);
    }
    
    public char[] getCharDigits() {
        return Long.toString(number).toCharArray();
    }
    
    public List getIntDigits() {
        List digits = new ArrayList();
        for(Character aChar : getCharDigits()) {
            digits.add(new Integer(Character.getNumericValue(aChar.charValue())));            
        }
        return digits;
    }
    
    public long getNumber() {
        return number;
    }
    public void setNumber(long number) {
        this.number = number;
    }
    public int getPriority() {
        return priority;
    }
    public void setPriority(int priority) {
        this.priority = priority;
    }
    public boolean isAssignedStatus() {
        return assignedStatus;
    }
    public void setAssignedStatus(boolean assignedStatus) {
        this.assignedStatus = assignedStatus;
    }
    public String getAssignedTo() {
        return assignedTo;
    }
    public void setAssignedTo(String assignedTo) {
        this.assignedTo = assignedTo;
    }

    public Long getId() {
        return id;
    }

    public void setId(Long id) {
        this.id = id;
    }

    public int getVersion() {
        return version;
    }

    public void setVersion(int version) {
        this.version = version;
    }
    
    public static void main(String[] args) {
        BugNumber aBug = new BugNumber("266912");
        aBug.getCharDigits();
        aBug.getIntDigits();
        System.out.println(aBug.getNumber());
    }
}
DisplayConsoleClient.java

package org.dataparallel.device.driver;

public class DisplayConsoleClient {

    /**
     * @param args
     */
    public static void main(String[] args) {
        // get port if passed in
        String portName = SerialDriver.DEFAULT_COMM_PORT_NAME;
        int baud = SerialDriver.DEFAULT_COMM_PORT_BAUD;        
        if(null != args) {
            if(args.length > 1) {
            portName = args[0];
            }
            if(args.length > 2) {
                baud =Integer.parseInt(args[1]);
             }
            
        }
        ApplicationService aService = new ApplicationService(
                ApplicationService.CAPTURE_URL_DEFAULT,
                portName, 
                baud);
         aService.processLoop();
    } // main
}
ApplicationService.java
package org.dataparallel.device.driver;

import java.io.BufferedInputStream;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.net.HttpURLConnection;
import java.net.MalformedURLException;
import java.net.URL;
import java.net.UnknownServiceException;
import java.util.ArrayList;
import java.util.List;

/**
 * The ApplicationService class handles download and processing of HTTP data
 * to be downloaded to the hardware device.
 * @author mfobrien
 */
public class ApplicationService {
 private String captureURL = null;
 private boolean useHTTPProxy = true;
 private String port = null;
    private SerialDriver serialDriver; // lazy load the serial driver

    // number of total digits on the display device (currently 4 rows of 8)  public int[] propellerLoadMessage = {10,10,2,6,6,9,1,2, 10,3,1,6,5,1,3,10, 10,2,2,4,1,9,2,10, 3,1,4,5,1,9,10,10};
    
    // Invariant constants
    // buffer size when reading from the ftp source 
    private static final int INPUT_BUFFER_SIZE = 1024;
    private static final int SLEEP_TIME = 15 * 60;
    public static final String CAPTURE_URL_DEFAULT = "https://bugs.eclipse.org/bugs/buglist.cgi?query_format=advanced;bug_status=ASSIGNED;component=Documentation;component=Examples;component=Foundation;component=JPA;classification=RT;product=EclipseLink";
    // The current length of bug numbers - as of 2010 we are still at 316000 - we should not hit 999999 until around 2015
    private static final int BUG_LENGTH = 6;

    public ApplicationService() { this(null, null, 0);    }
    public ApplicationService(String aUrl) { this(aUrl, null, 0);    }
    public ApplicationService(String aUrl, String aPort) { this(aUrl, aPort , 0);    }
    
 public ApplicationService(String aUrl, String aPort, int aBuadRate) {
     captureURL = aUrl;
     port = aPort;
  // initialize logging
  if(useHTTPProxy) {
    // inside a firewall only
    System.getProperties().put("proxySet","true"); 
    System.getProperties().put("proxyHost", "www-proxy.yourorg.com"); 
    System.getProperties().put("proxyPort",  "80");
   }

  bugs = new ArrayList();
 }

 /**
  * PUBLIC:
  * Return the URL that will be parsed for device data.
  * @return
  */
 public String getCaptureURL() {
     // lazy load default if not set
     if(null == captureURL) {
         captureURL = CAPTURE_URL_DEFAULT;
     }
     return captureURL;
 }
 
 /**
  * Set the URL that will be parsed for device data.
  * @param aURL
  */
 public void setCaptureURL(String aURL) {
     captureURL = aURL;
 }
 
 public SerialDriver getSerialDriver() {
     return getSerialDriver(null);
 }
 
 public SerialDriver getSerialDriver(String portOverride) {
     if(null == serialDriver) {
         if(null == portOverride) {
             serialDriver = new SerialDriver(port);
         } else {
             serialDriver = new SerialDriver(portOverride);
         }
     }
     return serialDriver;
 }
 
 public void resetBugs() {
     bugs = new ArrayList();
 }
 
    private boolean write32digits(String portOverride) {
        for(int i=0;i<32;i++) {
            getSerialDriver(portOverride).push(i, propellerLoadMessage[i], portOverride);
        }
        return true;
    }
 
    private String captureBugList(String urlString) throws Exception {
        /** this stream is used to get the BufferedInputStream below */
        InputStream abstractInputStream = null;
        /** stream to read from the FTP server */
        BufferedInputStream aBufferedInputStream = null;
        /** stream to file system */
        FileOutputStream aFileWriter = null;
        /** connection based on the aURL */
        HttpURLConnection aURLConnection = null;
        /** URL object that we can pass to the URLConnection abstract factory */
        URL     aURL = null;
        long byteCount;         
        // mark the actual bytes read into the buffer, and write only those bytes
        int bytesRead;
        String line;
        StringBuffer pageBuffer = new StringBuffer();
        // regular expression objects
        try {
            // Clear output content buffer, leave header and status codes
            // throws IllegalStateException
            aURL  = new URL(urlString);
                            
            // get a connection based on the URL
            // throws IOException
            aURLConnection = (HttpURLConnection)aURL.openConnection();
    
            // get the abstract InputStream from the URLConnection
            // throws IOException, UnknownServiceException
            abstractInputStream = aURLConnection.getInputStream();
            aBufferedInputStream = new BufferedInputStream(abstractInputStream);
            // signed byte counter for file sizes up to 2^63 = 4GB * 2GB
            byteCount = 0;
                
            System.out.println("Downloading quote from: " + urlString);
            // buffer the input
            // Note: the implementation of OutputStream.write(,,)
            // may not allow the buffer size to affect download speed
            // Also: the byte array is preinitialized to 0-bytes
            // Range is -128 to 127
            byte b[] = new byte[INPUT_BUFFER_SIZE];
            
            // Read a specific amount of bytes from the input stream at a time
            // and redirect the buffer to the servlet output stream.
            // A -1 will signify an EOF on the input.
            // Start writing to the buffer at position 0
            // throws IOException - if an I/O error occurs.
            while ((bytesRead = aBufferedInputStream.read(
                    b,              // name of buffer
                    0,              // start of buffer to start reading into
                    b.length        // save actual bytes read, not default max buffer size
                )) >= 0) {
                /**
                 * We will use the write() function of the abstract superclass
                 * OutputStream not the print() function which is used for html
                 * output and appends cr/lf chars.
                 * Only write out the actual bytes read starting at offset 0
                 * throws IOException 
                 * - if an I/O error occurs. 
                 * In particular, an IOException is thrown if the output stream is closed.
                 * IE: The client closing the browser will invoke the exception
                 * [Connection reset by peer: socket write error]
                 * IOException: Software caused connection abort: socket write error
                 * 
                 * If b is null, a NullPointerException is thrown.
                 * 
                 * Note: The default implementation of write(,,) writes one byte a time
                 * consequently performance may be unaffected by array size
                 */
                // keep track of total bytes read from array
                byteCount += bytesRead;
                // read to \r\n, \r or \n
                line = new String(b);
                pageBuffer.append(line);
            } // while
            System.out.println("\nHTML capture/processing complete: bytes: " + byteCount);
            // we successfully streamed the file 
            aBufferedInputStream.close();
            aURLConnection.disconnect();
            if(null != aFileWriter) {
                aFileWriter.flush();
                aFileWriter.close();
            }
        } catch (IllegalStateException e) {
            e.printStackTrace();
            throw e;                            
        } catch (UnknownServiceException e) {
            // testcase: remove ftp prefix from the URL
            e.printStackTrace();
            throw e;                            
        } catch (MalformedURLException e) {
            // testcase: remove ftp prefix from the URL
            e.printStackTrace();
            throw e;            
        } catch (IOException e) {
            // 403 testcase: add text after ftp://
            e.printStackTrace();
            throw e;                            
        } catch (Exception e) {
            e.printStackTrace();
        } finally {
            // close input stream
            if(aBufferedInputStream != null) {
                aBufferedInputStream.close();
            } // if
            // close file stream
            if(aFileWriter != null) {
                aFileWriter.flush();
                aFileWriter.close();
            }
            // dereference objects
        } // finally
        return pageBuffer.toString();
    } // captureURL
 
    private void processPage(String page) {
        String searchBugFragment = "show_bug.cgi?id=";
        String searchPriorityFragment = ")
        int searchPosition = 0;
        int priorityIndex;
        int userIndex;
        // keep a potential bug around until i verify it is assigned properly on the next while loop pass
        BugNumber potentialBug = null;
        // clear bug list for repeated URL parsing
        resetBugs();
        while(searchPosition < page.length()) {
            int thisBugPosition = page.indexOf(searchBugFragment, searchPosition);
            if(thisBugPosition < 0) {
                searchPosition += searchBugFragment.length();
            } else {
                // get bug #
                String bugString = page.substring(thisBugPosition + searchBugFragment.length(), 
                        thisBugPosition + searchBugFragment.length() + BUG_LENGTH);
                // if we get a new bug # before matching the user - discard old one
                potentialBug = new BugNumber(bugString);
                // verify we did not skip an intervening bug # by checking for the next one
                int nextBugIndex = page.indexOf(searchBugFragment,thisBugPosition + searchBugFragment.length());
                if(nextBugIndex < 0) {
                    // we are at end of doc - no more bugs
                    nextBugIndex = page.length() - 1;                    
                } 
                // Bug # found, check "assigned" field between them
                userIndex = page.substring(thisBugPosition + searchBugFragment.length() + BUG_LENGTH, 
                        nextBugIndex).indexOf(searchUserFragment);
                if(!(userIndex < 0)) {
                    System.out.println("Found " + potentialBug.getNumber());
                    bugs.add(potentialBug);
                }
                // move pointer
                searchPosition = thisBugPosition + searchBugFragment.length();
            }
        }
        // get the Priority
        // keep the values if assigned to me
    }
    
    /**
     * Select 4 out of (x) bugs to send to device
     */
    private void reduceAndPrepareBugList() {
        // sort by priority
        int bugCount = 0;
        int bugDisplayOffset = 1; // where to start writing bug digits
        // preload display buffer with position markers
        for(int i=0;i@lt;SerialDriver.NUMBER_DISPLAY_DIGITS;i++) {           propellerLoadMessage[i] 
        }
        for(BugNumber bug : bugs) {
            // preload display line with blanks
            for(int i=0;i<SerialDriver.NUMBER_DISPLAY_DIGITS_PER_LINE;i++) {
                propellerLoadMessage[bugCount * SerialDriver.NUMBER_DISPLAY_DIGITS_PER_LINE + i] 
                                     = SerialDriver.CHAR_CODE_BLANKING;
            }
            // only display the first four bugs
            if(bugCount < SerialDriver.NUMBER_DISPLAY_LINES) {
                List<Integer> digits = bug.getIntDigits();                
                for(int i=0;i<BUG_LENGTH;i++) {
                    propellerLoadMessage[
                        (bugCount * SerialDriver.NUMBER_DISPLAY_DIGITS_PER_LINE) + i + bugDisplayOffset] =
                        digits.get((BUG_LENGTH - bugDisplayOffset) - i);  
                }
                bugCount++;                                     
            }
        }
    }    /**
     * Download and process the message extracted from the URL to the device.
     * @param port
     */
    public void processLoop() {
        processLoop(null);
    }
    
    public void processLoop(String port) {
        String page;
        try {   
            for(;;) {
                page = captureBugList(getCaptureURL());
                // try again if the service is down after 5 sec
                int retries = 10;
                while(retries-- > 0) {
                    if(page == null || page.isEmpty()) {
                        System.out.println("> connection timeout - try again in 5 sec");
                        Thread.sleep(5000);
                        page = captureBugList(getCaptureURL());
                    } else {
                        retries = 0;
                    }
                }
                // process page
                processPage(page.toString());
                // reduce bug list from x bugs to 4
                reduceAndPrepareBugList();
                // do it twice (as first 2 may not write - give time for FTDI to initialize)
                write32digits(port);
                write32digits(port);
                Thread.sleep(SLEEP_TIME * 1000);
            }
        } catch (Exception e) {
            e.printStackTrace();
        } // try        
    }
 
 public static void main(String[] args) {
     // Test applicationService
  ApplicationService aService = new ApplicationService();
  aService.setCaptureURL(CAPTURE_URL_DEFAULT);
        // get port if passed in
        String portName = null;
        if(null != args && args.length > 0) {
            portName = args[0];
        }
        for(;;) {
            aService.processLoop(portName);
            }
 } // main
} // ApplicationService

Spin code:
CON
  _clkmode = xtal1 + pll16x
  _xinfreq = 5_000_000
VAR
  ' shared display RAM for the 4 display cogs
  long buffer[32]                                         
  long Stack0[64]                                         ' Stack Space
  long Stack1[64]                                         ' Stack Space
  long Stack2[64]                                         ' Stack Space
  long Stack3[64]                                         ' Stack Space
  long Stack4[16]                                         ' Stack Space
  long Stack5[16]                                         ' Stack Space
  long Stack6[16]                                         ' Stack Space
  byte Cog[6]                                             ' Cog ID
  byte flashFlag                                          ' visual clock bit
  long  randomNum
  
OBJ
   SER    : "FullDuplexSerial"    
  
PUB main | mIndex, mValue, lRec, i, tPin
  {{
    Each line of 8 LED 7-segments are driven by a 595 SIPO pair by a single cog
    No resistors are required as the inherent resistance of the 595 output pins
    is enough to limit the current below 20ma.
    The 3 pin parameters below are for the Shift Clock, Register Clock and Data pins.
  }}
  Cog[0] := cognew(Push8seg(0,buffer,2,3,4, 24_000,0), @Stack0) + 1
  Cog[1] := cognew(Push8seg(8,buffer,5,6,7,50_000,0), @Stack1) + 1
  Cog[2] := cognew(Push8seg(16,buffer,8,9,10,24_000,0), @Stack2) + 1
  Cog[3] := cognew(Push8seg(24,buffer,11,12,13,30_000,0), @Stack3) + 1

  ' echo Comm port data back to the sender
  ' comm control needs 2 cogs
  ser.start(31,30,0,38400)
  lRec := 48
  i := 0
   repeat
     ' wait for char
     lRec := ser.rx
     ' check for command 32 with index:value next
     if lRec == 32
       ser.tx(32)
       ' get index
       lRec := ser.rx
       mIndex := lRec - 96
       ' get value
       lRec := ser.rx
       mValue := lRec - 48              
       lRec := ser.rx       
     ' check for EOL
     if lRec == 13
       if i < 32
         i := i + 1
       else
         i := 0
       ser.tx(mValue + 48)
       ser.tx(mIndex + 96)
     else
         ser.tx(lRec) ' this represents the value
         buffer[mValue - 48] := lRec
{{
7seg CA CC (China) blue=ca, red=cc

    g  f  +  a  b
    1  2  3  4  5
 
    -a
  f|  |b
    -g
  e|  |c
    -d.dp
    
   10  9  8  7  6
    e  d  +  c  dp
}}
PUB Write595BitD(Data2,cpin,dpin,delay)
  ' write data
  outa[dpin] := Data2
  ' toggle clock
  outa[cpin] := 0
  waitcnt(cnt + clkfreq / delay * 1)
  outa[cpin] := 1
  waitcnt(cnt + clkfreq / delay * 1)
PUB Write595Bit(Data2,cpin,dpin)
  ' write data
  outa[dpin] := Data2
  ' toggle clock
  outa[cpin] := 0
  outa[cpin] := 1
PUB Push8seg(dOffset, Data,cpin,rpin,dpin, delay,mux) | dValue,loop, index, muxOffset, dByte, blueOn, cBit, dBit, sft595, tempPartAddress, segValue, colVal, hiddenColumn
  ' LED display is driven by 2 595 shift registers
  ' 595-1 is the column driver
  ' 595-2 is the data driver
  
  dira[cpin] := 1
  outa[cpin] := 0
  dira[rpin] := 1
  outa[rpin] := 0
  dira[dpin] := 1
  outa[dpin] := 0
  ' Hardware configuration
  ' 76543210 <--  sweep direction
  ' BBBBRRRR(BlueCA/RedCC)
  ' In format --> shift direction
  ' 0.............15
  ' gfab.cde0000c000
  
  muxOffset := 0
  repeat
   if muxOffset > 0
    muxOffset := 0
   else
    muxOffset := 1
   repeat loop from 16 to 100
    
    ' initially blank display
    ' shift 16 bits into 1 of 4 rows 0-7=column, 8-f=data
    ' blue 4 digits - common anode
     repeat dByte from 0 to 7'buffSize
       dValue := buffer[dByte + dOffset]
     ' do PWM
      if mux > 0
       repeat 1'loop / 16
        'waitcnt(clkfreq / 1000 + cnt)
        if muxOffset > 0
         repeat 16
          Write595Bit(0,cpin,dpin)
        ' push storage reg to output reg
         outa[rpin] := 0
         outa[rpin] := 1
       
      waitcnt(clkfreq / (delay) + cnt)
       if dByte < 4
         colVal := 0
       else
         colVal := 1       
       ' shift in data bits first
       repeat dBit from 0 to 7
         'segValue := segDigit[8 * buffer[dByte] + (7 - dBit)]
         if dValue < 16
           segValue := segDigit[(8 * dValue + (7 - dBit))]
         else 
           segValue := segDigit2[(8 * (dValue - 16) + (7 - dBit))]
         if colVal < 1       
            Write595Bit(segValue,cpin,dpin)
         else
            Write595Bit(1 - segValue,cpin,dpin)
          
        
       ' shift in column bits last
         {
       ' 0...:...7
         1000:1111
         0100:1111
         0010:1111
         0001:1111
         0000:0111
         0000:1011
         0000:1101
         0000:1110
         }
       ' skip initial blue
       hiddenColumn := 0
       blueOn := 1
       if dByte < 4 ' red
         ' 4 blank blue
         repeat 4
           Write595Bit(1 - blueOn,cpin,dpin)          
         repeat (3 - dByte)
           Write595Bit(blueOn,cpin,dpin)
         Write595Bit(1 - blueOn,cpin,dpin)          
         repeat dByte
           Write595Bit(blueOn,cpin,dpin)
       else ' blue
         repeat (3 - (dByte - 4))
           Write595Bit(1 - blueOn,cpin,dpin)
         Write595Bit(blueOn,cpin,dpin)          
         repeat (dByte - 4)
           Write595Bit(1 - blueOn,cpin,dpin)
         ' 4 blank red
         repeat 4
           Write595Bit(blueOn,cpin,dpin)          

     ' push storage reg to output reg
      outa[rpin] := 0
      outa[rpin] := 1
DAT
  'gfab.cde                                                                  ' 0 1 2 3 4 5 6 7 8 9 a b c d e f g h i j k l m n o p q r s t u v w x y z - +
  segDigit byte 0,1,1,1, 0,1,1,1,  0,0,0,1, 0,1,0,0,  1,0,1,1, 0,0,1,1,  1,0,1,1, 0,1,1,0,  1,1,0,1, 0,1,0,0,  1,1,1,0, 0,1,1,0,  1,1,1,0, 0,1,1,1,  0,0,1,1, 0,1,0,0,  1,1,1,1, 0,1,1,1,  1,1,1,1, 0,1,1,0, 0,0,0,0, 0,0,0,0, 0,0,0,0, 1,0,0,0

Running the code:
- download the SPIN code to the propeller microcontroller
- run the Java client on the Windows/Linux PC
c:\jdk1. 6.0_32bit\bin\java -cp .;javax.persistence_2.0.3.v201010191057.jar;eclipselink.jar;comm.jar;derbyclient.jar org.dataparallel.device.driver.ApplicationService COM15

Logs/Console Output
[EL Finest]: 2010-12-08 16:38:18.527--ServerSession(27379847)--Thread(Thread[main,5,main])--End deploying Persistence Unit dataparallel.derby; session file:/F:/wse/view_w35e/org.dataparallel.device.DisplayDriver/bin/_dataparallel.derby; state Deployed; factoryCount 1
Metamodel: MetamodelImpl@14105722 [ 6 Types: , 1 ManagedTypes: , 1 EntityTypes: , 0 MappedSuperclassTypes: , 0 EmbeddableTypes: ]
Downloading quote from: https://bugs.eclipse.org/bugs/buglist.cgi?query_format=advanced;bug_status=ASSIGNED;component=Documentation;component=Examples;component=Foundation;component=JPA;classification=RT;product=EclipseLink

HTML capture/processing complete: bytes: 68041
Found http://bugs.eclipse.org/307105
Found http://bugs.eclipse.org/325167
Found http://bugs.eclipse.org/326728
2010.8.12_16:38:20.385: COM15 opened for Propeller chip communications
OUT: 10@0
OUT: 5@1
OUT: 0@2
OUT: 1@3
OUT: 7@4
OUT: 0@5
OUT: 3@6
OUT: 10@7
OUT: 10@8

Circuit Diagram:
Pending... along with refactored fault-tolerant code and JPA persistence for DB access.

Update: 20101207:
Added Java JPA 2.0 persistence (PC side) via the EclipseLink 2.1.2 implementation of the spec.  The database is currently Derby 10.5.3.0 - which ships with most application servers and the JDK itself.

Update: 20101121: The Propeller controller on this board will be replaced and enhanced with the Spinneret Web Server which contains both the 32-bit Propeller processor and a WIZnet W5100 TCP/IP Ethernet controller - allowing updates to occur over the network instead of via USB.
Also, this board relies on the host PC to gather internet data from a URL - with the Spinneret we can do this all in the embedded device (doing away with the PC alltogether).  We will need proxy support though.

Parallax Spinneret product page:
http://www.parallax.com/Store/Microcontrollers/PropellerDevelopmentBoards/tabid/514/ProductID/710/List/0/Default.aspx?SortField=ProductName,ProductName
Parallax Spinneret discussion/development forum
http://forums.parallax.com/forumdisplay.php?f=82
http://forums.parallax.com/showthread.php?121494-Locating-amp-Identifying-propeller-chip-via-USB&p=1071158&viewfull=1#post1071158
http://forums.parallax.com/showthread.php?132944-Java-comm-and-basic-stamp&p=1071159&viewfull=1#post1071159
http://forums.parallax.com/showthread.php?109333-HowTo-Single-host-computer-and-multiple-propeller-serial-comm-and-programming&p=775612&viewfull=1#post775612
http://forums.parallax.com/showthread.php?137636-Quickstart-Board-reboots-when-PC-GUI-program-starts-and-stops&p=1071154&viewfull=1#post1071154


thank you
/michael