XUPV5

XUPV5

The XUPV5 is a cheap commodity FPGA platform provided by Xilinx. It is generally free (donated) for university users.

Post-11.09

The XUPV5 driver is now available in apt (following the instructions here):

sudo apt get install leap-platforms-xupv5

No special setup is required in newer LEAP releases, as we have migrated to dkms. You will need to reboot your machine after the first time you program the XUPV5, but the executable will prompt you to do so.

Note, you must still set your environment correctly:

export LIBRARY_PATH=$BLUESPECDIR/SceMi/g++4_64/:$LIBRARY_PATH
export CPATH=$BLUESPECDIR/SceMi/:$CPATH

Pre-11.09

The XUPV5 PCIE requires some bootstrapping. First, the FPGA must be programmed with a valid PCIE bitfile using Impact. Producing the first bitfile in AWB needs a little setup as building for SceMi requires linking against Bluespec provided libraries. Add the following to your initialization script (assuming you’re building on a 64-bit machine).

export LIBRARY_PATH=$BLUESPECDIR/SceMi/g++4_64/:$LIBRARY_PATH
export CPATH=$BLUESPECDIR/SceMi/:$CPATH

Building a suitable bitfile can now be accomplished by building the hello_hybrid_xupV5 model in the leap/demos directory in AWB and loading the bitfile (located in $WORKSPACE/build/ with impact. After programming the board and rebooting, lspci -vv should show:

  04:00.0 RAM memory: Device b100:b5ce (rev 07)
  Subsystem: Device b100:b5ce
  Flags: bus master, fast devsel, latency 0, IRQ 16
  Memory at db000000 (32-bit, non-prefetchable) [size=8M]
  Memory at db800000 (32-bit, non-prefetchable) [size=8M]
  Memory at dc000000 (64-bit, non-prefetchable) [size=32M]
  Expansion ROM at da000000 [disabled] [size=1M]
  Capabilities: <access denied>

We can now load the PCIE driver. Currently we are using PCIE drivers provided in the Bluespec installation. We are working on improving them for our own usage.

 $cd $BLUESPECDIR/lib/board_support/drivers
 $make
 $sudo ./load_scemi_pcie_driver.sh

This builds and loads the driver. We recommend loading the kernel module at init time.

Now, lspci -v will show bsemu as the kernel driver in use for the RAM memory device:

  04:00.0 RAM memory: Device b100:b5ce (rev 07)
  Subsystem: Device b100:b5ce
  Flags: bus master, fast devsel, latency 0, IRQ 16
  Memory at db000000 (32-bit, non-prefetchable) [size=8M]
  Memory at db800000 (32-bit, non-prefetchable) [size=8M]
  Memory at dc000000 (64-bit, non-prefetchable) [size=32M]
  Expansion ROM at da000000 [disabled] [size=1M]
  Capabilities: <access denied>
  Kernel driver in use: bsemu

You should now be able to run the null benchmark with hello_hybrid_xupV5.

Testing and Errata

PCIE works well after rebooting, but if we then decide to reprogram the board, the next time we try to run a test an error may be generated:

Error: ID register read returned ffffffffffffffff (expected 426c756573706563)

In some PCIE chipsets, the address registers may be misconfigured during reprogramming as low-end desktop chipsets do not expect hot-swapping. Some chipsets will require a reboot, although newer desktops (vintage Core2) can be repaired by manually configuring the address space that the Bluespec PCIE expects. The script template is as follows:

  #!/bin/sh
  setpci -d b100:b5ce COMMAND=7
  setpci -d b100:b5ce CACHE_LINE_SIZE=10
  setpci -d b100:b5ce BASE_ADDRESS_0=db000000
  setpci -d b100:b5ce BASE_ADDRESS_1=db800000
  setpci -d b100:b5ce BASE_ADDRESS_2=dc000000
  setpci -d b100:b5ce ROM_ADDRESS=da000000

This needs to be customized for each physical machine, by entering the correct base address reported for Regions 0, 1, and 2. After running the script lspci -vv should give:

  04:00.0 RAM memory: Device b100:b5ce (rev 07)
  Subsystem: Device b100:b5ce
  Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
  Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
  Interrupt: pin A routed to IRQ 16
  Region 0: [virtual] Memory at db000000 (32-bit, non-prefetchable) [disabled] [size=8M]
  Region 1: [virtual] Memory at db800000 (32-bit, non-prefetchable) [disabled] [size=8M]
  Region 2: Memory at dc000000 (64-bit, non-prefetchable) [disabled] [size=32M]
  [virtual] Expansion ROM at da000000 [disabled] [size=1M]
  Capabilities: <access denied>
  Kernel driver in use: bsemu

Once modified for your local setup, you should install the local version of the pci script in /share/leap/scrips/XUPV5. When running XUPV5 targets, leap-fpga-ctrl will automatically invoke this script at root privilege after programming the FPGA in order to reset the PCI-E registers. You should now be able to run all XUPV5 targets from the AWB GUI.

If you notice that the values reported do not changes after running the script, your chipset is one that requires a reset after reprogram.