
OpenPCIE – FPGA Root Complex, OpenBackplane, EduBOS5 RISC-V, Co-SIM
Computing is about communicating. Some would also say about networking. Digital independence tags along on the wave of "Recommendations and Roadmap for European Sovereignty in open source HW, SW and RISC-V Technologies …
以下正文同步自 Hacker News 最新,版权归原站所有,已转换为易读排版。
Computing is about communicating. Some would also say about networking. Digital independence tags along on the wave of "Recommendations and Roadmap for European Sovereignty in open source HW, SW and RISC-V Technologies (2021)", calling for the development of critical open source IP blocks, such as PCIE Root Complex (RC). This is the first step in that direction. And, if you are looking for an opensource, soft PCIE EndPoint (EP) core, please check our other repo.
This project opens Artix7 PCIE Gen2 RC IP blocks for use outside of proprietary tool flows. While still reliant on Xilinx Series7 Hard Macros (HMs), it surrounds them with open-source soft logic for PIO accesses — The RTL and, even more importantly, the layered software Driver with Demo App.
All that with full HW/SW opensource co-sim. Augmented with a rock-solid openBackplane in the basement of our hardware solution, the geek community thus gets all it takes for building their own, end-to-end openCompute systems.
The project‘s immediate goal, now accomplished, was to empower the makers with ability to drive PCIE-based peripherals from their own soft RISC-V SOCs.
Given that the PCIE End-Point (EP) with DMA is already available in opensource, the opensource PCIE peripherals do exist for Artix7. Except that they are always, without exception, controlled by the proprietary RC on the motherboard side, typically in the form of RaspberryPi ASIC, or x86 PC. This project changes that status quo.
Our long-term goal is to set the stage for the development of full opensource PCIE stack, gradually phasing out Xilinx HMs from the solution. That’s a long, ambitious track, esp. when it comes to mixed-signal SerDes and high-quality PLLs. We therefore anticipate a series of follow on projects that would build on the foundations hereby set.
This first phase implemented an open source PCIE Root Complex (RC) for Artix7 FPGA, utilizing Xilinx Series7 PCIE HM and GTP IP blocks, along with their low-jitter PLL.
PCIE Primer by Simon Southwell ✔
HW Architecture
The project relies on a modular hardware ecosystem that combines our custom-designed openPCIE Backplane with SQRL Acorn CLE-215+ FPGA modules and various PCIE adapters to create a flexible testing platform.
Please, refer to 1.pcb for additional detail, and to 1.pcb/KiCad-library for the self-contained KiCad library. Plain-text versions of the PDF and slide documents are in 0.doc/1.pcb.
The hardware platform for this project is the SQRL Acorn CLE-215+, a versatile FPGA development board. Although originally designed as a crypto-accelerator, its powerful Artix-7 FPGA and modular design make it an excellent choice for general-purpose PCIE development.
The system consists of two main components:
The fully assembled Acorn CLE-215+ development board, ready for use in a PCIE slot.
It is important to note that the Acorn CLE-215+ is functionally identical to the more widely known NiteFury board, with the primary difference being the amount of onboard memory. The Acorn model features 1 GB of DDR3 RAM, while the standard NiteFury has 512 MB. Therefore, the NiteFury schematic serves as a direct and accurate reference for the board's hardware layout.
The central component of the SQRL Acorn CLE-215+ system is the Xilinx Artix-7 XC7A200T-FBG484 chip. This FPGA is crucial for implementing the PCIE Root Complex - and, on the other card, the Endpoint - possessing a range of features that make it highly suitable for this purpose.
The key specifications are summarized below:
¹ The 'Logic Cells' count is a Xilinx metric derived from the physical 6-input LUTs to provide an estimated equivalent in simpler 4-input LUTs for comparison purposes. The number of physical LUTs and other resources are the exact counts for the XC7A200T chip.
Acorn-CLE215+ QuickStart
NiteFury
fpgas.online-test-designs PCIE ENUMERATION by RPI, for Acorn-CLE215+ EP
RTL Architecture
The opensource Root Complex RTL is a chain of blocks joined by SystemVerilog interfaces: the RISC-V SoC hands TLPs over AXI-Stream to the PCIE stack, which converts them for the PCIE_2_1 hard block and drives the GTP transceiver over PIPE.
Details are in the RTL README. Per design: common RTL, EP, RC-direct, RC-switched. The AMD/Vivado-IP reference designs are in 2.amd-rtl-with-Vivado-build (EP, RC-direct, RC-switched).
Basic PCIE EP for LiteFury
Regymm PCIE
LiteX PCIE EP
PCIE EP DMA - Wupper
Xilinx UG477 - 7Series Integrated Block PCIe
Xilinx DS821 - 7series_PCIE Datasheet
Xapp1052 - BusMaster DMA for EP
SW Architecture
The software stack is designed to run bare-metal on the soft RISC-V SoC embedded within the FPGA. In a standard PC, the Operating System (Linux/Windows) manages PCIE enumeration automatically in the background, hiding the complexity from the developer. In this project, our baremetal open-source driver takes full control, manually performing every step of the enumeration process to act as the PCIE Host.
The architecture adopts a clean, well-defined, layered approach:
Note: This structure allows developers to treat PCIE devices just like any other local peripheral, abstracting away the complexities of the physical link.
Sim Test Bench
The openPCIE test bench aims to have a flexible approach to simulation which allows a common test environment to be used whilst selecting between alternative CPU components, one of which uses the VProc virtual processor co-simulation element. This allows simulations to be fully HDL, with the RISC-V RTL core that goes into the bitstream (eduBOS5 by default, or PicoRV32), or to co-simulate software using the virtual processor, with a significant speed up in simulation times.
The test bench has the following features:
More details on the architecture and usage of the openPCIE test bench can be found in the README.md in the 5.sim directory.
The co-simulation can also be followed at TLP level in the WaveCrux waveform viewer, with an open-source PCIE TLP decoder plugin and a ready-made session -- here the firmware sizing BAR0 by readback and placing it at 0x80000000 (more):
One level down, Ferrite's PCIe PIPE and Data Link Layer decoders show the same link on the wire: link training, flow-control credits, and every TLP with its sequence number and Ack (more). With all three layers in one view, you can also see how long a TLP spends inside the hard macro: about 0.4 µs each way (more).
The openPCIE control and status register hardware abstraction layer (HAL) software is auto-generated, as is the CSR RTL, using peakrdl. A single SystemRDL specification, 4.build/csr_build/csr.rdl, describes the TLP transmit/receive window the RISC-V firmware drives, and everything else is derived from it: the register block RTL (csr.sv, csr_pkg.sv), which soc_csr.sv bridges to the picorv32 memory bus, and the software header. For co-simulation purposes an additional layer is auto-generated from the same SystemRDL specification using systemrdl-compiler that accompanies the peakrdl tools. This produces two header files that define a common API to the application layer for both the RISC-V platform and the VProc based co-simulation verification environment. The details of the HAL generation can be found in the README.md in the 4.build/ directory.
More details of the test bench, the pcievhost component and its usage can be found in the 5.sim/README.md file. See also the pcievhost model, its LTSSM support code, and the superseded DUT stubs.
pcieVHost
WaveCrux waveform viewer, decoder plugin interface
wavecrux-decoders: Ferrite's open-source PCIE PIPE and Data Link Layer decoders for WaveCrux
Implementation Workflow
The openPCIE implementation workflow is organized into several stages, covering the complete process from setting up the development environment to verifying the design on physical hardware.
Before building the project, install and configure the required development tools. These include an FPGA toolchain such as AMD Vivado or openXC7, a bare-metal RISC-V cross-compilation toolchain, Python, and the project-specific dependencies listed in the corresponding directories.
Before deploying the design to physical hardware, run the available simulation environment to verify the interaction between the PCIE hardware design and the software components.
The co-simulation flow is used to test essential operations such as device initialization, register access, PCIE link behavior, and data exchange between the Root Complex (RC) and Endpoint (EP). This stage helps identify hardware and software integration issues before synthesis and hardware deployment.
Compile the bare-metal RISC-V driver and test application using the configured cross-compilation toolchain, see 4.build/sw_build.
The software is responsible for initializing the required hardware components, configuring the PCIE communication flow, and transferring test payloads between the RC and EP. The generated executable can be used during simulation or deployed together with the FPGA design for hardware verification.
The FPGA design can be synthesized and implemented using either the commercial AMD Vivado toolchain or the open-source openXC7 toolchain. Depending on the selected flow, the process includes design elaboration, synthesis, placement, routing, timing analysis, and final bitstream generation.
The resulting .bit file contains the complete hardware configuration for the selected FPGA board. The appropriate board-specific constraints, build configuration, and target design must be selected depending on whether the board operates as the Root Complex (RC) or Endpoint (EP).
Detailed build instructions for both the Vivado and openXC7 flows are provided in the corresponding project directories.
Program the RC and EP FPGA boards with their corresponding bitstreams. After both designs have been loaded, start the software application and allow the boards to initialize the PCIE connection.
The deployed system can then be verified by checking the PCIE link-status indicators and comparing the transmitted payload with the data displayed or received on the Endpoint.
Detailed commands, configuration options, supported boards, and stage-specific instructions are provided in the corresponding project directories.
FPGA Board Setup
Properly programming and operating the Artix-7 FPGA on the SQRL board requires some soldering and work on open desktop motherboard.
The JTAG connector on the Acorn CLE-215+ is non-standard and not directly compatible with the standard 14-pin connector on the Xilinx Platform Cable. A custom adapter cable is therefore required.
The connector on the board is a Molex Pico-Lock 1.50mm pitch male header. This is not a standard 2.54mm or 2.00mm header, so standard DuPont-style cables will not fit.
To simplify making the cable, we highly recommend purchasing a pre-assembled cable with the correct female connector.
This cable has the correct female connector on both ends. The easiest method is to cut the cable in half, which gives you two connector cables with open ends. You can then splice one of these cable ends onto the wires of your Xilinx programmer cable, matching the signals according to the following wiring diagram.
JTAG Connection Guide: Physical Pinout and Wiring Diagram.
The board cannot be programmed or operated solely from the PCIE/M.2 slot power. It requires an external 12V supply to function correctly, especially when complex designs and high-speed transceivers are active. Power is provided via a standard 6-pin PCIE power connector from an ATX power supply.
The complete system, including the custom cabling, is mounted in a test PC chassis for verification.
The complete FPGA system mounted in a PCIE slot.
After the hardware was prepared, the connection was verified using the Vivado Hardware Manager. As shown below, the tool successfully detected the JTAG programmer and identified the xc7a200t_0 FPGA chip. This confirms that the physical connections are correct and the board is ready for programmin
正文由 探How 从来源站点 RSS 同步,内容未经改写;遇到排版缺失或需要图片、视频时请以原文为准。