De-Risking and Accelerating OS Boot for Arm SystemReady SoCs: Morello Case Study

22 Feb 2022 • 8 minute read

Introduction

Booting a generic off-the-shelf operating system (OS) on any SoC is a significant milestone and requires the SoC to meet a set of minimum hardware and firmware standards. Otherwise, long debug time, software workarounds/customization would lead to delays in the delivery of products if the OS boot does not work just “out of the box.” For commercial OSes such as Red Hat Enterprise Linux, VMware ESXi, and Microsoft Windows, such per-SoC customization would not be possible. Not meeting the minimum hardware standards may result in requiring a re-spin of silicon. Common issues are the OS may fail to boot or crash or hang. In devices that integrate PCIe interfaces, the hierarchy may be invalid or have no Root Port or there may be End Point interoperability issues if the minimum PCIe hardware standard requirements are not met.

Demonstrating pre-tapeout OS-boot can be an important test for building confidence in the design; however, there are several problems in doing that. This blog discusses in detail the major challenges that hinder achieving “out-of-the-box” OS boot. It introduces EDA tools from Cadence and content that accelerate the process of PCIe integration, enabling Arm SystemReady architecture compliance tests to be run on bare-metal RTL using hardware (HW) acceleration and thus achieve pain-free OS boot ahead of SoC tapeout. Arm recently reached a major milestone with the hardware now available for testing.

Although there are a lot of debugging methodologies and pre-silicon verification techniques available today, OS boot is still a very tedious process and must be addressed right from the SoC architecture stage. This blog post discusses how to overcome the challenges faced during the SoC boot operation of standard operating systems such as Linux and Microsoft Windows.

Figure 1: Arm Morello Development Boards

What Is the Morello Program?

The Morello program originated at Arm, which involved the creation of a prototype Morello SoC deployed on a board and the associated firmware – see Figure 1. The SoC had a requirement to be compliant with Arm’s Base System Architecture (BSA) and Server Base System Architecture (SBSA) standard. The primary focus is to describe the challenges faced in proving compliance and how a Cadence solution based on the Cadence Perspec system traffic library helped us to verify PCIe integration in an Arm SoC. The program demonstrated successful pre-silicon booting of OpenSUSE Linux distribution and WinPE Operating Systems on an FPGA platform. Subsequently both Linux and full Windows Operating Systems have been booted on the SoC. Read more to learn about the BSA/SBSA system standard, the benefits of complying with such architecture standards, and how we demonstrated compliance on Morello demonstrator boards. In this post, I describe the challenges in PCIe integration in Arm systems and explain how the usage of Perspec libraries accelerate the FFT implementation.

Morello is a research program, funded by the UK government’s Industrial Strategy Challenge Fund (ISCF) Digital Security by Design (DSbD) initiative and led by Arm.

Morello focuses on improving built-in security. It is based on capability-based architecture, developed by the University of Cambridge and SRI International, called CHERI (Capability Hardware Enhanced RISC Instructions). The Morello program brings this CHERI architecture into a prototype version of Arm architecture which introduces new architectural features to enable fine-grained memory protection and highly scalable software compartmentalization. Read more about the Morello project and architecture.

Arm SystemReady Program

Arm SystemReady is a compliance certification program based on a set of hardware and firmware standards. It enables operating systems to ‘just work’, so that SystemReady certified devices have a base level of interoperability, while still encouraging innovation and differentiation across a broad set of devices ranging from server and hyperscalers to embedded IoT and cloud edge-based devices.

This ensures that subsequent layers and common components like OS, hypervisors, and middleware just work, and partners can focus on innovating and deploying differentiating layers and help in improving the time to market. The compliance certification program tests and certifies that systems meet the SystemReady standards, giving confidence that OS and subsequent layers of software just work off-the-shelf. The SystemReady program is a set of key specifications consisting of Base System Architecture (BSA), XBSA (market segment hardware supplement requirements) and Base Boot Requirements (BBR) specifications, plus a selection of supplements.

Arm SystemReady SR provides a solution for servers or workstations so software "just works," allowing partners to deploy Arm servers or workstations with confidence. The program is based on the BSA/SBSA specification and SBBR recipe in the BBR specification, alongside the Arm Architectural Compliance Suite (ACS). Arm SystemReady SR ensures that Arm-based servers or workstations work out of the box, offering seamless interoperability with standard operating systems, hypervisors, and software. For the Morello program, it is important to ensure that the pre-silicon platform is BSA/SBSA compliant. This will create a low-risk path to SystemReady certification of the physical board.

Pre-silicon BSA/SBSA compliance is key to SystemReady SR and software that just works. Typical SoC design starts with architecture design, verification, and design leading to tapeout, and once silicon is available, there are some further bring-up exercises conducted. In the SystemReady SR program, the architecture must follow BSA/SBSA, then in addition to verification testing, it is required to run the compliance testing based in ACS test suite. As shown below, compliance testing is not a substitute for verification testing and must be done when silicon is available as a part of SystemReady compliance testing.

In this blog we will be discussing pre-silicon aspects of compliance testing, which involves implementation at the microarchitecture level. The overall pre-silicon result needs to include both the successful design verification as well as the compliance testing suite.

System compliance tests are mandatory in addition to standard verification and testing as they check whether the architectural intent and generic set of rules as per BSA/SBSA have been met. This architectural intent must be tested pre-silicon to avoid any re-spins and involves cost.

The layered pre-silicon BSA/SBSA compliance testing as shown consists of ACS tests (generic tests) on the top (by Arm) while at the bottom end there are SoCs, i.e., DUTs (designs under test) that we are testing the compliance to.

For Morello, the PCIe IP has been provided by Cadence and others parts like the CPU, MMU, and interrupt controllers are from Arm. To post the test from a generic platform to a specific implementation of various architectures, we need a middle layer that contains different drivers customized for given a implementation. This layer is created either by silicon partner/SoC creators or EDA vendors like Cadence who can help in creation of this middle layer.

Challenges in Pre-Silicon Verification for PCIe Integration

Server-class SoCs have a common interface to PCIe devices, which is one of their key features. There are certain interoperability issues when integrating Arm with PCIe, including:

Complex verification of environment, namely verification of Arm and PCIe IP together
Complex software setup in verification environment due to preloading for SMMU, GIC, etc.
Longer software sequences demand more time to simulate and debug
Very high simulation run times
PCIe and AMBA interoperability issues
Debugging complex scenarios can be highly time-consuming

Apart from these, some of the common hardware compliance issues while working with PCIe devices on an Arm platform are:

OS fails to boot/crashes/hangs
Invalid PCIe hierarchy, no root port discovered at all
Incorrect ECAM enumeration and devices not discovered
Endpoint interoperability issues

Solution

Measuring the completeness of verification is a well-known challenge. Combining PCIe and AMBA has been a challenge, despite several efforts some of the common hardware compliance issues have been noticed. This creates the demand for SystemReady standards where such issues are addressed at architecture definition itself. So, after architecture definitions, it should be ensured that the ACS tests that are being supplied can run on a given platform. Cadence's Perspec System Verifier Tool comes into this picture to resolve such issues, as fundamentally it is a SoC verification solution. It enables the capturing of use cases and helps in generating C test cases that can be used in each platform.

The Perspec System Verifier is used in Morello SoC for verifying PCIe integration to enable the capture of use cases to generate C-test cases that can be used in a given platform.

Benefits

Saves time by reducing development effort of complex verification scenarios
Enables automated system use case generation in form of C-tests that provide software perspective
Debug aids available

Perspec environment setup

Describe PCIe topology
Describe SoC configuration
Scenario description for PCIe use cases
Test information
Software Library setup for SoC components
Test bench creation with external components

The above figure illustrates how the Perspec tool was used in the Morello program, with the initial process of environment setup shown.

Outcome

The program was able to demonstrate successful pre-silicon booting of OpenSUSE Linux distribution and WinPE Operating Systems on an FPGA platform. The chip passed the pre-silicon BSA/SBSA compliance test. Subsequently, both Linux and full Windows have been successfully booted on the Morello SoC. Arm has announced the release of the Morello Evaluation Boards for more secure processors that involves the creation of a prototype SoC deployed on a board and the associated firmware. This solution made use of the Perspec library solution from Cadence.

Summary

The close Collaboration between the Arm and Cadence engineering teams on Morello has helped result in an environment to execute BSA/SBSA compliance testing of a pre-silicon design. Cadence Perspec technology-generated tests work with the BSA/SBSA compliance tests for the development of the middle software layer as well as in standard SoC design verification. The program was able to demonstrate successful pre-silicon booting of OpenSource Linux distribution and WinPE Operating Systems on an FPGA platform prior to tape-out. Linux and full Windows boot have now also been proven on the Morello SoC

Subscriptions