Increased CHI Coherency Verification Challenges

Cache coherency is not unique to the new ARM® AMBA® 5 CHI (Coherent Hub Interface) specification that enables processors to work together in high-performance, coherent processing “hubs”, and to deliver the high data rates that are common in enterprise markets, such as servers and networking. Coherent architectures have existed for many generations of CPU designs, but verifying adherence to coherency rules has always been one of the most complex challenges faced by verification engineers. However, it becomes even more challenging with increasing number of cores and the introduction of the embedded L3 (level 3) cache to the interconnect device, both of which are hallmarks of CHI-based SoCs.

The fundamental challenge of coherency is that when data is requested by a core (a "master" in the language of coherency), it is not known where the data will be found. The data may return from various sources: system memory, the interconnect's L3 cache, or one of the caches associated with the other masters in the SoC.

To gain an appreciation for this, let's look at an example of a few read commands to the same address in a multi-core system, as shown in the figure below.

Let's assume that Master 2 performs a ReadUnique operation to a specific address, a "line" of memory. In this example, no other masters have this line so the interconnect will fetch the data from main memory (a "slave" component). After the data comes back from main memory, Master 2 sets this cache line to a UniqueDirty state, meaning that it is the only master to hold valid data for this line.

Subsequently, Master 3 attempts to read the same data and performs a ReadShared operation to the same line. Since Master 2 holds the data, it will return the data through the interconnect to Master 3.

The outcome of this operation will be that now both Master 2 and Master 3 hold the data in a SharedClean state. However, if the interconnect has a L3 cache (like ARM's CCN-504), it will be updated with the same data.

Finally, Master 1 (a core that does not possess its own cache) performs a ReadOnce operation to the same line. This time the internal L3 cache of the interconnect rather than other masters will return the data.

So in this simple example, three read commands were issued to the same line, but the data returned from three different components. The implication for verification engineers is that the exact state of the system must be known in order to verify that the right component returned the data and that data coherency was properly maintained in the system. Such an insight in the state of the system is achieved by using both VIP for ARM CHI and Cadence's Interconnect Validator (IVD) in the verification environment. Stay tuned for in-depth information about VIP for CHI and Interconnect Validator in the upcoming blog posts!

Dimitry Pavlovsky